Analyzing your Google App Engine Logs with Google BigQuery

Developers know that logging and logs analysis can often mean the difference between delighting and disappointing users. With the Google App Engine LogService API, it’s easy to add logging to your App Engine App with just a few lines of code. But of course, logging events is only the beginning, and today we’re particularly excited to highlight using Google BigQuery to analyze your App Engine logs.

Google BigQuery is an externalized version of Google’s own logs analysis framework that allows developers to run queries across billions of rows of data in seconds via a RESTful API. BigQuery uses a familiar SQL-like query language and is able to scale to datasets that are terabytes in size and beyond. We see BigQuery as a natural fit for logs analysis, and at I/O this year, our developer relations team led a codelab demonstrating how to import and analyze App Engine logs with BigQuery.

Our customers have also had success with this technique, and App Engine developers at Streak.com posted their own walkthrough and Java framework, called Mache, for automatically exporting App Engine logs into BigQuery. Mache provides a simple interface for scheduling cron jobs that parse and ingest log file data into BigQuery at user-defined intervals.

If you’re interested in trying out Google BigQuery with App Engine, check out the getting started guide and the sample code from our I/O codelab. Happy logging!


- Posted by the Google App Engine Team


Also, if you’re interested in analyzing Datastore data in BigQuery, check out our article that shows how to use App Engine MapReduce to manage the transformation and export of Datastore entities.

AdMob SDK 6.1 Released

Last week, we released AdMob SDK v6.1 for both Android and iOS. This SDK contains a number of important bug fixes and exciting new features including:

Additional DoubleClick support

DoubleClick publishers will be happy to know that AdMob now provides support for app events, giving them the ability to execute custom code in their application when a creative dispatches an app event. Additionally, the new SDK provides support for multiple ad sizes using the same banner.

Easy access to Google Analytics

You’ll notice we’ve included the latest Google Analytics package in the “Add-Ons” directory. The new mobile app analytics provides the same best-in-class Google Analytics reporting, but for mobile apps.

If you have any questions about the AdMob SDK, please let us know on the forum or hang out during our upcoming AdMob/DFP office hours. For more information about this new SDK, take a look at our release notes.

 - Raj Parameswaran, AdMob Team

Making Google Analytics Content Experiments Even Better

A few weeks ago, we announced Google Analytics Content Experiments.  Since our announcement, we have been busy making Content Experiments available to Google Analytics users and improving it based on your feedback.  We'd like to tell you about a few changes that we have recently introduced:

Content Experiments available to everyone. Every Google Analytics user can now access Content Experiments. You can find this feature under "Experiments" in the "Content" section of your Standard Reports.

Support for relative URLs. Using relative URLs affords you increased flexibility when defining the location of variations.  This is particularly useful if you have experiments running on multiple domains, subdomains, or pages. You can learn more about using relative URLs in Content Experiments by reading our Help Center article.

Ability to copy experiments. You can now copy experiments by clicking the Copy experiment button on the Edit Settings screen of the experiment you want to copy. If you are running an experiment on a page, this allows you to run additional experiments after the original one finishes without having to add experiment code to your page or otherwise modify it.

Improvements to the experiments report. We've added regular Analytics-report capabilities to the experiment report, such as: Site Usage, Goal Set, and Ecommerce tabs, and the option to choose which variations you want to plot in the graph.

Click the above image to view the full report

We hope that you find these improvements useful. Our team is working hard to make Content Experiments in Google Analytics even better. Stay tuned for more news!

Posted by Inna Weiner, Software Engineer

I/O 2012: Innovation for Agencies

Google I/O 2012, held last month in San Francisco, was a veritable smorgasbord of innovation – from Chrome’s iOS debut to the unveiling of the Nexus 7, from the launch of Google+ Events to our most ambitious product demo yet, our annual developers’ conference wowed its 5,500 attendees with over 130 technical sessions, 20 code labs, and 155 Sandbox partners. This year, we were also pleased to introduce I/O Live, which brought each day’s events and announcements directly to our agency partners via live streaming, along with exclusive online content such as backstage interviews, product demos, some easter egg surprises, and a live blog on +Creative Sandbox with Christian Haas, Executive Director at Goodby, Silverstein & Partners.



AdWeek’s Tim Peterson recently sat down with Torrence Boone, Managing Director of Agency Business Development – Americas, to understand how a developers’ conference can hold as much significance for agencies as it does for tech enthusiasts. Long a passionate advocate of integrating digital platforms into the agency landscape, Torrence explains:

On agencies, digital, and Google platforms:

We believe that we’re creating this amazing toolkit, or palette, for agencies to do really fantastic things. And we have also seen agencies become a lot more technologically oriented, and they are investing in technology-driven approaches to the way that they’re developing campaigns.


On collaborating with agency partners:

We absolutely spend a lot of time with agencies in soliciting their input and perspective on existing products and on our product road map…They bring fantastic insights at scale because they’re working across various industries and they’re confronting different marketing problems on a day-to-day basis, so we actively solicit their perspective. Their feedback is a powerful source for us in driving the innovation engine that we have going at Google.


On the introduction of I/O Live:

The time felt ripe…to really elevate the new things that we’re introducing because, again, they enable this amazing opportunity to create new experiences and highlight for them that it’s not just about developers; that Google, even though fundamentally we’re a technology company, our desire is to allow agencies in partnership with their clients to create beautiful, seamless, immersive experiences that connect brands to consumers. Particularly given the nature of the announcements this year, it felt particularly important to amp up the visibility of the conference with our agency partners and get them in the mindset of thinking of Google I/O as a way for them to stay on the cutting edge and make sure they’re constantly pushing the envelope with what they’re doing for their clients.

Check out the full interview to learn more about how agencies are leveraging Google platforms, and what we’re doing to help take that innovation to the next level!



Posted by Kelly Duane, Americas Agency Team

Map data updates for parts of Europe, Africa, and Asia

More maps!



As part of Google's mission to provide the most complete and authoritative maps to everyone, we’re excited to announce upgraded maps for Croatia, the Czech Republic, Greece, Ireland, Italy, Lesotho, Macau, Portugal, San Marino, Singapore, and the Vatican City.



Today’s update follows previous updates in North America, Europe, and Australasia, and is part of an ongoing project we call Ground Truth: in which we source data from regional partners, supplement the data with satellite and Street View imagery, and apply a mix of algorithms and human judgement to produce a consistent and reliable geographic dataset which we can update quickly and corresponds as closely as possible to, well, the truth on the ground. In this case you’ll find our maps now include more precise area boundaries, greater coverage of road networks, and more comprehensive local points of interest. To capture and quickly fix errors reported by our users, we’ve also enabled the Report a Problem tool for these countries.



What does this mean for you, our 800,000 developers? For most of you, these updates will simply roll out over the next 24 hours and your Maps API applications will automatically display the latest imagery; however, for those developers who have cached results from any of our Maps API Web Services, please refresh your cache as soon as possible to ensure your results include the latest information.



As always, if you have any questions or concerns, please post them to our Google Maps API forums and we’ll be happy to respond.



Posted by Arthur Patterson, Software Engineer, Google Maps

Protecting yourself and others in YouTube videos

[Cross-posted from the Google Public Policy Blog]



YouTube is proud to be a place where citizens and activists come to tell their stories -- stories that may otherwise go unnoticed. A study released this week by the Pew Research Center’s Project for Excellence in Journalism found that YouTube is a top destination for news and that “citizens play a substantial role in supplying and producing footage.”



But this level of exposure can be risky to the citizens shooting the footage and the people who appear in their videos.



Today, we announced a new face blurring tool that represents a first step toward providing visual anonymity in video.



Of course, anonymity is never a guarantee, and people who capture sensitive video footage should consider taking other precautions to keep themselves and their subjects safe. Here are three suggestions:




1. Assess your risk. You and the people you film may face risk by posting video online. You may risk your own safety and that of your subjects while filming sensitive footage, during the editing process, and when you distribute your film online. After assessing the vulnerability you and your subjects face, you can make more informed decisions about when to film, whether to distribute your footage, and how widely you want to share it.



2. Consider other information which may give away identity. Video footage of your face is not the only way someone can detect your identity. Other factors that may be caught on video can also identify you or your subjects. Watch out for vocal identifiers, like recognizable voices or saying someone’s name on camera. Other footage can give away identity like a license plate, a name tag, or even the background scenery. Make sure that the imagery in your videos does not give away information about your location or identity.



3. Protect yourself when uploading. Consider, for example, local laws that may allow authorities to track the mobile device from which you upload. In certain countries, merely purchasing a sim card puts users at risk of tracking by government.



Over the past seven years, YouTube has evolved into a destination for citizen reporting. Along with curating projects like the Human Rights Channel and CitizenTube, we hope that the new technologies we’re rolling out will facilitate the sharing of even more stories on our platform.



Posted by Victoria Grand, Director, Global Communications and Policy, YouTube

Let Us Know How To Make Social More Useful For You

Google Analytics users are a digitally savvy bunch. As such we’re lucky to have a passionate community of marketers, analysts and developers that follow us across social platforms. Thousands of you read our blog posts, Tweets, and Google+ updates daily and we always use engagement data to refine what we share (and how we share it).

However, in social numbers are best paired with qualitative feedback and understanding more about who you, our most passionate users are. That’s why for the first time we’re taking feedback directly from you on how we can make our social participation even more useful.

Want more tips? More trends and opinion pieces? Video guides? Let us know and we’ll try and fulfill it. 

As a thank you for your time, we’ll be shipping 50 random respondents who complete the entire survey Google Analytics t-shirts. 

Posted by Adam Singer, Google Analytics Team




Update: this survey is now closed and we've removed the link -- thanks everyone for your feedback! We'll be using your ideas to improve how we participate in social.