The official Google Code blog

2009 was a remarkable year for developers. Vic Gundotra, VP of our developer team declared at Google I/O, "The web has won!" and this year was full of launches and announcements that remind us how the web has become the platform of our day. We found lots of inspiration from the developers at Google I/O in San Francisco and at our Google Developer Days in Japan, China, Brazil, Russia and the Czech Republic.



Here's a look back at some of our favorite highlights from 2009:

• Faster Apps Faster - the theme for the recent Campfire One event letting developers know how GWT 2.0 decreases development time (it's being used by Google Wave & Adwords for this reason) and how Speed Tracer gives you new insight into speeding up your apps


• Make the Web faster - as part of an effort to Make the web faster we launched tools like Page-Speed and our DNS resolution service. We also made our internal JavaScript tools available, called Closure Tools, so you can rev up your js like the guys on Gmail. With PubSubHubbub all your feeds can be real-time streams.


• A faster browser - A few weeks ago we launched Google Chrome for Mac and Linux, along with an extension framework for Chrome and a gallery for developers to distribute their extensions -- there are over 900 to check out already! Chrome has a set of developer tools built on WebKit to help optimize your code. Plus, we released details of Chrome OS...a netbook operating system for speed freaks. Along the way, we've worked with the larger web community contributing to the exciting future of the web with HTML5.


• Mobile - Android launched on one device in one country on one carrier in 2008. This year, 20 devices in 26 countries on 32 carriers have been launched or announced. Developers have created over 16,000 apps in the Android Marketplace. Some were winners of our second Android Developer Challenge. We launched new mobile versions of gmail and calendar using HTML5. You can take a picture to search using Google Goggles and get turn-by-turn navigation with Google maps. Plus, we recently announced our mobile lab for Android hardware.


• Hosting - App Engine apps were allowed to scale beyond free quotas and Java developers were invited to try it out at Campfire 1 in April. Developers responded, with over 100K apps  built on App Engine. Open source developers using Project Hosting were able to host projects on Mercurial, in addition to Subversion.


• New APIs - Google Wave was released early to developers at Google I/O in May. Wave is "equal parts conversation and document" and showcases what is possible in the browser. Developers can build robot extensions to automate common tasks or build gadget extensions to provide a new way for users to interact. Google Maps released v3, Friend Connect got more personalized and then got cozy w/ Twitter, Google Analytics released an API and then added asynchronous tracking.


• Google Web Elements - From maps to search to translate and more, Google Web Elements made it even easier than ever to add Google products to your site.


• Labs - Google Code Labs was created to let developers know which APIs and products are cutting edge labs "experiments" and which we are committed to supporting for the long term.

It is a very exciting time to be a developer...we are just starting to see what is possible with the web as the platform. It will be a lot of fun to see where all of us, together, can take the web in 2010!

Happy Holidays from the Google Developer Team!

By Mike Marchak, on behalf of the Google Developer Team

We know it's easier for developers to program in the languages they know. So we updated the Google Analytics API Python Client library with all the new API version 2 features and added reference exampels for both the Account Feed and Data Feed. Now it's easier than ever to automate your analysis workflow using our API.

Taking The Library For a Spin

With the updated library, we thought it would be a great time to highlight the power of the new v2 features. So we created a sample application to do just that. The application uses the new Google Analytics Python client library to retrieve metrics for a series of segments. It then performs some calculations on the data and creates bar charts using the GChartWrapper package, an open source Python wrapper for the Google Charts API. Finally, it uses the Python Imaging Library to add a title and legend, and stitches all the charts together into a single image. We decided to release this application as open source so you can create visualizations with your own data.

Solving Business Problems

With social media all the rage, we wanted to use this new application to help Avinash Kaushik, our Analytics Evangelist, to measure "engagement" on his popular Occam's Razor blog. We also wanted to determine if the time he spends participating in social media sites is valuable and sends new readers to his blog.

First we created segments to pull all the referrals from Facebook and Twitter. Second, we chose five calculations and corresponding metrics to compare the performance of thee two segments. We then compared the segments to each other and, for context, to all the visits to the site as a control.

They say a picture is worth a thousand words, here are the results:



Let's Analyze

Some interesting observations become apparent.

• Far more visits originate from Twitter (3.6x) when compared to Facebook, perhaps not surprising given Avinash's Twitter followers (~16,120)
• Visitors from Twitter tend to be new visitors, a good thing, but they view fewer pages and spend significantly less time on the blog.
• On the other hand Facebook delivers an audience that is loyal. These visitors come back to the site more often and spend a significant time on the blog (compared to Twitter and all other visitors).

The bottom line? Even though social networking sites are all the rage, they actually contribute very little to Avinash's blog. If this blog were a company, it would be wise to ensure the time and effort put into driving traffic from social media is proportionate to the actual volume of traffic and goal conversions from those sites.

Hopefully this example shows how powerful our new features can be.

If you're interested in running this report against your own data, the application is free and open sourced. Additionally, we made it really easy to change the metrics, segments, calculations and all the other visual properties to power your own visualizations. So please download it here and give it a whirl, we would love to hear your feedback.

By Nick Mihailovski, Google Analytics API Team

When I started work at Google, I visited the Google Earth team, hoping to find a 20% project on my favorite Google product. There I met Bruno Bowden, who introduced me to a problem I had never thought much about: how to take browser sizes into account when designing a page.

Bruno had noticed that many people who visit the “Download Google Earth” page never actually download, even though, as you can see, the button is pretty hard to miss:



He wondered if a significant number of users might have their browser windows too small to see the button:



To analyze this, Bruno looked at how large people's browser windows were when they visited this page. His first key idea was to measure not the entire browser window, but just the client area -- no toolbars, status bars, or other chrome.

Bruno's second key idea was to render several weeks' worth of page visitor browser sizes in a contour visualization:



Using this visualization, Bruno confirmed that about 10% of users couldn't see the download button without scrolling, and thus never noticed it. 10% may not sound like a lot, but in this context it turns out to mean a significant number of people weren't downloading Google Earth. Using this data, the team was able to redesign the page to good effect.

Bruno and I realized that Web designers might benefit from this information if it could be made more generally available. We constructed a page that could overlay a DIV containing the contour visualization atop an IFRAME containing any other Web page:



This turned out to be a good way to see which controls were and weren't visible at typical browser sizes. The only problem was, the overlay DIV prevented mouse events from getting to the page IFRAME, so it wasn't possible to interact with the page.

To solve this, we split the overlay DIV into four:



Each of the outlines above (red, yellow, blue, green) represents a separate DIV. As the mouse pointer moves, we resize and reposition the DIVs to leave a small window of blank space around the pointer, and adjust background offsets for each DIV to make the overlay look like one seamless graphic. (We originally did this on a timer, but we found a simpler way: when the mouse touches any of the DIVs, resize/reposition all of the DIVs.) End result: a designer can click and otherwise interact with the page with the mouse, and thus interact with the site normally instead of repeatedly typing in URLs.

We are now making this tool available to the public on Google Labs. To try it, simply visit browsersize.googlelabs.com and enter the URL of a page you'd like to examine. The size overlay you see is using latest data from visitors to google.com, so this should give you a pretty good indication of what parts of your UI are generally visible and what aren't.

We look forward to receiving your comments at browser-size-external-feedback!

By Arthur Blume on behalf of the Google Browser Size team

Over the past few months we've received a lot of great feedback from our developers about what they wanted to see in the Google Analytics API. Today we're excited to announce new powerful and flexible features to the Google Analytics Data Export API including:

Support for Advanced Segments

With advanced segmentation, you can look beyond the totals and into the nuances of the data for your site. For example, the average time on site for all visits could be 60 seconds, but when you segment by country, you might learn that average time on site of visits from Poland is over 2 minutes.

So we've added two new ways to use advanced segments through the API:

1. Create them on the fly by specifying their expression directly through an API query.
2. Use advanced segments created in the Google Analytics web interface through the API.

This video describes exactly what advanced segments do and how you can use them with the API.



Goal 5-20 and Configuration Data

With the recent Google Analytics v4 launch enabling up to 20 goals, many of you asked for access to this valuable data, and we listened. Now there are 48 new metrics to access goal performance. We've also added all the goal configuration data, including name, type, step names for each profile.

Here's a great video describing the depth of the goal configuration data.



Custom Variables

Custom variables are powerful new ways to describe visitors, visits and pages within Google Analytics. In this new release, we've added 10 new dimensions to access custom variable data. In addition, every custom variable that you've used is now available through the Account Feed.

All the details of this release can be found on our public changelog and public notify group. We've updated all our documentatation at http://code.google.com/apis/analytics. Please continue to give us feedback to improve our product through our public google group.

Thanks!

By Nick Mihailovski on behalf of the Google Analytics API Team

Today I'm excited to announce that Google Fusion Tables is releasing its own API.

What is Fusion Tables? A product launched recently in Google Labs, Fusion Tables is a free service for sharing and visualizing data online. It allows you to upload data, share and mark up your data with collaborators, merge data from multiple tables, and create visualizations like charts and maps.

Do you have data you need to share with other organizations? In Fusion Tables, you can share all or part of a table with other people. Does your data mean more when seen together with other datasets you don't own? By merging your data with other people's shared tables, you can see the whole picture in one place, discuss the data in embedded comments, and mark up the data with your collaborators. Fusion Tables keeps track of who contributed each part of the data and who has permission to edit.

Watch Circle of Blue's video description of how they use Fusion Tables to combine and visualize water data.



Often the real meaning and potential impact of a database can be hidden behind all the raw names and numbers, but a well-chosen visualization can bring the data to life. Fusion Tables has automatic data visualization built in: we've integrated with the Google Maps API and the Google Visualization API so you can view your data in maps, motion charts, and graphs. All of these can be embedded in your webpage, your Google Site, your blog...any Web page you want! The visualizations even update automatically as data is updated or corrected. Embed the visualization once, and the latest version will always be shown automatically.

Let other people help spot outliers and unexpected values in your dataset by linking them directly to data that is filtered, aggregated, and visualized for various angles of examination. Fusion Tables' data discussion features help you gather feedback from your community.

Is your dataset active, always changing? Is it being collected right now on cell phones or websites? With the new Fusion Tables API, you can update and query your dataset in Fusion Tables programmatically, without ever logging in to the Fusion Tables website. The API means you can import data from whatever data source you may have, whether a text file or a full-powered data base. On the more exotic side, imagine you're collecting data via survey software on GPS-enabled cell phones, as the Open Data Kit project is doing. Open Data Kit uses Google App Engine and the Fusion Tables API to instantly map locations of survey results.

Are you a data exhibitionist? Put your data in Fusion Tables and make it available for the world to see! Fusion Tables will maintain your attribution as your data participates in other tables, enforce your choices about sharing and exporting the data, and invite Google Web Search to index the table.

Fusion Tables allows datasets to play together in a safe, collaborative, and privacy-controlled environment. We can't wait to hear about the amazing things you will make happen with Fusion Tables.

By Anno Langen, Jayant Madhavan and Rebecca Shapley, Google Fusion Tables Team

We've concluded Google Developer Days 2009, a set of one-day developer events that travel around the world. This year, they started in Beijing, China on June 5 and ended in Moscow, Russia on November 10. At each event, attendees had the opportunity to learn about Google's newest web technologies with products such as Google Wave, Android, Chrome, Google Web Toolkit, OpenSocial, and App Engine. They also interacted with Google developers during "office hours" and had the chance to see how other developers are using Google technologies for their own applications at various demo areas.

We have finished posting the presentations and photos from our final events of the year, and hopefully each location's page will continue to be a useful resource for you. Thank you for making these such great events! We look forward to seeing you in 2010...

By Alyssa England Sachs, Google Developer Team

Earlier this year we introduced Google Web Elements, an easy way to embed Google products on your site by simply copying and pasting a snippet of code. Today we're excited to announce the addition of three new Web Elements:

• Translate element
  
  The Translate element enables visitors to automatically translate your pages into their native language, extending the reach of your website to a global audience. Even better, it will know when their language doesn't match your site's. Have some multi-lingual visitors? Don't worry, they'll be able to turn it off completely.
  
  Note that whether or not the Translate element pops up is based on the language of your web browser. In order to test what other language visitors will see on your site, you'll have to follow the directions here to change your browser's language.



• Reader element
  
  The Reader element embeds the headlines from your recent shared items on Google Reader right on your website. You can customize the element by choosing how many items to show and the color scheme. The headlines shown in the Reader element will automatically update as you share or unshare items.








• Orkut element
  
  The Orkut element makes it easy for your website's visitors to share your site on their Orkut page. When visitors to your site click the button, they'll be able to share your website content with their Orkut friends with a couple clicks

We've also added a couple new features that we hope you'll enjoy:

• Custom Search element themes:  The Custom Search element lets visitors search your site (or other content defined by you) and enables you to earn revenue (via AdSense) from highly relevant ads that are displayed in the search results. You've always been able to richly style this element (we described a couple ways in this blog post), but that required knowing JavaScript and CSS. Now you can adjust the look and feel of the element to better integrate with your site, without having to code anything.


• Login integration:  The Google Web Elements website is now integrated with your Google account.  By logging in (if you're not logged in already), you'll be able to more easily select your Calendar, Presentation or Spreadsheet in creating your element.  Login is also required to personalize your Reader element.


• Document selector: Once logged into the Google Web Elements website, you'll be able to easily search through your data when creating a Calendar, Presentation or Spreadsheet element.  All you have to do is click the "Choose" button and select the one that you would like to use with your element.  You still have the option of pasting in a URL (for instance, if you wish to use a document not associated with your account), but this is no longer required.
  
  

We hope you enjoy these new additions to the Web Elements family! To get started, check out www.google.com/webelements.

By Christine Tsai, Web Elements Team

Tonight at a Google Campfire One we released Google Web Toolkit 2.0, aiming to do two main things for developers:

• Make it easier to build faster apps
• Speed up the overall development cycle

This is a very exciting release because it's the cumulation of a year and a half working with teams like Google Wave, AdWords, and Orkut (among many others inside and outside of Google) to evolve GWT to meet the needs of today's web applications. There are many features and improvements, but let me call out three which we're especially excited about.

Faster Apps

Introducing: Performance profiling with Speed Tracer
The first thing you'll notice in 2.0 is that we've added a new tool called Speed Tracer. Speed Tracer is a performance profiler for Google Chrome that allows developers to see what's going on in a way which hasn't been possible before. We've worked closely with the Webkit community to add instrumentation in the browser to enable developers to gain deep insights into how code behaves, uncovering problems which have been hidden up till now.

Introducing: Incremental app download with code splitting
Another feature we've added into Google Web Toolkit is developer-guided code splitting. Code splitting allows a developer to split up their application for much, much faster startup times. Imagine if you have a settings page that users go to once a week. Why download that JavaScript when the application starts up? With code splitting, your users download just the JavaScript they need to get started.

Faster Development

Introducing: Declarative UI with UiBinder
UiBinder is a new declarative UI framework in Google Web Toolkit which enables rapid design iteration and a clean separation between presentation layer and application logic.

Dive into the details and more features in GWT 2.0.



By Andrew Bowers, GWT Product Manager

Today, we launched Google Chrome's extensions system in the beta channel for Windows and Linux (Mac is in progress). We've also opened up our brand new gallery, which contains more than 300 extensions!

We aspired to create an extensions system that is easy to use, stable, more secure and that wouldn't slow down Google Chrome. We're really happy to release a beta that begins to deliver on our initial vision. If you want to learn more about Google Chrome extensions, you can start by reading our docs and joining our mailing list. You can also check out the videos below:



You can also meet us in person - let us know if you want to join us in a small get together tomorrow on our campus in Mountain View (or via VC in New York and Kirkland). Space is limited - we'd love to see many of you there so RSVP early and we'll email you more information if you are selected to attend. You can also meet the team at Add-on Con, where we will participate in a couple of panels.

Finally for those of you who are far away, we are planning some online developer tutorial sessions. If you are interested in attending, please fill in this form.

By Erik Kay and Aaron Boodman, Google Chrome Software Engineers

Today, we're excited to announce the release of the Google Translator Toolkit Data API. Translator Toolkit is a powerful but easy-to-use editor that enables translators to bring a human touch to machine translation through translation search, bilingual dictionaries, and custom terminology databases. Using Translator Toolkit, you can translate HTML, Word, AdWords, Wikipedia, and other documents in a WYSIWYG ("what-you-see-is-what-you-get") editor, share them with other users, and download their translations onto your desktop.

Through our new API, you can upload, share, download, and delete your documents, glossaries, and translation memories using the Google Data Protocol. That means that integrating human translation into your translators' workflow just got easier! Here are a few things you can do with the Translator Toolkit API:

• Automatically connect your content management system (CMS) with Translator Toolkit. You can transform your content into HTML files, protect sections of HTML from translation through the class="notranslate" attribute, upload the HTML files, share the files with your translators, download the translated HTML, and then transform and upload the documents back into your CMS.


• Automatically connect your file system with Translator Toolkit. You can create a cron job that uploads files into Translator Toolkit, shares the files with your translators, then downloads completed files back to the file system.

For now, the API is available in labs as we rapidly add features based on your feedback. Check out our documentation, where you'll find our Java client library and a developer guide to get you started. Please visit our new developer forum if you have questions.

By Mayank Gupta and Paneendra Ba, Google Translator Toolkit Team

Today, as part of our efforts to make the web faster, we are announcing Google Public DNS, a new experimental public DNS resolver.

The DNS protocol is an important part of the web's infrastructure, serving as the Internet's "phone book". Every time you visit a website, your computer performs a DNS lookup. Complex pages often require multiple DNS lookups before they complete loading. As a result, the average Internet user performs hundreds of DNS lookups each day, that collectively can slow down his or her browsing experience.

We believe that a faster DNS infrastructure could significantly improve the browsing experience for all web users. To enhance DNS speed but to also improve security and validity of results, Google Public DNS is trying a few different approaches that we are sharing with the broader web community through our documentation:

• Speed: Resolver-side cache misses are one of the primary contributors to sluggish DNS responses. Clever caching techniques can help increase the speed of these responses. Google Public DNS implements prefetching: before the TTL on a record expires, we refresh the record continuously, asychronously and independently of user requests for a large number of popular domains. This allows Google Public DNS to serve many DNS requests in the round trip time it takes a packet to travel to our servers and back.


• Security: DNS is vulnerable to spoofing attacks that can poison the cache of a nameserver and can route all its users to a malicious website. Until new protocols like DNSSEC get widely adopted, resolvers need to take additional measures to keep their caches secure. Google Public DNS makes it more difficult for attackers to spoof valid responses by randomizing the case of query names and including additional data in its DNS messages.


• Validity: Google Public DNS complies with the DNS standards and gives the user the exact response his or her computer expects without performing any blocking, filtering, or redirection that may hamper a user's browsing experience.

We hope that you will help us test these improvements by using the Google Public DNS service today, from wherever you are in the world. We plan to share what we learn from this experimental rollout of Google Public DNS with the broader web community and other DNS providers, to improve the browsing experience for Internet users globally.

To get more information on Google Public DNS you can visit our site, read our documentation, and our logging policies. We also look forward to receiving your feedback in our discussion group.

By Prem Ramaswami, Public DNS Team

Did you hear that Friend Connect and Twitter are more closely integrated than ever? To learn more, check out our post on the Social Web Blog.

By James Reilly, Google Friend Connect Team

Today we're excited to announce our new Google Analytics Asynchronous Tracking Code snippet as an alternative way to track your websites! It provides the following benefits:

• Faster tracking code load times for your web pages due to improved browser execution
• Enhanced data collection & accuracy
• Elimination of tracking errors from dependencies when the JavaScript hasn't fully loaded

Here is the JavaScript source of the new tracking snippet:

<script type="text/javascript">

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXXXX-X']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script');
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 
        'http://www') + '.google-analytics.com/ga.js';
    ga.setAttribute('async', 'true');
    document.documentElement.firstChild.appendChild(ga);
  })();

</script>

The first part of the asynchronous tracking code snippet assigns the _gaq variable to a JavaScript array. After that, two tracking API calls (encoded as arrays) are pushed onto _gaq. When the tracking code initializes, it transforms the _gaq object from a standard array into a new object and executes all the tracking API calls initially collected in the array. With this feature, you can immediately store all necessary tracking calls even before the Google Analytics tracking code is downloaded! No more worrying about race conditions or dependency issues on the ga.js tracking code.

The second half of the snippet provides the logic that loads the tracking code in parallel with other scripts on the page. It executes an anonymous function that dynamically creates a <script> element and sets the source with the proper protocol. As a result, most browsers will load the tracking code in parallel with other scripts on the page, thus reducing the web page load time. Note here the forward-looking use of the new HTML5 "async" attribute in this part of the snippet. While it creates the same effect as adding a <script> element to the DOM, it officially tells browsers that this script can be loaded asynchronously. Firefox 3.6 is the first browser to officially offer support for this new feature. If you're curious, here are more details on the official HTML5 async specification.

Once loaded, the tracking code, transforms the _gaq array into an Analytics _gaq object. This object acts as a wrapper for the underlying _gat object and executes all the commands, sending data to your Google Analytics account. Your page code can ignore this fact though, because the _gaq.push syntax can be used at any time. See the Asynchronous Tracking Usage Guide for more details.

The new tracking code is now in Beta and available to all Google Analytics users. Keep in mind that use of the code is also optional: all your existing Google Analytics code will continue to work as-is should you decide not to adopt the new tracking method. But if you want to improve the speed of your website and the increase accuracy of your Analytics data, then we think you'll love this new option.

Learn more about this new tracking code in our Google Code developer docs and get started with our migration guide.

By Brian Kuhn, Google Analytics Team

The last few weeks, Google Chrome's developer tools have become much more useful. Besides benefiting from the work the WebKit team has done to improve Web Inspector (our developer tools are partially based on Web Inspector), we also recently released the heap profiler and the timeline tab in Google Chrome's Developer Channel.

With the heap profiler you can now take a snapshot of the JavaScript heap at any point in time. A heap snapshot helps you understand memory usage, and by comparing snapshots you can also follow memory usage over time. You will find the heap profiler in the profiles tab along with the sample-based CPU profiler.

The new timeline view gives you a complete overview of where time is spent when loading a web app. All events -- ranging from loading resources over parsing and executing JavaScript to calculating styles and repainting -- are plotted on a timeline.

Besides these product improvements, we've tried to make the Google Chrome Developer tools easier to find and understand by putting together mini site with tutorials and videos.



To take our newest release for a spin, get Google Chrome from the Developer Channel and you'll automatically be brought up to date. We welcome your feedback and your contributions to improve developer tools in WebKit and Google Chrome even more.

By Pavel Feldman, Software Engineer and Anders Sandholm, Product Manager

We work hard to make sure that the videos on the GoogleDevelopers channel on Youtube are captioned, but when I/O added over a hundred hours of video content, we got a little behind. I'm happy to announce that we're finally caught up! Every English and Spanish video from I/O now has captions that you can turn on in YouTube.

Didn't know we had captions? Just click to select captions from the menu in the lower right corner of the video player.

Some caption and subtitle-related news:

• A group of volunteers from Russia used the translated.by software to crowdsource translation for Google Wave video captions. Thank you, habratranslation! Check out one of the Wave videos with Russian subtitles. (You have to choose Russian from the caption menu in YouTube to see them.)


• If you'd like to help translate captions for any of our videos, please email google-video-captions@googlegroups.com with a request. We'd be happy to share any caption files that you might be interested in under a creative commons attribution license. If you send us the translation, we'll credit you in the video caption track and blog about how awesome you are.


• In addition to machine translation for captions, YouTube now provides experimental automatic caption transcription using the same speech recognition algorithms found in Google Voice. The GoogleDevelopers channel is part of the initial pilot, so this feature is available on many of our videos. To learn more, check out the blog post on the Official Google Blog.

By Naomi Bilodeau, Google Developer Team

Did you know Google has released more than 300 open source projects to date? Yesterday, we announced the latest addition to Google's open source projects - YouTube Direct, a new tool that enables any developer to solicit video submissions, moderate and display them on their website, all powered by YouTube. We recognize the role that open source plays at Google and how it helps us create better applications and we try to give back to the community as much as possible.

YouTube Direct was built on top of YouTube's public APIs and is designed to run on Google App Engine - Google's highly scalable platform. To date, several media organizations like ABC News, The Huffington Post and Politico have taken advantage of the open platform to deploy their own version of YouTube Direct to empower citizen journalism and enrich their site in the process. We look forward to see for more creative usage of the tool.

By Amanda Surya, YouTube Direct team

A couple days ago, Google welcomed Don Dodge to our Developer Relations team, where he joins us as a Developer Advocate working with developers, startups, and other Google Apps partners. We're expecting Don to be a fantastic addition to our team. He's already a prominent voice in the developer community, well-known and highly-regarded among entrepreneurs, technologists, and the media.

In the TechCrunch post first announcing Don's availability, Michael Arrington wrote how Don, "makes a big effort to give young startups the attention they deserve. This is a guy who gives a heck of a lot more to the community than he ever takes back." This dedication to the community of developers and the businesses they build is one of the things that excites us the most about having Don on our team. These businesses have been central to Google's success over the years, so we already know that Don's attitude will fit right in with our efforts. Don has deep experience working in startups from his days at companies like AltaVista, Napster, and Groove Networks, and has always continued to maintain the connection and passion for that community since leaving their ranks to join Microsoft, and now Google. We are eager for Don to share his personal experience and professional insights with developers and small businesses integrating with Google Apps, and be an advocate for developers and partners inside the company.

Don already wrote about his first day on the job at Google. Tomorrow you can hear him speak on the Enterprise Cloud Summit Panel in New York City. You can follow Don on his personal blog, email him at dondodge at google.com, or follow @dondodge on Twitter.

By Michael Winton, Google Developer Relations

Have you heard about Go? We released a new, experimental systems programming language today. It is open source and we're excited about sharing it with the development community. For more information, check out the Google Open Source blog.

By Robert Griesemer, Rob Pike, Ken Thompson, Ian Taylor, Russ Cox, Jini Kim and Adam Langley - The Go Team

Every day, more than 99 human years are wasted because of uncompressed content. Although support for compression is a standard feature of all modern browsers, there are still many cases in which users of these browsers do not receive compressed content. This wastes bandwidth and slows down users' interactions with web pages.

Uncompressed content hurts all users. For bandwidth-constrained users, it takes longer just to transfer the additional bits. For broadband connections, even though the bits are transferred quickly, it takes several round trips between client and server before the two can communicate at the highest possible speed.  For these users the number of round trips is the larger factor in determining the time required to load a web page. Even for well-connected users these round trips often take tens of milliseconds and sometimes well over one hundred milliseconds.

In Steve Souders' book Even Faster Web Sites, Tony Gentilcore presents data showing the page load time increase with compression disabled.  We've reproduced the results for three highest ranked sites from the Alexa top 100 with permission here:



The data from Google's web search logs show that the average page load time for users getting uncompressed content is 25% higher compared to the time for users getting compressed content. In a randomized experiment where we forced compression for some users who would otherwise not get compressed content, we measured a latency improvement of 300ms.  While this experiment did not capture the full difference, that is probably because users getting forced compression have older computers and older software.

We have found that there are 4 major reasons why users do not get compressed content: anti-virus software, browser bugs, web proxies, and misconfigured web servers.  The first three modify the web request so that the web server does not know that the browser can uncompress content. Specifically, they remove or mangle the Accept-Encoding header that is normally sent with every request.

Anti-virus software may try to minimize CPU operations by intercepting and altering requests so that web servers send back uncompressed content.  But if the CPU is not the bottleneck, the software is not doing users any favors.  Some popular antivirus programs interfere with compression.  Users can check if their anti-virus software is interfering with compression by visiting the browser compression test page at Browserscope.org.

By default, Internet Explorer 6 downgrades to HTTP/1.0 when behind a proxy, and as a result does not send the Accept-Encoding request header. The table below, generated from Google's web search logs, shows that IE 6 represents 36% of all search results that are sent without compression.  This number is far higher than the percentage of people using IE 6.


There are a handful of ISPs,  where the percentage of uncompressed content is over 95%.  One likely hypothesis is that either an ISP or a corporate proxy removes or mangles the Accept-Encoding header.  As with anti-virus software, a user who suspects an ISP is interfering with compression should visit the browser compression test page at Browserscope.org.

Finally, in many cases, users are not getting compressed content because the websites they visit are not compressing their content.  The following table shows a few popular websites that do not compress all of their content. If these websites were to compress their content, they could decrease the page load times by hundreds of milliseconds for the average user, and even more for users on modem connections.


To reduce uncompressed content, we all need to work together.

• Corporate IT departments and individual users can upgrade their browsers, especially if they are using IE 6 with a proxy. Using the latest version of Firefox, Internet Explorer, Opera, Safari, or Google Chrome will increase the chances of getting compressed content.  A recent editorial in IEEE Spectrum lists additional reasons - besides compression - for upgrading from IE6.
• Anti-virus software vendors can start handling compression properly and would need to stop removing or mangling the Accept-Encoding header in upcoming releases of their software.
• ISPs that use an HTTP proxy which strips or mangles the Accept-Encoding header can upgrade, reconfigure or install a better proxy which doesn't prevent their users from getting compressed content.
• Webmasters can use Page Speed (or other similar tools) to check that the content of their pages is compressed.

For more articles on speeding up the web, check out http://code.google.com/speed/articles/.

By Arvind Jain, Engineering Director and Jason Glasgow, Staff Software Engineer

Millions of Google users worldwide use JavaScript-intensive applications such as Gmail, Google Docs, and Google Maps. Like developers everywhere, Googlers want great web apps to be easier to create, so we've built many tools to help us develop these (and many other) apps. We're happy to announce the open sourcing of these tools, and proud to make them available to the web development community.

Closure Compiler
Closure Compiler is a JavaScript optimizer that compiles web apps down into compact, high-performance JavaScript code. The compiler removes dead code, then rewrites and minimizes what's left so that it will run fast on browsers' JavaScript engines. The compiler also checks syntax, variable references, and types, and warns about other common JavaScript pitfalls. These checks and optimizations help you write apps that are less buggy and easier to maintain. You can use the compiler with Closure Inspector, a Firebug extension that makes debugging the obfuscated code almost as easy as debugging the human-readable source.

Because JavaScript developers are a diverse bunch, we've set up a number of ways to run the Closure Compiler. We've open-sourced a command-line tool. We've created a web application that accepts your code for compilation through a text box or a RESTful API. We are also offering a Firefox extension that you can use with Page Speed to conveniently see the performance benefits for your web pages.

Closure Library
Closure Library is a broad, well-tested, modular, and cross-browser JavaScript library. Web developers can pull just what they need from a wide set of reusable UI widgets and controls, as well as lower-level utilities for the DOM, server communication, animation, data structures, unit testing, rich-text editing, and much, much more. (Seriously. Check the docs.)

JavaScript lacks a standard class library like the STL or JDK. At Google, Closure Library serves as our "standard JavaScript library" for creating large, complex web applications. It's purposely server-agnostic and intended for use with the Closure Compiler. You can make your project big and complex (with namespacing and type checking), yet small and fast over the wire (with compilation). The Closure Library provides clean utilities for common tasks so that you spend your time writing your app rather than writing utilities and browser abstractions.

Closure Templates
Closure Templates grew out of a desire for web templates that are precompiled to efficient JavaScript.  Closure Templates have a simple syntax that is natural for programmers.  Unlike traditional templating systems, you can think of Closure Templates as small components that you compose to form your user interface, instead of having to create one big template per page.

Closure Templates are implemented for both JavaScript and Java, so you can use the same templates both on the server and client side.


Closure Compiler, Closure Library, Closure Templates, and Closure Inspector all started as 20% projects and hundreds of Googlers have contributed thousands of patches. Today, each Closure Tool has grown to be a key part of the JavaScript infrastructure behind web apps at Google.  That's why we're particularly excited (and humbled) to open source them to encourage and support web development outside Google. We want to hear what you think, but more importantly, we want to see what you make. So have at it and have fun!

By the Closure Tools team

Today, we're excited to announce several new features for Google Friend Connect that make it possible for website owners to get to know their users, encourage users to get to know each other, and match their site content (including Google ads) to visitors' interests. Check out the Google Social Web Blog for an overview of these new features.

We also want to point out that there are APIs for developers who want to play with the interests data programmatically. With the new interest data described on the Social Web Blog, developers can write custom polls and access the interests data directly. Friend Connect provides API level access to both individual interests information as well as aggregate information for all users of a site. Interests information can be added programmatically for the signed-in user or via the poll gadget, and it can be accessed via both the JavaScript API and the OpenSocial REST API. The Guitar Universe example site should give you an idea of some of the things that are possible with this new launch.

As always, feel free to ask technical questions related to the Friend Connect APIs in the developer forum.

By Mussie Shore, Product Manager, Google Friend Connect

Google has recently added three important enhancements to our OAuth support:

1. The ability to use OAuth without registration
   
2. Support for software apps installed on a computer or mobile phone
3. Additional controls for our Google Apps Premier and Education customers which allows administrators to give another web application access to a subset of the data Google stores for that organization

Below is an overview of each enhancement, or you can refer to our updated OAuth documentation.

1. The ability to use OAuth without registration

Based on consistent feedback from our developers, we added the ability to use OAuth without having to register the website ahead of time. This change is especially helpful for developers working on test servers that cannot be accessed directly from the Internet.

2. Support for software apps installed on a computer or mobile phone

Many of the larger enterprises that use the Google Apps service choose to run their own login system. They accomplish this by leveraging our support for the SAML protocol which defines a way for Google to redirect the user to the company's login system to be authenticated before accessing their mailbox at Google.  However, in this situation Google normally does not have a password for the user — especially if the enterprise authenticates the user with a password and with a second factor of authentication (such as a token generator they carry on a keychain). Unfortunately, there are many installed software applications created by both Google and ISV developers that use Google's APIs, and those applications are hardcoded to ask a user for their email and password using Google's ClientLogin API. With this new OAuth feature, the software application can now launch a web browser and start a process that both logs the user in through their central SAML login system, and that also gets the user's consent to access their data hosted at Google. Because the user authentication is done in the web browser, it will work with the enterprise's existing login system.  Google is encouraging any ISV that uses the ClientLogin API to add support for this new OAuth flow, enabling usage by the large enterprise customers described above. Google is also planning to enhance our Google Apps Sync for Microsoft Outlook to support this feature such that Outlook can be used with both Google Apps and an enterprise's central login system.

3. Additional controls for our Google Apps Premier and Education customers which allows administrators to give another web application access to a subset of the data Google stores for that organization

This feature for our Google Apps Premier customers enhances our existing OAuth for Google Apps domain administrators, also known as 2-legged OAuth. This feature enables domain administrators to allow specific IT apps or third party web services limited access to user accounts via a centralized permissions system under the control of the  domain administrator. For example, with this new system, an administrator can use the Google Documents API to configure every user in the domain to have a Google Docs folder named "Human Resources" that is automatically populated with common employee forms.  The company might also sign up with an Enterprise SaaS vendor such as Manymoon and specify that Manymoon can access the Google Calendars of all of their users, providing tighter integration with Manymoon's project scheduling features. Previously, this feature required giving the third party vendor access to all of the data that Google stored for that organization, but with this new feature, administrators can limit access to particular data sources (Calendar, Documents, etc). Refer to our documentation for more information.

By Eric Sachs, Product Manager, Google Security

Do you operate a website and wish you could increase the percentage of users who finish the registration process? As discussed on Google's main blog, Google has been working with Plaxo and Facebook to improve the registration success rate for Gmail users. We now see success rates as high as 90%, compared to the 50-60% rate that most websites see with traditional registration mechanisms. This result was achieved using a combination of our OpenID, OAuth and Portable Contacts APIs. While those APIs have been available for over a year, we have added a number of refinements based on our experience with Plaxo and Facebook. Our documentation now has information on those new features, including:

• OpenID User Interface Extension 1.0 (including the ability to display the favicon of the website)
• x-has-session, which is an enhacement to checkid_immediate requests via the UI extension. If the request includes "openid.ui.x-has-session," it will be echoed in the response only if Google detects an authenticated session
• Support for the US Government's GSA profile for OpenID
• PAPE (Provider Authentication Policy Extension) to support forced password reprompts
• Support for not only Google Accounts, but also our Google Apps customers, as discussed on the Enterprise blog

For more details, please refer to our OpenID documentation.

While these technologies are all standards-based, the methods for how to combine them to achieve this success rate are not obvious, and took a while for the industry to refine. More information is available in the Hybrid Onboarding Guide, but below is a quick summary of some of the best practices for this hybrid onboarding technique:

• The technique is primarily for websites with an existing login system based on email addresses.
• It also assumes the website will send email to users who are not yet registered, whether it is through traditional email marketing or social network invitations.
• The website owner then needs to choose a small set of email providers such as Yahoo and Google that support these standards.
• Whenever the website sends email to a user at one of those providers, any hyperlinks that promote registration at the website should be modified to communicate the email address (or at least domain) of the user back to the website's registration page.
• If the registration page detects a user from one of these domains, it should NOT start the traditional process of asking the user to enter a password, password confirmation, and email. Instead, it should prominently show a single button that says "Sign up with your Google Account" — where Google is replaced with the name of the email provider.
• If the user clicks that button, the website should use the OpenID protocol to ask the email provider to authenticate the user, provide their email address, and optionally ask for access to their address book using the hybrid OpenID/OAuth protocol and the Portable Contacts API. More details about this flow are available on the OpenID blog.
• Once the user returns to the website, it can create an account entry for the user. The website can also mark the email address as verified without having to send a traditional "email verification" link to the user. If the website received the user's permission to access their address book, it can now download it and look for information about the user's friends.
• In the unusual case where an account already exists for that email address, the website can simply log the user into that pre-existing account. 
• For any newly registered user, the website should then display a page that confirms the user is registered and that indicates how they should sign in in the future.
• To make the login process simple, the website should modify their login box to include a logo for each of the trusted email providers it supports, or use one of the other user experiences for Federated Login.
• If a user clicks the email provider button, they can again be sent to that provider's site using the OpenID protocol. When the user comes back, the website can either detect that they previously registered, or if it is a new user, the website can create an account for them on the fly.
• In some cases the account may already exist for that email address, but it was not initially registered using OpenID. In that case, the website can simply log the user in to that pre-existing account.

By Eric Sachs, Product Manager, Google Security

It's Friday, time for some fun!

Here is a captivating way to visualize your Google Analytics data in a Treemap visualization and you can visualize your own data with our live demo.
(note: IE currently not supported for visualization part)





The goal of this example was to teach people how to use the Google Analytics API on App Engine in Java. As well as demonstrating how to use both OAuth and AuthSub along with the App Engine's various services. The code looked great, but the output was a boring HTML table. So I used some open source tools to transform the table into a pretty tree map visualization!

All the code has been open sourced on Google Project hosting. I also wrote an article describing how this application works making it easy for developers to use this example as a starting point for new data visualizations and other Google Data projects.

For the data retrieval part, this example uses the App Engine Java SDK and the Google Analytics Data Export API Java Client Library to retrieve data from Google Analytics. The example code implements both unsigned AuthSub and registered OAuth authorization methods allowing developers to get up and running quickly in development environments and later switch to a secure authorization method in production environments. The application also uses the Model-View-Controller pattern, making it flexible and allowing developers to extend the code for new applications. (like adding support for other Google Data APIs)

For the visualization part, I used the open-sourced Protovis SVG Visualization Library to create the Treemap. This JavaScript library is maintained by the Stanford Visualization Group and excels at creating brand new visualizations from a data set (in this case a boring HTML table). To handle all of the interactions, including rollover, tooltips and slider controls, I used JQuery. Here is the JavaScript source to the visualization part of the sample.

Enjoy!

By Nick Mihailovski, The Google Analytics API Team

P.S. If you have created any cool new visualizations using the Google Analytics Data Export API, email us so we can highlight them as well.

Custom Search themes make it easy for you to customize the look and feel of your search results pages. And if you want to take the customization gig further, you can also customize the result snippet—a small sample of content that gives search users an idea of what's in the webpage—by using structured data.

When you are reading a webpage that reviews a film, you can figure out what the title is, what reviewers thought of the film, and how they rated it. You can even search for stores with the best prices for the DVD. Structured data can convey the meaning of such key information to computers.

Structured data formats—such as microformats, RDFa, and PageMaps—are semantic markup that you add to your HTML page. Structured data make web content more meaningful to machines. These attributes do not change the formatting of your website, they just make the text enclosed within the XHTML tags "understandable" by computers and influence what shows up in the result snippets.

When you tag your webpages with structured data, Custom Search indexes them and sends the metadata back in the XML results for your page. You can then take this XML feed and transform it into HTML results that showcase key information—such as image thumbnails, summaries, dates, authorship, ratings, and prices. Having the most relevant information in your search results makes the webpages in your site more compelling to your users.

You can, for example, create the following kind of rich snippets:


You can even add thumbnails and actions that let your users download files or make purchases.


To learn more, read the Custom Search Developer's Guide.

By Kevin Griffin Lim, Custom Search Team