Surfacing forum posts in search results

Today, we introduced a new search feature that makes it easier for you to find forum posts or discussions related to what you're searching for. This new addition to Google search results applies to sites that tend to have a large number of posts on a specific topic. When several different discussions on a site are relevant to your query, we indent them under the primary result and include the date of each post.<br /><br />So for instance, if you search for [<a href="http://www.google.com/search?q=getting+from+rome+to+florence&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a">getting from rome to florence</a>] you will see, below the third result, a list of relevant discussions on various ways to get between these cities.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJOxl3eqY0Nx68H4S8yuwL6ImiY3oba9s80tLmuKNh1A4qH2JORzush7rrrABv2VEhcNNECA_NF5iqu6kyN_3NZKxdtQqJm7AWko6fl-D2GD9HPkCF7lnhkW4puvZv1vCvyYUaT5vcG5fN/s1600-h/search1.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5387449568174270866" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJOxl3eqY0Nx68H4S8yuwL6ImiY3oba9s80tLmuKNh1A4qH2JORzush7rrrABv2VEhcNNECA_NF5iqu6kyN_3NZKxdtQqJm7AWko6fl-D2GD9HPkCF7lnhkW4puvZv1vCvyYUaT5vcG5fN/s400/search1.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 118px;" /></a><br />It's always nice to know what others are saying about the best ways to get around (by boat or train) and how recent those comments are — especially if it's your first time traveling to Italy.<br /><br />We hope this feature gives you a deeper view into the relevant content available on sites throughout the web — even when that content spans multiple pages or discussions.<br />At the same time, the main search results are diverse as always — so if you can't pinpoint a useful comment there's a list of relevant sites there to help.<br /><br /><span class="byline-author">Posted by Johanna Wright, Director of Product Management and Devin Mullins, Software Engineer<br /></span>

Surfacing forum posts in search results

Today, we introduced a new search feature that makes it easier for you to find forum posts or discussions related to what you're searching for. This new addition to Google search results applies to si…

Đọc thêm »

Jump to the information you want right from the search snippets

For most search results, Google shows you a few lines of text to give you an idea of what the page is about — we call this a "search snippet." Recently, we've enhanced the search snippet with two new features that make it easier to find information buried deep within a page.<br /><br />Normally, a search snippet shows how a page, as a whole, relates to a your query by excerpting content that appears near and around where your query terms show on the page. But what if only one section of the page is relevant to your search?<br /><br />That's where these new features can help, by providing links within the snippet to relevant sections of the page, making it faster and easier to find what you're looking for. Imagine, for example, that we're researching trans fats and cholesterol, and their effects on the body. If we start with a generic query like [<a href="http://www.google.com/search?q=trans+fats">trans fats</a>], Google returns several results with lots of information about trans fats in general, including this result from Wikipedia:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQMZqrLTIR6bluy9LnoE6oBiFK-lpahwM01JqKdG7utOmY6kZf9jZLbGw_MBdTCt_AmBTE8XzohjMrk5KnVHUPv2fiXVmSA2T0TGpkS-dPCIdkfcLkqVcnwkv8w8bzD0Kfxi7atu7KV6ub/s1600-h/sp1.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5385463367921356546" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQMZqrLTIR6bluy9LnoE6oBiFK-lpahwM01JqKdG7utOmY6kZf9jZLbGw_MBdTCt_AmBTE8XzohjMrk5KnVHUPv2fiXVmSA2T0TGpkS-dPCIdkfcLkqVcnwkv8w8bzD0Kfxi7atu7KV6ub/s400/sp1.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 62px;" /></a>Now, included with the snippet are links to specific sections within the page, covering different subtopics of trans fats. Since we're particularly interested in what's healthy and what's not, "<a href="http://en.wikipedia.org/wiki/Trans_fat#Nutritional_guidelines">Nutritional guidelines</a>" is probably where the most relevant information is. Clicking this link will take you directly to that section, midway down the page.<br /><br />Now imagine we're particularly interested in learning about good cholesterol and what levels of it are healthy, so we try a more specific query, [<a href="http://www.google.com/search?hl=en&q=good+cholesterol+level">good cholesterol level</a>]. The top result is from the American Heart Association and has tons of information about cholesterol levels. The specific information about good (HDL) cholesterol, however, is contained in one section titled "Your HDL (good) cholesterol level"‎. Since the query was more specific, the snippet for this result now provides the option to "jump to" just this section of the website.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5VcPPQbqIlfOURqVafhHRRyLk3CWV2RrtC0ZnUgopbRBiA3VokfA1yAYuO1SE9ULD70VNq5fqrANHhhD1TCH1uYJWcdERAFtiP96UWfAsJTG4T2RKsKPxDWx5IWeOPvOpP5rUr4Nq5UEI/s1600-h/sp2.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5385463377597364946" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5VcPPQbqIlfOURqVafhHRRyLk3CWV2RrtC0ZnUgopbRBiA3VokfA1yAYuO1SE9ULD70VNq5fqrANHhhD1TCH1uYJWcdERAFtiP96UWfAsJTG4T2RKsKPxDWx5IWeOPvOpP5rUr4Nq5UEI/s400/sp2.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 63px;" /></a><br />Clicking on "<a href="http://www.google.com/url?q=http://www.americanheart.org/presenter.jhtml%3Fidentifier%3D183%23HDL&usg=AFQjCNEfCN2c_HMJHmvxopdKBvrfQ3avLA&ei=99q7SsvzJoPANYqxlMAL&sa=X&oi=section_link&resnum=1&ct=legacy">Jump to Your HDL (<span style="font-weight: bold;">good</span>) <span style="font-weight: bold;">cholesterol level</span></a><span style="font-weight: bold;">‎</span>" takes you directly to the most relevant information on the page:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhq80a4lQpmThjfenE6Bei331kGrGxVna2QLOyOQWVusWkNv5oYsjPfd09zY8n54B-LE8ty15r9m8W8n4MALJsWE4PjleNac2mjaw_ATbovOjrFKM-zga0zCr2692h1CFoBWKMEXw54tInf/s1600-h/sp3.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5385463382285453714" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhq80a4lQpmThjfenE6Bei331kGrGxVna2QLOyOQWVusWkNv5oYsjPfd09zY8n54B-LE8ty15r9m8W8n4MALJsWE4PjleNac2mjaw_ATbovOjrFKM-zga0zCr2692h1CFoBWKMEXw54tInf/s400/sp3.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 201px;" /></a><br />Clicking on the title of the snippet ("What Your Cholesterol Levels Mean") still takes you to the top of the page, as always.<br /><br />If you're a webmaster and would like to have these links appear for your webpages, take a look at the <a href="http://googlewebmastercentral.blogspot.com/2009/09/using-named-anchors-to-identify.html">Google Webmaster Central Blog</a> for info on some of the things you can do. And in the meantime, we hope these enhancements help you find the information you're looking for faster.<br /><br /><span class="byline-author">Posted by Chris Kern, Snippets Team</span>

• 
• 
• 

1. 1
2. 2
3. 3

• Previous
• Next

Jump to the information you want right from the search snippets

For most search results, Google shows you a few lines of text to give you an idea of what the page is about — we call this a "search snippet." Recently, we've enhanced the search snippet with two new …

Đọc thêm »

Two new improvements to Google results pages

Today we're rolling out two new improvements to Google search. The first offers an expanded list of useful related searches and the second is the addition of longer search result descriptions -- both of which help guide users more effectively to the information they need.<br /><br /><span style="font-weight: bold;">More and better search refinements </span><br /><br />Starting today, we're deploying a new technology that can better understand associations and concepts related to your search, and one of its first applications lets us offer you even more useful related searches (the terms found at the bottom, and sometimes at the top, of the search results page).<br /><br />For example, if you search for [<a href="http://www.google.com/search?hl=en&q=principles+of+physics" id="m9yf" title="principles of physics">principles of physics</a>], our algorithms understand that "angular momentum," "special relativity," "big bang" and "quantum mechanic" are related terms that could help you find what you need. Here's an example (click on the images in the post to view them larger):<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzKS5-M37xQav-4XE78L8LwbW-7HnyLTXuRR3OR-qgTdAf9myLkGZjTt94kaNY178O1UZLv-Eob_RAhg1X2Y0AHhIQVX8ObBhZYcKUVQYe6PvUyYp3TlgGQtk9r2xpqoLxNkwb39KnNWCS/s1600-h/Picture+5.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5316754908836560034" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzKS5-M37xQav-4XE78L8LwbW-7HnyLTXuRR3OR-qgTdAf9myLkGZjTt94kaNY178O1UZLv-Eob_RAhg1X2Y0AHhIQVX8ObBhZYcKUVQYe6PvUyYp3TlgGQtk9r2xpqoLxNkwb39KnNWCS/s400/Picture+5.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 177px;" /></a>Let's look at a couple of examples in other languages. In Russian, for the query [<a href="http://www.google.ru/search?hl=ru&q=%D0%B3%D0%B0%D0%B4%D0%B0%D0%BD%D0%B8%D0%B5+%D0%BD%D0%B0+%D0%BA%D0%B0%D1%80%D1%82%D0%B0%D1%85" id="crp4" title="гадание на картах">гадание на картах</a>] (fortune-telling with cards), the algorithms find the related terms "таро" (tarot), "ленорман" (lenormand) and "тибетское гадание мо" (tibetan divination mo). In Italian, if you search for [<a href="http://www.google.com/search?hl=it&q=surf+alle+canarie" id="pmxj" title="surf alle canarie">surf alle canarie</a>] (surf at the canary islands), we now offer suggestions based on the three most famous Canary Islands: "lanzarote," "gran canaria," and "fuerteventura":<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo64eF-Sn4mXoAzWCEIFCKLICLqCektnrO1WwRtayOkxyf_j_JfU6QM6pNOz1r4IygOMF7tfG1HOh12AKW4Y0odjmDh4xZo8Ol1algClS5ngN0SF0DXosHfx1n4Z2DBVC4-dHjWrQGxA2L/s1600-h/Picture+6.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5316755205123011730" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo64eF-Sn4mXoAzWCEIFCKLICLqCektnrO1WwRtayOkxyf_j_JfU6QM6pNOz1r4IygOMF7tfG1HOh12AKW4Y0odjmDh4xZo8Ol1algClS5ngN0SF0DXosHfx1n4Z2DBVC4-dHjWrQGxA2L/s400/Picture+6.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 232px;" /></a>We are now able to target more queries, more languages, and make our suggestions more relevant to what you actually need to know. Additionally, we're now offering refinements for longer queries — something that's usually a challenging task. You'll be able to see our new related searches starting today in 37 languages all around the world.<br /><br />And speaking of long queries, that leads us to our next improvement...<br /><br /><span style="font-weight: bold;">Longer snippets</span><br /><br />When you do a search on Google, each result we give you starts with a <span style="color: rgb(0, 0, 153);">dark blue</span> title and is followed by a few lines of text (what we call a "snippet"), which together give you an idea of what each page is about. To give more context, the snippet shows how the words of your query appear on the page by highlighting them in <span style="font-weight: bold;">bold</span>.<br /><br />When you enter a longer query, with more than three words, regular-length snippets may not give you enough information and context. In these situations, we now increase the number of lines in the snippet to provide more information and show more of the words you typed in the context of the page. Below are a couple of examples.<br /><br />Suppose you were looking for information about Earth's rotation around the sun, and specifically wanted to know about its tilt and distance from the sun. So you type all of that into Google: [earth's rotation axis tilt and distance from sun]. A normal-length snippet wouldn't be able to show you the context for all of those words, but with longer snippets you can be sure that the first result covers all those topics. In addition, the extra line of snippets for the third result shows the word "sun" in context, suggesting that the page doesn't talk about Earth's distance from the sun:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi821rOPwKFqRKsUY_YOuXiELw0iY8gjBM8MKT3wwnNK4GH5l3ZjYZrcrBKiMFCivQbK-1WM3PfX1ecap9ukOSbHjH5MxBm_PBFM3JCXDM3bgfpvt3_FsGJDN70_12rINzqPkR00Tkt12Ge/s1600-h/Picture+7.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5316755430475690962" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi821rOPwKFqRKsUY_YOuXiELw0iY8gjBM8MKT3wwnNK4GH5l3ZjYZrcrBKiMFCivQbK-1WM3PfX1ecap9ukOSbHjH5MxBm_PBFM3JCXDM3bgfpvt3_FsGJDN70_12rINzqPkR00Tkt12Ge/s400/Picture+7.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 242px;" /></a>Similarly, if you're looking for a restaurant review that covers all the parts of the meal, longer snippets can help:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoLwcTE17ciQgnNo6n8wcAj0ORHqwhRaYG21PfcMw-SGlZf7Cfn_XA0QJRTFxsVF9g3P9ljoltEfD9D83tsMHhu93wEkKMfqIBsBXv8I8AXhbsh0iSkdc0wfxIuciX0ehY4pLfTRDsc5fU/s1600-h/Picture+8.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5316755749449536610" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoLwcTE17ciQgnNo6n8wcAj0ORHqwhRaYG21PfcMw-SGlZf7Cfn_XA0QJRTFxsVF9g3P9ljoltEfD9D83tsMHhu93wEkKMfqIBsBXv8I8AXhbsh0iSkdc0wfxIuciX0ehY4pLfTRDsc5fU/s400/Picture+8.png" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 400px; height: 221px;" /></a>But don't just take our word for it — try it out yourself with your favorite long, detailed query.<br /><br />These are just two recent examples of improvements we've made. We are constantly looking for ways to get you to the web page you want as quickly as possible. Even if you don't notice all of our changes, rest assured we're hard at work making sure you have the highest quality search experience possible.<br /><br /><span class="byline-author">Posted by Ori Allon, Technical Lead, Search Quality Team, and Ken Wilder, Snippets Team Engineer</span>

• 
• 
• 
• 

1. 1
2. 2
3. 3
4. 4

• Previous
• Next

Two new improvements to Google results pages

Today we're rolling out two new improvements to Google search. The first offers an expanded list of useful related searches and the second is the addition of longer search result descriptions -- both …

Đọc thêm »

Eye-tracking studies: more than meets the eye

Imagine that you need a refresher on how to tie a tie. So, you decide to type [how to tie a tie] into the Google search box. Which of these results would you choose?<br /><br /><img alt="" border="0" id="BLOGGER_PHOTO_ID_5297226899518548434" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ3zxth2piK3LXkTi_GBpK47lDI5N86_bWUTURY5k8uzVuBoceKedp39isy1Rs3ZgclIxwvUThArfmrUBUzto3M8Vcfk7H9sM4y7vnsQ5P78GWDeV63aw2ihudo6MndmhAZgDLgUzCEMRu/s400/1-howtotieatie.jpg" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 382px; height: 400px;" /><br />Where did your eyes go first when you saw the results page? Did they go directly to the title of the first result? Did you first check the terms in boldface to see if the results really talk about tying a tie? Or maybe the images captured your attention and drew your eyes to them?<br /><br />You might find it difficult to answer these questions. You probably did not pay attention to where you were looking on the page and you most likely only used a few seconds to visually scan the results. Our User Experience Research team has found that people evaluate the search results page so quickly that they make most of their decisions unconsciously. To help us get some insight into this split-second decision-making process, we use eye-tracking equipment in our usability labs. This lets us see how our study participants scan the search results page, and is the next best thing to actually being able to read their minds. Of course, eye-tracking does not really tell us what they are thinking, but it gives us a good idea of which parts of the page they are thinking about.<br /><br />To see what the eye-tracking data we collect looks like, let's go back to the results page we got for the query [how to tie a tie]. The following video clip shows in real time how a participant in our study scanned the page. And yes, seriously — the video is in real time! That's how fast the eyes move when scanning a page. The larger the dot gets, the longer the users' eye pauses looking at that specific location.<br /><br /><object height="344" width="425"><param name="movie" value="http://www.youtube.com/v/w29DrEEsqT4&hl=en&fs=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/v/w29DrEEsqT4&hl=en&fs=1" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object><br /><br />Based on eye-tracking studies, we know that people tend to scan the search results in order. They start from the first result and continue down the list until they find a result they consider helpful and click it — or until they decide to refine their query. The heatmap below shows the activity of 34 usability study participants scanning a typical Google results page. The darker the pattern, the more time they spent looking at that part of the page. This pattern suggests that the order in which Google returned the results was successful; most users found what they were looking for among the first two results and they never needed to go further down the page.<br /><br /><img alt="" border="0" id="BLOGGER_PHOTO_ID_5297226902585795138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH5AuBvxX4x9tc6hjlMcoVB-rrWOqXh__FcYZDY-AxgHx09-F98z_Lp0xiUoMVDIArMWJYKL1w0lhrGeH8LxkS_zfEH62L_pY-zGVN61xC2aRngLBvA4bll1lmDewHLxrrMI-5DUskaEH0/s400/2-goldentriangle.jpg" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 374px; height: 400px;" /><br />When designing the user interface for <a href="http://googleblog.blogspot.com/2007/05/universal-search-best-answer-is-still.html">Universal Search</a>, the team wanted to incorporate thumbnail images to better represent certain kinds of results. For example, in the [how to tie a tie] example above, we have added thumbnails for Image and Video results. However, we were concerned that the thumbnail images might be distracting and disrupt the well-established order of result evaluation.<br /><br />We ran a series of eye-tracking studies where we compared how users scan the search results pages with and without thumbnail images. Our studies showed that the thumbnails did not strongly affect the order of scanning the results and seemed to make it easier for the participants to find the result they wanted.<br /><br />The thumbnail image seemed to make results with thumbnails easy to notice when the users wanted them (see screenshots below — page with the thumbnail image on the right)...<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ2q3FuqakUGfxLkKrz47Uk_oKuEh7ilNWFC6lBTxBnI1tLmoamtMDBtAAyCn_AKOZqEepNHsZU07n1p5NVQJOHANVmL3Z3FoiC2IeC30Y1uZTL4231V57Yb8p2d5EzHzsn5BCRk56YKCT/s1600-h/3-aninconvenienttruth.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5299742939311627346" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ2q3FuqakUGfxLkKrz47Uk_oKuEh7ilNWFC6lBTxBnI1tLmoamtMDBtAAyCn_AKOZqEepNHsZU07n1p5NVQJOHANVmL3Z3FoiC2IeC30Y1uZTL4231V57Yb8p2d5EzHzsn5BCRk56YKCT/s400/3-aninconvenienttruth.jpg" style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 142px;" /></a><span class="Apple-style-span" style="font-size: x-small;"><div style="text-align: center;">Click the images to  view them larger.<br /></div></span><div><div style="text-align: center;"><br /></div><div>...and the thumbnails also seemed to make it easy for people to skip over the results with thumbnails when those results were not relevant to their search (page with the thumbnail on the right).<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd4dlQKPJZm8KK0ccza5j3tg0p2HzJycQRMC1s_rT_w5bZ66JiAPZ2Bc-ZF22UyUQypBoeqrwjQfYObRNDs6JEAgRebfVdLk2OdiiTBulM-FZRSY29iXE3JZahXpEr7mpS9Pjx-K6pSZQ4/s1600-h/4-bees.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5299742942596031810" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd4dlQKPJZm8KK0ccza5j3tg0p2HzJycQRMC1s_rT_w5bZ66JiAPZ2Bc-ZF22UyUQypBoeqrwjQfYObRNDs6JEAgRebfVdLk2OdiiTBulM-FZRSY29iXE3JZahXpEr7mpS9Pjx-K6pSZQ4/s400/4-bees.jpg" style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 155px;" /></a><br /></div><div>For the Universal Search team, this was a successful outcome. It showed that we had managed to design a subtle user interface that gives people helpful information without getting in the way of their primary task: finding relevant information.<br /><br />In addition to search research, we also use eye-tracking to study the usability of other products, such as <a href="http://news.google.com/">Google News</a> and <a href="http://images.google.com/">Image Search</a>. For these products, eye-tracking helps us answer questions, such as "Is the 'Top Stories' link discoverable on the left of the Google News page?" or "How do the users typically scan the image results — in rows, in columns or in some other way?"<br /><br />Eye-tracking gives us valuable information about our users' focus of attention — information that would be very hard to come by any other way and that we can use to improve the design of our products. However, in our ongoing quest to make our products more useful, usable, and enjoyable, we always complement our eye-tracking studies with other methods, such as interviews, <a href="http://googleblog.blogspot.com/2008/11/art-of-field-study.html">field studies</a> and <a href="http://googleblog.blogspot.com/2006/04/this-is-test-this-is-only-test.html">live experiments</a>.<br /><br /><span class="byline-author">Posted by Anne Aula and Kerry Rodden, User Experience Researchers</span></div></div>

• 
• 
• 
• 

1. 1
2. 2
3. 3
4. 4

• Previous
• Next

Eye-tracking studies: more than meets the eye

Imagine that you need a refresher on how to tie a tie. So, you decide to type [how to tie a tie] into the Google search box. Which of these results would you choose?Where did your eyes go first when y…

Đọc thêm »

Our international approach to search

In previous posts in <a href="http://googleblog.blogspot.com/search/label/search%20quality">this series</a>, you have read about the challenges of building a world-class search engine. Our goal is to make Google’s search be relevant to all people, regardless of their language or country. As my colleague Amit Singhal <a href="http://googleblog.blogspot.com/2008/07/introduction-to-google-ranking.html">described</a>, we use statistical data as the basis for making sweeping algorithmic changes. Many of these changes can be rolled out across all languages we support, but in some cases the unique characteristics of each language require some algorithmic considerations and tuning. And to make things really interesting, there are cases where the same language is different across countries. Obvious examples are "color" in the U.S. vs. "colour" in the U.K., or "camião" in Portugal vs. "caminhão" in Brazil.<br /><br />My name is Daphne Dembo, and my focus is improving Google's international search. This is a tough challenge, since Google search is used in many countries and languages where our engineers have little personal knowledge. Initially, the international search improvements were done by Search Quality engineers who were passionate about their languages and countries: Lina from Sweden improved our parsing of compound words in German and Swedish; Dimitra from Greece introduced diacritical support; Ishai from Israel worked on transliteration corrections for Hebrew and Arabic; Trystan from Australia created methods for identifying local search results and ranking them together with foreign ones from the same language; Alex, a bilingual Ukrainian and Russian, introduced morphological understanding of these languages. As the importance of our international search grew, we solicited help from Googlers in all our offices. Finally, we are leveraging an international network of search specialists who help us understand search within the unique combination of their language and country.<br /><br />Our first step in providing search support for a language is to train our language model on a large collection of documents in that language. This ensures that our language model is more precise and comprehensive — for example, it incorporates names, idioms, colloquial usage, and newly coined words not often found in static dictionaries. For instance, we recently started identifying Swahili, and used pages such as this one for the <a href="http://www.bunge.go.tz/bunge/Supp_Qs_Ls.asp?PTerm=2000-2005&vpkey=1174&page=5">Parliament of Tanzania</a> to train our system with the language's nuances. Having a trained language model helps to categorize documents during crawling and indexing of the web and to parse the user's query. Once this stage was complete, we launched Swahili search in countries such as <a href="http://www.google.co.tz/">Tanzania</a> and <a href="http://www.google.co.ke/">Kenya</a>, enabling local searches for the "Dar es Salaam stock exchange" [<a href="http://www.google.co.tz/search?hl=sw&q=Soko+la+hisa+dar+es+salaam">Soko la hisa dar es salaam</a>], and "cure for Malaria" [<a href="http://www.google.co.ke/search?hl=sw&q=Tiba+ya+malaria">Tiba ya malaria</a>]. (As always, we are using square brackets to denote a search query. For example, you can search for "soccer" in Hamburg, Germany by clicking on [<a href="http://www.google.de/search?hl=de&q=fu%C3%9Fball+in+hamburg">fußball in hamburg</a>]).<br /><br />We learn some things from our users, so as people start using our search engine, we can improve the way we rank in that language. Here are few examples:<br /><ul><li><span style="font-weight: bold;">Spell corrections</span>: We recently launched spell corrections in Estonian. If your Estonian is rusty, and you don't remember how to spell "smoke detector," we can suggest a spell correction for [<a href="http://www.google.ee/search?hl=et&q=suitsuantur">suitsuantur</a>], leading to <a href="http://www.google.ee/search?hl=et&q=suitsuandur">better</a> search results.</li><li><span style="font-weight: bold;">Diacritical marks</span><span>:</span><span style="font-weight: bold;"> </span>Many languages have diacritical marks, which alter pronunciation. Our algorithms are built to support them, and even help users who mis-type or completely ignore them. For example, if you're a resident of Quebec, Canada and would like to know the weather forecast in Quebec City, we'll serve good results whether you type with diacritical signs [<a href="http://www.google.ca/search?hl=fr&q=M%C3%A9t%C3%A9o+%C3%A0+Qu%C3%A9bec">Météo à Québec</a>] or without [<a href="http://www.google.ca/search?hl=fr&q=meteo+quebec">meteo quebec</a>]. Czech users can read the same excellent results for a popular kids' cartoon by searching for [<a href="http://www.google.cz/search?hl=cs&q=krtecek">krtecek</a>] and [<a href="http://www.google.cz/search?hl=cs&q=krte%C4%8Dek">krteček</a>]. On the other hand, sometimes diacriticals change the meaning of the word and we have to use them correctly. For example, in Thai, [<a href="http://www.google.co.th/search?hl=th&q=%E0%B8%82%E0%B9%89%E0%B8%B2%E0%B8%A7+">ข้าว</a>] is "rice," with completely different results than [<a href="http://www.google.co.th/search?hl=th&q=%E0%B8%82%E0%B9%88%E0%B8%B2%E0%B8%A7+">ข่าว</a>], which is "news"; or in Slovakia, results for "child" [<a href="http://www.google.sk/search?hl=sk&q=die%C5%A5a">dieťa</a>] are different than results for "diet" [<a href="http://www.google.sk/search?hl=sk&q=di%C3%A9ta">diéta</a>].</li><li><span style="font-weight: bold;">Synonyms</span><span>:</span><span style="font-weight: bold;"> </span>A general case of diacritical support is the handling of synonyms in different languages. Korean searches showed that "samsung" can be viewed as a synonym of "삼성", so that when users search for [<a href="http://www.google.co.kr/search?&q=samsung">samsung</a>], they find results which have the company's name in Korean. </li><li><span style="font-weight: bold;">Compounding</span><span>:</span><span style="font-weight: bold;"> </span>Some languages allow compounding, which is the formation of new words by combining together existing words. You can see a nice example in Swedish, where we return documents about a Swedish credit card for both compounded [<a href="http://www.google.se/search?hl=sv&q=Visakort">Visakort</a>] and non-compounded [<a href="http://www.google.se/search?hl=sv&q=visa+kort">visa kort</a>] queries. </li><li><span style="font-weight: bold;">Stemming</span><span>:</span><span style="font-weight: bold;"> </span>Google has developed morphological models that can receive compound words as queries, and return pages which contain their stem, possibly as part of a different compound. For example, when searching for cars in Saudi Arabia, you can search for [<a href="http://www.google.com.sa/search?hl=ar&q=%D8%B3%D9%8A%D8%A7%D8%B1%D8%A9">سيارة</a>] and [<a href="http://www.google.com.sa/search?hl=ar&q=%D8%B3%D9%8A%D8%A7%D8%B1%D8%A7%D8%AA">سيارات</a>] because both are variants of the same stem, and both return many common results. A Polish user can search for "movie" [<a href="http://www.google.pl/search?hl=pl&q=film">film</a>], and get back results that contain other variants of the stem, such as "filmów," "filmu," "filmie," "filmy." A user from <a href="http://www.google.com.by/">Belarus</a> will find results for all word forms of the capital, Minsk [<a href="http://www.google.com.by/search?hl=be&q=%D0%9C%D1%96%D0%BD%D1%81%D0%BA&btnG=%D0%9F%D0%BE%D1%88%D1%83%D0%BA+Google&meta=">Мінск</a>]: "Мінску," "Мінска," "Мінскага." </li></ul>In addition to these semantic factors, Google does even more to parse documents and queries. Understanding the details of language usage in a country is important. Notation of acronyms is different across languages: In Hebrew it is double quotes before the last (left-most) character, as in "prime minister" [<a href="http://www.google.co.il/search?hl=iw&q=%D7%A8%D7%94%22%D7%9E">רה"מ</a>]; in Thai — a dot at the end of the word, as in police station [<a href="http://www.google.co.th/search?hl=th&q=%E0%B8%AA%E0%B8%99.">สน.</a> ]; while in the U.S. — dots after each character, as in [<a href="http://www.google.com/search?&q=i.b.m">I.B.M.</a>]. Chinese users quote works of art with a "《", as in: [<a href="http://www.google.cn/search?&q=%E3%80%8A%E6%89%8B%E6%9C%BA%E3%80%8B%E5%89%A7%E6%83%85">《手机》剧情</a>], and denote dates with a "日", as in: [<a href="http://www.google.cn/search?&q=2006%E5%B9%B41%E6%9C%8813%E6%97%A5">2006年1月13日</a>].<br /><br />Beyond the linguistic elements of a language, we consider how people enter a query. For example, some languages that do not have Latin scripts require keyboards with dual alphanumeric keys. The user can switch between language input modes by typing special keystrokes. In case the user forgets to type this sequence, the queries end up being gibberish. You can see correct handling of these mistakes in Arabic ([<a href="http://www.google.com.eg/search?hl=ar&q=hgsuv">hgsuv</a>] corrected to [<a href="http://www.google.com.eg/search?&q=%D8%A7%D9%84%D8%B3%D8%B9%D8%B1&spell=1">السعر</a>]) and ([<a href="http://www.google.com.sa/search?&q=presidential+elections">حقثسهيثىفهشم ثممثؤفهخىس</a> ] corrected to [<a href="http://www.google.com.sa/search?hl=ar&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=presidential+elections&spell=1">presidential elections</a>]), Hebrew ([<a href="http://www.google.co.il/search?hl=iw&q=vdrk,+kuyu">vdrk, kuyu</a>] corrected to [<a href="http://www.google.co.il/search?&q=%D7%94%D7%92%D7%A8%D7%9C%D7%AA+%D7%9C%D7%95%D7%98%D7%95">הגרלת לוטו</a>]), and Cyrillic ([<a href="http://www.google.ru/search?&q=rehc+ljkffhf">rehc ljkffhf</a>] corrected to [<a href="http://www.google.ru/search?&q=%D0%BA%D1%83%D1%80%D1%81+%D0%B4%D0%BE%D0%BB%D0%BB%D0%B0%D1%80%D0%B0">курс доллара</a>]).<br /><br />Another way of avoiding the inconvenience of switching keyboard modes is by typing the phonetic sounds of the query in Latin characters. Recreating the correct query in the target language isn't trivial, since there might be many possibilities. We can see several such examples in which we suggest the same query in the intended language for Russian ([<a href="http://www.google.ru/search?&q=biskvitnyi+rulet">biskvitnyi rulet</a>] to [<a href="http://www.google.ru/search?&q=%D0%B1%D0%B8%D1%81%D0%BA%D0%B2%D0%B8%D1%82%D0%BD%D1%8B%D0%B9+%D1%80%D1%83%D0%BB%D0%B5%D1%82">бисквитный рулет</a>]), "movies" in Chinese ([<a href="http://www.google.cn/search?&q=dianying">dianying</a>] to [<a href="http://www.google.cn/search?&q=%E7%94%B5%E5%BD%B1">电影</a>]), and "Bank of Attica" in Greek [<a href="http://www.google.gr/search?hl=el&q=trapeza+attikhs">trapeza attikhs</a>] returns good results for "Τράπεζα Αττικής". Users of 8 Indic languages (such as Hindi, Gujarati, Telugu) can type the phonetic sound of the query, and choose the words in Hindi script:<br /><br /><center><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4BjCeMuBWqz6tsZaCbdv5A-lKoCEgmHIDrxJ4xSJsY88PPjsEbi_URwu4EQWJlcmwDPVHzGcNnYYRH0-7LsImh-FRh3TZBc2gfmLmN4XL5XDZUwQ9P-BbQIBLmbetbhbwo5DIzQgh_Mo/s1600-h/IN-transliteration.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5271187474612225186" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4BjCeMuBWqz6tsZaCbdv5A-lKoCEgmHIDrxJ4xSJsY88PPjsEbi_URwu4EQWJlcmwDPVHzGcNnYYRH0-7LsImh-FRh3TZBc2gfmLmN4XL5XDZUwQ9P-BbQIBLmbetbhbwo5DIzQgh_Mo/s400/IN-transliteration.jpg" style="cursor: pointer; width: 400px; height: 263px;" /></a></center><br />Ease of typing and reading is also influenced by the language used. Since every Chinese word requires several keystrokes on a standard keyboard, we provide <a href="http://images.google.cn/imgcat?&cid=6&seq=2_6_7_4_1">category browsing by Images</a> and <a href="http://www.google.cn/search?hl=zh-CN&q=%E5%95%86%E4%B8%9A%E7%AD%96%E5%88%92%E4%B9%A6%E8%8C%83%E6%96%87">related searches</a> so that people don't need to type as much. Similarly, we are now launching Google Suggest, or <a href="http://googleblog.blogspot.com/2008/08/at-loss-for-words.html">real-time completion of queries</a>, in many languages.<br /><br />So far I described how we improve the quality of search in a language. However, there is a strong effect of the location of the user, even if it is only approximated to the country, since in many cases local content is more relevant than global information. For example, searching for Spanish Yellow Pages [Páginas Amarillas] will result in several documents of global interest and several local results in <a href="http://www.google.com.pe/search?hl=es&q=paginas+amarillas">Peru</a>, <a href="http://www.google.com.mx/search?hl=es&q=paginas+amarillas">Mexico</a>, and <a href="http://www.google.es/search?hl=es&rls=EGLA,EGLA:2008-08,EGLA:en&q=paginas+amarillas">Spain</a>. Similar to that, searching for [Côte d'Or] in <a href="http://www.google.fr/search?hl=fr&q=C%C3%B4te+d%27Or">France</a> will return results for that region, whereas searches in <a href="http://www.google.be/search?hl=nl&q=C%C3%B4te+d%27Or">Belgium</a> will return results about the chocolate maker.<br /><br />Note that the display of information should conform to the standards in that country, so we display "," as a decimal notation for Croatian users who want to know how many millimeters are in an inch [<a href="http://www.google.hr/search?hl=hr&q=in%C4%8D+u+milimetrima">inč u milimetrima</a>], or for Italian users who are interested in currency exchange rates [<a href="http://www.google.it/search?hl=it&q=50+euro+in+dollari">50 euro in dollari</a>]. Similarly, temperatures in Norway [<a href="http://www.google.no/search?hl=no&q=V%C3%A6ret+i+Oslo">Været i Oslo</a>] will be displayed in Celsius, while in the U.S. — in Fahrenheit [<a href="http://www.google.com/search?hl=en&q=weather+boston">weather Boston</a>].<br /><br />If everything else fails, we provide cross-language translations based upon Google's translation technology described in this <a href="http://googleresearch.blogspot.com/2008/09/doubling-up.html">blog post</a>. We will translate your query to English, search English documents on the web, and translate the returned results from English back into the original query language. For example, Japanese users who are interested in viewing Halloween illustrations (Halloween is a holiday which originated in Ireland) can search for [<a href="http://www.google.co.jp/search?hl=ja&q=%E3%83%8F%E3%83%AD%E3%82%A6%E3%82%A3%E3%83%B3+%E3%82%A4%E3%83%A9%E3%82%B9%E3%83%88">ハロウィン イラスト</a>]. You can then request a Japanese translation of the English pages (at the bottom of the page), which will bring up the translation page in the screenshot below. Similarly, Korean users can search for the latest on Harry Potter [<a href="http://www.google.co.kr/search?&q=%ED%95%B4%EB%A6%AC+%ED%8F%AC%ED%84%B0">해리 포터</a>], and Arabic readers can search for the opening of the Sydney Opera house [<a href="http://www.google.com.eg/search?hl=ar&q=%D8%A7%D9%81%D8%AA%D8%AA%D8%A7%D8%AD+%D8%AF%D8%A7%D8%B1+%D8%A7%D9%84%D8%A7%D9%88%D8%A8%D8%B1%D8%A7+%D9%81%D9%8A+%D8%B3%D9%8A%D8%AF%D9%86%D9%8A">افتتاح دار الاوبرا في سيدني</a>]. (Click on the image to see a larger version.)<br /><br /><center><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_xXY_YTEBV_iwkjLAuoL61InpTE55KeWXpA4ofwffuaAZ58aAH1NrD6owR1rgaAAWERCcffn8NvNn2ryMHWMQ-9IpR9qNccKqQIv92Y3wc4mv0Ojj_jrPOqEILwcA3rSTNPNEM3kyGRo/s1600-h/CLIR.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5271189025565112818" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_xXY_YTEBV_iwkjLAuoL61InpTE55KeWXpA4ofwffuaAZ58aAH1NrD6owR1rgaAAWERCcffn8NvNn2ryMHWMQ-9IpR9qNccKqQIv92Y3wc4mv0Ojj_jrPOqEILwcA3rSTNPNEM3kyGRo/s400/CLIR.jpg" style="cursor: pointer; width: 400px; height: 128px;" /></a></center><br /><br />All in all, Google Search is being actively developed for more than 100 languages, in 150+ countries, with dozens of improvements launched each month. So far I've covered the basics of how international search works, but this is just the surface of all the international work we do. There are many other interesting topics that impact international markets like usability, homepage and results page layout, and connectivity. An understanding of real cultural and human factors is essential to creating a search engine that resonates with the people who use it. (Click on the image to see a larger version.)<br /><br /><center><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOd4ZKynjM02kfcGl0odSi8tJRV5eYuenh2P20HxqLcJxLqIH07y5wtb2_MpZuvNmoVwgDE_w0CKnAS22SNirA6f2Ftves1rvG2DD3tXMivKmXP_TjlIWCGG0VP3wz_EW0hvJodjAII-E/s1600-h/HPs.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5271187878255992642" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOd4ZKynjM02kfcGl0odSi8tJRV5eYuenh2P20HxqLcJxLqIH07y5wtb2_MpZuvNmoVwgDE_w0CKnAS22SNirA6f2Ftves1rvG2DD3tXMivKmXP_TjlIWCGG0VP3wz_EW0hvJodjAII-E/s400/HPs.jpg" style="cursor: pointer; width: 400px; height: 282px;" /></a></center><br /><br /><span style="font-style: italic;">(Update:</span> Replaced example in the 4th bullet point.)<br /><br /><span class="byline-author">Posted by Daphne Dembo, Engineering Director</span>

• 
• 
• 

1. 1
2. 2
3. 3

• Previous
• Next

Our international approach to search

In previous posts in this series, you have read about the challenges of building a world-class search engine. Our goal is to make Google’s search be relevant to all people, regardless of their languag…

Đọc thêm »

The art of the field study

I'm Dan Russell, a member of the Search Quality team doing user experience research. This post is part of our <a href="http://googleblog.blogspot.com/search/label/search%20quality">ongoing series</a> to talk about the Search Quality team at Google, showing a bit of what we do in the day-to-day course of improving the quality of the user experience.<br /><br />The role of "user experience" research is to try and get the inside story on what people do when they search. We're constantly asking: What's the user's experience of search? What works and doesn't work for them? What are they looking for? What DO they want?<br /><br />To understand the full richness and variety of what people do when they are using Google, we spend many hours in the field, watching people search and listening to what they say as they do this. We hear it when they're happy, and when they're terribly frustrated. And perhaps most importantly, we also pay attention to the things they don't say -- the inexpressible "gotchas" that slow users down or get in the way of their search.<br /><br />It turns out that people are masters of saying one thing and doing another, particularly when it comes to nearly automatic behavior. We find that searchers often turn so quickly to Google that they don't really think too much about what they're actually searching for. It's surprising, but often we'll see people trying to find out something about a topic, but then never actually mention the topic itself. That is, there's often a big discrepancy between what they'll tell me (the human observer) they're trying to do, and the search terms they enter into Google. One person I shadowed for the day spent ten minutes trying to find the schedule of the ferry that runs between San Francisco and Larkspur, but somehow only thought of adding the word "ferry" much later in their search.<br /><br />We also study eye tracking. The eye makes a complex scan path over the search results, building up a composite picture of what is presented on the page. It's clear that what actually happens is a very rapid scan and assessment of each result as they are seen. In those milliseconds between the eye landing on the first fixation and seeing a few results, all kinds of decisions and choices are made--nearly all of them subconsciously.<br /><br />In this short video, you can see three different searchers all looking for the same thing (in this case, a child's backpack). The red dot is the searcher's gaze moving around on the search results page. Notice how methodically the gaze moves from result title to title, occasionally inspecting the snippet text to gain more detail about the result.<br /><br /><object height="344" width="425"><param name="movie" value="http://www.youtube.com/v/bZ5Be93RTeY&hl=en&fs=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/v/bZ5Be93RTeY&hl=en&fs=1" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" height="344" width="425"></embed></object><span style="font-size:85%;"><br />(Video courtesy of Kerry Rodden) </span><br /><br />So the job of figuring out what people actually do when they search isn't as simple as asking someone what they search for during the day. It's basically impossible to give an accurate telling of what you saw (or didn't see) on the results page while actively searching for a high quality results.<br /><br />Memories of your own behavior are also notoriously unreliable. People's search behavior in the lab is often different than when they're at home or at work. This is a natural (and expected) side effect of lab studies: people will work especially hard to please a researcher. If we ask them to search for a pair of brown shoes they'd like to buy for themselves, in the lab they'll find the first pair that seems reasonable and then stop, satisfied. If it was real, they would go on and spend more time. We still do lab studies, but we know what to watch for, and what to ignore.<br /><br />Data from field studies gives us insight into how people respond to the Google experience in ways that we can't otherwise measure.<br /><br />For instance, in several field studies we discovered that many of the people who went to the previous version of the Advanced Search page had a strong, almost visceral negative reaction when the page appeared. The text of the original page had language that many people saw as intimidating--words like "Domain," "Usage Rights" and "Safe Search" can be a bit much if you're not sure what they mean.<br /><br />The old Advanced Search page <span style="font-style: italic;">was</span> a little off-putting (click on the image to see a larger version):<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheW61EfyCXnPcXef902wlidfZMatVZqxSFSyDUp3p0b-z-9OkJuXzM4HzUsyYu3LnWZE-auWL8oMlh8G1kgnyNmko8qbT85kuUj5RPyMzSjUki3RaBvKozXr2aTEZrSYiMIWM_Nf_JM-w5/s1600-h/Google-adv-search.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5265610667405350274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheW61EfyCXnPcXef902wlidfZMatVZqxSFSyDUp3p0b-z-9OkJuXzM4HzUsyYu3LnWZE-auWL8oMlh8G1kgnyNmko8qbT85kuUj5RPyMzSjUki3RaBvKozXr2aTEZrSYiMIWM_Nf_JM-w5/s1600/Google-adv-search.jpg" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 525px;" /></a><br />Based on our field studies, we dug more deeply into how people were actually using our Advanced Search page, and quickly discovered that, indeed, a large number of users were going to the page, and then leaving it without ever filling in any of the slots.<br /><br />Armed with this insight from field studies, we redesigned the page, simplifying it by removing terms that were unclear to the average user (the word "occurrences," for example, just didn't mean anything to many of the Advanced Search page users), moving rarely used features (numeric range searches, date searches, etc.) into a part of the page that was expandable with a single click. That made them easy to get to for people who knew they wanted to search with those restrictions, but out of the way in a non-threatening way.<br /><br />One of the other things we noted in the field study was that people often didn't understand how the Advanced Search page worked. So we added a "visible query builder" region at the top of the page. As you fill in the blanks, the box at the top of the page fills in with the query that you could type into Google. It was our way of making visible the effects of advanced search operators.<br /><br />The Advanced Search page post-redesign (click on the image to see a larger version):<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0CkUiPUDgpsAWsbrmnpIjkcNaS-hQ217KKpW4bBtou2Hk6ZP2Mef4hU7C38yXShRRoxnpbCx9V_BeRPnwPUn8XIvEGdVGRR8Qbj2fMpLHSjWG-rq2cHHBxoDlRCqTbWlS6f9tEoz61Qlv/s1600-h/Google-adv-search-Oct-2008.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5265611107869300194" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0CkUiPUDgpsAWsbrmnpIjkcNaS-hQ217KKpW4bBtou2Hk6ZP2Mef4hU7C38yXShRRoxnpbCx9V_BeRPnwPUn8XIvEGdVGRR8Qbj2fMpLHSjWG-rq2cHHBxoDlRCqTbWlS6f9tEoz61Qlv/s1600/Google-adv-search-Oct-2008.jpg" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 525px;" /></a><br />The good effect of these changes quickly became clear. The number of users that bounced out of the Advanced Search page dropped significantly. Interestingly, the total number of Advanced Search page users didn't increase significantly... at least not yet. By improving the UI on the page, we hope to attract even more searchers to the large range of search options available on Google.<br /><br />In the end, this example shows the kind of insights that field studies can bring. As with the eye-tracking example, asking someone about their emotional response to a web page just isn't a useful way to get that data. But watching them <span style="font-style: italic;">in situ</span>, as they actually use Google to go about their daily search lives can reveal all kinds of remarkable, otherwise undiscoverable, and actionable insights into searcher behavior.<br /><br /><span class="byline-author">Posted by Daniel Russell, Uber Tech Lead, Search Quality</span>

• 
• 

1. 1
2. 2

• Previous
• Next

The art of the field study

I'm Dan Russell, a member of the Search Quality team doing user experience research. This post is part of our ongoing series to talk about the Search Quality team at Google, showing a bit of what we d…

Đọc thêm »

Search evaluation at Google

<a title="This series" href="http://googleblog.blogspot.com/search/label/search%20quality" id="so8g">This series</a> of posts has described Google's search quality efforts in areas such as ranking and search UI. Now I'll describe search evaluation. Simply put, search evaluation is the process of measuring the quality of our search results and our users' experience with search.<br /><br />Let me introduce myself. I'm Scott Huffman, an engineering director responsible for leading search evaluation, working with a talented team of statisticians and software engineers. I've been here since 2005, and have been working on search in one form or another for the past fourteen years or so.<br /><br />When I'm interviewing folks interested in joining the search evaluation team, I often use this scenario to describe what we do: Imagine a Google ranking engineer bursts into your office. "I have a great idea for improving our search results!" she exclaims. "It's simple: Whenever a page's title starts with the letter T, move it up in the results three slots." This engineer comes armed with several example search queries where, lo and behold, this idea actually improves the results significantly.<br /><br />Now, you and I may think that this "letter T" hack is really a silly idea, but how can we know for sure? Search evaluation is charged with answering such questions. This hack hasn't really come up, but we are constantly evaluating everything, which can include: <div id="gd2q"> - proposed improvements to segmentation of Chinese queries </div> <div id="gd2q0"> - new approaches to fight spam </div> <div id="gd2q1"> - techniques for improving how we handle compound Swedish words </div> <div id="gd2q1"> - changes to how we handle links and anchortext </div> <div id="ucyj"> - and everything in between<br /><br />As Udi mentioned in his <a title="initial post" href="http://googleblog.blogspot.com/2008/05/introduction-to-google-search-quality.html" id="xxu8">initial post</a> on search quality, in 2007 we launched over 450 improvements to Google search, and every one of them went through a comprehensive evaluation process. </div> <div id="yygm12"> <br /></div> <div id="yygm14"> Not surprisingly, we take search evaluation very seriously. Precise evaluation enables our teams to know "which way is up". One of our tenets in search quality is to be very data-driven in our decision-making. We try hard not to rely on anecdotal examples, which are often misleading in search (where decisions can affect hundreds of millions of queries a day). Meticulous, statistically-meaningful evaluation gives us the data we need to make real search improvements.<br /><br />Evaluating search is difficult for several reasons.<br /> <ul id="yygm20"><li id="yygm24" style="margin-left: 15px;"> First, understanding what a user really wants when they type a query -- the query's "intent" -- can be very difficult. For highly navigational queries like [ebay] or [orbitz], we can guess that most users want to navigate to the respective sites. But how about [olympics]? Does the user want news, medal counts from the recent Beijing games, the IOC's homepage, historical information about the games, ... ? This same exact question, of course, is faced by our ranking and search UI teams. Evaluation is the other side of that coin.<br /> <br /> </li><li id="yygm27" style="margin-left: 15px;"> Second, comparing the quality of search engines (whether Google versus our competitors, Google versus Google a month ago, or Google versus Google plus the "letter T" hack) is never black and white. It's essentially impossible to make a change that is 100% positive in all situations; with any algorithmic change you make to search, many searches will get better and some will get worse.<br /><br /> </li><li id="yygm27" style="margin-left: 15px;"> Third, there are several dimensions to "good" results. Traditional search evaluation has focused on the <i>relevance </i>of the results, and of course that is our highest priority as well. But today's search-engine users expect more than just relevance. Are the results fresh and timely? Are they from authoritative sources? Are they comprehensive? Are they free of spam? Are their titles and snippets descriptive enough? Do they include additional UI elements a user might find helpful for the query (maps, images, query suggestions, etc.)? Our evaluations attempt to cover each of these dimensions where appropriate.<br /> <br /> </li><li id="yygm30" style="margin-left: 15px;"> Fourth, evaluating Google search quality requires covering an enormous breadth. We cover over a hundred locales (country/language pairs) with in-depth evaluation. Beyond locales, we support search quality teams working on many different kinds of queries and features. For example, we explicitly measure the quality of Google's spelling suggestions, universal search results, image and video searches, related query suggestions, stock oneboxes, and many, many more. </li></ul> </div> To get at these issues, we employ a variety of evaluation methods and data sources:<br /><ul><li> Human evaluators. Google makes use of evaluators in many countries and languages. These evaluators are carefully trained and are asked to evaluate the quality of search results in several different ways. We sometimes show evaluators whole result sets by themselves or "side by side" with alternatives; in other cases, we show evaluators a single result at a time for a query and ask them to rate its quality along various dimensions.<br /><br /> </li><li> Live traffic experiments. We also make use of experiments, in which small fractions of queries are shown results from alternative search approaches. Ben Gomes talked about how we make use of these <a href="http://googleblog.blogspot.com/2008/08/search-experiments-large-and-small.html" id="leq_" title="experiments for testing search UI elements">experiments for testing search UI elements</a> in his previous post. With these experiments, we are able to see real users' reactions (clicks, etc.) to alternative results.<br /> </li></ul> Clearly, we can never measure anything close to all the queries Google will get in the future. Every day, in fact, Google gets many millions of queries that we have never seen before, and will never see again. Therefore, we measure statistically, over representative samples of the query-stream. The "letter T" hack probably does improve a few queries, but over a representative sample of queries it affects, I'm confident it would be a big loser.<br /><br />One of the key skills of our evaluation team is experimental design. For each proposed search improvement, we generate an experiment plan that will allow us to measure the key aspects of the change. Often, we use a combination of human and live traffic evaluation. For instance, consider a proposed improvement to Google's "related searches" feature to increase its coverage across several locales. Our experiment plan might include live traffic evaluation in which we show the updated related search suggestions to users and measure click-through rates in each locale and break these down by position of each related search suggestion. We might also include human evaluation, in which for a representative sample of queries in each locale, we ask evaluators to rate the appropriateness, usefulness, and relevance of each individual related search suggestion. Including both types of evaluation allows us to understand the overall behavioral impact on users (via the live traffic experiment), and measure the detailed quality of the suggestions in each locale along multiple dimensions (via the human evaluation experiment).<br /><br />Choosing an appropriate sample of queries to evaluate can be subtle. When evaluating a proposed search improvement, we consider not only whether a given query's results are changed by the proposal, but also how much <i>impact</i> the change is likely to have on users. For instance, a query whose first three results are changed is likely much higher impact than one for which results 9 and 10 are swapped. In Amit Singhal's <a id="n.lb" href="http://googleblog.blogspot.com/2008/07/technologies-behind-google-ranking.html" title="previous post on ranking">previous post on ranking</a>, he discussed synonyms. Recently, we evaluated a proposed update to make synonyms more aggressive in some cases. On a flat (non-impact-weighted) sample of affected queries, the change appeared to be quite positive. However, using an evaluation of an impact-weighted sample, we found that the change went much too far. For example, in Chinese, it synonymized "small" (小) and "big" (大)... not a good idea!<br /><br />We're serious about search evaluation because we are serious about giving you the highest quality search experience possible. Rather than guess at what will be useful, we use a careful data-driven approach to make sure our "great ideas" really are great for you. In this environment, the "letter T" hack never had a chance.<br /><br /><span class="byline-author">Posted by Scott Huffman, Engineering Director</span>

Search evaluation at Google

This series of posts has described Google's search quality efforts in areas such as ranking and search UI. Now I'll describe search evaluation. Simply put, search evaluation is the process of measurin…

Đọc thêm »

Search experiments, large and small

In my <a href="http://googleblog.blogspot.com/2008/08/search-quality-continued.html" id="ydhu" title="previous blog post">previous post</a>, I described the components of your web search experience and the principles behind creating a great search experience. There are complex algorithms underlying simple features such as spelling correction and the two line snippets that describe each search result. We figure out what works by running experiments - tiny tests for a small number of users which help us determine whether that feature helps or hurts. <br /><br /><br /><br />Experimentation is a very powerful tool, and we use it very widely to test potential changes to search. At any given time, we run anywhere from 50 to 200 experiments on Google sites all over the world. I'll start by describing experimental changes so small that you can barely tell the difference after staring at the page, and end with a couple of much more visually obvious experiments that we have run. There are a lot of people dedicated to detecting everything Google changes - and occasionally, things imagined that we did not do! - and they do latch on to a lot of our more prominent experiments. But the experiments with smaller changes are almost never noticed. <br /><br /><br /><br />For example, can you tell the difference between the two pages below?<br /><br /><br /><br />Choice 1:<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbp7OBMFq4enVdvdPVzB6Iq9ZDXUCpTBqQR7Ro5XgKq3TKmfmqcc5lmSjIyOXVkaXNnhWXHeFoWYxnAItfkT5tJLr6KKuGOsBYZLCwMW16fnvAS8IGrt-SsLza7aHXAStY7pe3dPyXyKV0/s1600-h/Gomes_1.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1lXC3kfdZVjGDcsjWj2Eb9UeG1vY-yOJJaq8zT5qdIS22JR1EQGTe3lQtuM42orduR7Q_UOYzpln7zDm6aC8uomJ89Q_ubJjDQQIxkqe9eEXHSU0WWfl7bs-O8VzyvoNEWFIT3FmHEOZo/s320-r/Gomes_1.gif" /></a></div><br /><br />Choice 2:<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtM_EEhk2JQmdQBDQ6aM8q-8I1uA0L32gca7q4XZyAdQ-GHS7cYie3LFTMRmhzbFhPwzBqe0MEo7KxDHubpjOf0gETHK4F50sOJwth5UE-LJYzmfhLyuZH-rsdbkECpiJiE8g-IbnxyKhh/s1600-h/Gomes_2.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYCVLfgwgZ8yCkcLInmwmW0aZud2Yy3ieBdWtCCT-s9fsZ15DOMvtCotoYWb5niBD2Ba22HqHrcQ4i9GXRSVPKssOXP9mrDrEkz5b3NbLc_Rsrf_UT12QaaOjU5gIYGyhfOG8DD7Nc9JhM/s320-r/Gomes_2.gif" /></a></div><br /><br />I'm pretty sure I would not be able to tell the difference if I were to see each of them on their own. But apparently you can! At least in the aggregate, there is a measurable difference with a change like this. In case you can't tell after staring, the white space around the first search result has changed, which makes the first result in Picture 2 slightly more visually prominent. This visual prominence conveys the fact that according to our ranking signals, the first result is a substantially better match than the next result. On the plus side, it helps you focus on the first result. But if you were looking for one of the other results, it can disrupt your scanning of the page. An experiment helps us determine which effect is more prominent, and whether a change would help you search faster. <br /><br /><br /><br />Another change, almost as minimal visually, is between these two results:<br /><br /><br /><br /></div><div id="bk7m"> </div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjN7H1NSOpGTVoiixOeYMIA1z4DP9C9ygEqgCkolBFNk5CGm7QbHZjDBzE8K8jqgJsnreSM-JPKcdhOTb_QFOcsT39tnsq6w_anlT-axXPHhYL7qTuDr59g3g945xwMizeQpSDSInC75YUy/s1600-h/Gomes_3.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQ56W398v5CJlmllGTivY4LXx3fcHBahvk5IHpskqzyDiAAHZ2GjzR5Ipdx_xmUeeJa7vU7WLUvFfxQy_QMXzsLFsvpZapQXDoxfxzmVCOjOurEpGGysiCuYOs-9HxQpFv8q9bSeApjCTM/s320-r/Gomes_3.gif" /></a></div><div id="bk7m"> </div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhablNJ1mQb34wRLm66CpsaWh38Z2KQJHxrVk_ko-cAHObPB6rAPVQ_OLCwa25VYqaS4VY7rhPdv4nEXhqEs-qu9pXR5zVYrrBOYOF94K44002ldRtMv4BOB0Y8l9yz6lhyphenhyphenGu5ZHbKnJ1DN/s1600-h/Gomes_4.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOEByVS8JlLtlvj4gPIHRD_O8KdSqJmGhsxBEVg91Avo9shUocx1py0mc2uF92cx3IdFNaCRQqyVXCMQ76BHgi5Ym4xG38IJUZZWGjKjtCzo4-4s5ejNynHAljbBeupaCE30lrJTIpcZlC/s320-r/Gomes_4.gif" /></a></div><div id="bk7m"> </div><div id="bk7m">In this case, the difference in user interaction is so clear and marked we could tell extremely quickly which one worked better: the difference is in the thickness of the plus box next to the stock quote. Now, coming to the conclusion that one is "better" is tricky, and there's many a possible slip on the way there. Does more interaction with the plus box mean that it is better? How about if users then miss good results because they are distracted by the more prominent plus box? Keep watching Google to see which version won! If we've done our job right, almost without your noticing, things will work just that little bit better for you. The world will seem rosier. Birds will sing. Or maybe not - but at least you will have the best-designed plus box we can come up with :)<br /><br /><br /><br />Okay, so not all of our experiments are insane eye tests. My main point in highlighting the above experiments is that we test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care. <br /><br /><br /><br />Another class of experiments have to do with changes that are not purely visual, but rather involve changes to the underlying presentation algorithms. For instance, the algorithm that is responsible for the titles and snippets of result pages now highlights stems and some synonyms of the original query term. For the query [<a href="http://www.google.com/search?source=ig&hl=en&rlz=&=&q=hp+printer+drivers&btnG=Google+Search" id="h1cc" title="hp printer drivers">hp printer drivers</a>] we will also return results that include and highlight the word "driver".</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-3PkmzfAAhutzHnK2VCXehUdiqD6fFLexvVPPZCd9va0LtHorddr_V0rwAICyHBtZjIKUByByGTU7VVuzIWfhJCXyw1DTNzPArkiOpLq4V8qYL5DosUR7hOluGPPWTyGXgkJbl6iENMRm/s1600-h/Gomes_5.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifPaLdl-p2jTmuwoAzv6LgIeALP8g6rk-aKlR0DV9ZrGxkBf7MyIrmWkhoFncWLrQcRR2UCoD8-FMx3erK98rz77A1Xo6AFVrxb3ZyYgxLvLad1fkIT2VKwiyIw-TvILFnCRgCbpkiRxlq/s320-r/Gomes_5.gif" /></a></div><div id="z2ml" style="padding: 1em 0pt; text-align: left;">This sort of "stemming," as it's called, is generally a good idea, because it helps you better identify results that match your query, but not always. Experiments of this sort help us verify (or, occasionally, overturn) our assumptions regarding changes in these algorithms. <br /><br /><br /><br />There is a further class of experiments - the kind that are hard to miss - which introduces fairly prominent features. Even with these larger features, the goal of experimentation always remains the same: are we adding something that really helps people, or is this just another distraction? Google does not really come with a user manual (actually, there are some nicely-written help pages, but we're pretty sure most of you don't bother to read them!). So features need to stand on their own feet, without the help of a careful explanation. Part of the goal of an experiment is to understand just how a feature will be used, which might be quite different from what we initially intended. <br /><br /><br /><br />Here's an example of an experiment that lets you comment on search results and move them around on the result page:</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj70-9FyRBx96tLE34dyYi0VyTjGoFu4lf_P8t4cTN4t_G8SMccJW1C_x2gq35PTUdcqYdNlHTxJZDbO1DQJFqyUOTBhPBMaZPhJqA_z_RY-mfPjtFOBJnKv-Uv0sO7vFJefA4NMzoH9TAT/s1600-h/Gomes_6.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3dQRlXLRHYHryA0hPHvzscbel3QeG0SxAohn9XlU4NKxiVoP2T7HCI_O9bxEsaru0q2g3XkhNIRwdltFMHIrphtssu2UoDh993-Lq-NVaLKscOz25iIWGCQQuIl-GNhJF-dc0UR9HzjrT/s320-r/Gomes_6.gif" /></a> </div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: left;"><br /><br />At this point, I can't say what we expect from this feature; we're just curious to see how it will be used.<br /><br /><br /><br />These are a small sample of the kinds of experiments we run as we test everything from the barely visible to the glaringly obvious. So the next time you use Google and it seems a little different - well, maybe it is. Just for you!<br /><br /><br /><br />Posted by Ben Gomes, Distinguished Engineer

• 
• 
• 
• 
• 

1. 1
2. 2
3. 3
4. 4
5. 5

• Previous
• Next

Search experiments, large and small

In my previous post, I described the components of your web search experience and the principles behind creating a great search experience. There are complex algorithms underlying simple features such…

Đọc thêm »

Search quality, continued

A few weeks back Udi Manber <a href="http://googleblog.blogspot.com/2008/05/introduction-to-google-search-quality.html" id="vd4j" title="introduced the search quality grou">introduced the search quality group</a>, and the <a href="http://googleblog.blogspot.com/2008/07/introduction-to-google-ranking.html" id="e1kr" title="previous">previous</a> <a href="http://googleblog.blogspot.com/2008/07/technologies-behind-google-ranking.html" id="qbn7" title="posts">posts</a> in this series talked about the ranking of documents. While the ranking of web documents forms the core of what makes search at Google work so well, your search experience consists of much more than that. In this post, I'll describe the principles that guide our development of the overall search experience and how they are applied to the key aspects of search. I will also describe how we make sure we are on the right track through rigorous experimentation. And the next post in this series will describe some of the experiments currently underway.<br /> <br />Let me introduce myself. I'm Ben Gomes, and I've been working on search at Google since 1999, mostly on search quality. I've had the good fortune to contribute to most aspects of the search engine, from crawling the web to ranking. More recently, I've been responsible for the engineering of the interface for search and search features.<br /> <br />A common reaction from friends when I say that I now work on Google's search user interface is "What do you do? It never changes." Then they look at me suspiciously and tell me not to mess with a good thing. Google is fine just the way it is -- a plain, fast, simple web page. That's great, but how hard can that be?"<br /> <br />To help answer that question, let me start with our main goal in web search: to get you to the web pages you want as quickly as possible. Search is not an end in itself; it is merely a conduit. This goal may seem obvious, but it makes a search engine radically different from most other sites on the web, which measure their success by how long their users stay. We measure our web search success partly by how quickly you leave (happily, we hope!). There are several principles we use in getting you to the information you need as quickly as possible:</span><br /><ul id="e3ld" style="background-color: rgb(255, 255, 255);"><li id="e3ld0"> <b id="e4fw">A small page. </b> A small page is quick to download and generally faster for your browser to display. This results in a minimalist design aesthetic; extra fanciness in the interface slows down the page without giving you much benefit. <br /> </li><li id="vs1t2"> <b id="e4fw0">Complex algorithms with a simple presentation. </b> Many search features require a great deal of algorithmic complexity and a vast amount of data analysis to make them work well. The trick is to hide all that complexity behind a clean, intuitive user interface. Spelling correction, snippets, sitelinks and query refinements are examples of features that require sophisticated algorithms and are constantly improving. From the user's point of view search, almost invisibly, just works better. </li><li id="xrq1"> <b id="xrq10">Features that work everywhere</b>. Features must be designed such that the algorithms and presentation can be adapted to work in all languages and countries. Consider the problem of spell correction in Chinese, where user queries are often not broken up into words or Hebrew/Arabic, where text is written right to left (interestingly, this is believed to be an example of first-mover disadvantage -- when chiseling on stone, it is easier to hold the hammer in your right hand!).<br /> </li><li id="suk2"> <b id="t3c9">Data driven decisions - experiment, experiment, experiment</b>. We try to verify that we've done the right thing by running experiments. Designs that may seem promising may end up testing poorly.</li></ul><span id="f7-n7" style="background-color: rgb(255, 255, 255);">There are inherent tensions here. For instance, showing you more text (or images) for every result may enable you to better pick out the best result. But a result page that has too much information takes longer to download and longer to visually process. So every piece of information that we add to the result page has to be carefully considered to ensure that the benefit to the user outweighs the cost of dealing with that additional information. This is true of every part of the search experience, from typing in a query, to scanning results, to further exploration.<br /><br /></span>The start of your search is typing in a query. A common cause of frustration is if you don't know the correct spelling of a word! Spell correction -- which seems like a simple and obvious feature -- hides many technical challenges. No common English dictionaries would ever include the correct spelling of Britney Spears, for instance (who, probably completely unbeknownst<span id="f7-n9" style="background-color: rgb(255, 255, 255);"> to her, has become the poster child example for </span><a href="http://www.google.com/jobs/britney.html" id="hr2o" style="background-color: rgb(255, 255, 255);" title="this feature">this feature</a><span id="f7-n10" style="background-color: rgb(255, 255, 255);">). We do a huge amount of <span id="c.j9" style="background-color: rgb(255, 255, 255);">analysis</span> of the billions of pages on the web and our query logs to determine what are "real words" on the web, and what are likely to be misspellings. The system that gives you the spell correction has to, in a fraction of a second, consider a huge number of possible words you </span><i id="xrq12" style="background-color: rgb(255, 255, 255);">might</i><span id="f7-n11" style="background-color: rgb(255, 255, 255);"> have meant (vastly greater than any dictionary ever manually constructed) and determine if there is a more likely query you meant to type. When we are confident that you actually meant to type something else, we take a rare liberty with our search results: we <span id="w-__" style="background-color: rgb(255, 255, 255);">try to</span> distract you from looking at the top result on the page. The spelling correction is in your line of sight and colored a bright must-see red. Furthermore, we now make sure that </span><i id="jq5y" style="background-color: rgb(255, 255, 255);">nothing else on the page is red</i><span id="f7-n12" style="background-color: rgb(255, 255, 255);">, unless it is as important to you as spelling! (so far, nothing is). The algorithms involved in spell correction are constantly getting better. They now work in a large number of languages and are even better at detecting when you have made a spelling mistake. Getting the spelling of your query right is so important that we are considering showing you the results of the spell-corrected query in the middle of the page (just in case you missed our bright red text at the top and bottom!</span>). <span id="f7-n13" style="background-color: rgb(255, 255, 255);"><br /><br />Having formulated your query correctly, the next task is to pick a page from the result list. For each result, we present the title and </span>url<span id="f7-n14" style="background-color: rgb(255, 255, 255);">, and a brief two line snippet</span><span id="id_10" style="background-color: rgb(255, 255, 255);">. Pages that don't have a proper title are often ignored by users. One of the bigger recent changes has been to extract titles for pages that don't specify an HTML title -- yet a title on the page is clearly right there, staring at you. To "see" that title that the author of the page intended, we analyze the HTML of the page to determine the title that <span id="jthu0" style="background-color: rgb(255, 255, 255);">the author</span> probably meant. This makes it far more likely that you will not ignore a page for want of a good title. Below the title comes the snippet, and a key early innovation was in what Google showed for the snippet. At the time, search engines showed you the first two lines of the web page; Google, instead, showed you parts of the page where your actual search keywords showed up (information retrieval experts call this "keywords-in-context"). Showing keywords-in-context is visually simple and virtually indistinguishable from the simpler style of snippets, but vastly more useful in helping you decide which page to visit. This simplicity belies underlying complexity: when we create a snippet we have to go through the actual text from each result to find the most relevant part (which contain your keywords) rather than just giving you the <span id="ptc5" style="background-color: rgb(255, 255, 255);">first few lines. </span></span> <span id="f7-n15" style="background-color: rgb(255, 255, 255);"><span id="ptc50" style="background-color: rgb(255, 255, 255);"><br /><br />We have been making improvements to our snippets over time with algorithms for determining the relevance of portions of the page. The changes range from the subtle </span><span id="w5lq" style="background-color: rgb(255, 255, 255);">--</span><span id="ptc51" style="background-color: rgb(255, 255, 255);"> we highlight synonyms of your query terms in the results </span><span id="d8jt" style="background-color: rgb(255, 255, 255);">--</span><span id="ptc52" style="background-color: rgb(255, 255, 255);"> to more obvious. Here'</span>s an example screenshot where the user searched for "arod" and you can see that Alex and Rodriguez are </span>bolded<span id="f7-n16" style="background-color: rgb(255, 255, 255);"> in the search result snippet, based on our analysis that you might plausibly be referring to him:<br /><br /></span><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfgX4oPSooi56jGCtB05OyOZIB5frKJnGaSkdxvfZbraF6XOWygl0-QOkkIQ15pCE5f3ZW1Fkg1VdaL9LJRdlOxBagxV92i3NBLvtodmVQs4ODSlFXX0-Y6st0IAU4b4xTEtypJEOGgsxl/s1600-h/search_Aug1_ex1.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5229534971213884514" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfgX4oPSooi56jGCtB05OyOZIB5frKJnGaSkdxvfZbraF6XOWygl0-QOkkIQ15pCE5f3ZW1Fkg1VdaL9LJRdlOxBagxV92i3NBLvtodmVQs4ODSlFXX0-Y6st0IAU4b4xTEtypJEOGgsxl/s320/search_Aug1_ex1.gif" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" /></a>As a more obvious example, we now extract and show you the byline date from pages that have one. These byline dates are expressed in a myriad formats which we extract and present uniformly, so that you can scan them easily:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ5MhxQlJdZpwfG-PdSebgKAnr05wscgoZ7fDhYDjxt3RClHQOAcuQYteCNbgY4WRekKc90u4NsFr8BoRgrXBzGq2PaGhtgFWeqZ7n1OXS8MhdjciCly7fCb26AmhppZ0bGF407f6dpDH5/s1600-h/search_Aug1_ex2.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5229535610787666098" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ5MhxQlJdZpwfG-PdSebgKAnr05wscgoZ7fDhYDjxt3RClHQOAcuQYteCNbgY4WRekKc90u4NsFr8BoRgrXBzGq2PaGhtgFWeqZ7n1OXS8MhdjciCly7fCb26AmhppZ0bGF407f6dpDH5/s320/search_Aug1_ex2.gif" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" /></a>For one of the most common types of user needs, navigational queries -- where you type in the name of a web site you know -- we have introduced shortcuts (we refer to them as sitelinks). These sitelinks allow you to get to the key parts of the site and illustrate many of the same principles alluded to above; they are a simple addition to the top search result that adds a small amount of extra text to the page.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0dJ6ntM7T8VpDO56BQAVAouCw4UFoctZgc6Zade7UG51vfQXIIftXX32K-mu_BjLSPVSROlybxC9FnaY1ETbHa-bynQ_qGzo9Tdv-CmRo_a8HE1b4PeXFAJAWk_NqEB7IMcAM15SouiqT/s1600-h/search_aug1_ex3.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5229535892770837234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0dJ6ntM7T8VpDO56BQAVAouCw4UFoctZgc6Zade7UG51vfQXIIftXX32K-mu_BjLSPVSROlybxC9FnaY1ETbHa-bynQ_qGzo9Tdv-CmRo_a8HE1b4PeXFAJAWk_NqEB7IMcAM15SouiqT/s320/search_aug1_ex3.gif" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" /></a>For instance, the home page of Hewlett-Packard has almost 60 links, in a two-level menu system. Our algorithms, using a combination of different signals, pick the top ones among these that we think you are most likely to want to visit.<br /><br />What if you did not find what you were looking for among the top results? In that case, you probably need to try another query. We help you in this process by providing a set of query refinements at the bottom of the results page -- even if they don't give you the query that you need, they provide hints for different (likely more successful) directions in which you could refine your query. By placing the query refinements at the bottom of the page, the refinements don't distract users, but are there to help if the rest of the search results didn't serve a user's information need.<br /><br />I've described several key aspects of the search experience, including where we have made many changes over time -- some subtle, some more obvious. In making these changes to the search experience, how do we know we've succeeded, that we've not messed it up? We constantly evaluate our changes by sharing them with you! We launch proposed changes to a tiny fraction of our users and evaluate whether it seems to be helping or hurting their search experience. There are many metrics we use to determine if we've succeeded or failed. The process of measuring these improvements is a science in itself, with many potential pitfalls. Our experimental methodology allows us to explore a range of possibilities and launch the ones that work the best. For every feature that we launch, we have frequently run a large number of experiments that did not see the light of day.<br /><br />So let me answer the question I started with: We're actually constantly changing Google's result page and have been doing so for a long time. And no, we won't mess with a good thing. You won't let us.<br /><br />In the next post in this series, I'll talk about some of the experiments we are running, and what we hope to learn from them.<br /><br /><span class="byline-author">Posted by Ben Gomes, Distinguished Engineer</span>

• 
• 
• 

1. 1
2. 2
3. 3

• Previous
• Next

Search quality, continued

A few weeks back Udi Manber introduced the search quality group, and the previous posts in this series talked about the ranking of documents. While the ranking of web documents forms the core of what …

Đọc thêm »

Technologies behind Google ranking

In my previous post, I introduced the <a href="http://googleblog.blogspot.com/2008/07/introduction-to-google-ranking.html" id="p3.q" title="philosophies behind Google ranking">philosophies behind Google ranking</a>. As part of our effort to <a id="g4xf" href="http://googleblog.blogspot.com/2008/05/introduction-to-google-search-quality.html" title="discuss search quality more">discuss search quality</a>, I want to tell you more about the technologies behind our ranking. The core technology in our ranking system comes from the academic field of Information Retrieval (IR). The IR community has studied search for almost 50 years. It uses statistical signals of word salience, like word frequency, to rank pages. (See <a href="http://singhal.info/ieee2001.pdf" id="ae1w" title="Modern Information Retrieval: A Brief Overview">"Modern Information Retrieval: A Brief Overview"</a> for a quick overview of IR technology.) IR gave us a solid foundation, and we have built a tremendous system on top using links, page structure, and many other such innovations.<br /><br /> Search in the last decade has moved from <i id="spfg">give me what I said</i> to <i id="spfg0">give me what I want</i>. User expectations from search have rightly increased. We work hard to fulfill the expectations of each and every user, and to do that we need to better understand the pages, the queries, and our users. Over the last decade we have pushed the technologies for understanding these three components (of the search process) to completely new dimensions.<br /><br />When we talk about queries at Google, we use square brackets [ ] to mark the beginning and end of queries (see "<a href="http://www.mattcutts.com/blog/writing-google-queries" id="mc-." title="How to write queries">How to write queries</a>" by Matt Cutts), a notation I will use throughout this post. (Pages and search results change frequently, so in time, some examples used here may not behave as explained.)<br /><ul id="jdpk"><li id="jdpk0"> <b id="c0tu">Understanding pages:</b> Over years we have invested heavily in our crawl and indexing system. As a result we have a very large and very fresh index. In addition to size and freshness, we have improved our index in other ways. One of the key technologies we have developed to understand pages is <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="o2bf">associating </span>important concepts to a page even<span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="tf52"> when they are not obvious on the page</span>. We find the official homepage for <a href="http://www.sprovieri.com/" id="v.ps" title="Sprovieri Gallery">Sprovieri Gallery</a> in London for the Italian query [<a href="http://www.google.it/search?hl=it&q=galleria+sprovieri+londra" id="uhkl" title="galleria sprovieri londra">galleria sprovieri londra</a>], even though the official page does not have either London or Londra on it. In the U.S., a user searching for [cool tech pc vancouver, wa] finds the homepage <a href="http://www.cooltechpc.com/" id="fzgl12" target="_blank">www.cooltechpc.com</a> even though the page does not mention anywhere that they are in Vancouver, WA. <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="qofs">Other technologies we have developed include distinctions between important and less important words in the page and the freshness of the information on the page.</span><br /></li></ul> <ul id="jdpk1"><li id="jdpk2"> <b id="bmgn">Understanding queries:</b> It is critical that we understand what our users are looking for (beyond just the few words in their query). We have made several notable advances in this area including a best-in-class spelling suggestion system, an advanced synonyms system, and a very strong concept analysis system. </li></ul> <div id="jdpk3" style="margin-left: 40px;"> Most users have used our spelling suggestion system at one time or another. It knows that someone searching for [<a href="http://www.google.com/search?hl=en&q=kofee+annan" id="hz8c" title="kofee annan">kofee annan</a>] is really searching for Mr. Kofi Annan, and is prompted: <span class="p" id="brt_" style="color: rgb(204, 0, 0);">Did you mean: </span><a class="p" href="http://www.google.com/search?hl=en&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=kofi+annan&spell=1" id="brt_0"><b id="brt_1"><i id="brt_2">kofi</i></b> annan</a>; whereas someone searching for [<a href="http://www.google.com/search?hl=en&q=kofee+beans" id="usu6" title="kofee beans">kofee beans</a>] is actually looking for coffee beans. Doing this internationally with very high accuracy is hard, and we do it well.<br /></div><br /><div id="jdpk4" style="margin-left: 40px;"> Synonyms are the foundation of our query understanding work. This is one of the hardest problems we are solving at Google. Though sometimes obvious to humans, it is an unsolved problem in automatic language processing. As a user, I don't want to think too much about what words I should use in my queries. Often I don't even know what the right words are. This is where our synonyms system comes into action. Our synonyms system can do sophisticated query modifications, e.g., it knows that the word 'Dr' in the query [<a href="http://www.google.com/search?hl=en&q=Dr+Zhivago" id="qjfs" title="Dr Zhivago">Dr Zhivago</a>] stands for <i id="ebb9">Doctor</i> whereas in [<a href="http://www.google.com/search?hl=en&q=Rodeo+Dr" id="wpsc" title="Rodeo Dr">Rodeo Dr</a>] it means <i id="ebb90">Drive</i>. A user looking for [<a href="http://www.google.com/search?hl=en&q=back+bumper+repair" id="w9kh" title="back bumper repair">back bumper repair</a>] gets results about <i id="mig5">rear</i> bumper repair. For [<a href="http://www.google.com/search?hl=en&q=Ramstein+ab" id="riso" title="Ramstein ab">Ramstein ab</a>], we automatically look for Ramstein <i id="ebb91">Air Base</i>; for the query query [<a href="http://www.google.com/search?hl=en&q=b%26b+ab" id="fycv" title="b&b ab">b&b ab</a>] we search for <i id="xt46">Bed and Breakfasts</i> in <i id="xt460">Alberta</i>, Canada. We have developed this level of query understanding for almost <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="eauj">one</span> hundred different languages, which is what I am truly proud of.<br /></div><br /><div id="jv3g" style="margin-left: 40px;"> Another technology we use in our ranking system is concept identification. Identifying critical concepts in the query allows us to return much more relevant results. <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="d-:d">For example,</span> our algorithms understand that in the query [<a href="http://www.google.com/search?hl=en&q=new+york+times+square+church" id="d.yj" title="new york times square church">new york times square church</a>] the user is looking for the <a id="nre5" href="http://www.tscnyc.org/" title="well-known church in Times Square">well-known church in Times Square</a> and not for articles from the <i id="b7dc">New York Times</i>. We don't just stop at identifying concepts; we further enhance the query with the right concepts when, <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="h06y">for instance,</span> someone looking for [<a href="http://www.google.com/search?hl=en&q=PC+and+its+impact+on+people" id="a992" title="PC and its impact on people">PC and its impact on people</a>] is in fact looking for <i id="z7d-">impact of computers on society</i>, or someone who searches for [<a href="http://www.google.com/search?hl=en&q=rainforest+instructional+activities+for+vocabulary" id="yjid" title="rainforest instructional activities for vocabulary">rainforest instructional activities for vocabulary</a>] is really looking for <i id="ux8e">rain forest lesson plans</i>. Our query analysis algorithms have many such state-of-the-art techniques built into them, and once again, we do this internationally in almost every language we serve.<br /></div> <ul id="sx.0"><li id="sx.00"> <b id="sx.01">Understanding user<span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="h06y0">s</span>:</b> Our work on interpreting user intent is aimed at returning results people really <i id="qb0c">want</i>, not just what they <i id="qb0c0">said</i> in their query. This work starts with a world class localization system, and adds to it our advanced personalization technology, and several other great strides we have made in interpreting user intent, e.g. Universal Search.<br /></li></ul> <div id="qb0c1" style="margin-left: 40px;"> Our clear focus on "best locally relevant results served globally" is reflected in our work on localization. The same query typed in multiple countries may deserve completely different results. A user looking for [<a href="http://www.google.com/search?hl=en&q=bank" id="bn23" title="bank">bank</a>] in the US should get American banks, whereas a user in the <a href="http://www.google.co.uk/search?hl=en&q=bank" id="r09t" title="UK">UK</a> is either looking for the Bank Fashion line or for British financial institutions. The results for this query should return local financial institutions in other English speaking countries like <a href="http://www.google.com.au/search?hl=en&q=bank" id="wr74" title="Australia">Australia</a>, <a href="http://www.google.ca/search?hl=en&q=bank" id="di09" title="Canada">Canada</a>, <a href="http://www.google.co.nz/search?hl=en&q=bank" id="o:3b" title="New Zealand">New Zealand</a>, <a href="http://www.google.co.za/search?hl=en&q=bank" id="ic2g" title="South Africa">South Africa</a>. The fun really starts when this query is typed in non-English-speaking countries like <a href="http://www.google.com.eg/search?hl=ar&q=bank" id="zk-4" title="Egypt">Egypt</a>, <a href="http://www.google.co.il/search?hl=iw&q=bank" id="vk2o" title="Israel">Israel</a>, <a href="http://www.google.co.jp/search?hl=ja&q=bank" id="kfvy" title="Japan">Japan</a>, <a href="http://www.google.ru/search?hl=ru&q=bank" id="dkh4" title="Russia">Russia</a>, <a href="http://www.google.com.sa/search?hl=ar&q=bank" id="megx" title="Saudi Arabia">Saudi Arabia</a>, <a href="http://www.google.ch/search?hl=de&q=bank" id="j-5_" title="Switzerland">Switzerland</a>. Likewise the query [football] refers to entirely different sports in <a href="http://www.google.com.au/search?hl=en&q=football" id="kf::" title="Australia">Australia</a>, the <a href="http://www.google.co.uk/search?hl=en&q=football" id="d7ty" title="UK">UK</a>, and the <a href="http://www.google.com/search?hl=en&q=football" id="ouj2" title="US">US</a>. These examples mostly show how we get the localized version of the same concept correctly (financial institution, sport, etc.). However, the same query can mean entirely different things in different countries. <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="mtvm">For example</span>, [Côte d'Or] is a geographic region in <a href="http://www.google.fr/search?hl=fr&q=C%C3%B4te+d%27Or" id="yath" title="France">France</a> - but it is a large chocolate manufacturer in neighboring French-speaking <a href="http://www.google.be/search?hl=fr&q=C%C3%B4te+d%27Or" id="qcng" title="Belgium">Belgium</a>; and yes, we get that right too :-).<span id="t.-d" style="color: rgb(0, 0, 255);"><br /></span></div><br /><div id="ntam" style="margin-left: 40px;">Personalization is another strong feature in our search system which tailors search results to individual users. Users who are logged-in while searching and have signed up for Web History get results that are more relevant for them than the general Google results. For example, someone who does a lot football-related searches might get more football related results for [giants], while other users might get results related to the baseball team. Similarly, if you tend to prefer results from a particular shopping site, you will be more likely to get results from that site when you search for products. Our evaluation shows that users who get personalized results find them to be more relevant than non-personalized results.<br /></div><br /><div id="u8vo" style="margin-left: 40px;"> Another case of user intent can be observed for the query [<a href="http://www.google.com/search?hl=en&q=chevrolet+magnum" id="wdec" title="chevrolet magnum">chevrolet magnum</a>]. Magnum is actually made by Dodge and not Chevrolet. So we present the results for Dodge Magnum with the prompt See results for: <a href="http://www.google.com/search?hl=en&pwst=1&q=dodge+magnum&revid=1169821596&sa=X&oi=revisions_inline&resnum=4&ct=revision&cd=1" id="yi:q"><b id="yi:q0"><i id="yi:q1">dodge</i></b> magnum</a> in our result set.<br /></div><br /><div id="l1w_" style="margin-left: 40px;"> Our work on Universal Search is another example of how we interpret user intent to give them what they (sometimes) really want. Someone searching for [<a href="http://www.google.com/search?hl=en&q=bangalore" id="e3t3" title="bangalore">bangalore</a>] not only gets the important web pages, they also get a map, a video showing street life, traffic, etc. in Bangalore -- watching <a id="m8dr" href="http://www.youtube.com/watch?v=kbGkUFinntc" title="this video">this video</a> I almost feel I am there :-) -- and at the time of writing there is relevant news and relevant blogs about Bangalore.</div> <blockquote>Finally let me briefly mention the latest advance we have made in search: Cross Language Information Retrieval (CLIR). CLIR allows users to first discover information that is not in their language, and then using Google's translation technology, we make this information accessible. I call this advance: <i id="r:gg">give me what I want in any language</i>. A user looking for Tony Blair's biography in Russia who types the query in Russian [<a href="http://www.google.ru/search?hl=ru&q=%D0%A2%D0%BE%D0%BD%D0%B8+%D0%91%D0%BB%D1%8D%D1%80+%D0%B1%D0%B8%D0%BE%D0%B3%D1%80%D0%B0%D1%84%D0%B8%D1%8F" id="u5y0" title="Тони Блэр биография">Тони Блэр биография</a>] is prompted at the bottom of our results to search the English web with:</blockquote><div style="text-align: center;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf4ttWCrROWJH-4eJk820zBOpwf0VQuRqNsrEmPL9AQ1b4cXiXgCYpgHWt2-xDpULXs62ggaEjO5X0hSHom6dtR09mKLycHoooDgILi3MSEKO7gnZnAfBhkiDsa9xs2sBMvpEl7ZgdqTp_/s1600-h/RU_example_2.gif"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5223635052799224626" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf4ttWCrROWJH-4eJk820zBOpwf0VQuRqNsrEmPL9AQ1b4cXiXgCYpgHWt2-xDpULXs62ggaEjO5X0hSHom6dtR09mKLycHoooDgILi3MSEKO7gnZnAfBhkiDsa9xs2sBMvpEl7ZgdqTp_/s320/RU_example_2.gif" style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" /></a></div><blockquote>Similarly a user searching for Disney movie songs in Egypt with the query [<a href="http://www.google.com.eg/search?hl=ar&q=%D8%A3%D8%BA%D8%A7%D9%86%D9%8A+%D8%A3%D9%81%D9%84%D8%A7%D9%85+%D8%AF%D9%8A%D8%B2%D9%86%D9%8A" id="qs8d" title="أغاني أفلام ديزني">أغاني أفلام ديزني</a>] is prompted to search the English web. We are very excited about CLIR as it truly brings us closer to our mission to organize the world's information and make it universally accessible and useful.<br /><br />I could go on and on showing examples of state-of-the-art technology that we have developed to make our ranking system <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="cmxx">as good as it is</span>, but the fact is that search is nowhere close to being a solved problem. Many queries still don't get satisfactory results from Google, and each such query is <span class="Apple-style-span" style="background-color: rgb(255, 255, 255);" id="ay_e">an opportunity to improve our ranking system</span>. I am confident that with numerous techniques under development in our group, we will make large improvements to our ranking algorithms in the near future.</blockquote>I hope my two posts about Google ranking have made it clear that we live and breathe search, and we are more passionate than ever about it. Our fervor for serving all our users worldwide is unprecedented. We pride ourselves in running a very good ranking system, and are working incredibly hard every day to make it even better.<br /><br /><span class="byline-author">Posted by Amit Singhal, Google Fellow</span>

Technologies behind Google ranking

In my previous post, I introduced the philosophies behind Google ranking. As part of our effort to discuss search quality, I want to tell you more about the technologies behind our ranking. The core t…

Đọc thêm »

Introduction to Google Ranking

<span class="byline-author">Posted by Amit Singhal, Google Fellow</span><br /><br />In May, Udi Manber <a href="http://googleblog.blogspot.com/2008/05/introduction-to-google-search-quality.html" id="ck7l" title="introduced the search quality group">introduced our search quality group</a>, the group responsible for the ranking of search results. He introduced various teams within "Quality" (as we like to call the group) including Core Ranking, International Search, User Interfaces, Evaluation, Webspam, and other teams. In this post, I want to tell you more about one of these: the Core Ranking team.<br /><br />Let me introduce myself. My name is Amit Singhal. I'm a Google Fellow in charge of the ranking team at Google. I've worked in the field of search for the past eighteen years, having been introduced to search in 1990 as a graduate student in computer science. In the academic world, the field of search is known as Information Retrieval (or IR). After spending a decade as an IR researcher, I came to Google in 2000, and have worked on Google ranking ever since.<br /><br />Google ranking is a collection of algorithms used to find the most relevant documents for a user query. We do this for hundreds of millions of queries a day, from a collection of billions and billions of pages. These algorithms are run for every query entered into most of Google's search services. While our web search is the most used Google search service and the most widely known, the same ranking algorithms are also used - with some modifications - for other Google search services, including Images, News, YouTube, Maps, Product Search, Book Search, and more.<br /><br />The most common question I get asked about Google's ranking is "how do you do it?" Of course, there is a lot that goes into building a state-of-the-art ranking system like ours, and I will delve deeper into the technology behind it in a later post. Today, I would like to briefly share the philosophies behind Google ranking:<br /><blockquote>1) Best locally relevant results served globally.<br />2) Keep it simple.<br />3) No manual intervention.</blockquote>The first one is obvious. Given our passion for search, we absolutely want to make sure that every user query gets the most relevant results. We often call this the "no query left behind" principle. Whenever we return less than ideal results for any query in any language in any country - and we do (search is by no means a solved problem) - we use that as an inspiration for future improvements.<br /><br />The second principle seems obvious. Isn't it the desire of all system architects to keep their systems simple? Well, as search systems go, given the wide variety of user queries we have to respond to in multiple languages, it is easy to go down the path where more and more complexity creeps into the system to serve the next incremental fraction of the queries. We work very hard to keep our system simple without compromising on the quality of results. This is an ongoing effort, and a worthy one. We make about ten ranking changes every week and simplicity is a big consideration in launching every change. Our engineers understand exactly why a page was ranked the way it was for a given query. This simple understandable system has allowed us innovate quickly, and it shows. The "keep it simple" philosophy has served us well.<br /><br />No discussion of Google's ranking would be complete without asking the common - but misguided! :) - question: "Does Google manually edit its results?" Let me just answer that with our third philosophy: no manual intervention. In our view, the web is built by people. You are the ones creating pages and linking to pages. We are using all this human contribution through our algorithms. The final ordering of the results is decided by our algorithms using the contributions of the greater Internet community, not manually by us. We believe that the subjective judgment of any individual is, well ... subjective, and information distilled by our algorithms from the vast amount of human knowledge encoded in the web pages and their links is better than individual subjectivity.<br /><br />The second reason we have a principle against manually adjusting our results is that often a broken query is just a symptom of a potential improvement to be made to our ranking algorithm. Improving the underlying algorithm not only improves that one query, it improves an entire class of queries, and often for all languages. I should add, however, that there are clear written <a href="http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769#quality" id="ya_e" title="policies">policies</a> for websites recommended by Google, and we do take action on sites that are in violation of our policies or for a small number of other reasons (e.g. legal requirements, child porn, viruses/malware, etc).<br /><br />Stay tuned for my followup post, where I will discuss in detail the technologies behind our ranking and show examples of several state-of-the-art ranking techniques in action. Let me just conclude this post by saying, our passion for search is stronger than ever - and as a search researcher, I have the best job in the world :-).

Introduction to Google Ranking

Posted by Amit Singhal, Google FellowIn May, Udi Manber introduced our search quality group, the group responsible for the ranking of search results. He introduced various teams within "Quality" (as w…

Đọc thêm »