Search evaluation at Google

<a title="This series" href="http://googleblog.blogspot.com/search/label/search%20quality" id="so8g">This series</a> of posts has described Google's search quality efforts in areas such as ranking and search UI. Now I'll describe search evaluation. Simply put, search evaluation is the process of measuring the quality of our search results and our users' experience with search.<br /><br />Let me introduce myself. I'm Scott Huffman, an engineering director responsible for leading search evaluation, working with a talented team of statisticians and software engineers. I've been here since 2005, and have been working on search in one form or another for the past fourteen years or so.<br /><br />When I'm interviewing folks interested in joining the search evaluation team, I often use this scenario to describe what we do: Imagine a Google ranking engineer bursts into your office. "I have a great idea for improving our search results!" she exclaims. "It's simple: Whenever a page's title starts with the letter T, move it up in the results three slots." This engineer comes armed with several example search queries where, lo and behold, this idea actually improves the results significantly.<br /><br />Now, you and I may think that this "letter T" hack is really a silly idea, but how can we know for sure? Search evaluation is charged with answering such questions. This hack hasn't really come up, but we are constantly evaluating everything, which can include: <div id="gd2q"> - proposed improvements to segmentation of Chinese queries </div> <div id="gd2q0"> - new approaches to fight spam </div> <div id="gd2q1"> - techniques for improving how we handle compound Swedish words </div> <div id="gd2q1"> - changes to how we handle links and anchortext </div> <div id="ucyj"> - and everything in between<br /><br />As Udi mentioned in his <a title="initial post" href="http://googleblog.blogspot.com/2008/05/introduction-to-google-search-quality.html" id="xxu8">initial post</a> on search quality, in 2007 we launched over 450 improvements to Google search, and every one of them went through a comprehensive evaluation process. </div> <div id="yygm12"> <br /></div> <div id="yygm14"> Not surprisingly, we take search evaluation very seriously. Precise evaluation enables our teams to know "which way is up". One of our tenets in search quality is to be very data-driven in our decision-making. We try hard not to rely on anecdotal examples, which are often misleading in search (where decisions can affect hundreds of millions of queries a day). Meticulous, statistically-meaningful evaluation gives us the data we need to make real search improvements.<br /><br />Evaluating search is difficult for several reasons.<br /> <ul id="yygm20"><li id="yygm24" style="margin-left: 15px;"> First, understanding what a user really wants when they type a query -- the query's "intent" -- can be very difficult. For highly navigational queries like [ebay] or [orbitz], we can guess that most users want to navigate to the respective sites. But how about [olympics]? Does the user want news, medal counts from the recent Beijing games, the IOC's homepage, historical information about the games, ... ? This same exact question, of course, is faced by our ranking and search UI teams. Evaluation is the other side of that coin.<br /> <br /> </li><li id="yygm27" style="margin-left: 15px;"> Second, comparing the quality of search engines (whether Google versus our competitors, Google versus Google a month ago, or Google versus Google plus the "letter T" hack) is never black and white. It's essentially impossible to make a change that is 100% positive in all situations; with any algorithmic change you make to search, many searches will get better and some will get worse.<br /><br /> </li><li id="yygm27" style="margin-left: 15px;"> Third, there are several dimensions to "good" results. Traditional search evaluation has focused on the <i>relevance </i>of the results, and of course that is our highest priority as well. But today's search-engine users expect more than just relevance. Are the results fresh and timely? Are they from authoritative sources? Are they comprehensive? Are they free of spam? Are their titles and snippets descriptive enough? Do they include additional UI elements a user might find helpful for the query (maps, images, query suggestions, etc.)? Our evaluations attempt to cover each of these dimensions where appropriate.<br /> <br /> </li><li id="yygm30" style="margin-left: 15px;"> Fourth, evaluating Google search quality requires covering an enormous breadth. We cover over a hundred locales (country/language pairs) with in-depth evaluation. Beyond locales, we support search quality teams working on many different kinds of queries and features. For example, we explicitly measure the quality of Google's spelling suggestions, universal search results, image and video searches, related query suggestions, stock oneboxes, and many, many more. </li></ul> </div> To get at these issues, we employ a variety of evaluation methods and data sources:<br /><ul><li> Human evaluators. Google makes use of evaluators in many countries and languages. These evaluators are carefully trained and are asked to evaluate the quality of search results in several different ways. We sometimes show evaluators whole result sets by themselves or "side by side" with alternatives; in other cases, we show evaluators a single result at a time for a query and ask them to rate its quality along various dimensions.<br /><br /> </li><li> Live traffic experiments. We also make use of experiments, in which small fractions of queries are shown results from alternative search approaches. Ben Gomes talked about how we make use of these <a href="http://googleblog.blogspot.com/2008/08/search-experiments-large-and-small.html" id="leq_" title="experiments for testing search UI elements">experiments for testing search UI elements</a> in his previous post. With these experiments, we are able to see real users' reactions (clicks, etc.) to alternative results.<br /> </li></ul> Clearly, we can never measure anything close to all the queries Google will get in the future. Every day, in fact, Google gets many millions of queries that we have never seen before, and will never see again. Therefore, we measure statistically, over representative samples of the query-stream. The "letter T" hack probably does improve a few queries, but over a representative sample of queries it affects, I'm confident it would be a big loser.<br /><br />One of the key skills of our evaluation team is experimental design. For each proposed search improvement, we generate an experiment plan that will allow us to measure the key aspects of the change. Often, we use a combination of human and live traffic evaluation. For instance, consider a proposed improvement to Google's "related searches" feature to increase its coverage across several locales. Our experiment plan might include live traffic evaluation in which we show the updated related search suggestions to users and measure click-through rates in each locale and break these down by position of each related search suggestion. We might also include human evaluation, in which for a representative sample of queries in each locale, we ask evaluators to rate the appropriateness, usefulness, and relevance of each individual related search suggestion. Including both types of evaluation allows us to understand the overall behavioral impact on users (via the live traffic experiment), and measure the detailed quality of the suggestions in each locale along multiple dimensions (via the human evaluation experiment).<br /><br />Choosing an appropriate sample of queries to evaluate can be subtle. When evaluating a proposed search improvement, we consider not only whether a given query's results are changed by the proposal, but also how much <i>impact</i> the change is likely to have on users. For instance, a query whose first three results are changed is likely much higher impact than one for which results 9 and 10 are swapped. In Amit Singhal's <a id="n.lb" href="http://googleblog.blogspot.com/2008/07/technologies-behind-google-ranking.html" title="previous post on ranking">previous post on ranking</a>, he discussed synonyms. Recently, we evaluated a proposed update to make synonyms more aggressive in some cases. On a flat (non-impact-weighted) sample of affected queries, the change appeared to be quite positive. However, using an evaluation of an impact-weighted sample, we found that the change went much too far. For example, in Chinese, it synonymized "small" (小) and "big" (大)... not a good idea!<br /><br />We're serious about search evaluation because we are serious about giving you the highest quality search experience possible. Rather than guess at what will be useful, we use a careful data-driven approach to make sure our "great ideas" really are great for you. In this environment, the "letter T" hack never had a chance.<br /><br /><span class="byline-author">Posted by Scott Huffman, Engineering Director</span>

Search evaluation at Google

This series of posts has described Google's search quality efforts in areas such as ranking and search UI. Now I'll describe search...

Đọc thêm »

Alaska-bound!

As this post goes live, five Googlers are on their way to Barrow, Alaska, the northernmost inhabited location on the North American mainland. What brings us to such a far-off place? We're visiting three rural school districts to show teachers and students how <a href="http://earth.google.com/">Google Earth</a> and <a href="http://maps.google.com/">Maps</a> can be powerful tools in the classroom.<br /><br />There has been a lot of interest in the 49th state of late, but our trip has actually been months in the making. The idea for an educational outreach tour began when one active Geo developer, John Bailey of the University of Alaska Fairbanks (UAF), told us about his experiences teaching undergrads with Google Earth. We wanted to learn more and help spread the word on how educators are using these technologies, and knew we wouldn't find a better guide than the <a href="http://www.geographyua.org/">Geography Department at the University of Alaska</a>.<br /><br />In each town, we'll begin with a school-wide assembly with 9th-12th graders where our colleagues from UAF will explain our visit and help introduce Google's Geo tools, after which we'll inject some Googley fun into the mix with a trivia contest.<br />Then we'll break into smaller groups where we'll focus on three classroom activities: First, we'll use Google Earth and image overlays to see how Alaska has changed over time. Then, we'll get out in the field on a geo-caching mini-adventure where students will learn how to upload GPS tracks. Finally, students will collaborate to build a <a href="http://local.google.com/support/bin/answer.py?hl=en&answer=68480">My Map</a> showing their favorite spot in each town, some of which we'll capture with <a href="http://www.gigapan.org/?window_height=975&window_width=1920">GigaPan</a> technology.<br /><br /><iframe width="425" height="350" frameborder="0" scrolling="no" marginheight="0" marginwidth="0" src="http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=100898834134734910368.000456de638de51ce1c5d&ll=66.25382,-156.56341&spn=10.07252,17.68258&t=h&output=embed&s=AARTsJqt5TZqp7TMU5XpaNVR6qEuCkRQ4A"></iframe><br /><small><a href="http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=100898834134734910368.000456de638de51ce1c5d&ll=66.25382,-156.56341&spn=10.07252,17.68258&t=h&source=embed" style="color:#0000FF;text-align:left">View Larger Map</a></small><br /><br />In addition to the materials already available on the <a href="http://www.google.com/educators/geo.html">Google for Educators</a> site, we'll be posting updates and more tips for teachers on the <a href="http://google-latlong.blogspot.com/">LatLong blog</a> throughout the week, so be sure to check back often.<br /><span class="byline-author"><br />Posted by Anna Bishop, Geo Education 20% team</span>

Alaska-bound!

As this post goes live, five Googlers are on their way to Barrow, Alaska, the northernmost inhabited location on the North American mainland...

Đọc thêm »

Google Desktop 5.8 for Windows: increased performance

We love getting your feedback, and one of the themes that has cropped up in the <a href="http://groups.google.com/group/Google-Desktop" id="ez8a" title="Google Desktop Help group">Google Desktop Help group</a> is that you want a lighter, faster product. We heard the message loud and clear and decided that the <a href="http://desktop.google.com/">Google Desktop 5.8 for Windows</a> release would be based entirely on performance.<br /><br />In true Google fashion, we took a data-driven approach: Measure, then analyze, fix the most important issues and lowest hanging fruit, then rinse, lather and repeat.<br /><br />First, we built performance tests that simulate a typical user's behavior when using Google Desktop and measure the time spent on actions such as starting up, shutting down, searching, adding a gadget, adding a new document to the search index, and so forth. We then took a page from the <a href="http://www.google.com/googlebooks/chrome/small_08.html">Google Chrome playbook</a> by running the performance tests automatically for every single change we made to the software, on dozens of machines each time so that an average of the time measurements from all of them would give us a reliable comparison against previous versions.<br /><br />Next, we refined an option that helps us improve the product, with your agreement. To send us information about problems you might experience, you can opt in to our error reporting system. If you have opted in, the system will send us anonymous diagnostic information about such events as crashes. In the new version, we improved this system to also detect and report situations where our product could be making your computer slow to respond.<br /><br />We've learned a lot from these two new sources of data. The performance measurement system enabled us to discover some seemingly small changes that had a major impact on performance. We've been able to immediately fix changes that decreased performance, and learn from changes that improved performance. For example: <div style="text-align: left;"> <ul><li> We found a way to cut memory usage during startup by about 50%. </li><li> Shutdown now happens five times faster, and we were able to avoid a seemingly innocuous state that would have almost doubled startup time. </li></ul> </div> <div style="text-align: left;"> <div style="text-align: left; margin-top: 0px; margin-bottom: 0px;"> <div style="text-align: left; margin-top: 0px; margin-bottom: 0px;"> From automated error reports sent by our beta testers, we learned that while Google Desktop generally performs very well, certain types of usage and some interactions with 3rd party software can trigger problems. We've made dozens of improvements based on this data. Here are some examples of what we've been able to fix:<br /><br /></div> <div style="text-align: left; margin-top: 0px; margin-bottom: 0px;"> <ul style="margin-top: 0px; margin-bottom: 0px;"><li style="margin-top: 0px; margin-bottom: 0px;"> A few obscure cases where interaction with third party software could slow the computer down when certain types of files were being added to the search index. </li><li style="margin-top: 0px; margin-bottom: 0px;"> A rare issue that could cause slower computers to become temporarily unresponsive during Google Desktop startup. </li><div> <br /> </div></ul> </div> </div> </div> <div style="text-align: left;"> We still weren't satisfied, so we analyzed the design of our software to find ways to improve it. This led us to make a few more changes: </div> <div style="text-align: left;"> <ul><li> We realized that with the <a href="http://desktop.google.com/plugins/" id="zap7" title="hundreds of Google Desktop Gadgets">hundreds of Google Desktop Gadgets</a> available from third parties, there is always the possibility that one of them performs badly. To help you deal with this, we added a system that detects when a gadget is causing your computer to perform poorly, and asks you whether to remove the gadget. While we were at it, we added proactive security measures that prevent malicious gadgets from stealing information from other gadgets, and we made changes that allow Flash programs such as interactive games to be embedded in gadgets; for certain types of gadgets this can be a lighter weight approach than previously possible. </li><li> To reduce memory usage, increase stability and reduce memory fragmentation, we reduced the number of different processes that Google Desktop runs, and we now recycle some of our processes frequently (just like <a href="http://www.google.com/googlebooks/chrome/small_06.html" id="vtw9" title="Google Chrome">Google Chrome</a>). </li><li> Microsoft Outlook users will be glad to hear that we've reduced memory usage while at the same time improving the coverage and increasing the stability of our Outlook support. </li></ul> </div> <div style="text-align: left;"> We've had a fun and rewarding time improving performance, because at every step of the way we were sure that you would enjoy our product even more. Please give our <a href="http://desktop.google.com/" id="osdg" title="new version">new lighter, faster version</a> a try. We hope you'll enjoy it.<br /><br /><span class="byline-author">Posted by Jói Sigurðsson, Tech Lead, Google Desktop for Windows</span></div>

Google Desktop 5.8 for Windows: increased performance

We love getting your feedback, and one of the themes that has cropped up in the Google Desktop Help group is that you want a lighter, faste...

Đọc thêm »

The social web: All about the small stuff

<span style="font-style: italic;font-size:85%;" > The Internet has had an enormous impact on people's lives around the world in the ten years since Google's founding. It has changed politics, entertainment, culture, business, health care, the environment and just about every other topic you can think of. Which got us to thinking, what's going to happen in the next ten years? How will this phenomenal technology evolve, how will we adapt, and (more importantly) how will it adapt to us? We asked ten of our top experts this very question, and during September (our 10th anniversary month) we are presenting <a href="http://googleblog.blogspot.com/search/label/Google%20at%2010">their responses</a>. As computer scientist Alan Kay has famously observed, the best way to predict the future is to invent it, so we will be doing our best to make good on our experts' words every day. - Karen Wickre and Alan Eagle, series editors</span><br /><br />What makes two friends feel "close" to one another? I'd argue that a big part of it is the small details that you know about each other. The funny comment your friend made about a billboard they saw while driving down the road, what they had for dinner, a person they ran into on the street, their comments about the movie they saw two nights before. Closeness often comes from knowing the small things, not just the big things. Distance makes knowing those small things harder. When you live together, either with your family or your friends, knowing the small things is easy. They get conveyed when passing in the hall, sitting down to a meal or just hanging out. It's effortless.<br /><br />When you live apart, things change. Suddenly it takes effort. It used to take a <i>lot </i>more effort when writing a letter was the primary way to communicate over distance as opposed to email or IM or telephone. But, even with our current technology, it still takes work. As a result, we share less with our friends. And when we do share, we tend to share the big stuff (big shifts at work, major family events like birthdays or school milestones) and leave the small stuff behind. We start to feel less connected because we don't know the details.<br /><br />The promise of the social web is about making it easy to share the small stuff -- to make it effortless and rebuild that feeling of connectedness that comes from knowing the details. My wife recently sent out a public Picasa Web Album of baby photos to ten of her friends. Four of them wrote back saying "I didn't know Joe got a new car?!" (her friends browsed through my other public photo albums). While she would never hesitate to share the big event (new baby), she never would have shared the small detail of me getting a new car. This kind of thing is repeated again and again. The small details are left out. A weekend with Grandma and Grandpa? Thinking about selling my house? Are these things all "worth" sharing? Maybe. Sometimes. For some people.<br /><br />Fortunately, as the web becomes more social, I won't have to spend as much energy thinking about what's "interesting enough" to share with a certain group. The people who care about me and that I allow will increasingly be able to tune in to the parts of my life that interest them.<br /><br />It will be great when the instant I think of something to tell my friends, or something I need from my friends, they're available to me in some way. Remember when Google embedded IM into Gmail, and you could suddenly see -- without changing applications -- that the friend you were about to email was online and easily reachable right at that second? That little green bubble of presence right in front of our eyes brought a little extra ping of closeness that email hadn't had until then. That was in 2006, at the start of the AJAX-powered wave of dynamic web apps. Now, many sites and services are adding even more sophisticated plumbing (like profiles and friends and presence and comments) that brings the immediacy of social interaction to more and more places on the web. Reaching your friends can be really active, as IM is today, or it can be passive, like changing your status message.<br /><br />In the coming decade, the web will become as effortlessly social as chatting with your family or roommates at home is today. Social features will be embedded and around and through all variety of spaces and places on the web. Sometimes you'll go to a place because you want to see your friends, and sometimes the place you're in will get better because you can bring your friends there. It will make it easier to strike up new relationships, new communities, new expressions of what your life is about. The web will connect people to the small moments that in many ways matter most.<br /><br />We're just now starting to navigate all the intersections between sociology and engineering on the web. We -- meaning Google and many others in the web community -- are in the midst of a burst of energy around all things social that is teaching us more every day about what people want to do with their friends and where. How does iGoogle or Gmail get more powerful when you've got your friends present in some new way? What is possible on mobile devices when you can put better data about you and your friends in your pocket? What are the big plumbing problems -- like contact portability, or standards for user authentication and authorization -- that need to be solved for the whole web because no one site can do it on its own?<br /><br />Google is chipping in on all of these fronts, listening closely to our users to make our existing products more social in useful ways, and by working with the web community on software projects like <a title="OpenSocial" href="http://www.opensocial.org/" id="jrpp">OpenSocial</a> and <a title="OAuth" href="http://code.google.com/p/oauth/" id="yqhz">OAuth</a> to address some of the big infrastructure challenges that are best solved in the open, with the perspective of many developers and website owners represented. Fast forward ten years, and you'll feel even more at home on the web than you do today - because it will be a pretty good reflection of you.<br /><br /><span class="byline-author">Posted by Joe Kraus, Director of Product Management</span>

The social web: All about the small stuff

The Internet has had an enormous impact on people's lives around the world in the ten years since Google's founding. It has cha...

Đọc thêm »

Ad perfect

<div style="text-align: left;"><span style="font-style: italic;font-size:85%;" > The Internet has had an enormous impact on people's lives around the world in the ten years since Google's founding. It has changed politics, entertainment, culture, business, health care, the environment and just about every other topic you can think of. Which got us to thinking, what's going to happen in the next ten years? How will this phenomenal technology evolve, how will we adapt, and (more importantly) how will it adapt to us? We asked ten of our top experts this very question, and during September (our 10th anniversary month) we are presenting their responses. As computer scientist Alan Kay has famously observed, the best way to predict the future is to invent it, so we will be doing our best to make good on our experts' words every day. - Karen Wickre and Alan Eagle, series editors</span><br /><br />Google's advertising business was founded on the core principle that advertising should deliver the right information to the right person at the right time. This is very similar to our mission in search, and, like our colleagues in search, those of us on the ads team are constantly striving to achieve better results. We have hundreds of thousands of advertisers who collectively have millions of products and services, and out of that vast amount of information our goal is always to show people the best ads, the ones that are the most relevant, timely, and useful (and, from the advertiser perspective, measurable). Achieving this ideal has been difficult since the early days of ads, but now, with the Internet, it is within reach.<br /><br />What does it take to do this? We need to understand exactly what people are looking for, then give them exactly the information they want. Timing is an important component. For example, when a person is looking for a specific item (like those table lamps I got a couple of weeks ago), the best ads will give more specific information, like where to buy the item online and locally, along with other relevant information such as style, size, availability, and pricing. Regardless of the timing, the best ads might include images, videos, or other formats about products and services to provide the most relevant information to people to make their purchasing decision.<br /><br />In other cases, ads can help you learn about something you didn’t know you wanted. For instance, a few weeks ago I was researching roller coasters for my son when I saw a great text ad for software that actually lets you design your own roller coaster! It turned out to be the perfect gift (and I now have a budding roller coaster engineer in my house). So in this case, I was doing some basic research, and the ad helped me discover something I didn't know existed.<br /><br />As we look forward, one way to make ads better would be to customize them based on factors like a person's location or preferences. If you're in a particular neighborhood using your mobile phone to look for a specific type of restaurant or shop, ads from local vendors are likely to be very useful to you.<br /><br />Finally, it is very important that anyone be able to advertise. People benefit when they see ads from any type of business or organization regardless of its size or geography. The right product for a user might be from a company they had previously never heard of, so it needs to be very easy and quick for anyone to create good ads, to show them only to people for whom they are useful, and to measure how effective they are.<br /><br />In <a href="http://googleblog.blogspot.com/2008/09/future-of-search.html" id="wy.e" title="Marissa's post">Marissa's post</a> on Wednesday, she talked about how the science of search is still in its infancy, and how we still have many breakthroughs before us. The same applies to advertising. In coming years, as people find new ways to use the Internet and new devices with which to access it, we have the opportunity to get even smarter about the ads we show. As always, we will use the best and most innovative technologies available so we serve relevant ads for you. We will do so in a way that safeguards user privacy by honoring our commitment to transparency and choice. And most importantly, we will continue to live by the philosophy that has guided our work from the outset: getting the right ad to the right person at the right time matters.<br /><br /><span class="byline-author">Posted by Susan Wojcicki, VP, Product Management</span></div>

Ad perfect

The Internet has had an enormous impact on people's lives around the world in the ten years since Google's founding. It has cha...

Đọc thêm »

Follow your favorite blogs

<div class="separator" style="clear: both; text-align: center;"><a href="https://docs.google.com/a/google.com/File?id=c5jdf7p_470dfv63jfr_b" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" id="w_yn" src="https://docs.google.com/a/google.com/File?id=c5jdf7p_470dfv63jfr_b" style="height: 169px; width: 179px;" /></a></div><br />Some people dream of being a rock star, while others just want a few discerning souls to become avid readers of their blogs. Both are noble pursuits, but with some <a href="http://buzz.blogger.com/2008/09/new-stuff-from-picasa-folks.html">good pictures</a>, and a witty turn of phrase or two, building a Blogger fan-base may be a more attainable dream.<br /><br />At Blogger we're passionate about helping communities form around blogs. To further that goal, we've introduced a <a href="http://buzz.blogger.com/2008/08/show-off-your-followers.html">new feature</a> that lets you easily follow your favorite blogs and tell the world that you’re a fan. To follow a blog with the Followers' Gadget, simply click the “Follow This Blog” link. You can show your support for the blog by following it right from your Blogger Dashboard or in Google Reader.<br /><br />Following isn't just for bloggers, it's also designed to make it easier for readers to keep tabs on their favorite blogs. Maybe you're following a <a href="http://broomjumpingbride.blogspot.com/">wedding blog for tips</a>, have <a href="http://plushyou.blogspot.com/">a thing for plush toys</a>, or perhaps you want to keep track of <a href="http://googleforstudents.blogspot.com/">Google's university bus tour</a>.<br /><br />In addition, you can easily check out what blogs other fans are writing and following - you might even find another blog you’ll want to add to your reading list. And by the way, you can follow any blog, even if it isn't hosted on Blogger or doesn't have the Followers gadget. Just go to your Reading List in your <a href="http://www.blogger.com/">Blogger Dashboard</a>, click the “Add” button, and type in the blog’s URL. And you can even import existing subscriptions from <a href="http://reader.google.com/">Google Reader</a>.<br /><br />This feature is currently only available in English, but we plan to make it available in more languages soon. We’re also in the process of integrating with <a href="http://googleblog.blogspot.com/2008/05/friend-connected-web.html">Google Friend Connect</a> to add even more engaging social features to Blogger.<br /><br />So start following. We hope you become a fan.<br /><br /><span class="byline-author">Posted by Mendel Chuang, Product Marketing Manager</span>

Follow your favorite blogs

Some people dream of being a rock star, while others just want a few discerning souls to become avid readers of their blogs. Both are noble ...

Đọc thêm »

The future of search

<span style="font-style: italic;font-size:85%;" ><blockquote>The Internet has had an enormous impact on people's lives around the world in the 10 years since Google's founding. It has changed politics, entertainment, culture, business, health care, the environment and just about every other topic you can think of. Which got us to thinking, what's going to happen in the next 10 years? How will this phenomenal technology evolve, how will we adapt, and (more importantly) how will it adapt to us? <span style="font-style: italic;font-size:85%;" >We asked ten of our top experts this very question, and during September (our 10th anniversary month) </span>we are presenting their responses. As computer scientist Alan Kay has famously observed, the best way to predict the future is to invent it, so we will be doing our best to make good on our experts' words every day. - Karen Wickre and Alan Eagle, series editors.</blockquote></span>I am a search addict. I’m naturally inquisitive – I’ve always liked finding things out. Plus, I’ve worked at Google on search for the past 9 years and 3 months. Of course I search - a lot. Yet I would guess that on any given day, I only do about 20% of the searches that I could. This past Saturday, I kept track of the things that came up in conversation that I wanted to search for right then but couldn’t:<br /><br />Are "fab," "goy" and "eely" words? (There was a Scrabble game going on.) What time does J.C. Penney open on Saturday? Which school has a team called the Banana Slugs? What is the team mascot for San Jose State? How much power does that hydroelectric dam generate? What do you call a group of turkeys? What time does <span style="font-style: italic;">Tropic Thunder</span> show? What’s the name of that great Irish flute player, first name James? What’s the name of the largest city in Russia after Moscow and St. Petersburg? Which is older, a redwood or a cypress? What’s the oldest living thing and how old is it? Who sings “Queen of Hearts”? What kind of bird is that flying over there? Is the "LF" in San Francisco on Union Square or Union Street? What are the dance steps to the Charleston? What day of the week was <span style="font-style: italic;">The Lawrence Welk Show</span> on? What are the lyrics to “In the Mood”? How does Coumadin differ from aspirin in its blood thinning effects? What was the story behind the naming of the number "googol"?<br /><br />And those are just the ones that I remember. Looking at this list, two things are very clear: (1) I could do <b>a lot more</b> searches and (2) search still has a lot of opportunity for innovation, change, and progress. There are lots of ways that search will need to evolve in order to easily meet user needs. Let’s look at some of my unanswered questions from Saturday and consider how search might change over the next 10 years.<br /><br /><span style="font-weight: bold;font-size:130%;" >Modes</span><span style="font-size:100%;"><br />First, why couldn’t I do these searches right then, when I needed to? Because search still isn’t accessible enough or easy enough. Search needs to be more mobile – it should be available and easy to use in cell phones and in cars and on handheld, wearable devices that we don’t even have yet. For example, when the topic of the oldest living thing came up during a boat ride, everyone in the conversation was curious about it, but no one wanted to break out an awkward, slow device to do a search. It would be much nicer if we had a device with great connectivity that could do searches without interruption. One far-fetched idea: how about a wearable device that does searches in the background based on the words it picks up from conversations, and then flashes relevant facts?<br /><br />This notion brings up yet another way that “modes” of search will change – voice and natural language search. You should be able to talk to a search engine in your voice. You should also be able to ask questions verbally or by typing them in as natural language expressions. You shouldn’t have to break everything down into keywords.<br /><br />Further, why should a search be words at all? Why can’t I enter my query as a picture of the birds overhead and have the search engine identify what kind of bird it is? Why can’t I capture a snippet of audio and have the search engine identify and analyze it (a song or a stream of conversation) and tell me any relevant information about it? Services that do parts of that are available today, but not in an easy-to-use, integrated way.<br /><br />In the next 10 years, we will see radical advances in modes of search: mobile devices offering us easier search, Internet capabilities deployed in more devices, and different ways of entering and expressing your queries by voice, natural language, picture, or song, just to name a few. It’s clear that while keyword-based searching is incredibly powerful, it’s also incredibly limiting. These new modes will be one of the most sweeping changes in search.<br /><br /><span style="font-size:100%;"><span style="font-size:130%;"><b>Media</b></span><br />Then there’s the media aspect. The 10 blue links offered as results for Internet search can be amazing and even life-changing, but when you are trying to remember the steps to the Charleston, a textual web page isn’t going to be nearly as helpful as a video. The media of the results matters.<br /><br /><a title="Universal search" href="http://googleblog.blogspot.com/2007/05/behind-scenes-with-universal-search.html" id="py-o">Universal search</a>, which we released last May, was an important first step that included images, videos, news, books, and maps/local information in our main Google search results. Yet our presentation is still very linear (the results are just a list) and even (no one result is more important or larger than the next). What if the results page began to transform radically to really harness these different types of results into something that felt much more like an answer rather than just 10 independent guesses? What if results pages pulled the best media together and laid it out such that the most useful content was not only first but largest? What if we laid out content in columns to use more of the width available on newer, wider screens?<br /><br />We’ve barely scratched the surface with universal search, but it’s an important first step to exploring the full range of what we can do with rich media. For the past year, our goal has been to take advantage of these new types of results and evolve the interface design and user experience in response. You’ll see the fruits of this experimentation in the coming months, but even these changes are just the beginning. The face of search will change dramatically over the next 10 years. Maybe it should contain even more videos and images, maybe it should sharply differentiate the relative weight and accuracy of the results more, maybe it should be more interactive in terms of refinements? We’re not sure yet, but we do know that the one thing that the search experience can’t be - especially in the face of the online media explosion we’re currently experiencing - is stagnant.<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Personalization</span></span><br /></span></span><span style="font-size:100%;"><span style="font-size:100%;">Search engines 10 years from now will be a lot better than the ones we have now. We know this because Google itself gets a little better each day. We’re constantly writing and revising new notions of search relevance, and we release improvements almost daily. Those improvements add up for us and for other search engines, so it follows that search engines 10 years from now will be markedly better. Therefore, the real question is not <i>will </i>search be better, but rather<i> how</i> will it be better?</span><br /><br />One answer is clear: search engines of the future will be better in part because they will understand more about you, the individual user. Of course, you will be in control of your personal information, and whatever personal information the search engine uses will be with your permission and will be transparent to you. But even with the most rudimentary user information, search engines can and will provide drastically better search results. Maybe the search engines of the future will know where you are located, maybe they will know what you know already or what you learned earlier today, or maybe they will fully understand your preferences because you have chosen to share that information with us. We aren't sure which personal signals will be most valuable, but we're investing in research and experimentation on personalized search now because we think this will be very important later.<br /><br /><span style="font-size:100%;"><span style="font-style: italic;font-size:100%;" ><span style="font-weight: bold;">Location </span></span><br />Your location is one potentially useful facet of personalized information. Looking at my questions, the answers to a number of them (What time does J.C. Penney open? How much power does that hydroelectric dam generate? What time does <i>Tropic Thunder</i> play?) require the search engine to know that I was in Yankton, South Dakota and Crofton, Nebraska when I asked. Since location is relevant to a lot of searches, incorporating user location and context will be pivotal in increasing the relevance and ease of search in the future.<br /><br /><span style="font-weight: bold; font-style: italic;font-size:100%;" >Social</span><br />Another element of personalization is social context. Who am I friends with, and how do I relate to them? How can I harness their knowledge more efficiently? For example, I have a friend who works at a store called LF in Los Angeles (hence, the question about LF in San Francisco). By itself, “LF” is a very ambiguous acronym. According to the first page of search results on Google, it could refer to my friend’s trendy fashion store, but it could also refer to Leapfrog Enterprises, low frequency, Lebhar-Friedman, Li & Fung Investment Group, LF Driscoll Construction Management, large format, or a future concept car design from Lexus. Today, the person typing “LF” has to figure out which is the right result – to “disambiguate” the ambiguous term – but this is something that the search engine needs to get better at. Perhaps we’ll understand the semantics of the question about where LF in San Francisco is, and infer that LF is a store. Or maybe, search could analyze my social graph and realize that one of my friends works at LF, that I saw that friend this weekend, and that in that context “LF” refers to her place of employment. Algorithmic analysis of the user’s social graph to further refine a query or disambiguate it could prove very useful in the future.<br /><br />In addition, there are searches where actually asking a friend helps. I was having a hard time finding out the answer to the question about aspirin versus Coumadin because I was spelling it ‘cumitin’ and Google wasn’t correcting me. A quick email to a doctor friend, and I was back on the right track - equipped with the right spelling and his explanation of the difference, so I could search and learn even more about how these two drugs are used to thin blood. There’s a lot of expertise, knowledge, and context in users’ social graphs, so putting tools in place to make “friend-augmented" search easy could make search more efficient and more relevant.<br /><br /><span style="font-weight: bold;font-size:130%;" >Language</span><br />The above examples show how modes, media, and various forms of personalization have the potential to vastly improve search – but what about language? We know there are cases where an answer exists on the web, but not in a language you read. This is why Google is investing in machine translation. We want to be able to unlock the power of web search for anyone speaking any language. The basic concept is – if the answer exists online anywhere in any language, we’ll go get it for you, translate it and bring it back in your native tongue. This is an incredibly empowering idea that could really change the way that users experience the web and communicate with each other, particularly in languages where not a lot of native content is available. You can see our early explorations in this space here, by visiting our <a title="cross-language information retrieval tool" href="http://translate.google.com/translate_s" id="ki0d">cross-language information retrieval tool</a>.<br /><br /><span style="font-weight: bold;font-size:130%;" >Conclusion </span><br />We’re all familiar with 80-20 problems, where the last 20% of the solution is 80% of the work. Search is a 90-10 problem. Today, we have a 90% solution: I could answer all of my unanswered Saturday questions, not ideally or easily, but I could get it done with today’s search tool. (If you’re curious, the answers are below.) However, that remaining 10% of the problem really represents 90% (in fact, more than 90%) of the work. Coming up with elegant, fitting and relevant solutions to meet the challenges of mobility, modes, media, personalization, location, socialization, and language will take decades. Search is a science that will develop and advance over hundreds of years. Think of it like biology and physics in the 1500s or 1600s: it’s a new science where we make big and exciting breakthroughs all the time. However, it could be a hundred years or more before we have microscopes and an understanding of the proverbial molecules and atoms of search. Just like biology and physics several hundred years ago, the biggest advances are yet to come. That’s what makes the field of Internet search so exciting.<br /><br />So what's our straightforward definition of the ideal search engine? Your best friend with instant access to all the world’s facts and a photographic memory of everything you’ve seen and know. That search engine could tailor answers to you based on your preferences, your existing knowledge and the best available information; it could ask for clarification and present the answers in whatever setting or media worked best. That ideal search engine could have easily and elegantly quenched my withdrawal and fueled my addiction on Saturday. I’m very proud that Google in its first 10 years has changed expectations around information and how quickly and easily it should be able to be retrieved. But I’m even more excited about what Google search can achieve in the future.<br /><br />And here, in order, are the answers to my Saturday questions.<br /><br />Are fab, goy, and eely words? Yes, yes, and yes, according to Merriam-Webster:<br />Search: [fab site:m-w.com ]<br />Result: <a href="http://dev.m-w.com/dictionary/fab">http://dev.m-w.com/dictionary/fab</a><br />Search: [goy site:m-w.com]<br />Result: </span></span><span style="font-size:100%;"><span style="font-size:100%;"><a href="http://dev.m-w.com/dictionary/goy">http://dev.m-w.com/dictionary/goy</a></span></span><br /><span style="font-size:100%;"><span style="font-size:100%;">Search:[eely site:m-w.com ]<br />Result: <a href="http://dev.m-w.com/dictionary/eely">http://dev.m-w.com/dictionary/eely</a><br /><br />What time does J.C. Penney open on Saturday? 10 a.m.<br />Search: [jc penney yankton ]<br />Hours on results page: <a href="http://www.google.com/search?q=jcpenney+yankton">http://www.google.com/search?q=jcpenney+yankton</a><br /><br />Which school has a team called the Banana Slugs? University of California, Santa Cruz<br />Search: [banana slugs]<br />Result: <a href="http://en.wikipedia.org/wiki/University_of_California,_Santa_Cruz">http://en.wikipedia.org/wiki/University_of_California,_Santa_Cruz</a><br /><br />What is the team mascot for San Jose State? The San Jose State Spartans<br />Search: [san jose state mascot]<br />On results page: <a href="http://www.google.com/search?q=san+jose+state+mascot">http://www.google.com/search?q=san+jose+state+mascot</a><br /><br />How much power does that hydroelectric dam generate? $35M of electricity annually<br />Search: [hydroelectric dam crofton yankton]<br />Search: [gavins point dam]<br />Result: <a href="https://www.nwo.usace.army.mil/html/Lake_Proj/gavinspoint/welcome.html">https://www.nwo.usace.army.mil/html/Lake_Proj/gavinspoint/welcome.html</a><br /><br />What do you call a group of turkeys? A rafter of turkeys<br />Search: [group of turkeys]<br />On results page: <a href="http://www.google.com/search?q=group+of+turkeys">http://www.google.com/search?q=group+of+turkeys</a><br /><br />What time does <i>Tropic Thunder</i> show? 7 p.m.<br />Search: [movies yankton mall]<br />Result: <a href="http://www.moviefone.com/theater/carmike-cinemas-yankton-mall-5/9346/showtimes">http://www.moviefone.com/theater/carmike-cinemas-yankton-mall-5/9346/showtimes</a><br /><br />What’s the name of that great Irish flute player, first name James? James Galway<br />Search: [irish flute player james]<br />On results page: <a href="http://www.google.com/search?q=irish+flute+player+james">http://www.google.com/search?q=irish+flute+player+james</a><br /><br />What’s the name of the largest city in Russia after Moscow and St. Petersburg? </span></span>Novosibirsk<br /><span style="font-size:100%;"><span style="font-size:100%;">Search: [largest Russian cities]<br />Result: <a href="http://en.wikipedia.org/wiki/List_of_cities_and_towns_in_Russia_by_population">http://en.wikipedia.org/wiki/List_of_cities_and_towns_in_Russia_by_population</a><br /><br />What’s older, a redwood or a cypress? Cypresses (4500 years old is oldest known) are older than redwoods (2200 years old is oldest known)<br />Search: [cypress tree age]<br />Result: <a href="http://www.payvand.com/news/08/apr/1253.html">http://www.payvand.com/news/08/apr/1253.html</a><br />Search: [redwood tree age]<br />Result: <a href="http://www.sempervirens.org/sequoiasemp.htm">http://www.sempervirens.org/sequoiasemp.htm</a><br /><br />What’s the oldest living thing and how old is it? The bristlecone pine, living for 5,000-11,000 years<br />Search: [oldest living thing]<br />Result: <a href="http://waynesword.palomar.edu/ww0601.htm">http://waynesword.palomar.edu/ww0601.htm</a><br /><a href="http://hubpages.com/hub/Oldest_living_thing">http://hubpages.com/hub/Oldest_living_thing</a><br /><br />Who sings “Queen of Hearts”? Juice Newton<br />Search: ["queen of hearts" song]<br />On results page: <a href="http://www.google.com/search?%20=%22queen+of+hearts%22+song">http://www.google.com/search? =%22queen+of+hearts%22+song</a><br /><br />What kind of bird is that flying over there? A turkey vulture<br />Search: [turkey vulture flying] on Google image search<br />Pictures that match on results page: <a href="http://images.google.com/images?q=turkey%20vulture%20flying">http://images.google.com/images?q=turkey%20vulture%20flying</a><br /><br />Is the LF in San Francisco on Union Square or Union Street? 1870 Union Street<br />Search: [lf san francisco]<br />Address on results page: <a href="http://www.google.com/search?q=lf+san+francisco">http://www.google.com/search?q=lf+san+francisco</a><br /><br />What are the dance steps to the Charleston? Show in video below<br />Search : [Charleston dance demonstration]<br />Video result: <a href="http://uk.youtube.com/watch?v=zzyg7l6qxNQ">http://uk.youtube.com/watch?v=zzyg7l6qxNQ</a><br /><br />What day of the week was <i>The Lawrence Welk Show </i>on? Saturday<br />Search: [lawrence welk show]<br />Result: <a href="http://en.wikipedia.org/wiki/The_Lawrence_Welk_Show">http://en.wikipedia.org/wiki/The_Lawrence_Welk_Show</a><br /><br />What are the lyrics to “In the Mood”?<br />“In the mood, that's what he told me,<br />In the mood, and when he told me,<br />In the mood, my heart was skippin',<br />It didn't take me long to say "I'm in the mood now".”<br />Search: [“in the mood” lyrics]<br />Result: <a href="http://www.lyricsdepot.com/glenn-miller/in-the-mood.html">http://www.lyricsdepot.com/glenn-miller/in-the-mood.html</a><br /><br />How does Coumadin differ from aspirin in its blood thinning effects? Aspirin is an anti-platelet agent that prevents clotting. Coumadin also prevents clotting but the mechanism is different. Both thin the blood, but Coumadin is stronger and much more effective in certain instances like atrial fibrillation.<br />Search: [aspirin Coumadin how different]<br />Result: <a href="http://www.stmaryhealthcare.org/body.cfm?id=250">http://www.stmaryhealthcare.org/body.cfm?id=250</a><br /><br />What was the story behind the naming of the number "googol"?<br />Search: [number googol named]<br />Result: <a href="http://en.wikipedia.org/wiki/Quintillion#The_googol_family">http://en.wikipedia.org/wiki/Quintillion#The_googol_family</a><br /><br /><span class="byline-author">Posted by Marissa Mayer, VP, Search Products & User Experience</span></span></span>

The future of search

The Internet has had an enormous impact on people's lives around the world in the 10 years since Google's founding. It has changed p...

Đọc thêm »