healthy4u

The Robots Exclusion Protocol

Posted by Dan Crow, Product Manager This is the second in a short series of posts about the Robots Exclusion Protocol, the standard for controlling how web pages on your site are indexed. This post provides more details and examples of mechanisms to control access and indexing of your website by Google. In the <a href="http://googleblog.blogspot.com/2007/01/controlling-how-search-engines-access.html">first post</a> in this series, I introduced robots.txt and robots META tags, giving an overview of when to use them. In this post, I'll look at some examples of the power of the protocol. These examples illustrate the detailed and fine-grain control online publishers have over how their websites are indexed. <h3> Preventing Googlebot from following a link </h3> Usually when the Googlebot finds a page, it reads all the links on that page and then fetches those pages and indexes them. This is the basic process by which Googlebot "crawls" the web. This is useful as it allows Google to include all the pages on your site, as long as they are linked together. Let's say you run the TheHighsteadPost.com website. Here's a map of part of the site: <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC5VlUtr2xKs7blO1bn74QT6CCU5kd3BCGflIEZe9PWXu7sXUfwWZDH22x1i16gQLwdapPvFf7LdT6k_VdEXizSgd8LWghNTr_nNP9IFBIzZlD3I0bJA78ExB6YdF34KzrjXrITHI1UnA/s1600-h/TheHighsteadPost.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5034461632325253906" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC5VlUtr2xKs7blO1bn74QT6CCU5kd3BCGflIEZe9PWXu7sXUfwWZDH22x1i16gQLwdapPvFf7LdT6k_VdEXizSgd8LWghNTr_nNP9IFBIzZlD3I0bJA78ExB6YdF34KzrjXrITHI1UnA/s400/TheHighsteadPost.png" style="cursor: pointer;" /></a> When Googlebot crawls the index.html file, it finds the links to breakingnews.html and articles.html. From breakingnews.html, it can find valentinesday.html and promnight.html and so on. What if you didn't want valentinesday.html and promnight.html appearing in Google's index? The articles in the Breaking News section may only appear for a few hours before being updated and moved to the Articles section. In this case you want the full articles indexed, not the breaking news version. You could put the NOINDEX tag on both those pages. But if the set of pages in the Breaking News section changed frequently, it would be a lot of work to continually update the pages with the NOINDEX tag and then remove it again when they moved into the articles section. Instead, you can add the NOFOLLOW tag to the breakingnews.html page. This tells the Googlebot not to follow any links it finds on that page, thus hiding valentinesday.html and promnight.html and any other pages linked from there. Simply add this line to the <HEAD> section of breakingnews.html: <META NAME="ROBOTS" CONTENT="NOFOLLOW"> However, there is an important caveat to NOFOLLOW that you should know about. It only stops Google from following links from one page to another. If one of the linked pages is also linked from somewhere else, Google can still find and index that page via that other link. For example if promnight.html is also linked from HighsteadCourier.com, Google can still find and index promnight.html when it indexes HighsteadCourier.com and follows the link from there to promnight.html. Using NOFOLLOW is generally not the best method to ensure content does not appear in our search results. Using the NOINDEX tag on individual pages or controlling access using robots.txt is the best way to achieve this. <h3> Controlling Caching and Snippets</h3> The Robots Exclusion Protocol allows you to specify, to some extent, how you would like your web pages should appear in Google's search results. Usually search results show a cached page link and a snippet, two features that our users tell us are very useful. Here, for example, is the first result I got when I searched for "Mallard duck": <h3><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheQJMwn_UsLsfo5hRRlljrMTBU3_7qQ0j7r8nkLZ5mRD4RO8wxi7QG4GB7X6AK7TAnLSd31RhWN0R7PGtMwfDGi9N2BXeVRjsjUHqCOGWwrlhzTquaTKEuK6X-W87NRIpvbYqniABHOhU/s1600-h/Picture+2.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5034464415464061730" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheQJMwn_UsLsfo5hRRlljrMTBU3_7qQ0j7r8nkLZ5mRD4RO8wxi7QG4GB7X6AK7TAnLSd31RhWN0R7PGtMwfDGi9N2BXeVRjsjUHqCOGWwrlhzTquaTKEuK6X-W87NRIpvbYqniABHOhU/s400/Picture+2.png" style="cursor: pointer;" /></a></h3> The snippet is the extract of text from the web page, in this case it starts "The mallard duck is found mostly in North America...". We know from user studies that users are more likely to visit your site if the search results show the snippet. Why? Because snippets make it much easier for users to see why the result is relevant to their query. If a user isn't able to make this determination quickly, he or she usually moves on to the next search result. Underneath the snippet is the URL of the page followed by the "cached" link. Clicking on this link takes you to a copy of the page stored on Google's servers. This is useful in a number of cases: for sites that are temporarily unavailable; for news sites that get overloaded in the aftermath of a major event, for example, 9/11; for sites that are accidentally deleted. Another advantage is that Google's cached copy highlights the words a person searched for, allowing them to quickly see how the page is relevant to their query. Usually you want Google to display both the snippet and the cached link. However, there are some cases where you might want to disable one or both of these. For example, say you were a newspaper publisher, and you have a page whose content changes several times a day. It may take longer than a day for us to reindex a page, so users may have access to a cached copy of the page that is not the same as the one currently on your site. In this case, you probably don't want the cached link appearing in our results. Again, the Robots Exclusion Protocol comes to your aid. Add the NOARCHIVE tag to a web page and Google won't cache copy of a web page in search results: <META NAME="GOOGLEBOT" CONTENT="NOARCHIVE"> Similarly, you can tell Google not to display a snippet for a page. The NOSNIPPET tag achieves this: <META NAME="GOOGLEBOT" CONTENT="NOSNIPPET"> Adding NOSNIPPET also has the effect of preventing a cache link from being shown, so if you specify NOSNIPPET you automatically get NOARCHIVE too. <h3> Learn more </h3> As usual the Google Webmaster Help pages have a lot of useful information: <ul> <li> <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=40364" title="More on Googlebot and robots.txt">More on Googlebot and robots.txt</a> </li> <li> <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=35248&topic=8846" title="Our robots.txt analysis tool">Our robots.txt analysis tool</a> </li> </ul> <h3> </h3><h3> Next time... </h3> The final post in this series will take some common exclusion problems that webmasters have told us about and show how to solve them using the Robots Exclusion Protocol.

The Robots Exclusion Protocol

Posted by Dan Crow, Product Manager This is the second in a short series of posts about the Robots Exclusion Protocol , the standard for con...

Đọc thêm »

You never know what you'll need to know

Posted by Anita Yuen, Senior Product Marketing Manager When people share their stories about how Google search has made a difference in their lives, we know we're doing our job. It's also given us an opportunity to learn about the breadth of information that you can find on Google. Such as how to find a lost tortoise -- as Jim Lyness did. Here's his story: <div><div style="direction: ltr;"><blockquote> "After Christmas, my son Sam wanted a turtle. We bought a Russian Tortoise instead and named him Rocky. Well, one day, we let Rocky out for a stroll around the house. We could not find him that night and into the afternoon the following day. After the boys went to school, my wife, Susan, and I were stumped. Did Rocky get out the front door? My wife told me I was crazy. Susan googled [how to find a Russian Tortoise] and bang -- we had a game plan. Russian Tortoises like warm, dark spaces. We started in the boys' bedroom, again. We pulled the bunk bed back and there was Rocky at the head of the bed. Case solved. When we tell friends and family about googling How to Find a Russian Tortoise, they bust a gut in laughter! </blockquote> </div> </div> If you have a story about how Google search has made an impact on you, we'd love to hear it. <a href="http://www.google.com/contact/success.html">Tell us here</a> or <a href="http://www.youtube.com/my_videos_upload">post a video</a> (be sure to tag it "google testimonial"). You never know when you'll need to search for a lost pet. p.s. While our <a href="http://googleblog.blogspot.com/2006/10/do-you-google.html">lawyers may not be happy</a> with Jim's use of 'googled' and 'googling', we are very pleased that Jim and Susan were able to find what they needed by searching on Google.

You never know what you'll need to know

Posted by Anita Yuen, Senior Product Marketing Manager When people share their stories about how Google search has made a difference in thei...

Đọc thêm »

Google Apps grows up

Posted by Derek Parham, Lead Software Engineer, Google Apps Back in 2005, Google Apps was conceived in a few lines of code, and then it was born in February 2006. Our team has had such a close relationship with it, you might understand how we have nurtured it as we would a child. So first there was <a href="http://googleblog.blogspot.com/2006/02/big-mail-on-campus.html">Gmail for your domain</a> -- a limited service that helped organizations like San Jose City College offer personalized Gmail inboxes to all their users. As our little guy picked up new skills (Calendar, Talk, and Page Creator) it grew out of its old name and into <a href="http://googleblog.blogspot.com/2006/08/get-your-people-talking.html">Google Apps for Your Domain</a>. A quick learner, by October Google Apps had perfected <a href="http://googleblog.blogspot.com/2006/10/get-your-people-talking-in-more.html">17 more languages</a>, so we could help bring our communication tools all around the globe. Later in the fall, we improved our organizational skills with the <a href="http://googleblog.blogspot.com/2006/11/start-your-day-off-right.html">Start Page</a>, which brought all the Apps together into a centralized place. Then it was time to start school. <a href="http://googleblog.blogspot.com/2006/10/score-one-for-sun-devils.html">Google Apps entered Arizona State University</a> and stood out as one of those high achievers. Today, students and administrators at large universities like ASU and <a href="http://communications.lakeheadu.ca/news/?display=news&nid=310">Lakehead</a> are raving about Apps -- how it saves money and IT resources, plus make students lives easier with bigger spam-free mailboxes and a set of tools for working together. Now, I'm excited to tell you that our baby has finally graduated and is entering the business world. <a href="http://www.google.com/a/enterprise">Google Apps Premier Edition</a> is a new version designed to take on all the challenges presented by businesses with complex IT needs. For $50 per account per year, you get the whole Google Apps package plus many new business-oriented features, including access to our APIs and partner solutions (so it’s easy to integrate with existing systems), conference room scheduling for Calendar, 10GB of inbox storage, extended business hours phone support, and mobile access to your email on BlackBerry devices (just in case you can't get enough at the office). Already, companies big and small, like Procter & Gamble, General Electric Corporation, Prudential, and SF Bay Pediatrics, are <a href="http://www.google.com/a/help/intl/en/admins/customers.html">talking about</a> how this new version of Google Apps makes it easy to offer low-cost communication and collaboration tools to all their employees so they can get on with what they do best. Google Apps also won't forget its roots anytime soon. The <a href="http://www.google.com/a">Standard</a> and <a href="http://www.google.com/a/edu">Education</a> Editions will continue to be offered for free, and we'll keep working on all three flavors of Google Apps with the help of feedback from all of you. As a start, we’ve just integrated Google Docs & Spreadsheets in all three editions so that everyone can share and edit documents online. Since August, we’ve also added five more major features you've requested, including customized service URLs (mail.yourcompany.com) and domain registration for organizations that don’t yet have a custom domain. Our appearance has matured too, with updates to the administrator control panel that make it easier to setup and manage your services.

Google Apps grows up

Posted by Derek Parham, Lead Software Engineer, Google Apps Back in 2005, Google Apps was conceived in a few lines of code, and then it was ...

Đọc thêm »

This is National Engineers Week

Posted by Shannon Madison, Instructional Designer It's that time of year: to <a href="http://www.eweek.org/">celebrate the engineers</a> who create the new technologies that revolutionize our world. Their work changes the way we work, play, learn, and communicate with each other. As part of our ongoing commitment to raise awareness of these contributions (here's more on our <a href="http://www.google.com/support/jobs/bin/answer.py?answer=57415">our K-12 initiatives</a>), and to get young students interested in engineering, we're hosting a <a href="http://www.eweek.org/2002/News/Eweek/girlsday.shtml">Introduce a Girl to Engineering Day</a> on February 22nd in several of our U.S. offices, including New York, Kirkland, and our headquarters in Mountain View. Hundreds of middle-school girls will come to these offices this week to buddy up with an engineer, attend interactive workshops, take a tour, and, of course, have lunch. (Just like last year - see photo.) <div style="text-align: center;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_UDmG3Mkm0Lu6htfaosa3DvyRh2ssVvfekWMcR3FZKihOg_VCC4j7O_LfO79_jPV-qaa0E76Qtccf5ygaDkcSPvzFuC3H_9kMawc5hHCrhm9UqZhXMvbvq-OYXWLkbPsV-6h5ZrKJ19N1/s1600-h/Girls_Eng_day.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5033736934979061090" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_UDmG3Mkm0Lu6htfaosa3DvyRh2ssVvfekWMcR3FZKihOg_VCC4j7O_LfO79_jPV-qaa0E76Qtccf5ygaDkcSPvzFuC3H_9kMawc5hHCrhm9UqZhXMvbvq-OYXWLkbPsV-6h5ZrKJ19N1/s320/Girls_Eng_day.jpg" style="cursor: pointer;" /></a> </div> Here's hoping the day will spark the interest of these young women to learn more about science, math and engineering -- and maybe even inspire them to pursue engineering studies, so that they too can change our lives. Here's <a href="http://www.eweek.org/site/Students/index.shtml">more for students</a> to do, and <a href="http://www.eweek.org/site/Teachers/index.shtml">a page for teachers</a> too.

This is National Engineers Week

Posted by Shannon Madison, Instructional Designer It's that time of year: to celebrate the engineers who create the new technologies t...

Đọc thêm »

Online child safety initiatives

Posted by Elliot Schrage, VP, Global Communications and Public Affairs A big topic of discussion lately is the increasingly large influence media of all kinds can have in the lives of children and teens. Last week I participated in a <a href="http://www.commonsensemedia.org/beyondprimetime/" title="conference">conference</a> in New York about this important issue, sponsored by <a href="http://www.commonsensemedia.org/">CommonSense Media</a> and <a href="http://www.aspeninstitute.org/site/c.huLWJeMRKpH/b.2280181/k.3A10/Beyond_Primetime_Kids_and_the_Media.htm">The Aspen Institute</a> together with several organizations that have a stake in child safety. (Here are <a href="http://www.commonsensemedia.org/beyondprimetime/webcasts.php">videos of the speakers</a>.) The conference organizers convened a panel of experts and executives to address the question "Does the Internet Change Everything?" I approached this question by suggesting a framework of four scenarios that characterize the online interactions of children and the content they encounter: <ol><li>When a child isn't actively seeking objectionable content online, and doesn't encounter any, no action is needed.</li><li>When a child isn't seeking objectionable content, but comes across it inadvertently, ISPs and other online services, like Google, and child safety organizations can provide tools and resources to help families effectively monitor their child's online interactions. </li><li>When a child is actively seeking out objectionable content online and finds it, parents are primarily responsible for devising a solution. </li><li>When a child isn't seeking out objectionable content, but someone deliberately forces such content on them, this amounts to exploitation -- and requires government involvement and cooperation by ISPs and other online services. </li></ol>So that you can see the types of things we're doing to promote safer online interactions, I've mapped some of Google's activities in this critical area of child safety online to these scenarios -- particularly 2 and 4, where we feel we can make the greatest difference. In connection with the second scenario, we have invested in developing family safety technology and tools, including <a title="SafeSearch" href="http://www.google.com/help/customize.html#safe">SafeSearch</a>, a filter that uses advanced technology to block pornographic and explicit content from Google search results. We've also partnered with child safety organizations to educate families about ways to use the Internet and other types of media safely. These efforts include joining forces with CommonSense Media to provide <a title="their movie reviews in Google search results" href="http://www.google.com/movies/reviews?cid=b86c7458ef13b529&fq=the+queen">their movie reviews in Google search results</a> to assist parents in identifying healthy content. We also work with organizations like <a title="i-Safe" href="http://www.isafe.org/">i-Safe</a> and <a title="iKeepSafe" href="http://www.ikeepsafe.org/iksc_kids/">iKeepSafe</a> to provide online public service announcements that promote access to resources about Internet safety. When the fourth scenario occurs, we work closely with law enforcement officials worldwide to identify, investigate and prosecute child pornography and exploitation. We remove child pornography immediately when we become aware of its presence on our search engine or content services, and report all instances of child pornography to law enforcement through the <a title="National Center of Missing and Exploited Children, or NCMEC" href="http://www.missingkids.com/">National Center of Missing and Exploited Children, or NCMEC</a>. We also respond to hundreds of child safety-related law enforcement requests each year, in addition to requests to preserve data related to these cases. Lastly, we donate hardware and software to improve NCMEC's ability to manage incoming reports of child exploitation and assist NCMEC in promoting its <a title="CyberTipline" href="http://www.cybertipline.com/">CyberTipline</a>, a resource for reporting cases of online child sexual exploitation. Keeping kids safe on the Internet is a huge task -- bigger than any single government, company or family. We're pleased to work with our industry partners, law enforcement and child safety advocates around the world (including the <a href="http://www.iwf.org.uk/">Internet Watch Foundation</a> in the UK and the Freiwillige Selbstkontrolle Multimedia (<a href="http://www.fsm.de/">FSM</a>) in Germany) to address this issue. As we develop new initiatives in this critical arena, we'll keep you posted.

Online child safety initiatives

Posted by Elliot Schrage, VP, Global Communications and Public Affairs A big topic of discussion lately is the increasingly large influence ...

Đọc thêm »

From Gmail with <3

Posted by David Murray, Associate Product Manager Gmail sign-ups are now open worldwide! No more waiting for someone to invite you—just create an account directly at <a href="http://www.gmail.com/" title="www.gmail.com">www.gmail.com</a>. What better way to share the love with the people you care about than with <a href="http://googleblog.blogspot.com/2006/02/gmail-hearts-you.html">Gmail chat with <3</a>. And Gmail is available in <a href="http://mail.google.com/support/bin/answer.py?answer=17091&query=languages&topic=&type=f&ctx=search%20%20">over 40 interface languages</a> (though <3 needs no translation). But seriously folks, good relationships are built on good communication. There's no reason you should struggle to reach the ones you love, and Gmail helps you communicate fast and easily. Update: Just for the heck of it, we enlisted the help of an ensemble cast we think you'll enjoy. <a href="http://www.youtube.com/watch?v=_YUugB4IUl4">Check out our 4-part video.</a>

From Gmail with <3

Posted by David Murray, Associate Product Manager Gmail sign-ups are now open worldwide! No more waiting for someone to invite you—just crea...

Đọc thêm »

Strawberries are red, stems are green...

Posted by Dennis Hwang, Google Doodler You may have noticed today's <a href="http://www.google.com/intl/en/holidaylogos.html">Google Doodle</a> on our <a href="http://www.google.com/">homepage</a> in celebration of romance and good food (something we consider extremely important). <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKB6VSEE66HNke_8OTx7FicT8jFOBunqfGFEk4JCIvW619VIUCafP0mnKSwmCAI4FynX8vg6ocwCDYlVHbL1-Q85g39aExMrnwV5oqY7e1dd9ipv7FplVlk1aRUOtabPxFoQ2za-R6eZEX/s1600-h/V+Day+image.jpg"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5031447915006321154" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKB6VSEE66HNke_8OTx7FicT8jFOBunqfGFEk4JCIvW619VIUCafP0mnKSwmCAI4FynX8vg6ocwCDYlVHbL1-Q85g39aExMrnwV5oqY7e1dd9ipv7FplVlk1aRUOtabPxFoQ2za-R6eZEX/s320/V+Day+image.jpg" style="cursor: pointer;" /></a> When you look at the logo, you may worry that we forgot our name overnight, skipped a letter, or have decided that "Googe" has a better ring to it. None of the above. I just know that those with true romance and poetry in their soul will see the subtlety immediately. And if you're feeling grouchy today, may I suggest eating a strawberry. A very happy Valentine's Day to everyone!

Strawberries are red, stems are green...

Posted by Dennis Hwang, Google Doodler You may have noticed today's Google Doodle on our homepage in celebration of romance and good ...

Đọc thêm »