healthy4u

Using data to help prevent fraud

Posted by Shuman Ghosemajumder, Business Product Manager for Trust & Safety We recently began a series of posts on how we harness the power of data. <a title="Earlier" href="http://googleblog.blogspot.com/2008/03/why-data-matters.html" target="_blank">Earlier</a> we told you how data has been critical to the advancement of search technology. <a title="Then" href="http://googleblog.blogspot.com/2008/03/using-log-data-to-help-keep-you-safe.html" target="_blank">Then</a> we shared how we use log data to help make Google products safer for users. This post is the newest in the series. -Ed. Protecting our advertisers against click fraud is a lot like solving a crime: the more clues we have, the better we can determine <a href="http://adwords.blogspot.com/2007/02/invalid-clicks-googles-overall-numbers.html" id="v4:f" title="which clicks to mark as invalid">which clicks to mark as invalid</a>, so advertisers are not charged for them. As we've mentioned before, our <a href="http://adwords.blogspot.com/2007/02/meet-click-quality-team.html" id="v3:6" title="Click Quality team">Ad Traffic Quality team</a> built, and is constantly adding to, our <a href="http://adwords.blogspot.com/2007/02/invalid-clicks-googles-overall-numbers.html" id="uta7" title="three-stage system">three-stage system</a> for detecting invalid clicks. The three stages are: (1) proactive real-time filters, (2) proactive offline analysis, and (3) reactive investigations. So how do we use logs information for click fraud detection? Our logs are where we get the clues for the detective work. Logs provide us with the repository of data which are used to detect patterns, anomalous behavior, and other signals indicative of click fraud. Millions of users click on AdWords ads every day. Every single one of those clicks -- and the even more numerous impressions associated with them -- is analyzed by our filters (stage 1), which operate in real-time. This stage certainly utilizes our logs data, but it is stages 2 and 3 which rely even more heavily on deeper analysis of the data in our logs. For example, in stage 2, our team pores over the millions of impressions and clicks -- as well as conversions -- over a longer time period. In combing through all this information, our team is looking for unusual behavior in hundreds of different data points. <a href="https://adwords.google.com/support/bin/answer.py?answer=6322" id="r4u0" title="IP addresses"> </a> <a href="https://adwords.google.com/support/bin/answer.py?answer=6322" id="r4u0" title="IP addresses">IP addresses</a> of computers clicking on ads are very useful data points. A simple use of IP addresses is determining the source location for traffic. That is, for a given publisher or advertiser, where are their clicks coming from? Are they all coming from one country or city? Is that normal for an ad of this type? Although we don't use this information to identify individuals, we look at these in aggregate and study patterns. This information is imperfect, but by analyzing a large volume of this data it is very helpful in helping to prevent fraud. For example, examining an IP address usually tells us which ISP that person is using. It is easy for people on most home Internet connections to get a new IP address by simply rebooting their DSL or cable modem. However, that new IP address will still be registered to their ISP, so additional ad clicks from that machine will still have something in common. Seeing an abnormally high number of clicks on a single publisher from the same ISP isn't necessarily proof of fraud, but it does look suspicious and raises a flag for us to investigate. Other information contained in our logs, such as the browser type and operating system of machines associated with ad clicks, are analyzed in similar ways. These data points are just a few examples of hundreds of different factors we take into account in click fraud detection. Without this information, and enough of it to identify fraud attempted over a longer time period, it would be extremely difficult to detect invalid clicks with a high degree of confidence, and proactively create filters that help optimize advertiser ROI. Of course, we don't need this information forever; last year we started <a href="http://googleblog.blogspot.com/2007/03/taking-steps-to-further-improve-our.html" id="yu6a" title="anonymizing server logs">anonymizing server logs</a> after 18 months. As always, our goal is to balance the utility of this information (as we try to improve Google’s services for you) with the best privacy practices for our users. If you want to learn more about how we collect information to better detect click fraud, visit our <a href="http://www.google.com/adwords/adtrafficquality/" id="bwg:" title="Ad Traffic Quality Resource Center">Ad Traffic Quality Resource Center</a>.

Using data to help prevent fraud

Posted by Shuman Ghosemajumder, Business Product Manager for Trust & Safety We recently began a series of posts on how we harness the po...

Đọc thêm »

Vote for the YouTube Awards

Vote for the YouTube Awards

Posted by Mia Quagliarello, YouTube Team We're thrilled to say that the second annual YouTube Awards are now live and ready for your vot...

Đọc thêm »

Keeping up with Google Apps

Posted by Jeremy Milo, Marketing Manager During 2007, we introduced 40 significant improvements to <a href="http://www.google.com/a/help/intl/en/business/applications.html" id="a4g:" title="Google Apps">Google Apps</a>, which is our suite of communication, collaboration, and security apps for businesses, schools, and organizations. Feedback from many of you, and even our own employees, inspired these changes. And there are many more in store -- the count this year is already up to 11. Since we introduced Google Apps a little more than a year ago, many of these features have been of particular benefit to businesses, like tools to migrate mail from an old email system to Google Apps; integration of Postini's security and compliance services; support for IMAP in Gmail; syncing tools for Google Calendar; support for dozens of new languages; new mobile access options; and just weeks ago, Google Sites. We're able to release these sorts of things quickly because all of our apps live "in the cloud." That means updates are continuous: you get improvements without having to wait for new versions, as you do with traditional software. Of course, it may also mean that IT administrators get a little dizzy from the pace of these changes. The latest improvement will help technology departments and those of you who already use Google Apps keep up with every change, no matter how often they appear: Now we're documenting all updates in an <a href="http://feeds.feedburner.com/GoogleAppsUpdates" id="ze6y" title="RSS feed">RSS feed</a>, so you can get notified as they happen. To subscribe to new updates and see details about past improvements, click this "Add to Google" button: <div style="text-align: center;"><a href="http://fusion.google.com/add?source=atgs&feedurl=http://feeds.feedburner.com/GoogleAppsUpdates"><img alt="Add to Google" border="0" src="http://buttons.googlesyndication.com/fusion/add.gif" /></a> </div> If there's something you'd like to see in Google Apps, <a href="http://google.com/support/a/bin/request.py?contact_type=suggest">let us know</a>. Feedback like yours shapes its future development.

Keeping up with Google Apps

Posted by Jeremy Milo, Marketing Manager During 2007, we introduced 40 significant improvements to Google Apps , which is our suite of commu...

Đọc thêm »

Space Jam

Posted by Lior Ron, Product Manager <a title="Code Jam" href="http://www.google.com/codejam/" id="tfny">Code Jam</a> is one of our most famous traditions. Programmers compete to hack and solve complex programming challenges in a very short time, and the winners are awarded fame, cash prizes and the opportunity to intern at Google. Diego Gavinowich from Buenos Aires was a finalist in our Latin America Code Jam, and joined us for a winter internship three months ago (missing summer break in Argentina!). Since he loves astronomy and web hacking, we gave Diego a new Code Jam challenge: code a web version of <a title="Google Sky" href="http://earth.google.com/sky/skyedu.html" id="iw.m">Google Sky</a> by the end of his time with us. Well, he did it... with the help of other engineers jamming along on their 20% time. We're very pleased to tell you that Google Sky is now available on the web at <a title="sky.google.com" href="http://sky.google.com/" id="fwfb">sky.google.com</a>. You can <a title="now search for planets" href="http://www.google.com/sky/#latitude=27.4491&longitude=-108.082107&zoom=8&Spitzer=0.00&ChandraXO=0.00&Galex=0.00&IRAS=0.00&WMAP=0.00&Cassini=0.00&slide=2&mI=1&oI=0&by=1" id="n_4f">search for planets</a>, listen to <a title="Earth & Sky podcasts" href="http://www.google.com/sky/#latitude=-2.410787363563224&longitude=8.301544189453125&zoom=10&Spitzer=0.00&ChandraXO=0.00&Galex=0.00&IRAS=0.00&WMAP=0.00&Cassini=0.00&slide=6&mI=1&oI=7&by=1" id="dj5j">Earth & Sky podcasts</a>, watch some beautiful <a title="Hubble telescope images" href="http://www.google.com/sky/#latitude=22.01276918882111&longitude=-96.36571884155273&zoom=13&Spitzer=0.00&ChandraXO=0.00&Galex=0.00&IRAS=0.00&WMAP=0.00&Cassini=0.00&slide=16&mI=1&oI=2&by=1" id="yxpw">Hubble telescope images</a>, or <a title="explore historical maps of the sky" href="http://www.google.com/sky/#latitude=-3.337953961416472&longitude=-54.140625&zoom=4&Spitzer=0.00&ChandraXO=0.00&Galex=0.00&IRAS=0.00&WMAP=0.00&Cassini=68.00&slide=1&mI=1&oI=2&by=1" id="mjun">explore historical maps of the sky</a> from the comfort of your browser. Sky in Google Earth, which <a href="http://google-latlong.blogspot.com/2007/08/sky-final-frontier.html">launched last August</a>, was originally available to our 350 million Google Earth users. This release brings the universe to every browser and makes Sky accessible to just about anyone with an Internet connection — from school children to professional astronomers — in 26 different languages. To learn more about Google Sky web edition, watch this short video, and read the full story on the <a href="http://google-latlong.blogspot.com/2008/03/organizing-more-than-one-worlds.html">Google Lat Long blog</a>. We'll miss you, Diego! <object height="355" width="425"><param name="movie" value="http://www.youtube.com/v/gX9MeF2Au9c&hl=en"><param name="wmode" value="transparent"><embed src="http://www.youtube.com/v/gX9MeF2Au9c&hl=en" type="application/x-shockwave-flash" wmode="transparent" height="355" width="425"></embed></object>

Space Jam

Posted by Lior Ron, Product Manager Code Jam is one of our most famous traditions. Programmers compete to hack and solve complex programmin...

Đọc thêm »

Using log data to help keep you safe

Posted by Niels Provos, Google Security Team We recently began two new series of posts. The first, which explains how we harness data for our users, started with <a title="this post" href="http://googleblog.blogspot.com/2008/03/why-data-matters.html" id="wkxh">this post</a>. The second, focusing on how we secure information and how users can protect themselves online, <a style="font-style: italic;" title="began here" href="http://googleblog.blogspot.com/2008/03/how-google-keeps-your-information.html" id="b3rt">began here</a>. This post is the second installment in both series.- Ed. We sometimes get questions on what Google does with server log data, which registers how users are interacting with our services. We take great care in protecting this data, and while we've talked previously about <a href="http://googleblog.blogspot.com/2008/03/why-data-matters.html" id="gjr1" title="some of the ways">some of the ways</a> it can be useful, something we haven't covered yet are the ways it can help us make Google products safer for our users. While the Internet on the whole is a safe place, and most of us will never fall victim to an attack, there are more than a few threats out there, and <a href="http://googleblog.blogspot.com/2008/03/how-google-keeps-your-information.html" id="syc9" title="we do everything we can">we do everything we can</a> to help you stay a step ahead of them. Any information we can gather on how attacks are launched and propagated helps us do so. That's where server log data comes in. We analyze logs for anomalies or other clues that might suggest malware or phishing attacks in our search results, attacks on our products and services, and other threats to our users. And because we have a reasonably significant data sample, with logs stretching back several months, we're able to perform aggregate, long-term analyses that can uncover new security threats, provide greater understanding of how previous threats impacted our users, and help us ensure that our threat detection and prevention measures are properly tuned. We can't share too much detail (we need to be careful not to provide too many clues on what we look for), but we can use historical examples to give you a better idea of how this kind of data can be useful. One good example is the <a href="http://www.citi.umich.edu/u/provos/papers/search_worms.pdf" id="ujjv" title="Santy search worm [PDF]">Santy search worm</a> (PDF), which first appeared in late 2004. Santy used combinations of search terms on Google to identify and then infect vulnerable web servers. Once a web server was infected, it became part of a <a href="http://en.wikipedia.org/wiki/Botnet" id="or.e" title="botnet">botnet</a> and started searching Google for more vulnerable servers. Spreading in this way, Santy quickly infected thousands and thousands of web servers across the Internet. As soon as Google recognized the attack, we began developing a series of tools to automatically generate "<a href="http://en.wikipedia.org/wiki/Regular_expression">regular expressions</a>" that could identify potential Santy queries and then block them from accessing Google.com or flag them for further attention. But because regular expressions like these can sometimes snag legitimate user queries too, we designed the tools so they'd test new expressions in our server log databases first, in order to determine how each one would affect actual user queries. If it turned out that a regular expression affected too many legitimate user queries, the tools would automatically adjust the expression, analyze its performance against the log data again, and then repeat the process as many times as necessary. In this instance, having access to a good sample of log data meant we were able to refine one of our automated security processes, and the result was a more effective resolution of the problem. In other instances, the data has proven useful in minimizing certain security threats, or in preventing others completely. In the end, what this means is that whenever you use Google search, or Google Apps, or any of our other services, your interactions with those products helps us learn more about security threats that could impact your online experience. And the better the data we have, the more effectively we can protect all our users.

Using log data to help keep you safe

Posted by Niels Provos, Google Security Team We recently began two new series of posts. The first, which explains how we harness data for ou...

Đọc thêm »

Book info where you need it, when you need it

Posted by Frances Haugen, Associate Product Manager and Matthew Gray, Software Engineer, Book Search Here at <a href="http://books.google.com/" target="_blank">Google Book Search</a> we love books. To share this love of books (and the tremendous amount of information we've accumulated about them), today we've released a new <a href="http://code.google.com/more/" target="_blank">API</a> that lets you link easily to any of our books. Web developers can use the <a href="http://code.google.com/apis/books" target="_blank">Books Viewability API</a> to quickly find out a book's <a href="http://books.google.com/intl/en/googlebooks/screenshots.html" target="_blank">viewability</a> on Google Book Search and, in an automated fashion, embed a link to that book in Google Book Search on their own sites. As an example of the API in use, check out the <a href="http://catalog.dpls.lib.or.us/record=b1022758" id="rx3l">Deschutes Public Library</a> in Oregon, which has added a link to "Preview this book at Google" next to the listings in their library catalog. This enables Deschutes readers to preview a book immediately via Google Book Search so that they can then make a better decision about whether they'd like to buy the book, borrow it from a library or whether this book wasn't really the book they were looking for. We think this API will be useful to all sites that connect readers with information about books, from <a href="http://nov9.scriblio.net/search/history?subj=20th+century&open_sesame=1" target="_blank">library catalogs</a> to <a href="http://catalog.dpls.lib.or.us/record=b1022758" target="_blank">public libraries</a> to <a href="http://catalog.lib.utexas.edu:2082/search/X?SEARCH=Landscapes+and+Communities+on+the+Pacific+Rim&searchscope=28" id="w4ew">universities</a>. To see more implementations in action, <a href="http://booksearch.blogspot.com/2008/03/preview-books-anywhere-with-new-google.html" id="x1:g">read this post</a> on the Book Search blog.

Book info where you need it, when you need it

Posted by Frances Haugen, Associate Product Manager and Matthew Gray, Software Engineer, Book Search Here at Google Book Search we love boo...

Đọc thêm »

Our solutions for ad serving

Posted by Rohit Dhawan, Senior Product Manager Earlier this week, we <a href="http://googleblog.blogspot.com/2008/03/weve-officially-acquired-doubleclick.html">completed</a> our acquisition of DoubleClick. Together, we're now focused on building a full suite of products and tools that help publishers of all sizes improve productivity, manage their inventory, generate additional revenue opportunities and save time so they can focus on what they do best -- creating great content and delivering an exceptional experience to their users. First, let us address the options that publishers have when it comes to selling and managing ad space on their websites. Some publishers use ad networks like <a href="https://www.google.com/adsense/login/en_US/?gsessionid=KDKfga-9mk0" title="Google AdSense" target="_blank">Google AdSense</a> to fill their ad space. Still others employ a direct sales force to manage and sell their ad inventory with solutions like the <a title="DoubleClick Revenue Center" href="http://www.doubleclick.com/solutions/publishers/index.aspx" target="_blank">DoubleClick Revenue Center</a>, and partner with third-party ad networks to fill in any unsold space. Regardless, it is a challenge for publishers to effectively manage their available inventory and ensure all of their clients' campaigns serve on time without a sophisticated ad management and ad serving solution. Today, we're announcing a new tool for publishers with the beta launch of <a href="http://www.google.com/admanager">Google Ad Manager</a>. Directed at addressing the ad management and serving needs of publishers with smaller sales teams, Google Ad Manager is a free, hosted ad and inventory management tool that can help publishers sell, schedule, deliver and measure their directly-sold and network-based ad inventory. It offers an intuitive and simple user experience with Google speed and a tagging process so publishers can spend more time working with their advertisers and less time on their ad management solution. And by providing detailed inventory forecasts and tracking at a very granular level, Ad Manager helps publishers maximize their inventory sell-through rates. Google Ad Manager effectively complements the DoubleClick Revenue Center, which is focused on publishers with larger sales teams. We're excited to add DART for Publishers to our suite of products, and we're committed to the continued development and enhancement of DoubleClick's offerings. Today's announcement demonstrates this promise, and at the same time furthers our goal of creating new opportunities for publishers of all sizes. Dozens of publishers have been using Google Ad Manager successfully in early trials. To hear what those publishers have to say check out some <a href="https://www.google.com/admanager/login/en_US/index.html#successstories" title="Ad Manager Success Stories" target="_blank">Ad Manager success stories</a> or take a product tour to learn more. As we are still in beta, Google Ad Manager is available to publishers by invitation only. If you're interested in learning more about it or would like to be considered for the program, visit the <a href="http://www.google.com/admanager">Google Ad Manager</a> site. Existing DoubleClick customers are not affected by this announcement. As we expand the Google Ad Manager beta program, we will be in touch again to include additional publishers and offer updates on our progress.

Our solutions for ad serving

Posted by Rohit Dhawan, Senior Product Manager Earlier this week, we completed our acquisition of DoubleClick. Together, we're now focu...

Đọc thêm »