Google Web Search Engine Engineering

Table of Content

But imagine that you’re unreasonable. For a moment, try to think like a Google engineer which pretty much requires being both insanely passionate about delivering the best search results and obsessive about how you do that. If you’re a Google engineer, you now that those nine words comprised about 120 bytes of data, enough to slow download time for users with modems by 20 to 50 milliseconds. You can estimate the stress that 120 bytes, times millions of searches per minute, put on Goggles 10,000 servers.

On the other hand, you can also measure precisely how many visitors took the tour, how many of those downloaded the Google Toolbar, and how many clicked through for the first time to Google News. This is what its like inside Google. It is a joint founded by geeks and run by geeks. It is a collection of 650 really smart people who are almost frighteningly single-minded. These re people who think they are creating something that’s the best in the world, says Peter Morning, a Google engineering director. And that product is changing peoples lives.

This essay could be plagiarized. Get your custom essay
“Dirty Pretty Things” Acts of Desperation: The State of Being Desperate
128 writers

ready to help you now

Get original paper

Without paying upfront

Geeks are different from the rest of us, so its no surprise that they’ve created a different sort of company. Google is, in fact, their dream house. It also happens to be among the best-run companies in the technology sector. At a moment when much of business has resigned itself to the pursuit of sameness and safety, Google proposes an almost joyous antidote to mediocrity, a model for smart innovation in challenging times. Goggles tale is a familiar one Two Stanford doctoral students, HYPERLINK http//vow. Postman’s. Mom/person/ screening Sergey Bring and Larry Page, developed a set of algorithms that in 1998 sparked a holy-sit leap in Web-search performance. Basically, they turned search into a popularity contest. In addition to gauging a phrases appearance on a Web page, as other engines did, it assessed relevance by counting the number and importance of other pages that linked to that page. Since then, newer search products such as Team and Fast have essentially matched Goggles advance. But Google remains the undisputed search heavyweight.

Google says it processes ore than 150 million searches a day and the true number is probably much higher than that. Goggles revenue model is notoriously tough to deconstruct Analysts guess that its revenue last year was anywhere from 60 million to 300 million. But they also guess that Google made quite a bit of money. As a result, there is constant, hopeful speculation among financiers around an initial public offering, a deal that could be this decades equivalent of the 1995 Netscape PIP. A few years back, such a deal might have valued Google at 3 billion or more. Even today, a Google offering might fetch 1 billion.

For now, though, most of the cars n the lot outside Goggles modest offices in a Mountain View, California office park are beat-up Volvo and Suburbs, not Porches. And while Google’s may relish their shot at impossible wealth, they appear driven more by the quest for impossible perfection. They want to build something that searches every bit of information on the Web. More important, they want to deliver exactly what the user is looking for, every time. They know that this wont ever happen, and yet they keep at it. They also pursue a seemingly gratuitous quest for speed Four years ago, the average search took approximately 3 seconds.

Now its down to about 0. Seconds. And since 0. 2 is more than zero, its not quite fast enough. Google understands that its two most important assets are the attention and trust of its users. If it takes too long to deliver results or an additional word of text on the home page is too distracting, Google risks losing peoples attention. If the search results are lousy, or if they are compromised by advertising, it risks losing peoples trust. Attention and trust are sacrosanct. Google also understands the capacity of the Web to leverage expertise. Its product-engineering effort is more like an ongoing, all-hands discussion.

The site features about 10 cosmologies in development, many of which may never be products per SE. They are there because Google wants to see how people react. It wants feedback and ideas. Having people in on the game who know a lot of stuff tells you earlier whether good ideas are good ideas that will actually work. But what is most striking about Google is its internal consistency. It is a beautifully considered machine, each piece seemingly true to all the rest. The appearance of advertising on a page, for example, follows the same rules that dictate search results or even new-product innovation.

Those rules are simple, governed by supply, demand, ND democracy which is more or less the logic of the Internet too. Like its search engine, Google is a company overbuilt to be stronger than it has to be. Its extravagance of talent allows it crucial flexibility the ability to experiment, to try many things at once. Flexibility is expensive, says Craig Silversides, a 30- year-old engineer who dropped his pursuit of a Stanford PhD to become Goggles first employee. But we think that flexibility gives you a better product. Are we right I think were right. More important, that’s the sort of company I want to work for.

And the sort of company that every company can learn from. What follows, then, is our effort to Google Google to search for the growth secrets of one of the worlds most exciting growth companies. Like the logic of the search-engine itself, our search was deep and democratic. We didn’t focus on Goggles big three CEO Eric Schmidt and founders Bring and Page. Instead, we went into the ranks and talked with the project managers and engineers who make Google tick. Hers what we learned. Rule Number One The User Is in Charge There are people searching the Web for spiritual enlightenment.

Peter Morning says this with such utter solemnity that its impossible to tell for sure whether he gets the irony. Then again, Morning is the guy who authored a hilarious Powering translation of Lincoln Gettysburg Address (available at www. Morning. Com), a geek classic. So maybe Hess having fun. But Hess also making a point. When someone enters a query on Google for spiritual enlightenment, its not clear what Hess seeking. The concept of spiritual enlightenment means something different from what the two words mean individually. Google has to navigate varying levels of laterality to guess at what the user really wants.

This is where Google’s live, amid semantic, visual, and technical esoteric. Morning is Goggles director of search quality, harmed with continuously improving peoples search results. Google tracks the outcome of a huge sample of the queries that we throw at it. What percentage of users click on the first result that Google delivers How many users click on something from the first page Nerving team members scour the data, looking for trouble spots. Then they tweak the engine. The cardinal rule at Google is, If you can do something that will improve the users experience, do it.

It is a mandate in part born of paranoia There’s always a chance that the Google destroyer is being pieced together by two more guys in a garage. By some estimates, Google accounts for three-quarters of all Web searches. But because its not perfect, being dominant isn’t good enough. And the maniacal attack on imperfection reflects a genuine belief in the primacy of the customer. That’s why Google must correctly interpret searches by Turks and Finns, whose queries resemble complete sentences, and in Japanese, where words run together without spaces.

It has to understand not only the meanings of individual words but also the relationships of those words to other words and the characteristics of those words as objects on a Web page. (A page that displays a search word in boldface r in the upper-right-hand corner, for example, will likely rank higher than a page with the same words displayed less prominently. ) Its why the difference between 0. 3 seconds and 0. 2 seconds is pretty profound. Most searches on Google actually take less than 0. 2 seconds. That extra tenth of a second is all about the outliers queries crammed with unrelated words or with words that are close in meaning.

The outliers can take half a second to resolve and Google believes that users productivity begins to wane after 0. 2 seconds. So its engineers find ways to store ever-more-arcane Web-text snippets on its servers, saving the engine the time it takes to seek out phrases when a query is made. And its why, most of the time, the Google home page contains exactly 37 words. We count bytes, says Google Fellow Ours Hole, who is on leave from the University of California at Santa Barbara. We count them because our users have modems, so it costs them to download our pages.

Just as important, every new word, button, or feature amounts to an assault on the users attention. We still have only one product, Hole says. That’s search. People come to Google to search the Web, and the main purpose of the page is to make sure that you’re not distracted from hat search. We don’t show people things that they aren’t interested in, because in the long term, that will kill your business. Google doesn’t market itself in the traditional sense. Instead, it observes, and it listens. It obsesses over search- traffic figures, and it reads its email.

In fact, 10 full-time employees do nothing but read emails from users, distributing them to the appropriate colleagues or responding to them themselves. Nearly everyone has access to user feedback, says Monika Henning, Goggles director of research. We all know what the problem areas are, where users are complaining. The upshot is that Google enjoys a unique understanding of its users and a unique loyalty. It has managed a remarkable feat appealing to tech-savvy Web addicts without alienating neophytes who type in Amazon. Com to find… Amazon-com. (Yes, people really do that. Google doesn’t know why. Google knows how to make geeks feel good about being geeks, says Core Doctor, prominent geek, flogger, and technology propagandist. Google has done that from the beginning, when Bring and Page basically laid open their stunning new technology in a 1998 conference paper. They invited in the geeks in and made them feel as if they were in on something special. But they didn’t forget to make everyone else feel special too. They still do, by focusing relentlessly on the quality of the experience. Make it easy. Make it fast. Make it work. And attack everything that gets in the way of perfection.

Rule Number Two The World Is Your RD Lab Paul Bausch is a 29-year-old Web developer in Corvallis, Oregon. He works with ASP, SQL Server, Visual Basic, XML, and a host of other geek-only technologies. He helped create Flogger, a widely used program that helps people set up their own Web log. And in a way that’s intentionally imprecise, Hess part of Goggles research effort. Isn’t this great exclaims Nelson Miner, a senior Google engineer. Miner and I are fooling with Bausch quirky creation called Google Commandos, where you can compare the volume of Google citations for any two competing queries. The New York Yankees slam the New York Mets war conquers peace. ) Google loosed Commandos and other eccentric Web novelties when it released a developers kit last spring that lets anyone integrate Goggles search engine into their own application. The download is simple, and the license is free for the taking. Hers the scary bit Basically, those developers can do whatever they want. The only intro that Google exerts is a cap of 1 ,000 queries per day per license to guard against an onslaught that might bring down its servers. In most cases, Miner and his colleagues have no idea how people use the code.

Its kind of frustrating, he concedes. We would love to see what they’re doing. Most companies would sooner let temps into the executive washroom than let customers much less customers who can hack anywhere near their core intellectual property. Google, though, grasps the power of an engaged community. The developers kit is a classic Trojan-horse strategy, putting Goggles engine in places that the many might not have imagined. More important, Bausch says, opening up the technology kimono turns the world into Goggles development team. Sites like Commandos, while basically toys, are an inkling of what Google could be used for, Miner says.

We can’t predict what will happen. But we can predict that there will be an effect on our technology and on the way the world views us. And more likely than not, it will be something pretty cool. Rule Number Three Failures Are Good. Good Failures Are Better. In Google Labs, just two clicks away from its home page, anyone can test-drive Google Viewer, sort of a motion-picture erosion of your search results, or Voice Search, a tool that lets you phone in a query and then see your results online. Is either ready for prime time Not really. (Try them out. On Voice Search, you’re as likely to get someone else results as your own. But that’s the point. The Labs reflect a shared ethos between Google and its users that allows for public experimentation and for failure. People understand that not everything Google puts on view will work perfectly. They also understand that they are part of the process They are free to tell Google what’s great, what’s not, and what might work better. Unlike most other impasses, observes Matthew Beer, a senior analyst at Jupiter Research, Google has said, Were going to try things, and some aren’t going to work. That’s okay. If it doesn’t work, well move on. In the search business, failure is inevitable. It comes with the territory.

A Web search, even Goggles, doesn’t always give you exactly what you want. It is imperfect, and that imperfection both allows and requires failure. Failure is good. But good failures are even better. Good failures have two defining characteristics. First, says Ours Hole, you know why you failed, and you have something you can apply to the next project. When Google experimented tit thumbnail pictures of actual Web pages next to results, it saw the effect that graphical images had on download times. That’s one reason why there are so few images anywhere on Google, even in ads. But good failures also are fast.

Fail, Hole says. But fail early. Fail before you invest more than you have to or before you needlessly compromise your brand with a shoddy product. Rule Number Four Great People Can Manage Themselves Google spends more time on hiring than on anything else. It knows this because, like any bunch of obsessive engineers, it keeps track. It says that it gets 1,500 Russ a day from wane-be Google’s. Between screening, interviewing, and assessing, it invested 87 Google people-hours in each of the 300 or so people that it hired in 2002. Google hires two sorts of engineers, both aimed at encouraging the art of fast failure.

First, it looks for young risk takers. We look for smart, says Wayne Rising, who heads Goggles engineering ranks. Smart as in, do they do something weird outside of work, something off the beaten path That translates into people who have no fear of trying difficult projects and going outside the bounds of what they know. But Google also hires stars, Pads from top computer-science programs and search labs. It has continually managed to hire 90 of the best search-engine people in the world, says Brian Davison, a Lehigh University assistant professor and a top search expert himself. The Pads are Goggles id.

They are the people who know enough to shoot holes in ideas before they go too far to make the failures happen faster. The challenge is negotiating the tension between risk and caution. When Rising started at Google in 2001, we had management in engineering. And the structure was tending to tell people, No, you can’t do that. So Google got rid of the managers. Now most engineers work in teams of three, tit project leadership rotating among team members. If something isn’t right, even if its in a product that has already gone public, teams fix it without asking anyone. For a while, Rising says, I had 160 direct reports.

No managers. It worked because the teams knew what they had to do. That set a cultural bit in peoples heads You are the boss. Don’t wait to take the hill. Don’t wait to be managed. And if you fail, fine. On to the next idea. There’s faith here in the ability of smart, well- motivated people to do the right thing, Rising says. Anything that gets in the way of that is evil. Rule Number Five If Users Come, So Will the Money Google as no strategic-planning department. CEO Eric Schmidt hasn’t decreed which technologies his engineers should dabble in or which products they must deliver.

Innovation at Google is as democratic as the search technology itself. The more popular an idea, the more traction it wins, and the better its chances. Hers how one Google service came into the world. In December 2001, researcher Krishna Brat posted an internal email inviting Google’s to check out his first crack at a dynamic news service. Although Google offered a basic headline service at the time, news was not a corporate mandate. This was simply Brats idea. As a respected PhD hired away from Compact and a member of the company’s 10- person research lab, coming up with new ideas is basically Brats job.

For an early prototype, it was quite a piece of work. Brat had built an engine that crawled 20 news sources once an hour, automatically delivering the most recent stories on in-demand topics something like a virtual wire editor. And within Google, it got a lot of attention. Importantly, it attracted the attention of Marimbas Mayer, a young engineer turned project manager. Mayer connected Brat with an engineering team. And within a month and a half, Google had posted on TTS public site a beefed-up version of the text-based demo, which is now called Google News and which features 1 55 sources and a search function.

Within three weeks of going public, the service was getting 70,000 users a day. One reason Google puts its innovations on public display is to identify failures quickly. Another reason is to find winners. For Brat and Mayer, those 70,000 users provided ammunition to build a case for News within Google. A public trial helps you go fast, Mayer says. If it works, it builds internal passion and fervor. It gets people thinking about the problem. Soon, Mayer had marshaled a handful of engineers to bulk up News. They expanded its reach to more than 4,000 sources, updated continuously instead of hourly.

They created an engine that was robust enough to support five times the anticipated early volume. And they prettied it up, designing an interface that displayed hundreds of headlines and photos but that was still easy to navigate. By September, the new News was up. Is Google News an actual product Not exactly. Its home page is still labeled Beta, as are all but a few of Goggles offerings. It may become a Google fixture, it may disappear, or it may recede into Google Labs. Mayer is still studying the traffic, and the engineers are still tweaking, reacting to users emails.

The company’s organic approach to invention bugs some onlookers. Google is a great innovator, says Danny Sullivan, editor of Search Engine Watch and an influential commentator. They keep rolling out great things. But Google News was an engineer deciding he wanted a news engine. Now Google has this product, and it doesn’t know how to make money off of it. Sullivan is onto something important At some point, all of this great stuff has to turn a profit. That was the one great moral of the dotcom blowout Monitoring eyeballs turned out to mean throwing money down a sinkhole.

When Mayer argues that the traffic will let us know whether News is a success, she echoing a long line of now-unemployed executives who thought that they had tamed the business cycle. But at Google, building and then following the traffic makes perfect sense. Its central to the company’s culture and its operating logic. Consider this For the first 18 months of its existence, Google didn’t make a penny from its basic Web-search service.

Cite this page

Google Web Search Engine Engineering. (2018, May 18). Retrieved from

https://graduateway.com/google-web-search-engine-engineering/

Remember! This essay was written by a student

You can get a custom paper by one of our expert writers

Order custom paper Without paying upfront