SUNDAY, MAY 28, 2006

... if I were to rate people on general smartness I would have put myself in the bottom 25%.  Hmmm,...how are you defining & rating Intelligence ???????  What is of grave concern is - if everyone there is so-o esoteric, and SEARCH has now become a "common persons" resource, Can they really empathize and understand the needs and habits of the so-called less extreme intelligent members of the Human Race who dominate their customers (and the world)?  The interview where you were asked for your SAT and GPA scores makes one wonder if they are only focussing a very LIMITED portion of Human Intelligence.  Homo Sapiens are so varied and verstile, hopefully the channels of communication will open up to include and understand the masses in what is becoming a universal source of information retrieval!  Hopefully the masses will ALSO be allowed to contribute to Google's evolution ;-)

3:55 PM

Ron said...

Hmmm,...how are you defining & rating Intelligence ?  A good question, and I'm actually planning a whole post to answer it.

4:30 PM

Ron said...

Oh, I guess I should mention too that they didn't ask me my GPA or my SAT score. (I suspect they reserved that for the non-technical folks.)

4:34 PM

Milly said...

Ron, the Swedish Chef lives! It's under "Bork, bork, bork!" in that prefs page dropdown. It even had some utility for a year or so: selecting that language brought back the old blue and white tabs (and less tricky Groups) design. But they've fixed that now (hey, some security holes take 'em longer) :  www.imilly.com/google-cookie.htm#bork  Once there's enough of you xooglers gathered, maybe you could do an exit reasons survey. I wonder if many would cite the growth from jetski to supertanker as a reason?  In July Marissa Mayer said "We are planning on releasing some alternatives in the next one to two months that make our current captchas more compatible with screen readers ...".  Yet here we are with most Google services, Blogger comments included, still using evil visually-oriented captchas: www.w3.org/TR/turingtest.  Apparently it takes time for a supertanker to change course ...

5:58 PM

Milly said...

Oh, and Doug - how about an RSS feed for the comments?

6:04 PM

Anonymous said...

I'm looking forward to seeing the comments on intelligence as I am very interested in how it manifests itself in terms of how Google makes decisions as well as how well the technology works.  Here is an example. A long time ago, some Google employees claimed that it was not possible for their index to be spammed because their algorithms were powerful enough to defeat any spam attempts. I think they did not take into account the economics of the Internet -- in particular, that web sites (and therefore links) could be built rather cheaply. What I want to understand is what aspect of intelligence does reaching this type of conclusion fall into and to what extent it factors in the hiring process.  Disclaimers: I have been turned down for employment by Google. No sour grapes -- I realize it is like getting into Harvard; some very intelligent people don't get in. Also, I used to work for another search engine that ran into trouble and got sold.

7:31 PM

Steve said...

 What is of grave concern is - if everyone there is so-o esoteric, and SEARCH has now become a "common persons" resource, Can they really empathize and understand the needs and habits of the so-called less extreme intelligent members of the Human Race who dominate their customers (and the world)?   Well, maybe not..  http://www.infoworld.com/article/05/12/06/HNgoogleuserstudy_1.html

4:00 PM

cayblood said...

On the other hand, there are so many obvious problems out there that don't necessarily require UI skills that they can just work on those and hire a few good UI people to write the interface to them.

8:45 AM

Anonymous said...

I also worked at Google and I understand the feeling of being in the bottom 25%. Everyone there was smart, but I didn't feel that the top 25% were that smart, and I didn't feel that the management was that smart either.  Basically, it felt like a graduate research group. Everyone was good or they wouldn't be there. There was a definite pecking order based on sucking up to the prof.

12:16 PM

SUNDAY, NOVEMBER 27, 2005

In the beginning...

POSTED BY RON AT 3:34 PM

The memory of my first encounter with Google is seared into my memory like JFK's assassination is supposed to be seared into the minds of many baby boomers.

As an aside, It's always seemed odd to me that the baby boom generation is supposed to include anyone born through 1964. Technically I'm a boomer, but JFK was long dead when I entered first grade. On the other hand, I do clearly remember what the world was like before there was Google, before there was the World Wide Web, and even before there were personal computers. I am a member of the last generation to know what that world was like. God, do I feel old now.

But I digress.

I was reading a usenet newsgroup (I'll tell you which one later -- it bears on a peculiar personality trait of mine that has some bearing on my Google experience later on) when someone answered a particularly obscure question and followed up with "Thank God for Google" or some such comment. (When Google acquired Deja and brought their database of old usenet postings on line I went looking for the post that started the whole thing and couldn't find it. This was just the first of many Twilight-Zonish (or X-Filesish for you younger readers) events that happened to me of the course of the next few years.)

Google? What's Google, I wondered. So I pulled up a web browser and took a wild guess (which was as good a way as any of finding things on the web in those days): www.google.com. Oh, it's a search engine, kind of like Alta Vista. But, holy shit, it's fast! And it has this uncanny way of putting just what you're looking for right at the top of the results list.

To understand what happened next I have to give you a little background about my professional life up to that point. I had moved to California from Virginia in 1988 along with my Ph.D. thesis advisor, David Miller, who had just been offered a job at JPL to do research for the Mars Rover program. Over the course of the next twelve years my career had a lot of ups and downs, and I had a lot of opportunities to leave JPL, but somehow the opportunities always seemed to come along at the "up" times when life seemed good and I wasn't much inclined to rip up my roots, which were growing deeper as the years went by.

But the day I found Google just happened to be during a "down" time.

My first thought was, "How the hell do they do that?" Alta Vista was astonishing enough in its day, but this took speed and accuracy to a whole new level. I had always been idly curious about how Alta Vista worked, and now I just had to know. In a fit of what Alan Greenspan would have called "irrational exuberance", I dashed off a resume.

Google got back to me with astonishing speed. It was early 2000, the dotcom bubble was just reaching its peak, it was a seller's market when it came to any kind of technical talent, and I looked pretty good on paper. If memory serves, it was only about two or three hours before my phone rang. A week later I was flying up for an interview.

I don't have nearly as many colorful interview stories to tell as Doug did. I met with half a dozen people. (Neither Larry nor Sergey interviewed me.) They grilled me on the usual things -- caches, hash tables, virtual functions, etc. It was a pretty standard technical interview as best I can recall, with a few Googley twists (how would you write a program that could identify news sites on the web?) I guess I must have hornswoggled them pretty good because they made me an offer. (That may sound like a bit of self-deprecating humor, but it isn't. The truth is I really wasn't qualified for the job. But that didn't become apparent until later.)

So now I'm in a pickle. On the one hand I've got this job offer and an opportunity to learn how this cool technology works, and maybe even make a few bucks on the stock options (though that was never the main motivation for going. It was pretty clear even in early 2000 that the internet bubble was gong to burst sooner or later, and besides, how was a search engine ever going to make money?) More to the point, I was worried that if I didn't get away from JPL now I never would, and I didn't really want to retire without ever having experienced anything but working for one organization.

On the other hand, I had a pretty cushy situation. My seniority at JPL made it possible for me to work on pretty much anything I wanted to. I was well paid (by my standards at the time). My job was (or seemed) secure. My wife and I had just bought a nice new house and gotten a dog and a cat. (The Southern California real estate market was just starting to pull out of a slump and we were able to buy for what now seems dirt cheap.) To take this job we would either have to move or I would be doing the commute from hell. Neither of those prospects seemed very appealing.

I made my decision while visiting some friends in upstate New York. One of my friends had just had her father die unexpectedly. He was an orthopedic surgeon. Very wealthy, or so it seemed to me at the time. Full of life. Commuted from his farm in Virginia to his job in his own Bell Jet Ranger. (I remember he flew in one night while I was visiting my friend on the farm. He was a real regular down-to-earth guy. Asked me if I'd like to join him tomorrow -- he was going to the Pittsburg Steeler's summer training camp. I'm not that much of a football fan, but it was pretty damn cool anyway. There were fans everywhere wondering who the hell we were that we got to go into all the VIP areas.)

One day he just keeled over while skiing. Heart attack. He was dead before they got him off the mountain.

It was during that reunion that I had an epiphany: life is short, and I had everything I ever wanted: a nice house, a secure job that I (mostly) enjoyed, why in the world would I want to put myself through hell just to hang out at some dotcom that would probably be bankrupt in a year or two?

When we got back from New York I called Google and told them that I had decided to decline their offer.

13 COMMENTS:

Loughlan said...

Must have been a tricky decision. I wonder if knowing what you had known now, would you have still done the same thing?  I realise it is probably a faux pas to post here. But I wanted to mention what a wonderful piece of writing this is.  Good luck with whatever you choose to do in life.  Kindest Regards Loughlan Burnett

Anonymous said...

Fascinating!

6:40 PM

Leah said...

Hi. Please don't remove the anonymous option for posting comments -- it allows for more interesting people to feel comfortable posting. I see that you've already reserved the right to remove comments; I think that's just, but please do so sparingly. Just my HO.  Great blog, btw. The exaggerated lovefest that people have had with Google gets tiresome.

6:56 PM

Lucian said...

I'm kinda facing the exact same situation, though probably not at the same scale or level as you.  The thrill of working on something everyone uses excites me to no end. I've a pretty cushy job at the moment; pretty inconsequential, but cushy.  I wonder about family life should I decide to take on the new, more exciting job. Whether I should just coast, and whether I'll regret that I never took the plunge. That is the question.

7:04 PM

Anonymous said...

Does anyone at Google ever stop to think that 99.99% of the company's revenue is the result of a cold hard reality-  The majority of websurfers out there don't have the experience to understand that those adsense ads or google search result ads aren't organic links?  There was a poll done by the BBC about 6 months ago that showed 68% of surfers were unable to point out what links on a page were ads and which were part of the content.  Banner ads had a huge clickthrough rate when they were introduced, then plummeted to almost nothing after a few years when even the newest of the newbies realized they were just advertisements. Text ads right now seem to be enjoying the same "novelty" in that plenty of people arent aware they are ads either.   Since almost every website using adsense matches the colors and fonts and background of the ad panel to look like the rest of the content, and google's engine puts ads that seem related to the rest of the text on the page, is it any surprise that the clickthrough rate is so high? 10 percent of Google's revenue comes from AOL users, who have a clickthrough rate that's more than 5x higher any other demographic on the net. That should tell you something.  Anyway, I guess my question is, how long until even the AOL crowd catches on and clickthroughs plummet?  Btw- why does it say ads by "Goooooooogle" ? Are you trying to be cute or is there some reason dealing with spiders that you want to be able to identify casual mention of Google from adsense ad mentions?

8:16 PM

Doug said...

I'll let Ron address the issue of ad clickthroughs, though it seems pretty evident at this point that the difference between banners and Google ads is relevance to the user. While there's undoubtedly confusion in some instances about what's an ad and what is not, the utility of the ads is unmistakable.  I can also state without hesitation that I worked very closely with Google's partner team to ensure that search partners followed our guidelines for clearly identifying what was a sponsored link and what was a search result. I'll likely post about those wars in the future.  As to "Ads by Goooooogle," we needed a way to make Google ads unique and thus protectable under the law. This was not a universally beloved way of doing that, but it seemed to be the best of the options available.

8:58 PM

Ron said...

I'll let Ron address the issue of ad clickthroughs  I'm not quite sure what I'm supposed to address here. At least two issues were raised: 1) the prediction that text ad clickthroughs are going to plummet the way banner ad clickthrough did, and 2) the possibility that people don't know that the ads are ads.  I don't have much to say about issue #2. The ads are pretty clearly labelled, but I will not dispute that there are some pretty dim bulbs out there.  As to issue #1 I'll defer to Carl Sagan who observed that prophecy is a lost art.

10:12 PM

Jorg Brown said...

Turned 'em down? Heh. For exactly the same reasons, I stayed as far away from dotcoms as I could. Especially in 2000, when some of the stupider ventures had already disappeared.  I eventually interviewed at Google in 2004, but only after two friends of mine went, and I could grill them on whether or not it was really a place they liked to work. Since Google had just gone IPO, and the stock had shot from 85 to 120, I figured the easy money was already gone from stock options. But my career wasn't going anywhere at work, and I didn't have to move, so why not?  I still want to know what Google told you when you said no. For me in 1998, I told Microsoft no, and they responded with a better offer. Given the job market in 2000, I'm surprised that Google didn't do the same to you.

11:06 PM

Anonymous said...

IMO, banner ad clickthroughs fell because they were (are) intrusive to the search engine user's experience. Intrusive ads seem to work on sites like ESPN (at least, I have not heard that they feel the need to abandon them). It would be interesting to learn what % of their regular users also use Google.  I think at some point the rate of "legitimate" clickthroughs will fall off (if it hasn't already) as the medium matures and users become more selective in their surfing and buying behavior.

11:51 PM

Red said...

I think the defining difference between Google ads and the more 'traditional' forms is utility, as Doug suggests. Google ads can be so darned interesting that you want to click on some, especially if you're searching hurriedly for product or services. I find myself scanning the page, then automatically scanning the Google ads alongside if I haven't found what I'm looking for. It's an interesting twist on the ad form.

2:37 AM

Stefan said...

"Some pretty dim bulbs"?  http://www.pewinternet.org/PPF/r/146/report_display.asp

1:27 PM

Anonymous said...

WHAT IS THE CONTINUATION OF THE STORY?????  DID U WORK FOR GOOGLE OR NOT?

11:54 PM

nick said...

great piece of writing! sorryabout your friend... i'm wondering uif you could include some pieces of writing for your spanish-speaking fans... specially the ones from mexico, like me... Kudos and good luck in life.

6:13 PM

SUNDAY, NOVEMBER 27, 2005

The wormgear turns

POSTED BY RON AT 5:07 PM

Ron sez:

It ocurred to me that by putting up my story piecemeal like this most people are gong to be reading it in reverse order. Hm. I wonder if there's a way to reorder postings -- or if people are so used to this that they tend to read blogs backwards anyway. Wow, the things you start thinking of.

Anyway...this is part three of my ongoing saga of my year at Google.

Of all the random shit that has affected my life surely none has had more impact than the fact that Google would not take no for an answer. They asked if they could fly me up again to try to change my mind. And they wanted me to bring my wife along as well. Well, heck, a free dinner for two, how could I turn that down?

The person assigned to win me over was Urs Hoelzle, at the time VP of Engineering. Three things impressed me about him. First, he told a really good story about how Google was planning to make money (one that turned out to be pretty much right on the money, so to speak). Second, his technical background was in programming languges, which was a big plus for me for reasons that I will explain shortly. And third, he had a really huge dog that he brought to work with him. (Yoshka, the Leonberger, has since become the stuff of legend.)

(Another interesting bit of trivia about Urs: he always wore red socks. Something to do with him being Swiss, I believe, but I never fully grokked it.)

To fully appreciate the horrible irony of what happened next, I have to tell you about this peculiar affectation (or perhaps the word I'm lookin for is affliction?) of mine: I am a fan of Lisp (the programming language, not the speech impediment).

For those of you reading this who are not programmers I have to digress even further and explain a little about programming langauges. Programmers are craftsmen, and craftsmen can sometimes get a little weird about their tools. It's funny to watch Tim Allen wax rhapsodic about the relative merits of the Binford model 2330 frabnobulator because, like all good comedy, it contains more than a grain of truth. Programmers are craftsmen, and programming languages are their primary tools, and so it is not uncommon (understatement of the day) to see spirited discussion of the relative merits of this or that language.

One of the reasons that I had chosen to stay safely cloistered in the ivory tower of JPL for over a decade was that my favorite tool/language had definitively lost the language wars. I got into Lisp in the 80's because it was the language used for artificial intelligence (AI) work, which was the field I wanted to get into. Unfortunately, AI failed to deliver on lofty promises, and so it fell out of favor with its (mostly government) sponsors. As a result, industry abandoned Lisp in favor of C, and later, C++, Java and Perl.

I don't want to rehash the language wars here. Suffice it to say that I attributed a significant portion of my own professional success to my use of Lisp over the years, and I was loath to give it up. But the industry had moved definitively away from Lisp and towards C++ and, in the Web world, Java. I found both of these languages very difficult and frustrating to master, and that was a large factor in my decision to stay at JPL during most of the dotcom boom. Why should I submit myself to all that pain when I had a perfectly good job where I could do as I pleased?

But in Urs Hoeltzle I saw a beacon of hope. I had done my homework on Urs and found that he had made a major portion of his career in the development of a language called Smalltalk, which was very similar to Lisp in many important ways. In particular, it was not a mainstream language, so I thought that in Urs I would find a kindred spirit, an engineer who knew what it was like to work on and perhaps even fall in love with a programming language that was not part of the mainstream.

My wife and I have different recollections about what happened next. I have very strong memories of my telling her about starting to change my mind about Google, and her responding that she wasn't too enthusiastic about the prospect, but that if it was what I wanted to do that she would support me. She insists that she never said any such thing.

It seemed like a good time to leave JPL. Things were changing there and I was beginning to get somewhat marginalized on the project I was working on. (That's a whole 'nuther long story. If you're interested you can read it here. Be aware that it contains some spoilers for the Google story, so if you're hanging on to the edge of your seat waiting for the next installment here at xooglers (yeah, right) you might want to give it a miss.) So I called Google and we set a start date in June of 2000, and gave notice at JPL.

To my astonishment, people started coming out of the woodwork to express their shock and dismay that I was leaving JPL. Several people who I was sure would be happy to see me leave actually asked me to reconsider. It did my ego a lot of good to hear it, but it was too late. I was off to Mountain View.

13 COMMENTS:

Anonymous said...

But Erann, in the end you didn't get to use Lisp (or Scheme, I presume). That's too bad! Is it still the case that a lisper has no chance at Google? They have Norvig there. Inquiring minds want to know.

11:54 PM

Anonymous said...

Ron,   Sorry -- I got Erann from your JPL/Lisp/Google piece, and didn't see yet you'd changed your name.   I'm really curious about languages at Google. E.g. is ML or CAML used? As I'm sure you know, there is neat and powerful software that runs in those languages. And Haskell too, I guess.   Thanks for any info -- and aren't you afraid of getting sued by Google? I heard they are paranoid -- but not psycho like Intel.

12:08 AM

TraderEyal said...

Hey Doug, Cool blog. I really enjoy reading your story. As for reading the blog backwards, you can consider putting a 'sticky' on top by way of a future dated post recommending readers to start from the oldest post..  All the best TraderEyal

12:09 AM

Ron said...

Is it still the case that a lisper has no chance at Google? They have Norvig there.  Peter Norvig came to Google from NASA's Ames Research Center to be Director of Search Quality. Peter was well known in the Lisp community, having written am excellent textbook on the topic but he, to my dismay, was already abandoning Lisp in favor of Python and Java long before he came to Google.  As far as whether a Lisp programmer has "no chance" at Google (by which I assume you mean no chance of being allowed to use Lisp), I have no idea. It's been nearly five years since I left; perhaps things have changed. But I wouldn't count on it. BTW, it is not at all clear to me in retrospect that using Java was not the right thing to do from the company's point of view. More on that in a later post.  aren't you afraid of getting sued by Google?  No. Fear that the First Amendment no longer applies is a self-fulfilling prophecy.

12:42 AM

Philipp said...

Yep, I'm currently reading the last few days' posts backwards... but once you're up to speed you can read the most recent post everyday and that means it'll be in chronologically correct order. That being said, I believe there's an option over at blogger to order things chronologically, but I'd rather turn this into a book later on (via Lulu.com or BlogBinders.com or even better, a publisher).

8:00 AM

Phil said...

Nat Friedman's blog has the first page with newest-first, and all previous months in chronological order. It's a bit disorienting at first, but it makes a whole lot more sense. I haven't gotten around to doing that on my own blog, but I really should.

9:45 PM

David said...

I actually prefer having the blog with most recent posts first. Otherwise when I came, I'd have to page through old entries.   And yes, I do start at the bottom and work my way up. ;)

6:11 PM

Charles said...

I hope you don't mind but I created a new RSS feed which will spill out the posts in the opposite order (one per day), ie. the oldest posts first then moving forward. Details here:  http://www.surfarama.com/?p=266  It sort of serializes the story. I'll be adding new posts they come.

6:40 PM

Doug said...

Hey, thanks! Neither Ron nor I know anything about RSS, and we had it way down the list of things to figure out. And to prove I'm clueless, let me ask you this: can I just add a link under the Atom feed link on Xooglers to the page you've set up? Is that what people would expect if it the link were labeled "RSS feed"?

7:01 PM

Charles said...

Just for everyone elses interest, I have sent code to Doug to display the serialized RSS feed link (as opposed to using the link above. Reason being it requires some javascript to generate unique urls so it can deliver feed content in a serialized fashion.  Or you can visit my blog to subscribe here.

10:02 PM

Anonymous said...

Reading the blog backwards on a computer screen ain't that big of a deal. However, I don't read it on a screen, I actually print it out every other day (in order to read it at home or/and on the bus), and this is where things start getting really complicated. So yes, I would love to be able to re-sort the entries prior to printing.  Ah, and by the way: this is a darn good blog, let me tellya that.

4:02 AM

Isaack said...

I was recommended your blog from another blogger. Once i got here, I saw that I had to take some time getting it from beginning, so I had the same browser with your page on opened for two days before I started it... now, an hour ago. I've gotten to to this point and really enjoying myself!  With the backwards posting... I just started reading cronologically, scrolled all the way down and started from there.

4:07 AM

Anonymous said...

Ron, another admirer of Heinlein. Hopefully science fiction is not dismissed as BS at Google

10:10 AM

SUNDAY, NOVEMBER 27, 2005

Day One

POSTED BY RON AT 11:34 PM

Ron sez:

My first day at Google went like this:

5:30 AM, wake up.

6:00 AM drive to Burbank airport.

6:30 AM stand in line at the Southwest airlines gate to get one of those now defunct plastic boarding passes

7:30 AM board the plane

9:00 AM or so land in San Jose

9:30 AM get the rental car

10:00 roll up to the Googleplex

It was a murderous commute. Four hours, which I eventually whittled down to three after a year of optimizing. I went through various permutations of cabs, rental cars, having my wife drive me to and from the airport (that didn't go over well at all) before I finally settled into something that vaguely resembled a routine. I ultimately ended up driving two different cars back and forth between LA and San Jose, and on a couple of occasions I did my commute in a Cessna 182RG (I'm a pilot) which was just too cool for words.

But getting to San Jose wasn't nearly as bad as trying to get back home. Originally I took Monday to be my telecommute day so that I could be at Google on Friday afternoons for the weekly TGIF meeting. TGIF is, as far as I know, still a Google tradition where everyone gets together to hear the latest developments from upper management (which back then meant hearing it directly from Larry and Sergey. Maybe it still does, I don't know.)

Trying to get on a Southwest flight at the San Jose airport on a Friday evening in mid-2000 was a serious nightmare. Heck, just trying to get to the airport was a nightmare. (I remember on one occasion the traffic was so gridlocked that I could not get to the rental car return, which back in those days was still in the main parking structure. I just left the car in a remote parking lot and walked from there to the terminal to make my flight. I guess the rental car company eventually found the car because only the normal rental charges ever showed up on my credit card.)

I found a place to stay, renting a room from Susan Wojcicki (now Google's director of product management) and her husband Dennis. Really cool people, and coincidentally, their house is also the one where Google first set up shop, so I really felt like I was in the bosom of history.

In the back of my mind there was one niggling little worry: I had neglected to ask Urs what I would actually be doing. Having decided to embark on this adventure I guess I decided it didn't matter. I was going to do something new and different, and, more to the point, I was going to learn how this incredible search technology worked. And I was going to knock Urs's red socks off with all the cool ideas I had for new features.

Things started off really well. The work environment at Google was everything that it has since been reputed to be. I learned to ride a unicycle while I was there, and my pool game improved considerably. My office had a spectacular view. Chef Charlie's cooking was delicious. (I still make some the recipes he gave me, but I can never get them to come out nearly as good as he did. I hear he's planning to open up his own restaurant soon. I'm looking forward to patronizing it.) The conference rooms were all color coded, except for the main one which was called the Lucky Lounge. Meetings, in stark contrast to my experience at JPL, started on time. It was all very refreshing and energizing. I was brimming with enthusiasm.

So I was a little disappointed when I found out on day 1 that I had been assigned to the ads group. But that disappointment turned to dismay when I learned what my assignment was to be: I was the lead engineer on a new advertising system code named "adstoo", what eventually became AdWords. That part wasn't so bad. The bad part was, this was going to be the inaugural Java project at Google. Google, which had until now been a Java-free zone (which was one of the reasons I took the job) was going for Java in a big way, and I, the consummate Java hater, was supposed to be its chief evangelist.

Just peachy.

On top of that there was trouble at home. My wife was having a really tough time trying to keep everything together back on the home front, what with the new house and four-legged kids and all. Our finances were starting to look a little rocky because I had taken a pay cut and taken on extra expenses for weekly plane tickets, rent, and a second car. (Google did give me a travel allowance, one of the factors in my decision to take the job, but my salary and travel allowance together were still less than my old salary had been at JPL.)

And then there were the stock options.

Yes, stock options can make you rich, but getting to that point can be pretty damned annoying. When you sign on to a company that has stock options as part of the compensation plan you get an option grant up front, but you don't actually vest any of those options until you've been at the company for a year. After that you vest a few more shares every month, but if you quit -- or get fired -- after eleven months and twenty nine days you're screwed.

Notwithstanding all that, you can actually choose to excercise your options up front even though they aren't vested, and it's often wise to do so because if you wait and the stock price goes up then when you do exercise the options then the difference between what you paid and what they were worth at the time counts as income for purposes of the alternative minimum tax despite the fact that you might be restricted from selling your shares because, for example, your company hadn't gone public yet. A lot of people got badly burned during the dotcom crash because they exercised their options when they couldn't sell the stock, generating a huge amount of AMT income on which they owed taxes. Then they watched helplessly as the stock price crashed, often to levels below what they had paid for the stock. So not only did they lose some or all of the actual cash they had paid to exercise the options, they also owed huge tax bills on paper "profits" that not only had they never actually earned, they could not have earned even if they wanted to because their stock was restricted. It was horribly unfair.

On the other hand, if you vest in your options but don't exercise them and then leave the company then the options expire and become worthless. Since I wasn't really planning to stay at Google for the long haul I figured it would be wise to exercise early to avoid the AMT trap. But to do that I had to come up with the cash to actually pay for the stock. And to add insult to injury, the day I joined Google they announced that they had finalized their deal with Yahoo, and the board raised the stock price by a factor of nearly 5! So not only did I have to come up with the cash, I had to come up with five times as much cash as I was expecting to have to come up with! My wife was not pleased.

In retrospect of course it all turned out OK, but at the time it was nerve-wracking. The dotcom crash was gathering steam, and Google's long-term prospects were far from clear. Still, growth was steady and there was electricity in the air, so I was fairly optimistic, even at the time. But writing that check was painful.

All in all, it was a stressful week.

7 COMMENTS:

Uri L. said...

A great post! thanks for sharing.  What a manic commuting routine...

6:40 AM

Kiltak said...

And I tought that 3 hours of car each day was bad enough. Personally, travel time is killing me, so taking the plane each week to go to work would be out of the question. Especially with a demanding wife and a baby.  You were very courageous my friend ;)

12:31 PM

Tuor said...

tuor sez:  the doug / ron thing is not working very well, and its sort of confusing to read things the way they are (though things are, of course, extremely intersting, otherwise I wouldn't be here) As one of the chief complainers on blogger, I note that MovableType has had a category function for the longest time now, so that you can only view things by category (e.g. click on "Ron" link and see only Ron's posts). Maybe you guys should go off Google (or companies that have been bought by Google, though I suppose its pretty much the same thing) and try some open-source fun!  Of course, despite my complaining I'm still here using blogger, so I'm not sure what right I have to make suggestions. Take it all with a grain of salt, if you will.

1:22 PM

Anonymous said...

Thanks for sharing all those "ancient" moments of Google with us.  "TGIF is, as far as I know, still a Google tradition where everyone gets together to hear the latest developments from upper management (which back then meant hearing it directly from Larry and Sergey. Maybe it still does, I don't know.)"  Yes, it still does. :-)

5:09 PM

Anonymous said...

Surprised you didn't take Caltrain or the light rail to get to/from SJC.

11:29 PM

Taco said...

iTs a great post! you should change the way the comments are viewed. allowing comments on a pop up window make me read both, all other post and comments because the main page takes a hwile to load because the text is too long!

3:35 PM

Chris Marino said...

Ron, not surprised that you were at Google only a year. As important as AdWords is, I don't think it can provide the ongoing technical challenge that is necessary to keep the best people engaged.

MONDAY, NOVEMBER 28, 2005

Into the breach

POSTED BY RON AT 12:36 PM

Ron sez:

I dove into the adstoo project with as much enthusiasm as I could muster, which I'm ashamed to say wasn't much. The situation was exacerbated by the fact that we had no Java development infrastructure. We were writing bareback, so to speak. We had no debugger. We were using JSP, but had no editor support for JSP syntax. (That turned into a real debugging nightmare. It could take many tens of minutes to find a single typographical error because the only indication that there was a problem was that the output just looked all wrong, but the actual problem could be very far away from the apparent source of the problem.)

Fortunately for me, I was assigned a junior engineer to work with/for me, and he actually knew what he was doing. While I struggled to learn the Java libraries and debugging techniques (I knew the basic language, but I had never done any serious development in it before) this guy just took the bull by the horns and pretty much just wrote the whole damn thing in a matter of weeks. I sometimes pull this old joke out of the dustbin, that in the ancient tradition of senior-junior relationships, he did all the work and I took all the credit.

That's not quite true. I did end up writing the credit card billing and accounting system, which is a nontrivial thing to get right. Fortunately for me, just before coming to Google I had taken some time to study computer security and cryptography, so I was actually well prepared for that particular task. Back in those days internal security was more or less nonexistent. All the engineers had root access to all of the servers. But I believe in planning ahead, and I anticipated the day when Google was not going to be a small company any more, and so I designed the billing system to be secure against even a dishonest employee with root access (which is not such an easy thing to do). I have no idea if they are still using my system, but if they are then I'd feel pretty confident that my credit card number was not going to get stolen.

But on the whole I was struggling, not just technically, but personally as well. The situation was exacerbated by my manager, who thought that the answer to my falling behind schedule was to start micromanaging me. That just made things worse. Much worse. I had been a senior scientist at a national research lab, essentially the equivalent of a tenured professor (but without the teaching responsibilities). I was used to being my own boss more or less, and I really resented being asked to make detailed lists of everything was going to do every single day. After all, Google had come asking me to join them, and so I felt I ought to be accorded more respect. But on the other hand I had to admit that I was not really performing, and so all the micromanagement was in some sense justified. I responded, I'm ashamed to admit, with some passive-aggressive head games, and so the situation deteriorated rapidly to the point where my manager and I were barely on speaking terms.

Things were made worse by the fact that I had been assigned an office mate who was also new to Google, and who was not part of the ads group. Most of the other ads group members were sharing offices (or cubicles) with other ads group members, and so I felt I wasn't really part of the club. On top of that, I was away from home and didn't really have a life up there in Northern California. The stress mounted. I started to get paranoid that I would get fired before reaching the one-year mark. I started experiencing stress-related health problems, some of which are still with me today. On more than one occasion I came that close to quitting. To this day I have no idea why I didn't.

It was about this time that I had my one and only meeting with Larry Page. It was to discuss the progress of the adstoo project and to set a launch date. My manager was there along with a couple of other people (including Doug I think). Things went smoothly until Larry suggested changing the way billing was handled. I don't remember the details, but my response was that this would be significant work. No one challenged me, but I found out later that the reaction of people in the room was something along the lines of, "Is he crazy? This ought to be a trivial change." This little incident turned out to have very far ranging repurcussions later, but that will have to wait for the next blog entry.

Somehow we actually managed to launch AdWords on schedule, in September of 2000. It still seems like a bloody miracle. Most of the credit goes to Jeremy, Ed and Schwim. It could not have been done without them.

I can still remember watching the very first ad roll in. It was for a company called Lively Lobsters. Two months ago, after five years of intending to do so, I finally bought myself a little toy stuffed lobster to commemorate the occasion. (Update on 12/9/2005: It appears that Lively Lobsters has gone out of business. There's some irony for you.)

About two weeks later all hell broke loose.

13 COMMENTS:

Philipp said...

You got me really curious now...

2:40 PM

Zoli Erdos said...

Ron, this should be illagal! Like in soap operas, at the most exiting line...stay tuned .. ahhhhhhhh :-)))

3:03 PM

One bit of information that would really be helpful to clarify  Who came up with the idea of SPONSOR LINKS The (two, now three) listings ABOVE the SERPs with the pastel background color..  Was there any concern/debate about ethics.   Whose idea was it to create the ADWORDS BOXES -then years later take them away.  The Google Sandbox (suggestion) tool - why did it NOT include numbers - like the inventory.overture tool?  How did the transition from Pay per View - to pay per click happen?

3:16 PM

AP said...

It appears that the boys at google didn't realize Ron's talent and lost out eh?

John said...

Man, I am hooked.. very interesting.. keep going...:-)

Anonymous said...

I have a question about the 20% time to pursue one's own interests. Must the project be a totally different project or may someone just do more work on their current projects?

11:40 PM

netron said...

aah man - this is like one of those cliffhangers that you see in soap operas...  compulsive reading!

3:40 AM

Anonymous said...

Ron,  The lisper in me wonders why you didn't write a lisp program to generate your Java program -- at least then, you'd get to use all your lisp tools/skills, and the stuff would be syntactically correct (as long as it didn't "blow out" the girly-man Java development tools).  But then again, you'd probably have only gotten into big trouble for that sort of smartass behavior.  Also, I sympathize with your AMT/options/salary woes. A lot of people can't deal well with making monetary sacrifices for risky businesses -- it keeps them up at night.   If you own risky securities, you can always sell them and buy less-risky ones (but perhaps get whacked by the 365-day capital gains tax) -- but if the risky securities come as part of your job, you are really stuck. And you got screwd with the 5x revaluation! What a shame!

8:15 AM

Anonymous said...

Your blog is interesting. The Google persona I read about on tech sites (being super friendly and laid back) seems to be at odds with your experiences. There is a lot more business than engineering involved in your work there.  -Ivan

9:18 PM

Kevin said...

I liked the lobster part

2:03 AM

markg said...

Doug, When do we get our Xooglers Tee Shirts. Love the stories can't wait until you start telling about the Google Ski trips to Squaw Valley!!

3:00 PM

Ryan said...

Ron,  Here is some more irony for ya... I founded Lively Lobsters in 1998 and was the guy who placed the first AdWords ad you speak of. I am not sure if you are aware, but a nice young lady named Allegra emailed me requesting a high-res version of my company logo for use on some t-shirts being made for the engineers to celebrate. (I never received one... I have asked around and heard mixed news on whether they were ever made, and if they were, I would love to have one now!) You correctly stated that Lively Lobsters went out of business, but I am reviving it this fall. The only reason I shut it down in the first place was because I became so involved with other profitable AdWords advertising that it no longer made sense to focus on Lively Lobsters. These other AdWords ventures have kept me a faithful advertiser ever since, and I am one of Google's top spenders all time, having purchased literally many millions of dollars worth of clicks through my various companies. The power of AdWords itself is responsible for my success, and those very fruits are being used to resurrect Lively Lobsters. In a nutshell... your system (AdWords) made Lively Lobsters, then me, a success and now that success is coming back to revive Lively Lobsters! I am not sure if this is interesting to you or anyone else, but I thought I'd share my story when I came across this blog via a Google search for "Lively Lobsters" while satisfying my curiosity about references to Lively Lobsters still floating around. Thanks for everything you did for Google and AdWords... I owe ya some long-overdue lobsters. Get in touch with me and they are yours!

10:42 PM

Ron said...

Hi Ryan,  Wow, what a great story! Glad to hear everything worked out so well. If you are spending millions on AdWords then you must be doing well for yourself indeed. Congratulations! And glad to hear the Lively Lobsters is rising from the broiler. Maybe I can return the favor by placing the first order on the new site :-)  I owe ya some long-overdue lobsters. Get in touch with me and they are yours!  Thanks! We'd gladly take you up on your offer, but we don't know how to reach you! Would you please send a clue to xooglers at gmail? Doug has some things he wants to ask you privately as well.

8:33 PM

MONDAY, NOVEMBER 28, 2005

The billing disaster

POSTED BY RON AT 7:48 PM

The AdWords launch went fairly smoothly, and I spent most of the next two weeks just monitoring the system, fixing miscellaneous bugs, and answering emails from users. (Yes, I was front-line AdWords support for the first month or so.)

The billing system that I had written ran as a cron job (for you non-programmers, that means that it ran automatically on a set schedule) and the output scrolled by in a window on my screen. Everything was working so well I didn't really pay much attention to it any more, until out of the corner of my eye I noticed that something didn't look quite right.

I pulled up the biller window and saw that a whole bunch of credit card charges were being declined one after another. The reason was immediately obvious: the amounts being charged were outrageous, tens of thousands, hundreds of thousands, millions of dollars. Basically random numbers, most of which no doubt exceeded people's credit limits by orders of magnitude.

But a few didn't. Some charges, for hundreds or thousands of dollars, were getting through. Either way it was bad. For the charges that weren't getting through the biller was automatically shutting down the accounts, suspending all their ads, and sending out nasty emails telling people that their credit cards had been rejected.

I got a sick feeling in the pit of my stomach, killed the biller, and started trying to figure out what the fsck was going on. (For you non-programmers out there, that's a little geek insider joke. Fsck is a unix command. It's short for File System ChecK.)

It quickly became evident that the root cause of the problem was some database corruption. The ad servers which actually served up the the ads would keep track of how many times a particular ad had been served and periodically dump those counts into a database. The biller would then come along and periodically collect all those counts, roll them up into an invoice, and bill the credit cards. The database was filled with entries containing essentially random numbers. No one had a clue how they got there.

I began the process of manually going through the database to clean up the bad entries, roll back the erroneous transactions, and send out apologetic emails to all the people who had been affected. Fortunately, there weren't a huge number of users back then, and I had caught the problem early enough that only a small number of them were affected. Still, it took several days to finally clean up the mess.

Now, it's a complete no-brainer that when something like that happens you add some code to detect the problem if it ever happens again, especially when you don't know why the problem happened in the first place. But I didn't. It's probably the single biggest professional mistake I've ever made. In my defense I can only say that I was under a lot of stress (more than I even realized at the time), but that's no excuse. I dropped the ball. And it was just pure dumb luck that the consequences were not more severe. If the problem had waited a year to crop up instead of a couple of weeks, or if I hadn't just happened to be there watching the biller window (both times!) when the problem cropped up Google could have had a serious public relations problem on its hands. As it happened, only a few dozen people were affected and we were able to undo the damage fairly easily.

You can probably guess what happened next. Yep. One week later. Same problem. This time I added a sanity check to the billing code and kicked myself black and blue for not thinking to do it earlier. At least the cleanup went a little faster this time because by now I had a lot of practice in what to do.

Stress.

And we still didn't know where the random numbers were coming from despite the fact that everyone on the ads team was trying to figure it out.

6 COMMENTS:

Anonymous said...

There's no need to apologise for your geekiness -- as if we don't all know what chron and fsck are.   I'd save the apologies and explanations for the "Steven Levy" style book. I'd guess that most know to go look at things like this for definitions.

12:20 PM

Jan said...

There are still problems with creditcards being declined without any reason - perhaps its all your fault :)  PS: support-team still can't explain the problem

12:48 PM

Philipp said...

Uhm, just to contrast the first comment, yeah thanks for letting us known what Fsck means right here and now without us having to jump around websites too much :) Who says you can't read a Steven Levy book online? Plus, I've got a suspicion this will turn into a book one day or another, and then it will come in handy to not rely on links for crucial points :)

2:26 PM

Greg Linden said...

I'm curious, what database were you using at that time?

3:16 PM

Anonymous said...

The problem is still there , they are supposed to bill every other day or so but sometimes they will let it accumulate and the bill the card - But many times the transaction get declined (as many cards have a 10k limit for a single transaction)

6:02 PM

Anonymous said...

PS. fsck is file system consistency checker.

7:50 AM

WEDNESDAY, NOVEMBER 30, 2005

Racing to the finish

POSTED BY RON AT 8:26 PM

Ron sez... oh wait, don't need that any more.

OK, time to wrap up this little soap opera.

The problem turned out to be something called a race condition, which is one of the most pernicious and difficult kinds of bugs to find. (Those of you who are technically savvy can skip to the end.)

Most modern server code is multi-threaded, which means that it does more than one computation at once. This is important because computers do more than just compute. They also store and retrieve information from hard disks, which are much, much slower than the computers. Every time the computer has to access the disk things come to a screeching halt. To give you some idea, most modern computers run at clock speed measured in gigahertz, or billions of cycles per second. The fastest hard disks have seek times (that is, the time it takes the drive to move the read/write head into the proper position) of several milliseconds. So a computer can perform tens of millions of computations in the time it takes a hard disk just to get into position to read or write data.

In order to keep things from bogging down, when one computation has to access the disk, it suspends itself, and another computation takes over. This way, one computer sort of "pretends" that it is really multiple computers all running at the same time, even though in reality what is happening is that one computer is just time-slicing lots of simultaneous computations.

The ad server, the machine that actually served up ads in response to search terms, ran multi-threaded code written in C++, which is more or less the industry standard nowadays for high-performance applications. C++ is byzantine, one of the most complex programming languages ever invented. I've been studying C++ off and on for ten years and I'm still far from being an expert. Its designers didn't really set out to make it that complicated, it just sort of accreted more and more cruft over the years until it turned into this hulking behemoth.

C++ has a lot of features, but one feature that it lacks that Lisp and Java have is automatic memory management. Lisp and Java (and most other modern programming langauges) use a technique called garbage collection to automatically figure out when a piece of memory is no longer being used and put it back in the pool of available memory. In C++ you have to do this manually.

Memory management in multi-threaded applications is one of the biggest challenges C++ programmers face. It's a nightmare. All kinds of techniques and protocols have been developed to help make the task easier, but none of them work very well. At the very least they all require a certain discipline on the part of the programmer that is very difficult to maintain. And for complex pieces of code that are being worked on by more than one person it is very, very hard to get it right.

What happened, it turned out, was this: the ad server kept a count of all the ads that it served, which it periodically wrote out to the database. (For those of you wondering what database we were using, it was MySQL, which leads to another story, but that will have to wait for another post.) It also had a feature where, if it was shut down for any reason, it would write out the final served ads count before it actually quit. The ad counts were stored in a block of memory that was stack allocated by one thread. The final ad counts were written out by code running in a different thread. So when the ad server was shut down, the first thread would exit and free up the memory holding the ad counts, which would then be reused by some other process, which would write essentially random data there. In the meantime, the thread writing out the final ad counts would still be reading that memory. This is why it's called a race condition, because the two threads were racing each other, with the ad-count-writer trying to finish before the main thread freed up the memory it was using to get those counts. And because the ad-count-writer was writing those counts to a database, which is to say, to disk, it always lost the race.

Now, here is the supreme irony: remember the meeting with Larry where he wanted to make a change to the billing model that I said would be hard and everyone else in the room thought would be easy? The bug was introduced when the ad server code was changed to accommodate that new billing model. On top of that, this kind of bug is actually impossible to introduce except in a language with manual memory management like C++. In a language with automatic memory management like Java or Lisp the system automatically notices that the memory is still in use and prevent it from being reused until all threads were actually done with it.

By the time this bug was found and fixed (by Ed) I was a mental wreck, and well on my way to becoming a physical wreck as well. My relationship with my wife was beginning to strain. My manager and I were barely on speaking terms. And I was getting a crick in my neck from the chip I was carrying around on my shoulder from feeling that I had been vindicated in my assessment of the potential difficulties of changing the billing model.

So I went to my manager and offered to resign from the ads group. To my utter astonishment, she did not accept.

18 COMMENTS:

Anonymous said...

Nice story. But I wish you did not end most of your posts with dramatic phrases like 'utter astonishment'. It is a really sophomoric style for a narrative.In fact, it reeks of the Dan Brown style that I totally detest.

12:04 AM

Anonymous said...

Dude, it's either a nice story or not. Stop nit-picking to show off and just enjoy!

2:12 AM

justin flavin said...

yeah - stop nit picking. just sit back and enjoy the cliff hangers.

2:54 AM

Anonymous said...

Actually, I doubt Ron is exaggerating at all when he says "utter astonishment."  Lispers are perfectionists -- and they drive lesser programmers nuts with their well-deserved superior attitude.  It doesn't surprise me that after Ron screwed up ("failed") because they made him use a lousy language, he felt disgusted with himself and the job, and figured that they wanted him out.  Indeed, if he'd been working with a lisper and the guy made an error with similar ramifications on a NASA mission, Ron would probably be disgusted with the loser.  But most normal programmers use lousy tools, and they tolerate errors more than the perfectionists. They were probably just psyched that Ron's stuff worked and was making them some real money.   But at no point did they ever decide that they needed to start using Lisp instead of their second-rate Fortran-derived languages. [Ron -- you can see I've suffered at the hands of management too.]

5:01 AM

Anonymous said...

Just curios whether it is on Microsoft or Linux.

9:16 AM

Anonymous said...

I'm surprised Larry and the others didn't realize this was a difficult problem requiring a careful design, testing, etc.

10:55 AM

Brinke said...

great story, but i totally have a headache after reding the C++ nightmare part. Time for a, uh, nap.

3:37 PM

Adi Oltean said...

Nice story.  Multithreaded programming is not for everyone. Finding bugs in MT code, or designing some new MT code require experience, a certain state of mind, and an ability to analyze methodically a large set of possible cases. Whenever I have to write C++/C# code that uses shared data structures by two or more threads, I am 10x more careful than usual.  I can certainly say that it takes years to become somewhat familiar with multithreaded programming, especially in things like finding common MT bugs, writing efficient MT code that won't trash the L1 cache or introduce false sharing in multi-proc systems, dealing in a defensive way with shared data (life-management, ownership issues of global objects on the heap, etc), and even more advanced topics like lock convoy issues, lock-free MT programming, etc.  And, no, don't even think that a high-level language like Java is going to automatically solve all your problems here. You can have can be lots of suble MT bugs in Java/C# code as well.

11:11 PM

Anonymous said...

Ugh, the nigthmare of race conditions. The story gives me shivers just thinking about when I had to track down similar bugs.

10:22 AM

Anonymous said...

Reminds me of my RA during my MS for some reason... Great posts, I'm addicted, keep it up.

3:23 PM

Anonymous said...

Definitely a great post. Carefully choosing language/platform is very difficult in business environment. I had an internship at of the giant software companies this summer, and was quite surprised by some of the executive decision made by managers. Essentially I had to design a test framework for a system that set up a certain environment for the production code. I wanted to do it in C# or perl, but everyone on the team favored C++ (with no STL libraries!!). As a result, I spent 3/4 of my time writing string manipulation functions, implementing various storage containers, and looking for memory leaks. At the end of the internship I got most of the things to work, but my life literally went to hell.

7:54 PM

Anonymous said...

7:54 PM

Anonymous said...

Dude, you worked for a woman?!!! Do they have those in geekdom?

10:57 PM

roschler said...

It is nice that Java has automatic memory management and for many, many apps it is a better choice for fast development without memory management headaches.  But it's lack of C++ style destructor brings in a host of problems of its own, especially when it comes to managing lockable resources. Destructors and the intelligent use of smart pointers (or shared pointers) can go a long way to making resource management much easier. Last I checked Java's finalizer wasn't much help here and in fact introduces problems of its own.  I agree that C++, especially when people try to turn STL into a borderline functional programming language, can be a nightmare. If someone could create a language with automatic memory management and C++ style destructors, that would be great.  Multi-threaded programming will always be a huge precarious chore until the Einstein of operating systems builds something we haven't seen yet into the next big O/S.

1:16 AM

Anonymous said...

use PERL

2:33 AM

saurav said...

This kind of reminds me of the pain i went thru writing a multithreaded external sorter for a dual cpu machine. Race conditions in multithreaded programs can be nightmarish. I wonder if anyone knows of a modelchecker for C++ which can allow us to express and validate multithreaded protocols to avoid such problems.

4:15 AM

Joshua Smith said...

"But at no point did they ever decide that they needed to start using Lisp instead of their second-rate Fortran-derived languages."  Of course, C++ is not Fortran-derived. It's ALGOL-derived: ALGOL begat Pascal begat C begat C++.  ALGOL is fundamentally different from Fortran in that ALGOL was the first computer language defined in BNF.

1:59 PM

Anonymous said...

"I wonder if anyone knows of a  modelchecker for C++ which can  allow us to express and validate  multithreaded protocols to avoid  such problems."  I think Coverity Prevent can  detect some of the problems  discussed in this thread  coverity.com/concurrency/index.html

1:51 PM

THURSDAY, DECEMBER 01, 2005

Interlude

POSTED BY RON AT 10:07 AM

I'm going to be away for a couple of days so no new posts from me before the weekend. I did want to say before I went, though, that it may seem that I am, as Doug put it, a disgruntled ex-employee. I know it's not apparent from the story so far, but I am in fact quite gruntled (now there's a word for you), and not just because everything turned out well in the end. Even at the time (well, more towards the end) I thought that my experience at Google was on balance very positive, though it won't be apparent why until I get a little further along in the story.

I wanted to say this up front because I got an email today from my old manager, the one I've been kind of harshing on in previous posts, and that made me realize that I may have some people worried that I'm writing all this because I have an axe to grind. I don't. It's just coming out negative now because it started out that way, and because, frankly, negative experiences make much more interesting stories. I'll have more to say about that later too.

Thanks for bearing with me. Hasta luego!

10 COMMENTS:

It is vital that this information be publicized - not only for historical research, but so that Managers and Administrators can discover what imperfections exists, and tweak the atmosphere.  Most would be less likely to risk offending some hierarchy by confronting them while you are a subordinate.  Whan a company is constantly seeking the creme' de la creme' of Engineers and Programmers - they have an obligation to be as concerned with the emotions and frustrations of this elite workforce - as much as they are concerned with perfecting the Search Engine Softwares.  From a historical perspective, it will be another piece of the puzzle in the history of one of the most globaly influential companies in the history of the World Wide Web.  The motives do not automatically invalidate the validity of these concerns being expressed, and archived for future generations.

12:25 PM

John Vanhara said...

I cannot even imagine someone would say that you are disgruntled ex-employee. It doesn't feel like that at all. Maybe that's because I am European, but I am used to much harsher criticism and it's still considered OK. Nothing is perfect in this world. Everybody is doing mistakes. Bad stuff and good stuff happens all the time. So there is no reason to be too careful about how to explain what happened. I have very positive feeling from the way you explain everything. It is really good story.

1:50 PM

wfzimmerman said...

P.G. Wodehouse's Bertie Wooster may have been the first modern figure to use "gruntled" as a verb. You are following in heroic footsteps.

1:56 PM

Anonymous said...

The flood is over, the land is dry, so why do you wear, such big balls to write all this stuff.

10:23 PM

amar said...

just wanted to let you guys know that i'm enjoying reading your recollections. you don't come across as disgruntled employees at all. looking forward to hearing more.

10:29 PM

KenP said...

"Do no Harm."  Interesting post that isn't on topic but what the heck...  Seems the bad guys over in Iraq have a GPS and a laptop with Google Earth with them most of the time. They know where they are and they know the land's appearance they are attacking. --from a young Marine just back from Iraq  On the good side the laptop often provides a lot of good intel.

12:28 PM

Anonymous said...

"I could see that, if not actually disgruntled, he was far from being gruntled." -- P. G. Wodehouse  I think Bertie Wooster was referring to D'Arcy CheeseWright :-)

12:47 AM

Purnell said...

That's so very true exspecially in America negative stories gains the most publicity and attention.

11:32 AM

doug in Seattle said...

Choices, decisions, consequences. If one consequence is a slam on /., I don't do it.

9:10 AM

r_wolfcastle said...

Whan a company is constantly seeking the creme' de la creme' of Engineers and Programmers - they have an obligation to be as concerned with the emotions and frustrations of this elite workforce - as much as they are concerned with perfecting the Search Engine Softwares.  'Tis true. And as someone who spent a combined 16+ years at Bell Labs and Adobe, both places where you couldn't swing a dead cat without hitting a PhD, I can tell you that managing herds of brilliant people is an art. Furthermore, although the best practitioners of that art are usually quite bright, they are not necessarily brilliant PhDs themselves. And that is something that I am not sure Google has yet been forced to grok, but they will have to eventually.  Google does have several distinct advantages in this arena, all of which Doug has pointed out. Among those are their egalitarian idea culture and their continual striving for excellence while doing no evil. All these things make even the worst prima donna geniuses happier, more productive, and much easier to manage. Google also seems to decidedly not be a place that hires the cream of the crop, burns them out before they are 30, and throws them away. That would be Oracle or, back in the day, Texas Instruments.

11:47 AM

SUNDAY, DECEMBER 04, 2005

Just do it!

POSTED BY RON AT 5:17 PM

Doug wrote:

The upside of this philosophy is that Google did many things quickly, most of which turned out to be positive. The downside is that Google sometimes did things with unforeseen effects and individuals occasionally misinterpreted exactly how much power they possessed. The tradeoff appears to have been worth it.

This reminds me of two anecdotes.

I didn't really appreciate it at the time, having never experienced anything different, but except for having to do it in Java and some basic functional requirements, I pretty much had absolute technical freedom to design and implement AdWords however I saw fit. I wasn't really able to take full advantage of this freedom with all the problems I was having, but no one ever second-guessed any of the decisions I made within the constraints that were given to me up front.

The second story is something that happened in my (one) meeting with Larry Page to discuss AdWords. The issue came up about whether or not ads should be screened before appearing on the site. There was almost universal consensus that they should. The fear was that neo-nazis or some such group would post an offensive ad and Google would get bad PR for publishing it. The lone dissenter in the room was Larry. He was adamant that he wanted to give users the instant gratification of seeing their ads live right away. And being the boss his view prevailed. But we all left the room thinking he was making a big mistake.

Turns out Larry was right and we were all wrong. The instant liveness turned out to be a major feature distinguishing AdWords from the competition, and the neo-nazis never came. Most organizations worry about the negative consequences of the things they do. Google, thanks in no small measure to Larry and Sergey's leadership, tends to worry more about missing out on the potential benefits of things they don't do. (Some of this approach was visible to the public for the first time during the IPO.) The net result is that they do more. Occasionally they hit a bump in the PR road (like during the IPO, or when GMail launched), but on balance, as Doug says, the tradeoff seems to have been worth it.

(There's an interesting book by the same person who wrote "The Tipping Point", which examines the benefits of not over-analyzing decisions. I'm about halfway through it. So far it's been a worthwhile read.)

22 COMMENTS:

Noam Samuel said...

How analytical is the book?

6:01 PM

Spontaneouty was indeed a likely factor in AdWords early success during its "Pay Per Impression" days.  What is not clear is.. "why did they NOT use some keyword filtering software technology - to decrease the probability that these potentially objectionable words would get online?  it would be interesting to elaborate on whose idea it was to ultimately CHANGE to the current "Pay Per Click" model :?

8:45 PM

Anonymous said...

I liked the book. Malcolm Gladwell came and gave a tech talk a few months ago and I was lucky enough to snatch a copy while he was there. Very inspirtional guy.

9:44 PM

Anonymous said...

Can any of you guys talk about the new compensation item called the "Google Unit" ?  Is it like a stock grant that vests over time?  Why did they start issuing those instead of regular options or grants?

12:22 AM

Robert Carnegie rja.carnegie@excite.com said...

Is this the same technology that decorated news stories about the Asian tsunami with Google ads for surfing equipment, until someone figured out how to stop that? Genuinely. I saw 'em.

3:29 AM

charlie_zoo said...

I just found your site Doug - really interesting stuff - and I wonder if any of you out there are UK-based Xooglers? Can you show yourselves? I'd be interested in having a conversation on a professional basis, either here or at Zoo Time.

5:14 AM

Matisse Enzer said...

I've often wondered how conciously Google set about creating the huge real-time auction system that drives adwords.  By now it must be one of largest auction systems in the world. Can you say something about the the various degrees of concious planning and accident that went into it?

7:50 AM

Technicator said...

Google's so different internally that it's probably why their stock is doing so well. I've covered that on my website and it's always so bullish. I give them credit. Is it true that Google has more pHDs hired than a typical clinic?

2:00 AM

Anonymous said...

why does everyone think that a PHD is a magic bullet to making infinite profits?  if so, half the faculties at universities around the country would be millionaires.  they arent.  all a PHD proves is that you can follow instructions.

1:24 PM

R2K said...

Isnt is a conflict of interest for you to host this on a google blog?  R2K..Bathrooms..Rocketry

Ddot the King said...

Oh ok cool....good day.

6:11 PM

Anonymous said...

well..ok, had to comment. OK i am dumb founded on this sunject...what exactly is adwords? I am lost..ahumm..sorry, clue me in?

7:41 PM

Anonymous said...

lol..OOPS not sunject...,haha subject*

7:42 PM

Andrew Goodman said...

I got a lot out of Blink. Beyond your post, which is great to learn from: we get into debates with clients about how much to analyze campaign data, etc.  Although Gladwell probably doesn't have all that much first-hand knowledge, his journalistic account touches a chord: experienced professionals see the benefit of reaching decisions based on "quick analysis" or "thin slicing" rather than a full rundown of all relevant data. The more your job involves seeing the same complex data patterns every day, the more impatient you become with the comprehensive approach.  This is not the same as acting on instinct.  Anyway...  I am not entirely convinced this relates to Larry's conviction on the instant ads.  Our impression has always been that "instant on" was and continues to this day to be a wonderful differentiator for AdWords. Larry is a marketer par excellence (without knowing it?). People still assume that "instant on" is a big unique feature of AdWords, even if this isn't entirely true and isn't particularly important.

11:33 PM

David Towers said...

Doug,  This blog is a goldmine, thank you so much. I'm currently writing my dissertation on the organisational culture of Google (I'm an undergraduate in International business commerce). Your posts have been so helpful in helping me understanding things from inside Google.  Also, as I will be applying to work with Google in Paris it makes it even more interesting to hear about these things.  Please keep the posts coming, thanks again,  David Towers

Anonymous said...

That "someone" would be Malcolm Gladwell, author of "The Tipping Point" and "Blink," and a frequent contributor to prestigious magazines. But that's a small aside. Although Google has prevailed with AdSense, I wish it wouldn't run ads for paper mills. It doesn't need the money and it encourages scummy behavior.

2:10 PM

Anonymous said...

Re: overanalyzing decisions, fair enough, but let's not forget that a lot of companies fell apart because they didn't analyze their decisions closely enough.

12:40 AM

doug in Seattle said...

Sounds to me like the whole premise of Blink is:  Go with your gut instinct  Or as is often dismissed "woman's intuition".

9:05 AM

Anonymous said...

No neo-nazis? The way I found out the ads weren't screened was when my search returned one for a *child porn* site. Naturally I contacted Google about it immediately. I didn't save my original email to them, but here's their response. The ad was removed, but reappeared a couple of times before they apparently decided to block it somehow...

9:41 AM

Anonymous said...

Offensive ads were definitely a real problem in the early days of Adwords. I understand the marketing reasons for instant approval, but, it let some really nasty stuff get through onto Google result pages. Always they would be "unapproved" (no one had reviewed them yet), but, not infrequently, unapproved ads would run for many hours and generate lots of impressions.

2:47 PM

Bob Aman said...

I'd have probably just used a bayesian filter to identify bad vs. good ads. If the filter let it through, it'd run until someone complained, at which point the filter could just be retrained with the new information. If it failed the filter, it would sit there and wait for approval or denial.

8:08 AM

R2K said...

Wow several clicks from this very old post. I wonder what happened?

5:12 AM

FRIDAY, DECEMBER 09, 2005

They say it's darkest before the dawn

POSTED BY RON AT 1:37 PM

The billing bug was the only major problem we ever had with AdWords. It actually astonished me how few major disasters there were given that the potential for them was so high. The Google system is mind-bogglingly complex and it grew up very fast, which is usually a recipe for building a huge collection of intractable bugs (look at all the problems Microsoft has with Windows and IE). But it didn't happen. In fact, other than the billing disaster, the worst problem I can recall that was visible to users was that they once pushed a version of the home page that caused it to appear without a logo. Other than that things seemed to go very smoothly. Before coming to Google I would have given long odds against that being possible for a development of this magnitude done in C++. I was beginning to have serious doubts about my professional worldview.

A lot of the credit goes to three of the most brilliant hackers I have ever known, Jeff and Sanjay who shared an office so I dubbed them the dynamic duo, and Craig Silverstein, director of technology (I'm making an exception to the xooglers policy of referring to people only by their first names because Craig has already been outed in the press). I didn't have many occasions to interact with them since I was working on things that weren't very technologically demanding, but on a few occasions I did go to ask their advice about one thing or another and they always blew me away with the sheer quantity of abstruse technical information that they had stashed away in their heads -- to say nothing of their ability to type fast. Craig especially was really something to see. He used this funky ergonomic keyboard and when he typed it sounded like one of those ratcheting noisemakers you spin over your head at new years. I once came to ask him a question that I thought would take me half an hour to figure out on my own and he replied, "Well, let's find out" and in less than a minute he whipped up a piece of code that answered my question. My jaw was on the floor.

With the billing problem fixed, AdWords settled into a nice steady growth pattern, and everyone involved got an engraved lava lamp. We hired some people to man the customer support lines and do ad screening, and I was left to relax a bit and build some internal tools for monitoring the system's performance. Google was a very open company in those days, and the results were visible for all to see, so anyone could pull up a web page on the internal server and see just how much money AdWords had made on any given day. It was very exciting to watch the numbers go up. And up. And up.

It was becoming clear that Google was on a serious roll, and one of the things that would soon have to be done was to internationalize AdWords so that it could serve non-English-speaking audiences. This presented a major headache. Because AdWords had been written in such a hurry, there was English text scattered all over the system. A lot of it was in HTML pages (actually JSP pages, which is like HTML with embedded Java code), and a lot of it (like error messages) was embedded in the Java code. Just finding all the English text would be a significant challenge.

Here again I hit a major frustration as a result of being forced to use Java and JSP. Just before I had come to Google I had written a bunch of infrastructure to write web servers in Lisp. One of the cool features of Lisp is that you can very easily change the way the language is parsed, so finding all the English text is easy. You can more or less have the system do it for you automatically. But in Java (or any other existing language for that matter) you can't do that. The way that the system processes programs is carved in stone and you can't change it. The best you can do is write another parser for your program and use that to do whatever automatic processing you want to do. But that's a lot of work. In Lisp it's literally five lines of code (maybe ten for a really industrial-strength version).

But, of course, that avenue was now closed to us, and because we had to deal not only with Java code but also with JSP, we pretty much had to do it manually. Fortunately for me I was able to weasel out of doing this incredibly tedious work, which was fortunate because if I had been forced to do it at that point in my life I very well might have thrown myself off the Golden Gate Bridge.

It was becoming very clear to me that I was not a good fit for the ads group. There was actually some very interesting work going on there. A fellow named Eric, who had hired on just a few weeks after I did, had come up with a brilliant idea for increasing clickthrough rates. It was so brilliant that it didn't even make me slap myself on the forehead and say, "Why didn't I think of that?" because I knew that I never could have thought of it. I didn't have the mathematical background. I was really beginning to have serious doubts about my ability to contribute anything meaningful in this company. But on the other hand I had already invested four months, and it was becoming clear that Google was not going to be a casualty of the dotcom crash any time soon. Besides, I had offered to resign from the ads group and my manager refused, so I figured that put the ball in her court to figure out what to do with me.

I was assigned a series of small projects and I pretty much screwed them all up. As this is not one of the proudest episodes in my life I hope you will forgive me if I don't go into too much detail (actually I think I've repressed a lot of the details). I can remember two incidents in particular: Jane (my manager) had asked for some statistics to be produced. She, unlike many managers, was actually quite a competent hacker in her own right (despite (or perhaps because of) the fact that her degree was in philosophy) and had taken a preliminary cut at the problem, but the results were not what she wanted. She handed the project over to me. It was just on the heels of a series of screwups on my part and I really wanted to redeem myself, so I threw myself into the work and got it done in an impressively short amount of time. Too short as it turned out. I sent her the results I had produced and didn't even notice until she pointed it out that they were identical to the (incorrect) results she had produced herself. I really wanted to crawl into a hole.

That was pretty much how November and December went. Anyone who watched me during those two months would have justifiably concluded that I was the world's biggest moron. I was beginning to believe it myself. I have no idea why Google didn't fire me. And given that they didn't fire me, I have no idea why they didn't try harder to help me find a place where I fit into the company. It still baffles me to this day. For a company that does so many things right, and in particular that takes such pains to make sure its technical employees are happy, why they chose to dump me in purgatory and leave me there is a great mystery.

UPDATE 12/11/2005: I'm putting this here rather than in the comments section because blogger won't let you put a <pre> tag in a comment so you can't format code properly.

Some anonymous coward wrote:

So it's trivial to write a piece of code that walks a program, replacing string constants with function calls that pull the correct string for the current language out of a database (for instance).

It's actually even easier than that. I'd do it with a read macro, something like this:

(defun my-string-reader (stream char)

(let* ( (s (ccl::read-string stream char))

(translations (rest (assoc s *translations* :test 'string-equal))) )

(if translations

`(or (getf ',translations *current-language*) ,s)

s)))

(set-macro-character #\" #'my-string-reader)

Here's an example of how it would work:

(defvar *translations* '(("Hello" :spanish "Hola" :german "Guten tag")

("Goodbye" :spanish "Adios" :german "Auf wiedersehen")))

(defvar *current-language* :english)

(defun test ()

(print "Hello")

(print "Goodbye")

(print "This string should not be translated")

(values))

? (test)

"Hello"

"Goodbye"

"This string should not be translated"

? (let ( (*current-language* :spanish) ) (test))

"Hola"

"Adios"

"This string should not be translated"

? (let ( (*current-language* :german) ) (test))

"Guten tag"

"Auf wiedersehen"

"This string should not be translated"

This is actual working code, and it took me about fifteen minutes to write.

19 COMMENTS:

Derrick said...

Ron, I completely admire you for your ability to write about this. You are a better man than I to be able to share what sounds like a really wrenching time in this forum.  I think you've said that it turned out OK. I'm really cheering for you!

4:59 PM

Anonymous said...

Your honesty is an asset. Google's willingness to allow failure is their asset. Both of you will work out fine on this planet of dishonesty and required perfection.

5:32 PM

Anonymous said...

My guess is that they assumed you would figure out something useful to do...

7:18 PM

Anonymous said...

xxxx

10:07 PM

Anonymous said...

This sort of gybes with my experience as a recent interview candidate. They never really explained what the job was, although I knew roughly the role. In retrospect, I believe this is because they expect smart people will find and essentially define the job, and if the current one isn't working out, then its just a matter of time before self-adjustment occurs ;-)  I am also learning that there's a pattern of low-balling candidates on pay. I think there are several motivations for this:  1. The total benefits package has to be taken into consideration, and the benefits are substantial, when compared to other companies.  2. It's like heaven. Once you're in, the living is good. And if you're smart, you'll only excel and advance.  3. They can get away with it!

10:23 PM

Ron said...

My guess is that they assumed you would figure out something useful to do...  Maybe. I actually did try to take the initiative several times. Twice I was shot down (remember, I tried to resign from the ads group so that I could go somewhere else, but I wasn't allowed to quit), and the other times my proposals were ignored (maybe I should have pushed harder, but I was getting more than a little gunshy by then). This was not a systemic problem, because other people were getting their ideas off the ground (like Eric with his clickthrough enhancement idea). I don't know, maybe all my ideas just sucked, but then I'm back to wondering why they didn't just fire me.

12:15 AM

amit Dixit said...

First of all, that’s very honest of you...What I would like to suggest is that, you should try to learn more about yr self (your -+) rather than google in first place....Always remember to start from grass root level of a problem and use your common sense to look for its solution.....and when you put forward your idea in front of your boss/colleges make sure you give ample justification for it. Question your own ideas, in your mind before putting it forward to avoid criticism. Above all always present yourself in such a way that you are looking for solutions, rather than proving your intellect and try to involve every1....as I think you must be surrounded by the mastermind of their field…………………………..And Maaan you are da best ….Cheeer up NOW!!

3:53 AM

Brian Slesinsky said...

Could you say a little more about how you would do i18n in Lisp? If I understand this correctly, it sounds like some kind of structured search-and-replace, which some Java IDE's do have now. (But not every string needs to be internationalized, so it seems like there'd still be some cleanup.)

10:26 AM

Anonymous said...

Programs are data in Lisp; or, more clearly: the structures built by the parser are part of the language specification, and are quite simple and easy to understand besides. So it's trivial to write a piece of code that walks a program, replacing string constants with function calls that pull the correct string for the current language out of a database (for instance). In fact, with a Lisp macro, this can be made to happen transparently as part of compilation, so you never have to see those calls at all.

7:29 PM

Anonymous said...

So, what was the great idea to help with clickthroughs!?!

10:42 PM

Ron said...

This post has been removed by a blog administrator.

12:40 AM

Ron said...

So it's trivial to write a piece of code that walks a program, replacing string constants with function calls that pull the correct string for the current language out of a database (for instance).  I've posted a response to this in the main body of the post (along with an explanation of why I put it there).

12:52 AM

Anonymous said...

Re: Lisp read macro. Very cool, but what if "Hello" happens to be a command sent to an internal server via RPC? When you change languages, your RPC fails. It seems dangerous to mark ALL strings to be translated automatically. I think it'd be better to explicitly mark strings that are for "external" use to distinguish them from internal strings (logging, RPC, etc.).

8:11 AM

DIE said...

fascinating, the lisp references reminded me of the yahoo store articles. I wish more webbased companies would look at lisp as an alternative you but tend to be stonewalled with zealots. Sysadmins preach Perl, and managers tout Microsoft dotnet.   I can work magic in any language but I'd could do so much more if given the right tool.   Word to the wise!! Math isn't much of a problem man, I was one of the ones that didn't sale the math books from college. When in doubt I use them as reference. Stats books are cheap. peace out and keep it real yo

10:50 AM

Matisse Enzer said...

OK, I need to get around to learning Lisp :-)  I sympathize very much with what you describe about being sort of "abandoned" an Google - I've had that happen to me, and for my particular personality, it is very tough. Very cool of you to post about it so thoughtfully.

2:09 PM

Ron said...

what if "Hello" happens to be a command sent to an internal server via RPC?   Then I'd suggest calling that code without rebinding *current-language*.

4:52 PM

Radha said...

Hi Ron, Sorry for taking off the veil of Jeff and Sanjay. I believe they are Jeffrey Dean and Sanjay Ghemawat. I read their papers on MapReduce and GFS.. Truly astounding..  One trend I noted is that a lot of early systems folks inside Google came from DEC(later Compaq , now HP labs) Western Reasearch lab in Palo Alto. Is it possible to shed some light on how Google(or Sergey/Larry) could have convinced these great minds to work for Google. Or was it Carly Fiorina that really drove them away??

9:05 AM

Joshua Daniel Franklin said...

That example looks a lot like how it's done in C with the poorly-named gettext.

10:06 AM

Anonymous said...

Python has some limited introspection abilities, but it would still be difficult to output "fixed" programs.  I am familiar with scheme, it was used in S&IoCP, great book, used for MIT freshmen to teach programming concepts. However, I've never run across a program where I thought "this is a perfect reason to use scheme!". So in reality, I've never used it. Which is to say, I've hardly ever needed to manipulate source code.

7:26 PM

FRIDAY, DECEMBER 09, 2005

Let's get a real database

POSTED BY RON AT 3:09 PM

This story is a bit of a tangent, but I think it's worth telling because there are some useful lessons to be learned.

AdWords was built using the MySQL database, which is open-source and therefore available for free. It is by now also nearly as full-featured as the best commercial databases, but back in 2000 this was not the case. MySQL was quite a capable system, but missing a few (what some would consider basic) features. These missing features were obviously not a show-stopper, as we managed to get AdWords to work without them, but in a few cases it did take some extra programming to work around one of these missing features. On the plus side, MySQL was fast and reliable and, as I have already noted, free.

After AdWords launched, Jane, the ads group manager, decided that now would be a good time to switch over to a "real" database. "Real" is one of those words that Doug ought to add to his list of words. It means "expensive". Many managers seem to have this idea that it is invariably true that you get what you pay for, and that therefore nothing that is available for free can possibly be any good. Using MySQL was acceptable as an expedient to get things up and running quickly and with a minimal of capital outlay, but now that things were settling down it was time to recognize that this was really, fundamentally, a mistake, and it should be fixed sooner rather than later.

The flip side of this philosophy is the one more commonly espoused by engineers, which is nicely summed up by the old aphorism, "If it ain't broke, don't fix it." Yes, MySQL was missing some features, but it wasn't broke(n). We had spirited debates in ads group meetings over what to do.

We finally decided to go with a commercial database (I won't say which one) over the objections of a number of engineers, including myself. To ease the transition it was decided to convert AdWords over to the new system first, and to do the main ads system later. It was a project on a par with the internationalzation effort in terms of the tedious work required to comb over nearly all of the AdWords code and change all of the database queries. (Databases are supposed to all be compatible with one another, but in reality they pretty much aren't.)

To make a long story short, it was an unmitigated disaster. The new system was slower than molasses in February. Some heroic optimization efforts eventually produced acceptable performance, but it was never as good as the old MySQL-based system had been. For a long time we were stuck with the worst of all possible worlds, with the two ads systems running on two different databases. It was still that way when I left Google in October of 2001, but I have heard through the grapevine that they eventually went back to MySQL. (Since then, MySQL has added many of the features that had been missing at the time.)

The moral of the story is that sometimes, and in particular with free software, you get more than what you pay for. There are a lot of companies out there paying dearly for commercial databases (and operating systems for that matter). As far as I'm concerned they might as well be flushing that money down the toilet. Actually, they might be better off. We certainly would have been.

As an aside, there is a raging debate in the hacker community about the overall economic merit of the open source model. (Making money producing free software is quite a challenge.) I am not taking sides in that debate here. All I am saying is that from the end user's point of view free software is often much better than the producers of commercial software would like people to think.

69 COMMENTS:

James A. Warholic said...

I am actually somewhat surprised that there was a push by some at Google to pay for another program in which the original was working quite well to begin with. After all, I thought the main search servers were built with Open Source Software such as Python. Anyways, I have thoroughly enjoyed reading the posts back from the beginning. I look forward to the next segments.

5:36 PM

Jonathan Ellis said...

holy crap, you ran adwords on a transactionless database?  you have balls. and you're lucky as hell nothing (else) went wrong.

9:48 PM

Ron said...

holy crap, you ran adwords on a transactionless database?  Yep.  you have balls. and you're lucky as hell nothing (else) went wrong.  Nope. Luck had nothing to do with it. If you don't have transactions you just roll your own. It's actually not hard at all.

10:21 PM

Anonymous said...

Aren't there issues with the GPL? PostgreSQL for example offers a BSD license, which is why many commercial users prefer it to MySQL.

1:45 AM

Rob said...

This neatly encapsulates my employer's experiences (and I was one of the tech guys who encouraged the company to move to *cough* Oracle *cough*!). You can get an elephant to tapdance, but the results will never be graceful. Asking one to do so in your living room is twice a mistake.

6:39 AM

Anonymous said...

For the life of me I don't understand why more folks don't use Postgres. The BSD license seems to be the way to go. Similarly, I can't comprehend why Google uses a Linux kernel when the BSD kernel is completely free of any GPL b.s.

6:47 AM

Ron said...

I can't comprehend why Google uses a Linux kernel when the BSD kernel is completely free of any GPL b.s.  Google isn't distributing software based on GPL code, so the GPL "b.s." doesn't come into play.

9:06 AM

seo black & white said...

Btw, MySQL supports transaction rather well since they added InnoDB storage engine. In the last three years it works for me like a charm...

Steve said...

This story says a lot about the culture at Google--especially in those early days. While LAMP (er non commercial platform based) architectures are today given the respect they deserve--it was a much more aggressive thing to do back then (and still be taken seriously). I recently left a moldy old company doing bank automation systems which collapsed under the weight of WebSphere, DB/2 and Oracle. Unfortunately it takes an engineering culture to cut through the hype and distill the issues to the essence of a particular tool or system's purpose--then to select the right tool for the job. Google's culture embraces this and is a fundamental reason for their success.

10:18 AM

Anonymous said...

In 1999 I gave a presentation to the bankers taking MP3.com public (July 1999, $370m offering). We used MySQL (wish I'd used Postgres and today, that's all I use) and the bankers were agast when I told them we did not use Oracle. Today your bankers would be agast if you did use Oracle in your startup. MP3.com also funded Hans Reiser to add journaling to Reiserfs so as to reduce dependencies on VxFS. An estimated savings of $8m over five years. Open source definately has its place and you've got to give back to the community to keep it fresh and moving forward.

6:36 PM

Anonymous said...

People who don't like the GPL must really like the reach around Microsoft and Apple give you when they are fucking you for your code.

6:51 PM

Anonymous said...

A former employer of mine got the "let's get a real language" fever. They decided to ditch Python for Java because of a whole bunch of pointy-haired boss reasons. The mentality seems the same as that of the "real database" story here...why does management so often disregard the opinions of the people that have actually been in the trenches doing the work? Why do they decide to throw away or overhaul working systems because of some marketing gobbledygook?   This is an apect of business I really don't understand. Even when the decision precipatates horrible consequences (unbearably slow systems, unmanageable hulking codebases), some folks just can't recognize that they made a mistake by not listening to the engineers.

7:02 PM

Jonathan Ellis said...

"If you don't have transactions you just roll your own. It's actually not hard at all."  You're scaring me.

7:05 PM

Jonathan Ellis said...

To elaborate: some transaction benefits can be simulated "easily," such as logging each update to make sure Bad Things don't happen when you, say, charge a customer and the db goes down before the ad gets inserted. Although if single updates aren't even atomic -- and if your db doesn't support transactions, they probably aren't, which is the case for MySQL's in non-innodb tables -- you're still in trouble if you're running something like "update users set balance = balance - delta" when it goes down.  The harder ones, though, such as isolation, I've never seen done outside the database, and I wouldn't trust the code of anyone who claimed to do it. Much safer to use a database that's already debugged their code many times over.

7:14 PM

Mike Hillyer said...

Jonathan Ellis said...  holy crap, you ran adwords on a transactionless database?  you have balls. and you're lucky as hell nothing (else) went wrong.  Actually, MySQL has had ACID transactions for many years now, including savepoints and four levels of transaction isolation. Best get current before criticizing.

7:24 PM

Anonymous said...

Erm, the article is about yr 2000. MySQL got transactions about 2001 if i'm correct, but don't really count on that :) However, we can presume that MySQL had no transactions back then :D

7:47 PM

Anonymous said...

Single MySQL updates have been atomic for a looong time (1999 at worst), maybe always. With those you can roll your own transactions.  It's funny when people trot out disaster scenarios that derive from premises that are incorrect. "Well, yeah, if not for Gravity, things might be different like you said..."

7:48 PM

Jonathan Ellis said...

MySQL does claim that updates are atomic even on MyISAM tables, but I've seen many people report that this is only true if nothing goes wrong, i.e., power loss, simple crashing, etc. Two such people have commented on the relevant doc page: http://dev.mysql.com/doc/refman/5.0/en/ansi-diff-transactions.html  I'm no expert on MySQL source but I've seen enough non-idiots report on MySQL corruption that I find toeing-the-party-line claims to the contrary difficult to believe.  In any case, my main point stands: atomic updates alone aren't going to help you improvise isolation support, etc., if your db doesn't have a concept of transactions.

8:22 PM

Hank said...

Actually Ron, "rolling your own" is not a smart way to go. Trying to emulate transactions in your app means that you don't actually have the guarentee of a transaction, since your app and the DB are seperate, and one or the other can go down in the middle of a pseudo-transaction, leaving you with inconsistant data.  Using such a crippled and broken database is even more crazy when you consider that it corrupts data (intentionally) and that there was a free alternative without these problems, and that handles many concurrent queries better.  And anon, they can't use BSD, they're from Stanford.

8:33 PM

Anonymous said...

MySQL is rock solid.

10:16 PM

Online Degree said...

We have been using MySQL on our site (www.degreeme.com) since its inception and have never had a single problem. Impressed? Just kidding... I know that isn't at all impressive :) But I am still a huge supporter of MySQL although I also have heard of instances where MySQL has had corruption problems. Hopefully the new version 5 has fixed that.

11:35 PM

Anonymous said...

i love this story... you do get what you pay for... of course how you pay is relative... because some of the best things in life are free... fun stuff

11:38 PM

Anonymous said...

I'd dare to guess that the reason they had such a big problem porting over to some other database was that their code base was full of "hand rolled transactions", including a huge number of calls to "lock table".  If they had gome with a database with transaction support from the get go, like PostgreSQL, the porting work would have been much simpler.  The fact that MySQL makes you code a lot of non-standard stuff which makes it difficult to port the code over to other databases is being presented as a good thing in this article. If the author had opened his eyes a bit he'd learn to use something else from the get go the next time.  And yes, PostgreSQL is a "real database", and it's not expensive, it's free.

11:53 PM

Ron said...

Actually Ron, "rolling your own" is not a smart way to go.  What can I say? It worked for us. Worked for this guy too.

12:33 AM

Anonymous said...

As some previous people said, I would be surprised if the problems porting to the 'other' database from MySQL where really any different from the problems developers generally get when porting from database A to B. Without a really good understanding of how things work in a given DBMS, especially locking and transaction logging, it is very difficult to get something to work as well when ported. I would wager than even the high calibre googleplex folk would find it a challenge to get that experience quickly. Tom Kyte's Oracle books contain numerous stories which illustrate this point from his experiences of going from Sybase to SQL Server to Oracle.

3:08 AM

sf said...

It's not "free" vs "proprietary". It's some features against other features and tradeoffs against other tradeoffs. MySQL has the single unique feature of being a speed demon amongst the databases. The tradeoffs are quite dire in the domain of functionality, and need to use specific MySQLisms instead of the "high way" to have things running fast, not slower than all the rest. Other databases are more feature-packed, but they are simply slower. Sometimes WAY slower. The change of the database should have considered this.

3:48 AM

Samuel Cochran said...

It is quite possible to make money from Open Source software. Rather than charging for software, one must charge for services. Installation and configuration of software, customisation of software and solutions, etc. can all be marketed rather than software. This is my approach.  By doing this one can contribute any changes and improvments to the Open Source community whilst providing solutions for companies, and make a living, too. :)

5:38 AM

Anonymous said...

I went through similar issues at a biotech startup. I was the only programmer there for the the first 1.5 years and had built everything with a postgres back end. I caught a lot of heat resisting managements wish to convert to Oracle, but persist I did, because there was no technical justification for the effort. I left for a much better opportunity six months later and my successor apparently got the job by promising the conversion. After 1.5 man-years, they still did not have the system working as well as it had been. Eventually the company was sold for pennies on the dollar.

8:09 AM

Anonymous said...

You have a team of engineers that are resisting the move to another database system pushed by management. These engineers are all happy with their current database, which is probably the one they use any chance they get, so they are really in tune with how it works. The new database has to be used and tuned in a completely different way, requiring its own years of experience to master. Of course the second solution using the new database is going to be worse, how could it not? It says nothing about the database products tbemselves.

9:01 AM

Anonymous said...

postgres advocates are delusional -- millions of mysql users have no time for crashgreslow.  Back in 2000, postgres didn't support the LIMIT keyword. I got hoodwinked by the FUD they spread about MySQL and tried using it for three projects: each time I got crashes and data corruption with small data sets. You could find very few sites that were using Postgres on the web, and the operators of many of those sites regretted that choice.  Somewhere around postgres 7.4, postgres became stable enough that you actually ~could~ use it in a production system. Still, it's pretty common for people to try a few queries on Postgres and mysql, notice that some very common queries run 60 times faster on mysql, and come to an obvious conclusion.  People who push the "no transactions" FUD also forget that transaction support often reduces the reliability of applications -- yes, it's the stupidity of the application programmers to blame, but an organization I've worked for spent $250,000 on some commercial junkware which doesn't do correct error handling with transactions, and this leads to strange, glitchy, sporadic problems... And these guys aren't using an "off brand database", but they're using Oracle.  The truth is that it's not the end of the world if you mess up a row or two in most databases: if you consider all the other ways an e-commerce site can screw up because of user error, human error in fulfillment, problems in shipping, other programming errors, losing a row because of the lack of transaction support is a minor problem... As the CEO of one company told me, "if we mess up one order in six months, we can just let the customer have it for free" -- that's a lot cheaper than the cost of a DBA and the extra hardware to run Oracle, never mind the cost to license Oracle.

9:22 AM

Anonymous said...

9:26 AM

Lauri said...

Ron, how much need did you have for transactions when working AdSense anyway?   It looks to me like it is mostly INSERT operations and lots of them. They get written to separate servers based on the user hash and the balance changes get calculated at X intervals offline.   Am I close? Or did you have some other setup.

10:42 AM

Anonymous said...

I'm not a developer, i'm a financier so please excuse my ignorance. I found this post quite interesting -- where would you suggest I go to read the best open source debates? I'm interested in what the technology community pitches as the pro's and con's. Thanks.

4:58 PM

Ron said...

Ron, how much need did you have for transactions when working AdSense anyway?   Well, that depends on what you mean. In some sense we had no need for them at all, obviously, because we built the system without them. If you want to know how many times we had to write extra code that we wouldn't have had to write had we had transactions, it's hard for me to recall exactly (it's been five years) but it wasn't very much. It certainly was not a major issue. And, like I said, in those situations where you need real transactions (fairly rare in most real world applications I believe), emulating them yourself isn't particularly hard.  Of course, it's all a moot point now that MySQL has transactions.

5:11 PM

Anonymous said...

The MySQL doc page says an update might not update ALL of the rows if the thread is killed during the update. That still does not stop single-row updates from working, as mentioned. Furthermore, if you are using your own transactions, you are writing some extra code to make sure multi-row updates work as well.

6:15 PM

Matisse Enzer said...

One way of setting up MySQL these days is to do all the INSERT/UPDATE/DELETE to an InnoDb table and then to have a pool of "query slaves" where the table is MyISAM.  The query-slaves are behind a load-balancer and are used by applications that need to do SELECTs, and this way they never have to wait for the locks that the INSERT/UPDATE/DELETE create.  The Master can be a MyISAM table if transaction-safety isn't as important as write-speed - INSERT into a MyISAM table is very very fast.

10:32 PM

Sugree Phatanapherom said...

Did they develop any lock server to simulate transaction in global scale?

1:53 AM

Anonymous said...

All I am saying is that from the end user's point of view free software is often much better than the producers of commercial software would like people to think.  "End user" being the developer or the manager who actually gets to pick and pay the product?

2:55 AM

Anonymous said...

People prefer MySQL over PostgreSQL for the same reason people prefer GNOME over KDE: Because the developers are arrogant bastards, and the supporters act like dicks.

5:16 AM

Jayson Vantuyl said...

In all seriousness, I don't see what the GPL vs BSD comments are about.  Google developed an app on MySQL on Linux. The GPL didn't come into play unless they tweaked and then distributed a modified Linux kernel. Even then, they CAN STILL LEGALLY SELL IT!  The only thing that someone else can't do is improve the source without providing a little quid pro quo. This is not unreasonable, it's a LICENSE choice. For the small developer, it's a damn sensible one as well.  That said, I prefer Postgres for reasons absolutely unassociated with the license. I feel it implements much "purer" SQL than MySQL. I like its administration better. I like how it plans queries better.  As for rolling your own transactions, you should have covered this if you took a CS degree. Between lockless techniques and a simple test_and_set routine, rolling your own locks (and thus everything Atomicity, Concurrency, and Isolation) is really childs play. As for Durability, a log table (or fsync'd file even) is really not that difficult either.

7:15 AM

Anonymous said...

Ron, I don't see anything at all about "that guy" trying to create safe transactions on top of year 2000 mysql. He says very clearly that he didn't use a database at all, he used files. Because he not using tables and rows accessed through SQL, he can ensure that his updates are completely atomic. You can't do that with year 2000 mysql. There is a very big difference.  And Jayson, I realize its easy to say you can make safe transactions, but its not easy to do. The only way to make it safe is to rely on a log, and if a query fails, take the database down and restore it from a known good point, then replay the log. This is not transactions, its recovery. There is simply no way with year 2000 mysql to run a bunch of queries in a single atomic transaction and guarentee that they either all succeed, or they all do nothing.

10:56 AM

Arqueaopterix said...

They were right in that MySQL is a toy database. It only got popularity because Slashdot uses it.  However, unless you're using Oracle, PostgreSQL is better than your database.

12:07 PM

Ron said...

There is simply no way with year 2000 mysql to run a bunch of queries in a single atomic transaction and guarentee that they either all succeed, or they all do nothing.  That is true, but irrelevant. What is important is not to guarantee that they do nothing, but merely to guarantee that whatever they do they do not affect the results of subseqent queries unless they all succeed. That is possible, even with y2k MySQL. Figuring out how is left as an exercise.

11:17 PM

Rob said...

What I don't understand is why modern huge database applications are written using the database engines that run on top of some universal OS? I mean, what else can be on that server except the database? So the ideal modern database should be the OS, possibly even without the file system as we know it. He hard-drive itself can be structured to suit the database needs so one would never have to rewrite any files, just the segment of the disk that was assigned to the particular data. The improvement is speed and reliability would be enormous.  This exists presently. It is called Microsoft SQL Server (presumably, as usual for MS products, its hooks dig deep beneath the OS into the kernel). It is of necessity a security nightmare, as are all MS products.  You have a team of engineers that are resisting the move to another database system pushed by management. These engineers are all happy with their current database, which is probably the one they use any chance they get, so they are really in tune with how it works. The new database has to be used and tuned in a completely different way, requiring its own years of experience to master. Of course the second solution using the new database is going to be worse, how could it not? It says nothing about the database products tbemselves.  Certainly there are elements of this, but then, if the DB requires an army of expensive high priests to keep it running smoothly, why bother? The design flaw of failing to self-tune becomes a business feature in that the customer now has to sign up for hours and hours of expensive consultant hours. I don't see this as a positive.

6:31 PM

r_wolfcastle said...

I have worked at 2 companies that rolled their own transaction logic in the application and used MySQL MyISAM tables -- one starting in 1999 and the other starting in 2001. Anyone remember AvantGo? That entire thing was a single MySQL instance with a load balancer and multiple query servers.  At the second job, the one starting in 2001, as jack-of-all-trades I got put in charge of database stuff. We wrote our own layer between the application logic and the database for ease of porting. Our app was essentially 100% database-centric, but we were able to port from MySQL to MS SQL Server in a couple of weeks because of the way we had written the layer. It turned out that none of our customers wanted MS SQL Server (actually, they didn't care what we used as long as it worked and it was cheap), so we let support for that lapse and just stuck with MySQL.  If you are selling a product that is substantially smaller in scope than, say, a comprehensive SAP solution or e-commerce infrastructure for eBay or Amazon, you cannot use Oracle for economic reasons. You can't use the customer's existing Oracle installation for your app, because that installation is already highly-tuned by expensive folks for SAP or CRM or whatever. They won't let you and your app and its processing and bandwidth requirements anywhere near it. So to use Oracle the customer has to have an Oracle instance just for your app, and it is pretty damned difficult to sell a customer an $80,000 product when you have to say, "Oh, and by the way, you'll need to fork over about $30,000 for Oracle and hire another admin to tune and babysit it." That is where open source databases like MySQL really shine.  Finally, someone said that INSERT into MyISAM tables is very fast, and that is true. But there are special circumstances (and they're not that rare) in which you can go even faster using LOAD DATA INFILE. MySQL claims is "up to 20 times faster" (last time I looked) than using MyISAM INSERTs, but which in practice is actually 5-8 times faster in apps I've measured.

3:18 PM

Anonymous said...

This has just got to be a wind-up, right?  BTW Oracle bought up InnoDB a while back. You stick to your, um, MySQL disaster-in-waiting. I'll stick with PostGres, or Firebird rather than a toy.

6:55 AM

Tyler said...

Just wanted to chime in and comment on some of the "Oracle is expensive" comments. There are really 4 versions of Oracle to consider: - Enterprise Edition = $1x - Standard Edition = $.38x - Standard Edition One = $.13x - Express Edition = FREE! (just released this month)  The .38 and .13 numbers are from the current price list.  I use x, since government pricing is typically MUCH lower than what you see on our price list.  Most people have seen the price for EE, but don't know about the other versions. MS has a VERY similar pricing structure.  Full Disclosure: I am an Oracle employee.

6:05 AM

Anonymous said...

What a sad commentary about how little so many in the IT industry know about data.  I've read that mysql was free, but commercial databases are expensive. Then the author admits that they had to write their own transaction logic. Where did this free labor come from?  I've read that "it's not the end of the world if you mess up a row or two in most databases". The issue is that the results are *unpredictable*. You don't know what will break when you violate your contract between components.   I've read that mysql is a speed demon. Ah, right. With no parallelism, no partitioning capability, inability to properly optimize queries with 4 joins, no memory tuning. Please. The only queries that it runs fast fall into a very small niche of read-only retrievels of highly selective data (<1%) via btree index. That's good - but only covers a subset of possibilites that you should be looking at.   I've read that mysql has all the features you really need. Back in 2000 it didn't support transactions, pk/fk constraints, views, unions, subselects, triggers, stored procedures, etc. And it still doesn't support query parallelism or partitioning. If you're writing database apps without any of these features you might as well stick to MS Access.  I've read that they wrote their own transaction logic (since mysql didn't support transactions well) and it worked fine. Really? How do you really know? How did you empiracly measure data accuracy? Hmmm, you didn't? Really, I wonder what kinds of problems (loss of accuracy of adsense) were caused by data quality problems. Note: I've never encountered a database that didn't have data quality problems, the lack of constraints and transactions almost guarantees their widespread existance - whether or not it was shoved in the face of a programmer.  I've read that conversion from mysql to the other database was tough. As though that was caused by the other database. FYI, mysql is the least ANSI-SQL compatible database in the market (especially so in 2000). Having to roll your own transactions, and getting by with mysql deliberately converting invalid data arbitrarily to work and then going to a standard solution would of course, be a nightmare.  Undeniably, oracle was (and still is) overpriced. Though it isn't tough to get discounts if you know what you're doing. DB2 and Informix are both cheaper and easier to work with, though they weren't VC darlings in 2000. And undeniably, you can write working software with mysql. But there are compatibility problems, data quality problems, and scalability problems. Maybe it's unfair to assume that this is obvious to junior programmers unfamiliar with database technology. Especially given the current hype around mysql.

9:35 PM

Anonymous said...

I've been using MySQL since 2000 to implement a community website (bicycle touring journals, now about 800 journals and over 60,000 pictures). The site gets somewhere in the region of 20-50,000 page requests (150,000 - 300,000 http reqs) per day. So it's not exactly a super busy site, but it does have quite a few active users. I've had the site up and running 24/7 (with some breaks for moves and hardware failure) for about six years now. Not once have I had any integrity or dataloss problems caused by MySQL. It's been really rock solid. All these people who say that using MySQL for anything significant is a "disaster in waiting" are simply blindly repeating religious dogma. That's what it is - religious. They believe that MySQL is crap, so no matter what other people demonstrate to the contrary (that it, uh, actually works, rather well in fact) they will still continue to twist reality around to suit their existing viewpoint. I don't really understand the outright hatred that exists for MySQL in the minds of some people. It's been working for me for six years now... how much evidence do you need before you accept that it really does work well? And as for the data integrity issues, lack of transactions etc... all I can say is this: Despite all the wailing and gnashing of teeth, the fact is that fixing data problems is really not all that hard. It's very occasional, and actually I haven't even seen any problems that were caused by MySQL itself - the biggest problems were caused by user error (mine, generally). This involves stuff like dropping the wrong table or field, or mistakenly deleting entries wholesale. Sorry, but arguing that "the database should stop you from making that kind of mistake" isn't valid at all. Any database that lets you change data will have the potential for user error, and that's the biggest source of data loss in my experience. Only on a couple of occasions has MySQL itself had a problem - and then it was just a corrupted index file, which was easily fixed. I'll take that kind of rock solid stability any day. Sure, MySQL could be better - I have wished for a while that it was able to utilize more than one index in a query. I think that is starting to happen. I'll be using MySQL for a while yet, I think.

Neil Gunton said...

Sorry, for some reason the post preview system reverted my identity on the previous msg to "anonymous" even though I had previously selected "other" along with my home page. Anyway, the community website I was referring to is crazyguyonabike.

9:26 AM

b7j0c said...

This post has been removed by a blog administrator.

10:11 PM

Anonymous said...

Rob said: What I don't understand is why modern huge database applications are written using the database engines that run on top of some universal OS? I mean, what else can be on that server except the database? So the ideal modern database should be the OS, possibly even without the file system as we know it. He hard-drive itself can be structured to suit the database needs so one would never have to rewrite any files, just the segment of the disk that was assigned to the particular data. The improvement is speed and reliability would be enormous.  The ability to create database "files" on raw disk slices/partitions has existed in Oracle for as long as I can remember (at least 13 years). This was a highly recommended way of installing Oracle some years back but modern Oracle documentation now downplays this method in favour of cooked file systems.

7:00 AM

Aminorex said...

I've been running dozens of websites offering commercial services, all backed by mysql, since 2001. I can attest that it is good enough and fast enough and plenty plenty cheap enough for everything I've ever needed in a database system. It scales on demand, with no incremental cost. Unless you have a very specialized need, anything else should draw the intense ire of your CFO, and maybe a shareholder lawsuit.

4:46 PM

Anonymous said...

Someone asked where the hatred of MySQL comes from? It started with the... I want to call them lies... that they continually repeated in their presentations, community websites, and documentation; like how transactions are not important, that ACID doesn't really matter, and that being strict about data integrity and features like foreign keys and views were only things that "purists" cared about. And the hatred grew as people watched them implement all these things over the last 5 years.   Sure you can make MySQL work, you can make GOTO work as well, but that doesn't change that 90%+ of MySQL users would be better served by choosing a different database.

12:35 PM

Paul Reilly said...

At the end of the day, MySQL WAS adopted by Google. OK, so additional transactional development was required but it worked, and more importantly, it worked more efficiently that the commercial counterpart! I see this attitude towards the adoption of commercial software Vs. the adoption of Open Source software.   Here within the corporate culture, where I work, I often see the same pro-commercial attitude. The key point here is that by adopting the commercial alternative, it provides a scape goat surely every corporate top director who answers to shareholders, surely needs to protect himself by having someone to sue in the event of a mission critical failure, a fine motivation to adopt a commercial database.

8:26 AM

Anonymous said...

A few thoughts about recent posts:  1. Why the hatred towards mysql? I don't think there is hatred, I think a few issues have really raised people's hackles: - hype vs reality - history of company for spreading worst-practices (don't use transactions or foreign keys, etc) As proof of the above - note that you almost never see so many criticisms of Postgresql - a database that is both freer and more standard than mysql.  2. It works great on my hobby website, it should be good enough for everyone: Ah, no. Go do a dozen business critical database projects - and for god's sake, use a few other products besides mysql to get a perspective. *Then* you're in a much better position to talk about mysql vs the rest of the options. In the meanwhile, you really have zero credibility on the subject of databases.  3. I've used it for years on my site and have never seen any data quality problems, therefore that just isn't a big deal: Really? How do you know you don't have data quality problems? Are you explicitly testing for them? Almost nobody does - they take it for granted that things are working right. The fact that they are only looking at a report with 95% of the data instead of 100% is quite difficult for them to know!  4. mysql is good enough for everyone, it's just the out of touch purists that don't like it: the people who say this are typically those with very little experience with databases, perhaps only with mysql. And they are complaining about the people with 10+ years of experience across a half-dozen products. These aren't purists - they're just experienced professionals.  5. I've been running dozens of mysql websites offering commercial services. Anything else is just a specialized application: if you're just supporting e-commerce packages for smallish companies at your isp you've touched on about 1% of what people use databases for. You haven't yet handled large volumes, 100% failover environments, reporting, complex queries, etc. It's great if it worked for you, but you really probably wouldn't notice the data corruption caused by silent data truncation/conversion, orphaned data allowed by lack of fk constraints, etc.   Sigh. The proponents of mysql really need to broaden their perspective by trying out a few other products and developing a few applications that have higher standards (not just performance, but also reliability & quality). Additional recommendation: work with people who have been doing this for a while and who can show you the many ways that data quality problems occur, how they can have unpredictable impacts on systems, and how to best prevent them.

10:36 AM

Anonymous said...

Sigh. The ignorance and lack of objectivity from commercial hucksters is disheartening. The commercial bigotry persists. The truth is many of those closed minds have their careers tied to a commercial product and have little or no knowledge of alternative products or solutions. The folk pushing commercial products need to ask themselves what features of a commercial product are 'really' needed in the majority of real world applications and STOP preaching vendor product marketing jargon and inventing FUD of open source.

5:53 PM

Neil Gunton said...

I thought I'd come back here to see what other people had posted since my prior comment. And, of course, there it is... the typical attitude of the PostgreSQL pusher, which can roughly be summarized as this: "You don't know what a database is, even if you think you've been using it for years to do non-trivial tasks, you don't really know what a real database is. There are certain things that any database MUST do, and if it doesn't do them then it is NOT a database and you should run along now and run your little hobby website. Leave the REAL database stuff to the professionals who know what they're doing". Wow! Talk about arrogance, not to mention obnoxious. It really shouldn't surprise anybody if I actually avoid PostgreSQL because of people like this. Also, I do hear from the odd person that actually tried shifting over to something like PostgreSQL that it was *very* slow. That little fact gets conveniently swept under the carpet by the fundamentalists. If you look at any discussion involving MySQL vs PostgreSQL over on slashdot (and it seems that any discussion that has to do with MySQL will bring out the PostgreSQL fanboys), then you'll see comments tucked away in the oblivion of "unmoderated hell" which talk about how difficult or slow PostgreSQL is in reality. I have learned to take the Open Source zealots with a huge pinch of salt these days. What you tend to see are comments that just keep repeating the same thing over and over - PostgreSQL is the only thing anybody should be using (even though it's actually still really slow)... or Mozilla/Firefox is the best thing since sliced bread (even though it was basically unusable on my old 450 MHz workstation, it was so slow, and moreover has had major bugs unfixed in the rendering engine for literally years)... seems like the whole Open Source crowd has a tendency toward the religious in terms of closed thinking. Something to do with a mixture of "us vs them" tribal attitude, with an added dose of dogma and hatred of anything that threatens their cosy little world view. You'll notice that these people won't acknowledge that MySQL is pretty good for many applications (the majority of database applications, actually). In fact, many of them won't even acknowledge that MySQL deserves to be called an RDBMS. Any post praising or supporting MySQL is rebuffed with a link to a "Gotchas" page listing irrelevant or out of date bugs or anomalies. Any post defending MySQL is picked apart, point by point - you can tell a zealot when they think you are *totally* wrong on *every* point, each of which has to be individually addressed, simply to show you just how *wrong* you are. No middle ground, no no no. Just "mysql totally sucks, PostgreSQL is reasonable and sane, anybody who says otherwise is delusional". Whatever. I'll just keep developing my "hobby site" quite happily, scaling it up as needed using MySQL. It works, thanks MySQL! Oh, and for the record: I'm sure PostgreSQL works just fine too. I'm sure you can tune it just so, so that it is ok speedwise. But a note for the PG fanboys: You don't win people over with overbearing, arrogant posts that say anybody who thinks that you can do quite a lot with MySQL is somehow less professional or just a hobbyist. I'm sorry to go on, but this kinda gets on my nerves after a while. It's like swatting annoying flies - every time someone mentions MySQL, these fanatics show up claiming that it can't possibly be used for anything. To which I reply: GET A LIFE MATE! Thanks.

10:41 PM

GoodBytes said...

I am coming a little late in the debate...  I wanted to congratulate you on this wonderful article and two excellent points:  "with free software, you get more than what you pay for", and  "from the end user's point of view free software is often much better than the producers of commercial software would like people to think".  I have used both MySQL and PostgreSQL, and have good opinion about them both. They have their good and bad sides, but you get to learn them in time and find workarounds. What's best, they are free and they are constantly improving with new versions.  Also, it is good to mention that not everyone needs a database for heavy-lifting applications. For simpler tasks, having one of these databases is priceless.  Thanks.

3:28 PM

Anonymous said...

I think someone made a comment a bit like that one. I've used both MySQL and Oracle and I must say that i'd rather work with an Oracle db that with MySQL. It has many advantages like having the "data integrity logic" directly in the database. This is priceless when two or more apps are accessing/writing to the database.  But, i would never use Oracle for an high number of concurrent sessions. By high number I mean as low as 1000. IMHO, Oracle for a complex and/or high data volume database and where transactions matters; MySQL for a simple but heavely queried database. Like the frontend-backend thing.  I still believe that you can achieve the same performance with Oracle or with MySQL. But with Oracle you need the knowledge and the resources to do so. And why would you need to put your hand in your pocket, MySQL is there to do just that (the performance thing). But I fear that while on the road of adding more and more "basic" features, MySQL will release a light version of itself.  js

8:22 PM

uri said...

Just wondering, when you talk about the "AdWords program" is this the one external customers work with, or something used for internal processing of ads etc.?  Thanks for the insightful story :)

6:06 AM

Andrei Lopatenko said...

Hard to believe. I certainly agree that MySQL may outperform untuned Oracle (sorry, I talk ONLY about database I am truly experienced with fort many years) for transaction-free database under 10 million records. I believe that AdWords, AdSense needs much large database to operate. Tuning of Oracle to real environment which depends on both hardware and software - RA memory, HDD, network, operating system, RAD is hard and time consuming but results maybe surpising. Performanse may increase scores times comparing to "default" untuned configuration. Proper programming like more functionality to PL/SQL then to Python may improve functionality up to 10 times in data intensive applications even for plain select statements without any database update functionality. I do understand that select processing is 99.9999% of time of functionality required from Google AdSense database, but remaining 0.0001% is not less important for such applications. Besides performance issues -- what is about fault recovery which includes transaction loging, backup and restore. If hardware and software fails, could Google restore it to the point just before fail or some users, customers, will loss their money, settngs etc What is about updates? MySQL is pretty efficient for SELECT, but not for updates especially for updates of indexed tables. How many update transactions (new AdWord customers? users? settings changes? clicks?) per day ar eperformed? Can MySQL handle them? If it can handle them will it handle in one year? Are updates done in parallel?  Again, let mt emphasize a point about perfortmance. It is hard to believe that good Oracle engineer can not tune Oracle to be faster then MySQL in case if you have a database more then 10 million tuples.  I am not again Open source and I do not sell Oracle Database :). I like Open Source, but my around 12-year experience proves that for large databases of mission critical data Open Source databases can not compete with Oracle.  I do not say anything about MS SQL or IBM DB2 for only the reason that I do not know how to tune them or how efficient they are for large databases.

12:04 PM

Andrei Lopatenko said...

But, i would never use Oracle for an high number of concurrent sessions. By high number I mean as low as 1000.  What do you mean "concurrent sessions"? User connections? You definitly do not need to use 1000 connection to run applications like AdSense 1000 parallel. My experience - Oracle is pretty efficient for parallel queries, especially if you try to apply cluster architecturw  about PostgresQL. It is reaLLy slow

12:08 PM

Anonymous said...

"To elaborate: some transaction benefits can be simulated "easily," such as logging each update to make sure Bad Things don't happen when you, say, charge a customer and the db goes down before the ad gets inserted. Although if single updates aren't even atomic -- and if your db doesn't support transactions, they probably aren't, which is the case for MySQL's in non-innodb tables -- you're still in trouble if you're running something like "update users set balance = balance - delta" when it goes down.  The harder ones, though, such as isolation, I've never seen done outside the database, and I wouldn't trust the code of anyone who claimed to do it. Much safer to use a database that's already debugged their code many times over."  ....  It's all code that makes it all work in the database. If the base goes out at the wrong time, no matter how many hundreds of safeguards happen, you're going to have a loss of data. All this advanced junk to prevent it is pretty meaningless, because all it does is move the point where a failure will cause a loss to a different spot in the program.

9:32 PM

Administrator said...

Thanks for your blog... http://best-casino-online.t35.com Best Online Casino

11:15 AM

Anonymous said...

Hmm,  After reading the mixed opinions, and very stern beliefs I found it rather difficult to not post a response. All in all the artical touts the success of Mysql Over a commercial database. This onto itself is a benchmark for success. I have been involved in a large number of commercial projects, including one of the worlds largest databases which happens to run on Mysql "EdgeHealth".  I have worked side by side with developers and dba's who have scratched their head at how well Mysql performs. Oracle, DB2, Postgres all seem to perform rather well until you compair them to Mysql. Although I do often enjoy the ability to extend postges with perl modules, and create objects for oracle in java.. The speed and reliability of Mysql cannot be denyed.  Consider the following, Cox, Google, IBM, Sun, Oracle, Microsoft, and a hoard of other companies deploy Mysql as thier database of choice for web and e-commerce applications, it speaks volumes when a vendor does not trust thier own products for production enviorments.  Mysql Postgres and Oracle will be around for years to come because they are good products. As for which database is the best.. thats just a matter of oppion. For the moment Mysql is the king of spead, and step child of features. My experiance says of the 3 Mysql wins out in deployment for the reasons, 1 performance, 2 scalability, 3 cost.   Don't forget products evovlve or die, so if all 3 of these products are still around in the next couple of years, I would expect them to evolve in the dirrection of what they don't have now.  Oracle and Postgres simply lack through put, Mysql lacks external scripting langauges. All of these products offer transactions, clustering, and high avalibility.  If Mysql ever adopts external scripting languages, will it still keep its speed? Only time will tell.

2:04 PM

Anonymous said...

Re: Database as OS  You are absolutely correct, the database server typically runs nothing else, and using the file system to store tables rather than a raw partition is inappropriate. The file system is tuned for small files; in the 1-8kB range. That's just what Unix normally holds.  OTOH, by keeping the databases as normal files, backup and manipulation don't require new tools; you just use mv and cp, like you would any other file (with the database turned off, natch).

5:58 PM

Anonymous said...

Speaking of SQL, I heard that no database actually implements the full feature set that was defined back in the 1970s.  I personally tried to write a python layer that implemented views for MySQL. It was quite a bit of work and I never finished it.  Does anyone else have their sense of aesthetics offended when they see SQL embedded in another language? Reminds me of inline assembler or something. There has to be a better way. I tried to invent an OO layer over MySQL in the views project, but it was very ambitious.

7:04 PM

Anonymous said...

"Real" simply doesn't mean expensive in the database case. PostgreSQL and Firebird are both just as free as MySQL, but are actually robust and reliable.

7:42 AM

SATURDAY, DECEMBER 10, 2005

Oops!

POSTED BY RON AT 8:57 AM

I accidentally posted a political rant on xooglers that I had meant to post to my personal blog. I have a BlogThis! button in my bookmarks, and I forgot that now that I have posting access to more than one blog I have to actually tell it which one I want to use. Sorry. It won't happen again.

(Interesting how this happened the morning after I wrote a xooglers enry about how I kept screwing up at Google.)

3 COMMENTS:

Tuor said...

actually, this may be a bug in blogger and not your own fault. I have observed it myself using the "blogthis" feature with multiple blogs. Sometimes it posts to my primary blog even when I have selected another blog in the select option.   I have probably filled out a bug report for this, but I have never heard back from blogger.com staff about any bug reports I have filed, so I don't know if they look at them.  (TIP: I have noticed that if the BlogThis window doesn't *blink* briefly, it has not updated and will post on the default blog instead of the blog you have selected)

11:29 PM

13TonGimp said...

I used to have tons of blogger related errors, so I switched to wordpress.  I have fewer problems now....

3:28 AM

Anonymous said...

My sites work fine 

SATURDAY, DECEMBER 10, 2005

Good things

POSTED BY RON AT 9:07 AM

Since my last posts painted a pretty gloomy picture I thought I'd devote a post to some of the good things that were happening at the time.

Despite all the stress I actually looked forward to my time at the 'plex. I didn't really have a life up there so I could really immerse myself in work, which I actually enjoy. I am an engineer at heart. I get a real kick out of watching something that I've built work. And the 'plex was the perfect environment, filled with smart, friendly people, state-of-the-art equipement, and half a dozen different kinds of coffee machines. (I actually had some interesting conversations about the finer points of espresso making, and at one point someone even brought in a book that was entirely devoted to the topic.) I learned a lot, more I think in my one year there than in the previous five.

As I said, I didn't really have a life, so I almost always stayed late (except when I felt like making the odd pilgrimage to Fry's). Charlie served dinner every evening around 5:00 or so, and there were always about twenty or thirty people who showed up for that. Late at night when there weren't people running around I'd try to learn to ride Sergey's unicycle (I'm pleased to report that neither I nor any of my colleagues sustained any permanent injuries), or work off some stress by riding one of the Zappys around the 'plex at full speed. (Nowadays they have Segways!)

One week I got to go on the annual Google ski trip. The year before it had been only a couple of dozen people. The year I went we had 150 or so, and filled three or four busses. It was a really strange week. I flew up from LA, got on a bus, rode to Tahoe, went skiing, got bussed back to Mountain View, and got back on a plane to LA. And I got paid for it!

But the best part of being at Google was the sense of being part of history. At the time no one had any idea how big Google would get, but I had been a computer geek all my life, and the Silicon Valley was always the Promised Land. I had wanted to be a part of it ever since I touched my first computer, a Sol-20. I had done a couple of short stints as a co-op employee at IBM (the Cottle road plant), and visited a lot (my first wife's family lived in Los Gatos), but this was the first time I was really a part of it. I even got a certain perverse thrill out of the insane commute. I would look around at my fellow passengers and think to myself that these were the movers and shakers. There were always Cool Things going on in the Valley, and these were the people making them happen. And I was one of them. I was finally living my lifelong dream. It felt great.

In the main, it felt great a lot more than it felt crappy. But the crappy parts make much better stories.

21 COMMENTS:

Tavarius J. Taylor said...

Wow, so you really had no idea of the sucecss to come for Goog? By the way I really enjoyed your posts.

9:44 AM

Anonymous said...

Hi,  I'm a mother of a Googler. I just wanted to say that I love your blog. Keep it up!

7:02 PM

HERE IS ANOTHER "GOOD THING"  Just been added to Slashdot today - your will get thousands of visitors during the next hours  http://slashdot.org/article.pl?sid=05/12/10/2125204&tid=217&tid=95

7:36 PM

Anonymous said...

I think its an awesome right. Interesting perspective but I disagree that all computer geeks dream of working in Silicon Valley. These days with the cost of living I think that is less true then ever. Especially since computers are so mainstream now and companies are moving about the country like other industries.

8:39 PM

Anonymous said...

Keep up the great blogging! I love these stories!

8:47 PM

Anonymous said...

This place reminds me of my time at Disney. Good luck with the blog. ^_^

9:19 PM

Anonymous said...

This blog is shit. You complain too much. Get over it, move on with your life.

9:24 PM

Anonymous said...

your post is shit, you criticize too much. Stop criticizing other peoples blogs and get on with your life.

9:28 PM

Anonymous said...

Why do people leave Google? It seems that some people after just one year. I would stay for at least 10. Are you working contractually?

9:39 PM

sickboy said...

These are an awesome inside view of google, I have always wanted to work for a company like google, kinda helps you see through the stuff everyone talks about to the reality.  http://redsoxunixgeek.blogspot.com

9:40 PM

Anonymous said...

Psst...Getting a job at Google isn't so hard if you don't mind working as a contractor for awhile and thinking outside of the valley. Trust me!

10:36 PM

Anonymous said...

I worked for the IBM on Cottle Rd for most of 2000. When were you there?

10:36 PM

Ron said...

I worked for the IBM on Cottle Rd for most of 2000. When were you there?  The summers of 1983 and 1984.

12:35 AM

Anonymous said...

"equipEment"?  Oh my, Google hires illiterate people. yuk.

2:18 AM

Anonymous said...

In general, and not a comment about the poster. But why does every web hack call themselves an engineer? I remember having to take a state PE cert then be required to work in the field for several years before taking the engineering exam. Fields had to be recognized engineering fields. Maybe the word has been regulated *joke* to the same professions as rocket scientist and brain surgeon.

7:15 AM

Anonymous said...

this is very funny  Google BADSENSE!  http://www.worldbankinginfo.com/credit.htm  http://img506.imageshack.us/img506/1133/badsense9ga.jpg

7:19 AM

Luke said...

I am a professional barista by day so of course I have to ask: which machines did you like using best and which book were you handed?  Thanks for the interesting read!

12:47 PM

Anonymous said...

Ron,  You are a true Lisper! Thanks for the awesome read-macro! That's awesome.  I hope you can find work where you can hack lisp and be happy.   And thanks for putting the transactions-must-be-in-the-db fool in his place. As if a DB guy, working in C/C++, is going to do manage the concurrency better than you guys. That Paul Graham reference was awesome!

1:06 PM

Orodemniades said...

I'm merely a fellow blogger (with a BA in Eng Lit!) but this is without doubt one of the best blogs I've come across.   Funny, I feel like I'm a part of history merely reading this blog.

3:47 PM

Anonymous said...

Hey Doug and Ron,  Just wanted to ;et you know that I found your blog on SlashDot Google feed.  I use gmail, Adwords, Adsence, Site Map and a few others. My home page has been google for years.  I have been using the Personalization feature and have added your Blog to 'Create a section'. However if you guys have any friends on this project tell them there is an anyoying bug. Everytime the page updates it forces 'always on top' to the browser I have it in. Seeing I am an avid user of the right click and open in new window this always seems to happen when I am typing in a form or something and screws me up. I am almost to the point of just going back to the old google home page but I figured someone would figure this out sooner or later.  Lastly, I work in the internet industry and work smack dab in the middle of Marketing and R&D so I can relate to both of your stories. I also always wondered how the "big boys" did things and it is nice to see my shop has a lot of the same things going on and the grass isn't greener...

SJ said...

Nice post. I would like wokr for Google or bettre yet start the next Google. I am tryign to see if I can pick any lessons from here.

12:21 AM

THURSDAY, DECEMBER 15, 2005

What he said

POSTED BY RON AT 12:06 AM

Funny, I was finally getting around to writing a sorry-I've-been-away-it's-these-damned-holidays-will-probably-be-gone-a-while-longer entry but Doug beat me to it. I'm probably going to be out of commission until Jan 1.

I'd like to send a great big thank you to all of you who have left supportive comments. Aw heck, it's Christmas -- thanks to everyone whether you've been supportive or not. Just knowing that someone is reading all this drivel that I'm writing makes it all worthwhile.

I wish you all a happy holiday season and a healthy and peaceful new year. Merry Christmas, happy Hanukka, splendid Solstice, stupendous Saturnalia, kick-ass Kwanzaa, a fun fabulous fortuitous and fortune-filled Festivus, and from the better-late-than-never department, a delightful Diwali.

Now stop surfing the net and go drink some egg nog!

42 COMMENTS:

AG said...

Excellent start guys. You both blog in an entertaining way which keeps me coming back for more. Please keep up the good work as you have a lot of fans now. AG

12:54 AM

la lara said...

Then call when you're back

3:48 AM

Matthew said...

Ya Salams!  here here, I too have been irrisitably drawn to return to glean more & more details about the conciousnesses involved in the big G juggernaut..... !  & don't forget 'eid al adha' for all Muslims just after Xmas!  God Bless!  MT

5:32 AM

Dave said...

and a happy Agnostica.

5:43 AM

Chrissie said...

I hope you have a happy holidays as well.

11:15 AM

Anonymous said...

No shout out to all us Muslims and Eid ul-Adha? Sure its in early January this year but still its almost the holiday season!

5:18 PM

Kiran said...

been going thro ur posts for some time now. its been interesting.  and abt Diwali it happened in Nov itself but anyways thanks and wish u all a very happy X'mas and New year.  Kiran

8:37 AM

jeffysspot said...

yah .....what you said

10:03 AM

Anonymous said...

Thanks for great blogg.

1:34 PM

Ron said...

No shout out to all us Muslims and Eid ul-Adha?  An unintentional oversight. My bad.  An excellent Eid ul-Adha to all!

3:30 PM

Noctis said...

This post has been removed by a blog administrator.

7:09 PM

Sports Bettor said...

Keep writing. Happy Holidays.

9:06 PM

Anonymous said...

i'll be watching my RSS =)  have a good holiday!

11:54 PM

Samer said...

great stuff! Happy holidays...

1:03 PM

Sam said...

Nice job... wonder why you call your blog, Xooglers? But I am just starting to type my own ideas and stuff on mine...

6:15 PM

omm said...

I am one of your big fans too. Merry Christmas to you too and thanks for the nice thoughts :-)

1:17 AM

Archosidf said...

one of the most fascinating things I've read on the web in the last months

9:19 AM

Dunzo said...

I'll stop drinking egg nog and start surfind the web right now.

10:16 AM

Anonymous said...

Here is a blurb from the 12/20 WSJ,  "Mr. Kordestani oversaw an important shift in advertising models at Google in 2003. Google moved from charging advertisers based on how many consumers saw their ads to its model of billing them only for users who clicked on their ads. That involved combining several sales databases -- an initiative known as "Project Drano" -- which Mr. Schmidt says "nearly killed" Google. Phasing out the old ad system entailed changing the incentive and commission structure for Google's sales staff -- a treacherous task, which Mr. Kordestani supervised."  Any juicy tidbits about this project or Mr K?

10:49 PM

Anonymous said...

Happy Holidays regardless what you celebrate!

10:37 AM

Anonymous said...

I'm curios about the relationship between Engineering and Product Managment. Was it cooperative? How was the PM group viewed internally? Did they fall under the 70/20/10 rule? What did they spend their time doing? How big was the team?

4:58 PM

JK & PK said...

Dude, you deserve some eggnog (you can have mine too, i think it tastes like crap)

4:59 PM

Anonymous said...

Diwali is already over in November in India .  Happy X mas to you.

6:38 AM

Rufus the Great said...

I love your blog! Please check out mine, http://placeofrufus.blogspot.com/

8:56 AM

Jennas Blog said...

I'm new to blogging, and internet business. So far, I've found google to be difficult and Yahoo much more user friendly.

7:04 PM

chuck said...

IT'S OFF TO EGGNOG I GO...

9:38 PM

Stenar said...

G-ZUS LOVES YOU

11:27 PM

rafiq said...

The New Google doodle page has an error on it.   http://www.rafiq.za.net/2005/12/24/google-holiday-doodle-page-has-an-error-on-it/

12:52 AM

rafiq said...

Wow! that was fast. Luckily I took a screenshot as proof (available at my blog)

1:19 AM

underpanter said...

you are hot, wanna link to or from my site?

3:55 PM

localudal said...

Merry Googlemas!  Even as the blog is dedicated to the Google past, here's something for the ominous Google future (just one excerpt from my blog http://goolocalizations.blogspot.com):  GOOGLEVISION  One Google opening for i18n specialist requires a candidate to possess a "vision". You want a vision, here you go.  In 1993-1994 I worked on Nostradamus TV script translation, and it was me who said that the scene of a global war of West against Islam must be dated 2001, not 2000, as faulty translation of Nostradamus' quatrains suggested. Check it out in Mosfilm archives, or in some Hollywood script hangars.  Here's what happens in the next decade from now:  1. Google buys Australia, Starbucks, Moffett Field, Antarctica, some parts of California -- from a weakening grip of Governator. Google finances the digging of the tunnel and the pipeline between Eurasian Empire of Czar Putin I and Alaskan Dominion.  2. On Moffett Field, Google builds the Babelfish Tower out of millions of nanocarbon tethers delivering food, water, replacement nuts and bolts to ISS by dozen molecules at a time. Ancient flotilla of space shuttles is grounded and converted into Starbucks restaurants.  3. Google buys CNN, FoxNews, replaces their field force by 360 deg Webcams put on every phone pole in US and in major critical points around the globe. The new service is called Google News, and its content, zoom, direction and choice of commenting blogs can be tailored by user's preferences. Hollywood movies can be watched in raw footage, or pre-director's cuts. Effects, actors and scripts can be edited by a viewer.  4. Cash is virtually abolished and used only in illegitimate trade of drugs, firearms, in presidential campains and in live prostitution. Numismatists grumble over cash substitute called AdCents: an ancient $2 bill is valued at 200 AdCents at GoogleBay: 1 AdCent is equal to 1 million of AdSense clicks. As the result, Google Cache replaces Fort Knox. Google's motto 'Don't Be Evil' now reads: 'Don't Be!" Accordingly, the final solution to private information and its theft is brought about: except for the name, approximate date of birth, and fingerprints there's nothing could be found on any individual. Legacy data like account numbers, PINs, SSN, Apt./House Numbers, etc., are known only to Google Index and corresponding agencies.  5. Google trades McDonalds' back to Canada for Québec. Resulting huge reserves of frying oil, with Persian Gulf's oil being almost finished, are used in mix with corn schnapps to fuel 4-wheeled segways, or SUVs, as in Segway Utility Vehicle. Ford leads the pack of auto manufacturers with a nostalgically smelling Segway-T model capable of 1 mpg.  6. Googlers (see section below) are routinely shot off the skies over China and Northern Korea by ancient DoS ACKACKers. Google retaliates by using Mandarin over the Continental China. Meanwhile, with Google's backing Al Gore, the old Internet inventor, buys EU Internet back from European rebels. A new resulting Internet is called Google Index.  7. Google buys LOC, NYPL, SJPL, Stanford Library, and copyright debates cease with an acceptance of Copygoogle Law. Google buys Wikipedia and Britannica, publishes the resulting Brikiwikipedia in 100 set in Katull paper volumes of 5000 pages each. To enhance readability of such a tome, Googlese it is printed in (see below) has lost many useless letters and signs like 'c', 'j', 'q', 'v', 'w', 'y', 'z', '$', comma, colon, semicolon, ellipsis, dashes, etc. Sites insisting on using old, or so called correct English, are progressively excluded from Google Index.   8. Google launches Goobble, a space telescope armed with hurricane battling laser. First fire test was partially successful: the Omega hurricane was dissipated over Cancún, Transtexas, which was severely fried in the process. Goops!  9. Google's AdSense program is challenged by AbSence ideology by Teoma, the most used search engine of 2015. Teoma's success is based on abolishing spamming and sponsored links in its search results.  10. At Googleplex and next to Vint Cerf's, Google unveils a bronze monument of me, the best Google Borshcht Chef whose after-borshcht burp renders a better "Goooooooogle" sound out of majority of contest jury for 10th year in a row.

11:28 AM

Shaun said...

Awesome blog :)  "Happy HOlidays"  Shaun - ohpunk.blogspot.com

3:59 PM

JR said...

Merry 2 miutes afta christmas!!!    comment to me at hairston.blogspot.com

12:03 AM

JR said...

u guyz no how to make a good blog!

12:05 AM

Rufus the Great said...

jr, I cannot understand you. And with my superior intellect, you would think I could understand anyone. What is wrong with you?

6:49 AM

Mel Logan said...

happy holidays, and thanks for the good reading material.

12:46 PM

Miyagi said...

Great blog dude!

10:18 PM

Anonymous said...

Egg Nog rocks  Signature: Get $25 free on first deposit at the world's largest poker room with 70,000+ players http://www.partypoker.com use code: shefa

10:27 AM

todd said...

wow you have a lot of fans i just read this blog. i like it. i'm a fan of courtesy. your a writer thats cool i want to be a writer. i noticed you did the whole naming of every holiday to be politicaly correct. just say happy holidays. or merry christmas. reading your blog i can tell your smart. i didn't get bored reading your blog like i do with others. you kept me interested.

10:35 AM

frank white said...

hey hey now. i like your blog. please explain to a google browser search user what xoogler is all about? what does the word mean? is there a site? maybe a program? hey kiran where are u from dude? you need to wear slightly loose brand of jeans. those balls need to breath bro. by the way i can see that you have exactly 2 dimes and 4 pennies in your front right pant pocket

8:10 PM

JC said...

You left out Agnostica!

2:46 PM

Harry said...

And may the force be with you :p

4:51 PM

MONDAY, DECEMBER 26, 2005

We interrupt this vacation...

POSTED BY RON AT 4:13 PM

... to bring you an important message.

I had a little down time today and so I decided to hook up my laptop and check my email.

Big mistake.

It seems that Xooglers has attracted the attention of all sorts of people who think that because I used to work at Google I now know Everything there is to know about Anything. I try to be helpful when I can, but listen up folks: it's been nearly five years since I worked there. Everything I know about the current state of the company I know from reading the papers (or Bob Cringely). I cannot offer you any particular insight about how to get a job at Google, or how to prepare for an interview, or how to get better tech support for your blogger account.

But as long as I've gone to the trouble of getting on my soap box, here's a bit of advice I can offer: when sending a cold-call email, a little common courtesy can go a long way. Introduce yourself. (A simple "Hello, my name is..." is fine.) Tell the person you are writing to how you found them. Just because you found someone through (say) their blog doesn't mean everyone has. Tell them a little bit about yourself and what you are doing that leads you to ask whatever it is that you are asking. And finally, you can get a surprising amount of leverage out of the odd "please" and "thank you." (But don't overdo it. Being demanding isn't cool, but neither is groveling.)

And for God's sake, never ever send a cold-call email from a mail account with a self-aggrandizing alias like "Lord and Master of the Known Universe." You may just as well tattoo the word "dweeb" on your forehead if you're going to do [expletive deleted] like that.

We now return you to our regularly scheduled intra-holiday lull.

34 COMMENTS:

Anonymous said...

More eggnog!

5:09 PM

Captured Intrigue said...

Hahaha....  Signed,  Princess of The Dark Realm Of Pixies That Hunt People Who Worked For Google But Are Expected To Know It All Still.  That is like asking somebody who used to work for McDonald's 7 years ago what this week's prices are for a chicken nugget meal.

5:38 PM

SJ said...

Hey you, get me a job at Google ok. And make it fast.   :))  Happy holidays!!

7:30 PM

NN said...

Hi, i'm am errr... a student of business. i read you blog because it gives a lot of insight on how google works/worked. i don't remember where i came from. but i come here often :D -me  PS: can please make my professors stop talkign about google!

2:16 AM

Pasamio the Geek said...

Hello, my name is Pasamio. I found your blog when it was on slashdot and had a slow and gentle read through the posts that the two of you have made here.   I'm a geek and I'm wondering if calling myself 'Pasamio the Geek' is too dweeb like.   I await your answers! :p  On a more serious note, very interesting historic read from a time that I often wish I was a part of, but didn't get the chance. Season Greetings guys, thanks for providing more information for the world to digest!

8:12 AM

Anonymous said...

/haha  can u teach how hack? source code plz.  send to lordofall@hotmail.com

8:15 AM

Anonymous said...

umm ok why is whereever I go, google is there!! ?? !! ah

9:45 AM

Anonymous said...

HOW CAN I GET A JOB AT GOOGLE??? I WANT A JOB AT GOOGLE> GET ME ONE NOW> hahahahaha

9:46 AM

Anonymous said...

A little courtesy goes a long way. Great Blog...Thanks  Signature: Get $25 free on first deposit at the world's largest poker room with 70,000+ players http://www.partypoker.com use code: shefa

10:26 AM

Dr. Ægustüs Ŗiviŋiallli said...

What a glorious place the World Wide Internet is! In this vast "web" of information, we find expertise in the most mundane of areas, such as searching for jobs! Oh, the joy of this cornucopia of knowledge!

12:33 PM

Talha Masood said...

:)  interesting  thank You!

5:03 PM

Anonymous said...

Can you blog about the growth currently at Google? Your thoughts? Why does this company function as a institution rather a business? Is Google the replica of Stanford but in Search form?

9:22 PM

Trish said...

Hello lord and master my name is nobody, any idea how many hits you get on google when you type in buffalo tack glass and why?

6:52 AM

khaled said...

SJ  me too!!!!!!

7:21 AM

Steve said...

Surely if you were the best and the brightest and worked at el Goog you knew everything at that time, and therefore still know everything now? And I DEMAND you tell me EVERYTHING! (As I understand, CAPITALISING some WORDS is likely to get you a FASTER response).

10:02 AM

Writer on Board said...

Trish says her name is Nobody. I'm Nobody too. Will the real NOBODY please stand up. What happens when you type NOBODY into GOOGLE?   Tell me Jason.   Tell me Mr. Google-Man, Mr. Googl-ator, Senor Goog-alito! Tell me!   Sorry. I'm having a really hard day. Good Blog.  -Jason Nobody

3:37 PM

Silyth said...

Sometimes I think a hint is lost on others since you state clearly your response on some things and people just come back asking the same things. Oh well, how tedious it must be for you. As you mentioned before introductions are nice so...hello I'm Silyth. It's a pleasure reading about what you want type. Now I want to get back to reading and I wish you a good holiday. Thanks for the good read!  Silyth http://silyth.blogspot.com

6:28 PM

W. said...

I am a Berkeley educated Social Scientist. I can tell you how to get a job at Google. I have cold emailed Google in the past to evaluate how they respond to various personalities. As well, I have been inside the mind of some of their stars. LOL  Ask yourself if you want to have some guy who is socially dysfuctional say things like "shit" to you because he got some false confidence after his equity in the company grew. I would rather work as a park ranger with people that actually are not so insecure.

1:15 PM

Anonymous said...

I don't mind reading interesting stuff from someone even if they are potentially socially dysfunctional or use words like "shit". Not to imply that the former is the case with xooglers, I can't see why someone would even care.

1:45 PM

submergent said...

I know what you mean. Once people find out I once bagged groceries for a couple years out of high schools, it's non stop questions about spacial relationships and weight ratios. It's a curse.

4:49 PM

Sarah Khokhar said...

My neighbours think I can solve all their artistic needs just cuz I learned most of the Adobe stuff in class. All I really know is how to crop an image.

9:49 PM

samuel said...

I kept reading, waiting for the guy with the problem with a word. That shit is fucking weak! It was actually kind of a let down when I did finally read it. I was ready for moralizing and I got some dimwit that ran out of real things to bitch about. So is it true that you know the HR guy at Google? email me at drunkanddisorderly.com Dude, totally hook me up!

12:26 AM

speed_demon said...

Hi there,  No offense, but how do you feel when google went IPO?  Man, working at google must be great. It must have felt like "walking among the giants"!  Happy New Year!

3:22 AM

The Gradster said...

This is really stupid... http://toypalace.blogspot.com/ go there!!!!!!

7:00 AM

remcgregor said...

Just wanted to comment on what a great Blog you have here. Google as a company has always fascinated me. Looking forward to when you guys are back and blogging again! :)

4:24 AM

IanShorten said...

Google is a sweet company. I've used google for so fucking long and so naturally when Gmail comes out I flipped. Wish they would make a win 98 compatible google talk.

1:18 AM

Anonymous said...

Isn't blogger owned by google?

1:24 AM

The Urban Kunoichi said...

Hello there,  I began to read your blog since it was in the featured list at blogger.com.   Great blog!  But I have to admit, I get as much enjoyment reading your comments as I do from your own words!  Thanks for the entertainment!

5:26 PM

Anonymous said...

To the "Berkley educated Social Scientist" who could probably tell everyone a little bit about everything: A really big ego is often a sign of insecurity. So, you seem to have earned your false confidence at Berkley.  You probably couldn't fit your ego into a park ranger's shirt.

10:12 PM

Kimberly said...

Lord and Master,  Please please please please please please please please please please please please please please please please ...ignore this post.  Thank you, thank you thank you thank you thank you thank you thank you thank you thank you thank you.

4:03 AM

b said...

please can u give us ignorant folks some fast fast search advice.thank u in advance

1:13 PM

Anonymous said...

Jokes.

Harsha said...

That is funny..

4:52 PM

Adbloggers.com said...

I love this blog. I check it often for updates. Keep blogging!  Ron Wilson  www.adbloggers.com

6:38 PM

WEDNESDAY, JANUARY 04, 2006

We are experiencing technical difficulties

POSTED BY RON AT 9:47 AM

Happy new year everyone!

If you've been watching my personal blog you will have seen that I've actually been back for a while now. The reason I've been quiet on xooglers is that Google legal has asked to talk to me about it (no big deal I am assured) and we've been having trouble making contact.

[UPDATE: I was hoping this would just pass, but I guess I'd better clarify before it spins wildly out of control: Google has not asked me to stop blogging. They have not even remotely hinted anything of the sort. All they have done is ask to talk to me, and I have entirely on my own initiative stopped writing Google stories as a courtesy to them until we're able to make contact and I hear what they have to say. So I'm sorry to disappoint all you conspiracy theorists looking for a juicy tidbit, but there is nothing more sinister going on here than a game of phone tag.]

[UPDATE2: We finally made contact. The conversation lasted all of five minutes, and the message was basically, "We think Xooglers is great, but please be careful not to reveal any confidential information, and if you have any doubts about what might be confidential please feel free to call us and ask." No big deal. So I should be picking up the story again later today.]

So in the interim I thought I'd tell a story that Google can't possibly object to because it happened in 1997, a year before there was a Google.

Doubtless what fascinates many people about Google is the vast amount of money it has generated. Larry and Sergey are now billionaires. They've bought a 767 (which is not quite the splurge it might at first glance appear to be). What is it like to have all that money?

Well, I have no idea what it is like to have that kind of money, though I can report from firsthand experience that having some money is awfully nice. But it's no panacea. Money can let you eliminate some of life's headaches, but it comes with some of its own. Don't get me wrong, I definitely place these in the category of "nice problems to have", but they are nonetheless problems.

My most direct encounter with "that" kind of money came in 1997 when I attended a technical conference in Santa Barbara. Because SB is pretty close to where I live and I'm a pilot, I decided to fly myself there. At the time I was flying an old Cessna 182RG, which is a pre-dark-days four-seater with retractable gear (hence the "RG". No, it's just a coincidence that my new initials are RG. The plane I'm flying now is a Cirrus SR22, which has fixed gear.)

(As long as I'm digressing I guess I should explain that the "dark days" were the period in the late eighties to early nineties when no small planes were being manufactured in the U.S. because of liability concerns. At the time, common law held that an airplane manufacturer was liable for manufacturing defects in the airframe no matter how old the airplane was. In 1994 Congress passed the General Aviation Revitalization Act which limits the liability of aircraft and parts to 17 years after the date of manufacture. 17 years still seems like a long time to me, but apparently it's enough to let you make a business case, and since then the U.S. aircraft industry has undergone a renaissance.)

Anyway, it turned out that I was not the only one who had flown their own plane to the conference. There was a very highly ranked executive of a major software company there as well. I won't say who it was or what company he was with. I'll just call him "Chuck." I had actually met Chuck several years earlier when I interviewed for a job (which I didn't get) so I re-introduced myself and we started chatting. It turned out that both our planes were parked at the same place at Santa Barbara airport, so I offered to give him a ride to the airport when the conference was over, an offer which he declined.

So the conference ends, I get in my little rental car and start to drive out of the parking lot when here comes Chuck running after me waving his arms. It turns out that his ride to the airport hasn't shown up and can he take me up on my offer after all? Sure, I say, hop on in, which he does. He then goes on to make himself comfortable by taking off his shoes and putting his feet up on the dash. OK, whatever.

We get to the airport and we both walk into the FBO. My plane is off in the boonies, but his is parked right out front. It's a Falcon 2000, pretty much top of the line in its day (about $30M worth of plane I believe). The red carpet is rolled out (literally) and a little army of crew people are running around getting the thing ready. I pick my jaw up off the tarmac and wish Chuck a pleasant journey.

I walked over to my plane, which seemed very, er, cozy at this point, and started preflighting. In the distance I can hear the Falcon spooling up its engines, and then spooling them down again. Then once more, up and down, and again. And again.

Hm, that's odd, I think. Why don't they just leave?

Then they shut the engines down and open the door. Hm. Doesn't look good.

I go back inside the FBO and there's Chuck sitting on the couch looking very unhappy. "Disaster," he says to me. One of the engines has a broken part. They can't take off. And he has to be in Europe in eighteen hours for a wedding.

Since he had no where to go, Chuck invited me for a tour of his plane. It was a pretty sweet ride. Overstuffed leather upholstery, a nicely stocked bar, I think there was even a shower in back. But the part that really got my attention was in front: it was the first time I'd ever seen a glass cockpit up close.

Being in a plane like that without being able to go flying is kind of like being in a candy store without being able to eat anything, so I hung out for a few minutes, then thanked him for the tour, and took my leave. I offered to fly him to LAX where he could catch a commercial flight to Europe but he declined, which was probably the right thing to do. Flying in to LAX is theoretically possible in a small plane (I've actually flown to JFK in a four-seater once), but it's not something you'd want to do lightly, and at the time I was probably not experienced enough to do it right.

So I flew home. It was a perfectly clear night, and the city lights were spectacular. I marveled at the fact that I was going home while behind me a man worth many billions of dollars was stuck at the airport like an ordinary shmoe.

Postscript: thinking back on it now, I have no idea why he didn't just call a charter company and arrange for another plane to come pick him up. For a few hundred thousand (which should have been chump change by this guy's standards) you can have your pick of the fleet at your beck and call. Maybe if I ever run into Chuck again I'll have a chance to ask him.

22 COMMENTS:

EmailHosting.com said...

What is a xoogler?

6:25 PM

That is sad that there are potential concerns about the blog express by Google legal - they must understand the effect Google is and has had on the planet - and Google history should be more than press releases and PR speeches given at keynotes....Good Luck  By the way, read Matts blog about the new BigDaddy

6:29 PM

Penguin said...

are Xooglers , former employees of google?

7:25 PM

Anonymous said...

I wonder - was that Larry Ellison, or Paul Allen ? Hard to think of many people with that kind of money and interest.

7:48 PM

tby said...

In my experience, people that have achieved wealth (as opposed to being born into it) sweat the little things. $30M plane, no problem. $12K for last-minute first-class airfare to Europe, problem.  I knew a guy that sold the company he co-founded and walked away with nine figures. He got the Ferrari, the sailboat, a small island with a log cabin of McMansion proportions in Canada. He made his wife go back to dial-up because the ISDN bill was a couple hundred bucks one month.

8:43 PM

Anonymous said...

the glegal stuff is lame! this blog has a following. are they stalling on purpose? anyway, i say give them another few days to say whatever they need to say, and then start writing again.

Meme chose said...

Ha! My friends, who owned two planes, used to say: "If you really have more than enough time, take your own plane".

10:27 PM

Anonymous said...

I hope the "little talk" with Google isn't going to disrupt this blog.   Keep it up, I love reading it :)  TF

12:05 AM

rafiq said...

A few more clicks and reading would tell you what xooglers are.   The Xooglers blog is one of the best blogs I've ever come accross. Legal Shmegal, I think we should have the right to know what the guys over at the big G are up to seeing that they know what we are up to if we use their services, gmail, personalised homapges blah blah. Who know our lives online are are up to. Welcome back & Keep it up.  Instead of asking I will google for your personal blog or just check this one if it is somehwere here.  off topic: If friends or family ever use you notebook/workstation/tablet or pc be sure to be logged into your google account :) I wonder if there is a study comparing your personality to how & what you search for?

12:28 AM

Anonymous said...

My guess would be Charles Simonyi

1:06 AM

Tim said...

Yeah, I'm going for Charles Simonyi too. Doubt it was Paul Allen - I've met both and Charles is more sociable than Paul. It didn't sound like Paul. Well, he said something, for a start :)

3:44 AM

Anonymous said...

Keep up the great posts! Please keep us updated on the legal stuff. We may not be lawyers, but I am guessing your dedicated readers can help make some noise if you need it.

8:12 AM

TwistedNoggin said...

The little planes are SO much more fun to ride in. I wish I could pilot. Sorry about the Google complications. Hope that all works out well. :)

9:39 AM

Anonymous said...

Interesting that you'd choose to call me "Chuck". Heh.

10:10 AM

Ron said...

Interesting that you'd choose to call me "Chuck".  Well, I didn't want to have to keep referring to you as "this person" so I had to call you something, and I figured you wouldn't want me to use your real name. If you prefer I used a different name contact me by email and let me know and I'll change it.  (For the record, I have no reason to believe that this particular anonymous really is Chuck, but you never know.)

10:22 AM

Ron said...

This post has been removed by a blog administrator.

10:42 AM

Ron said...

This post has been removed by a blog administrator.

10:43 AM

Ron said...

OK, I can hear the conspiracy nuts going to work on the two deleted comments above. What happened was that I was trying to respond to twistednoggin who wrote "I wish I could [be a] pilot" by saying but for some reason the link disappeared and the result looked really weird. Hopefully this more verbose version will work...

10:48 AM

Ron said...

Wow, something really seems to be broken here!

10:49 AM

Anonymous said...

Are you implying that the owner of the plane was a Google employee? And that the fact that the plane did not work is your way of suggesting that Google does not work? You better be careful there Mr. Blogger.

2:07 PM

Bernoulli said...

Kind of off topic but...... does anyone know about the new IPv6 and do you think Google is looking at this? Heres the link that made me curious. http://www.imakenews.com/signal/e_article000497585.cfm?x=b11,0,w

12:02 PM

Anonymous said...

I do have the best guess of who Chuck is. It was Elon Musk former paypal's largest shareholder right? I know he owns a falcon 2000. His spacex rocket's name is also falcon.. He is not a billionaire though, I think

10:24 PM

THURSDAY, JANUARY 05, 2006

Lost in translations

POSTED BY RON AT 9:38 PM

In December of 2000 I had my first performance review. I don't really remember it all that well. Both my manager and I were keenly aware that on the one hand I wasn't performing very well, but on the other hand the project that I was nominally in charge of had launched on time and was a huge success. AdWords was starting to bring in a significant portion of the company's revenue.

There was something else had started to notice in the weeks leading up to the performance review: I wasn't being asked to do interviews any more. Interviewing was widely viewed as a chore. It was time-consuming and disruptive, but the company was growing, and someone had to vet the new hires. Personally I enjoyed doing interviews, probably because they afforded be a break from writing Java code. No one ever said anything to me about it, but after about September or October I noticed that I wasn't being put on the interview roster any more.

I remember only two significant things from that first review. First, I didn't get a raise, which is customary (or at least so I'm told) unless you've really screwed up. I suppose that was fair enough; I would be the first to admit that I'd really screwed up. But the second thing floored me: I was being kicked out of the ads group.

I was furious, though I tried hard to cling to the tattered remains of my professionalism and not to show it. Three months earlier I had offered to resign from the ads group and my offer had been refused, and now they were kicking me out.

I spent that winter vacation in a deep funk not sure at all what to do. I was getting so many mixed signals. On the one hand I had twice tried to walk away from the situation with some honor intact and twice I'd been refused. But on the other hand I was getting so many signals that the company really wanted to be rid of me: no raise, no interviews, and to top it off, getting kicked ignominiously out of the group.

I cannot for the life of me remember where I was between Christmas and New Years of 2000.

I remember my wife and I having an earnest discussion about whether I should resign. The situation seemed to be getting untenable, but I'd already invested six months, and it was starting to look as if Google was going to do reasonably well, and that the stock might actually be worth something some day. I didn't really want to walk away from that. So in January of 2001 I once again got up at 5 AM to catch the 7:25 Southwest flight from Burbank to San Jose.

Oddly, one of the things I remember most clearly from this time was this big patch of impatiens that were growing in a flower bed in one of the parking lots adjacent to the Googleplex. I had hooked up with a ridesharing partner who was doing the same commute I was (you'd be amazed how many people do insane commutes in California!) and he would drop me off at the corner of Shorline Boulevard and Amphitheater Parkway and I would walk the rest of the way to the 'plex. It's a beautiful area, right next to a wildlife preserve.

Imaptiens are annnuals, and I thought that they were supposed to die, and yet here was this batch of flowers just growing like gangbusters. The colors were brilliant. And as the winter wore on they just seemed to grow and grow and grow. It's amazing how a little color can cheer you up.

To mark the start of the new year and to really put the cap on my ignominy, the manager with whom I had had all the friction went on maternity leave shortly after the new year (as best I can recall -- I could be a little off on the timing of events here. My memory of this period is not the best.) She was replaced with a fellow named Ross, with whom I ended up getting along much better. At about the same time there was this big reorganization of the engineering department, and I ended up -- wait for it -- back in the ads group! Or at least I was reporting to Ross along with all the rest of the old ads group members. I wasn't actually working on ads stuff any more.

My new project was the Translation Console. At the time, Google was available in about a dozen languages. The translations were provided by a professional translating service which was very expensive and (I was told) not very reliable. The translations were also not particularly good. The solution was typically audacious: build an interface to allow Google's users to translate the site contents into their own languages for free. The result would be higher quality translations in more languages (over 100 at last count, including Klingon!) for free.

The problem with this approach is, how do you do quality control? If nobody at the company happens to speak, say, Serbo-Croatian, how can you tell if the ostensible Serbo-Croatian translation of "I'm feeling lucky" really means "I'm feeling lucky" instead of "Visit my hot new porn site"?

Again, Google audaciousness: let the users do the quality control too. The translation interface allows you to not only translate new content, but also to correct errors that you find in already translated content. As long as there are more honest translators than there are people trying to introduce spam, the spam will quickly get corrected away. Just as the Google search engine computes a "Page rank" based on how many links a page has pointing to it, the translation console computes a "translator rank" based on how many translations you've submitted and how many of your translations have been corrected by other translators. It works like a charm. To my knowledge, there has not been even a single instance of translation spam that has ever gone live onto the Google site.

I wish I could take credit for all this brilliance but I can't. I wrote the code, but the idea was conceived by Marissa Meyer (whose name I feel safe mentioning because her role in Google has been well documented in the press) and Deb (whose last name I'll withhold because her's hasn't). Marissa did most of the design, and I did the actual implementation. It was not the most challenging thing I've ever done, but at least I got to do it in Python instead of Java. If you can't have Lisp, Python is the next best thing. I also got to do some hacking on the Google Web Server code so that it could use the new translations, which was very cool and a little bit scary, because if you screw that up you will definitely hear about it.

It was probably good for me to have something a little less demanding to do at that point in my life. I got to decompress a little, and I had some free time to explore other parts of the company. In particular, I got to know people in the research group, which is where I thought I probably would be the best fit, but it soon became clear to me that I was way out of my league here. The sad fact of the matter is that my own field, autonomous robots, is (or at least was -- things may have changed) rather badly screwed up, and there was an awful lot of bogosity that passed for scientific research. (I actually published a paper about this at one point, which went pretty much unnoticed.) But the people in the Google research group were actually doing real stuff. I would have had to work for months, maybe years, to catch up to them, and it was becoming more and more obvious that my days at Google were numbered, so it seemed pointless to try.

I set my sights on my one-year anniversary in June when I would finally vest my first (and only) batch of stock options, gritted my teeth, and settled into the routine.

I can hardly remember any of it any more.

21 COMMENTS:

Anonymous said...

MORE ABOUT GOOGLE :  Unofficial Google Blog Google news

11:09 PM

Steve said...

You surely had a lot more strength than I did. I aimed for doing at least two years with my previous employer, but I just could not hang on. I made it to my 18th month with the company, and although I promised my peers that I would try my best to stick around, I just couldn't.  The people who were working there did not drive me away from the company. It was the environment. I honestly could not give a feasible prediction on what I would be doing there over the next five years, and if it was me doing the same thing I was doing at the time, that future is simply unacceptable.  I am not even sure if I still have any stock options, but obviously the number of stock options and their potential value could not keep me from leaving. The promised quantity of options was also low enough not to create incentive for me to push for the advancement of the company.  So I left.

11:45 PM

Veky said...

Yes, your translation console worked pretty well. It's interesting that you mention "I'm feeling lucky" translation to Serbo-Croatian, because there was some controversy about it (in Croatian at least -- Serbian and Croatian are two "different" languages now). However, it wasn't about spam, but about clumsy translations. I guess you English-spaeking people don't really get that feeling: when something just stabs you in the eye as a bad translation. Yes, I know that "I'm Feeling Lucky" is very hard to translate (there was even a whole entry in your FAQ about that particular phrase, IIRC), but I thought I had a translation that sounded better -- at least, it evaded problems with gender and cases, which are so common in automatic translation to Croatian.   However, I couldn't communicate it to Google well. The problem in your ranking scheme, as I perceived, was that for many "small" languages, one person had translated mostly everything in the beginning, and if just one his/her (you see? gender problem:) translation was bad (and he/she was convinced it was good), there wasn't much you could do -- because his/her "rank" was too high compared to your one.  But, I'd like to tell it was overall a pleasant experience (much the same as your work at Google -- a human mostly talks about the bad side of things). And it was surely much better than what we have today (e.g. for Gmail) -- where you can't even enter the circle of translators except if you are "chosen". :-(

2:39 AM

Danny said...

There was at least one translation problem I know of that, if not spam, certainly wasn't meant to help the program. The search button used a slang word in Malta that means "penis in a can." More on that here: http://blog.searchenginewatch.com/blog/050412-160229 cover

4:01 AM

Greg Linden said...

Minor correction. "Marissa Meyer" should be Marissa Mayer, I suspect.

8:23 AM

Anonymous said...

You write: "I got to know people in the research group..." and "...the people in the Google research group were actually doing real stuff." This is what gets me about Google. Four years ago I talked to some senior folks at Google, about working there. I wanted to do research. I was told, in uneqivocal terms, that "Google does not do Research and Development.. it does Engineering and Development". One senior fellow told me that Google does not hire researchers, it hires engineers. That was enough for me to hear, and so I have never bothered applying for a job.   So what is this "research group" you're talking about? Development, yes. Engineering, yes. But research? Hadn't I been told that "Google doesn't do that"?  Pshaw. Google makes no sense.

9:13 AM

Bernardo said...

Hey. I've been reading this blog with much interest since it got started. After working at Google for about 9 months, I quit in November (on the 17th, the first day of this blog, coincidentally). While the people I worked with were really awesome, there were things about how Google is run that I could not stand. (No, I will not talk about those things - I'm too afraid of Google's Legal dept, despite (or maybe because of) having done some work with them). Because of my less-than-great experience working there, I am naturally curious for any information about how Google evolved from a small start-up to a secretive, not-too-flexible, not-too-friendly large company. Hence my interest in your blog. Beyond this, I should say you guys' writing is very good indeed - the blog would be almost as interesting if it were about some other company. So keep it up! (And Ron, through this blog I discovered your personal blog. Your views on lots of things are very similar to mine (least of which is the wonder of flight), so I'm enjoying that a lot).  The reason I finally wrote just now was because I did some work with Google involving translation, so I thought I'd throw in my two cents. Most of that work involved revising the Portuguese translations made by that "very expensive and not very reliable" service of translators. The translations were indeed sub-optimal. I had trouble believing that someone was being paid a lot of money to translate this poorly, inconsistently, and uncreatively. (When I started working at Google, Portuguese was moving from being a Console-translated language to an officially-translated language, but instead of having everything re-translated, we inherited some material from the Translation Console days, which was also (predictably) not expertly translated, but (surprisingly) not much worse than the professionally translated material).  I was also one of the few people doing end-user support in Portuguese (and actually the only person doing end-user support in Portuguese for some products), so I was the first one to hear complaints from users about our bad translations. There were so many examples of poor translation, we (the Brazilian folks at Google) could not catch all of them. But we did try.  I liked Veky's comment about how "you English-speaking people don't really get that feeling: when something just stabs you in the eye as a bad translation". I think enough of the world speaks good-enough English that, when something needs to be translated into English, it's usually done pretty well (which is not so true of other languages) so English speakers just aren't exposed to really bad translation (with a few humorous exceptions, compiled in a handful of websites like Engrish.com). English vocabulary is also flexible enough that, if a slightly-wrong word is used, it doesn't look too bad. But I would CRINGE looking at some of the Portuguese material on Google's pages and products. I would then write somewhat angry emails to the people in charge of this material, saying "This is TERRIBLE. You're sending a message that says 'International users don't really matter to us'." I knew that they were busy, and that having the Portuguese material be perfect was not their highest priority, but I had to give them a hard time. =]  However, later in Veky's comment, there's a part about how "you can't even enter the circle of translators except if you are 'chosen'." I disagree. Well, the REAL reason that I'm writing is in the defense of the people in charge of the international side of the products. When I wrote them an email pointing out a mistake in translation, it usually got fixed. So it would be incorrect to paint an image of a Google that is too lazy, too skeptical, or for some other reason too reluctant to correct its language mistakes. International users matter a lot to Google. If it doesn't look like it, then this is just because there is just TOO MUCH to be translated and revised, and the few people at Google who speak (insert your not-super-common language here - like Portuguese) have things to do other than revise mountains of dubiously-translated material. Until that material gets revised, it's better to have that poor translation on our products and pages than no translation at all, right? But my point is, if you see a mistake, write Google about it (using the "contact us" on the website of the product/service where the mistake is), and they'll usually do something about it. As more and more international people are hired, and as more and more material is revised and updated, I am sure Google's foreign-language material will eventually be extremely well written. That's the current tendency anyways.  Google is working very hard right now to be as international as they can be - doing things like being sensitive to each region's culture while adapting a product to new regions and languages. The responsibility to be sensitive to local language and culture is not one Google takes lightly - there are a lot of very smart people dedicated to this. However, they are only human and have a LOT of work to do.  While I have some perspective into how the company operates, what it wants, how it makes decisions, etc, I would not share those insights publicly (I congratulate Ron and Doug on their courage). So I don't think I'd be comfortable sharing any more than I just did. Besides, almost all the work I did at Google was not very interesting at all. So don't bother asking me about it...  All right. Keep up this great blog!

11:28 AM

Bernardo said...

Ah, one more thing. Ron, you mentioned that the Translation Console languages numbered at "over 100 at last count, including Klingon!". But I don't think a user can ADD a new language to the console - only the Googlers in charge of the console can. I heard that Klingon, Elmer Fudd, Bork Bork Bork (Swedish Chef), Pig Latin, etc, were added while the console was being tested so that the people doing the testing and the debugging could understand the "languages" they were working on. I'm not sure whether or not this is true. (Is it?)  Some of the languages on the console are obscure enough that it would be hard for me to imagine they were suggested by the Googlers in charge of the console. Guarani, for example, is one of the languages spoken by Native Brazilian tribes. I wonder if a Brazilian person wrote in, saying they were willing to translate everything into Guarani, at which point the Googlers in charge of the console said "Why not" and added it. That to me seems to be the most likely explanation. (While working at end-user support, I once received an email from someone offering to translate the Google pages and interface into Qenya (Tolkien's Elvish language). I see this has not been implemented, probably because most computers do not have the Tengwar alphabet language pack...

11:40 AM

localudal said...

You say 'To my knowledge, there has not been even a single instance of translation spam that has ever gone live onto the Google site.'  Oh really? Who presented you with that 'priceless' knowledge? My estimate (see http://goolocalizations.blogspot.com, sections 'Goolocalizations' and 'Googlese') there's tons of 'translational spam' and 'junk translations' in Russian, Ukarainian, Japanese, Simplified Chinese. Someone with inflated 'page rank' and on a local competition assignment produced translational frauds, so, say, Google in Russian is just a silly joke compared to native Yandex.ru. Simplified Chinese of Google by the same scheme is either laughable, or illiterate, compared to native Baidu.com. Now, just for a second: Chinese market is no way Serbian, or Croatian, to believe so easily to any crook 'experts'. This is the price of un-lingustic approach to translations and localizations, so even outstanding Python, Java, and whatever Lisp knowledges can't repair the damage done.  Besides, who said that professional (OK, maybe expensive, but never so costly as the loss of Chinese market) translators did poor job? Your miserable page rank of their Web site, lack of links to and from them? Who are you kidding with that?  Measurements of text qualities based on authors' claims/appearances/page ranks is a dangerous illusion. Say, I presume that page rank of the Google.com itself goes through the roof, right? In your scheme, it makes next to impossible then to really copyedit the junk American English used, say, in originals of AdSense/AdWords descriptions.

Anonymous said...

all your base are belong to us

6:26 PM

These scenarios appear to have been a result Poor Communication and lack of individual attention....  There HAD to have been a place at Google for you - but perhaps the Management style was just too solipsistic, perhaps everyone was just too focussed on SELF to see the entire perspective of things...  The poor communicationm, may have also been a result of being too individually self focussed...  It takes a superhuman to be Extremely technically skilled and also be a people-oriented generalist Manager....  This excerpt is really confusing...  I remember only two significant things from that first review. First, I didn't get a raise, which is customary (or at least so I'm told) unless you've really screwed up.

8:30 PM

Ron said...

I heard that Klingon, Elmer Fudd, Bork Bork Bork (Swedish Chef), Pig Latin, etc, were added while the console was being tested so that the people doing the testing and the debugging could understand the "languages" they were working on. I'm not sure whether or not this is true. (Is it?)  It certainly helped with testing, but that was not the main reason those "languages" were added to the list. They were always intended to go live, and indeed they were among the earliest languages to go live.  Some of the languages on the console are obscure enough that it would be hard for me to imagine they were suggested by the Googlers in charge of the console.  As best I can recall, we got a list of languages from some official-seeming website somewhere (maybe the UN or something like that?) I really don't remember. I just know that the list came from outside Google.  So what is this "research group" you're talking about?   [Shrug] maybe they disbanded it. Don't forget this was five years ago. Or maybe they want to discourage academic-style researchers who think that publishing papers is a good measure of productivity. I don't know. All I know is that when I was there, there was a group called the research group and they did researchy kinds of things.  This excerpt is really confusing...  What I meant was this: Companies generally don't like to fire people except in the most egregious of circumstances because that exposes them to potential wrongful termination lawsuits. Instead they send subtle signals that you aren't working out and hope that you leave on your own initiative. Not giving someone a raise, even a small one, at their first performance review is one such signal.

9:45 PM

Anonymous said...

Google has hired a number of "research scientists," many of them belonging to the research team. (Try googling for "research scientist at google" with the quotes.) It's an impressive set of people.

9:51 AM

Anonymous said...

Oh, I know Google has hired a number people who are top research scientists. It's just that they don't call them researchers. They call them engineers.   Check out the Google Jobs listing: Where is the "Research Department" on this list?   Google Jobs  I see no research department. I see an Engineering department, yes. But no research.   Maybe there really is a research department, too, but they're just not hiring right now? Is that why the "research lab" doesn't appear on the list? Maybe they've solved all the tough information retrieval problems with all the researchers they already have, and don't need any more? Uh huh.  Look, admittedly this is a minor point in the grand scheme of things. But it's one that gets under my skin. It's mild schizophrenia on Google's part.. to continually talk about how they only hire engineers, and then also talk about their fantastic research lab. Just call it for what it is, is all I'm saying. If a researcher has to do more coding at Google than in a comparable position at AT&T, IBM, or academia, just make that part of the job requirements. But you're still hiring a researcher, not an engineer.

10:39 AM

Anonymous said...

Whoever posted that it would take $100 million to keep Mountain View's Slater Elementary open didn't know what he was talking about. I have this from a VERY reliable source:  The cost of keeping Slater opened is estimated by the savings from reduced expenditures that would otherwise exist in keeping the school open. This savings in reduced expenditures is approximately $360,000. Additionally, the cost of keeping Slater open is the loss of the potential revenue of leasing the site. The potential revenue of leasing the site is estimated at $500,000. Therefore, the savings in expenditures and the estimated increase in revenue adds up to $830,000. This “cost” is ongoing; meaning, the district would need to find this amount not just for the 2006-07, but for every year thereafter.  Surely that is within reason and even easily doable by a small group of Googlers and/or Google corporate right?   Who is the right person to ask about this?

10:49 AM

Anonymous said...

Ah, sorry, I'm the anonymous guy above, and I think I need to eat my words. A little bit more scratching on this particular itch led me to find, buried in a non-immediately obvious location in the engineering department, the following opening for "Research Scientist"  So I guess it does exist. I do stand corrected.  However, I also stand by the fact that some top Google tech folks I've spoken with really have said "Google does not do research and development; it does engineering and development". I still wish they would call it for what it is. But I suppose that's just part of the whole Google corporate personality. Anyway, back to this fascinating blog :-)

10:57 AM

The Irrational Investor said...

What about the "bio-info-nano" initiative? That's engineering too? It actually sounds scary.

10:32 PM

Greg said...

Regarding the "research" department at Google: back when Ron was at Google, there was a group of people we called "Research". These days, I don't know that most people think of there being an official research group any more, but researchers are still employed at Google (at least in the Mountain View and New York offices.) There are actually a couple of different groups that these people belong to (including, for example, the Systems Lab, which is where a lot of the Bell Labs people ended up.) And at least a few of the people I know from research have moved on (for example, the director of engineering in New York used to be in the research group.)  As far as the comment that Google doesn't hire researchers, only engineers, I believe that you could have been told that. But to clarify it a bit, Google doesn't want people who come up with ideas and then don't implement them: a researcher has to also implement at least a prototype of their system. So Google didn't want people who weren't willing to actually build something, and that's all that's meant by "engineering and development as opposed to research and development".  Maybe the research the anonymous poster posted about above was not concrete enough. Or maybe he/she was talking to the wrong people. But there certainly was a research group at Google, and without it, we wouldn't be seeing Froogle or Google News, among other contributions that might be less visible.

10:00 AM

Steve said...

I think this is a great blog. I'm very interested in the inner workings of Google, and somewhat angry with them right now since they kicked me out of Adsense because my friends and family clicked on my ads without my knowing it. Next time I will keep my mouth shut. But I digress...  It has been interesting to see Google rise from a little-known search engine to some kind of Internet... conglomerate, for lack of a better term... I don't really know what to call them now. Keep up the good work guys.  gonights.blogspot.com

6:42 PM

Anonymous said...

Greetings again.. I'm the anonymous "research" guy from above. I'd posted three times in a single day, so I decided to give it a rest for a few days, and let the majority of the readership move on, lest I come off as a troll.   Anyway, in response to both Ron and Greg, above, I very much understood at the time, and still understand today, that Google is not interested in hiring someone who thinks productivity is measured in the number of papers published.   My own area of research is multimedia information retrieval. In order to evaluate the research, I have had to build, over the years, a number of functioning prototype systems. With no system, there is no evaluation. With no evaluation, there are no results. With no results, you really don't have any research. So the prototyping component has always been a part of the research. That wasn't an issue.  So again, my main agitation was just that Google really just called everyone "engineers", and everything they did "engineering", even though I clearly knew that Google had hired researchers.   If that were the only issue, I wouldn't be posting a comment again. However, in the past two days my mind has really started to change and clarify on this whole matter. Something clicked in me, when Google released Video. In just the past two days, I have begun to think that Google really does not do research. That they really only spend their efforts on engineering. Even with that 20% time. Let me explain.  See, when I talked to these senior Google managers back in 2002, the thing they told me about the research/engineering schedule is that the idea-to-prototyping schedule lasts 5 months. (I've since hear the same thing repeated from other sources, so I don't think I'm giving away any NDA here by talking about the 5-month cycle.)  In 5 months, Google expects you to come up with a new idea, work the kinks out of your idea, and then implement the idea in a working prototype, so carefully crafted that it could go live on the Google main page one or two months later, after being passed off to the hardcore coders.   So whether or not Googles calls them "researchers" or "engineers", they basically only have a month or two to really be able to come up with new ideas, before you have to start hacking away, debugging, etc. to come up with this "soon-to-go-live" prototype.   I think this is too little time to really think up and do the exploratory data analysis that you need to do, to come up with really great new ideas. Real research is not just about the prototyping. Real research is about having the time to do the exploration and bounce all the ideas around, to come up with something good. If you're worried about going live in four months, you just don't have -time- to do real research. You will be forever stuck in this shallow mode of slightly incremental improvement, but never a real breakthrough.  So I've been thinking about this in the context of the new Google Video store that was released just a few days ago. I've watched as the blogosphere has greeted the offering with a yawn. Everything from "We Sat Around Waiting For Google Video And All We Got Was This?" to "Google and CBS release embarrassment of a video store".   And it makes me sad.   I was there, in 2002, with my multimedia information retrieval experience, ready to buckle down and get some real ideas flowing, get some new, working notions going about how to index, retrieve, and present/display multimedia data. I was ready, with all that Google idealism, to help change the world.   But even if I had applied to Google (which I didn't) and gotten an offer, it wouldn't have worked. If the expectation was that all the "research" someone in my position was doing had to go live to the Google main site every half year, then any research at all that was multimedia-related would not have been allowed... because Google Video itself didn't go live until just a few days ago. There would have been no place on the Google page to integrate those working prototypes, back in 2002, in 2003, 2004, and 2005.   If Google did research, -really- did research, the kind that requires a dedication window larger than 5 months, they might actually have a video offering that wasn't so universally greeted as lackluster.   And search, which I presume means all kinds of search, video included, is at the core of Google's mission statement. That is where the research had been needed. Not in developing interfaces for Google Talk. Not in incremental text search hacks such as letting you type in a Fedex tracking number. But in really developing the ideas (research) that will allow us to overcome the semantic gap in multimedia search.  And it makes me sad because what attracted me to Google in the first place, not just as a researcher but as a user, was this wonderful dedicated focus on search. Now, I feel like nothing has really happened in that area (esp multimedia search), when it should have been researched for years now. Maybe it has been. I just get the feeling it wasn't.   Maybe there is a lot going on under the scenes. But.. with all that, where is my ability to do relevance feedback? This is a well-known, extremely useful IR technique. Why do we get GTalk before we get relevance feedback mechanisms, for example?  So Google.. if anyone is actually reading this post by this point.. you're a black hole. If you really are doing wonderful stuff, you gotta let us know somehow. If you're doing all this fantastic research, you gotta help us users see it, so that we get excited about it. As an outside observer, I see what you've been offering in the last 2+ years, and I don't see that you've really got anything that has pushed information retrieval forward.

10:39 PM

Eric Gometz said...

I read the comment above about Guarani being an "obscure language" spoken only by the native tribes of Brazil. Actually, it is the second official language of the country of Paraguay. In addition to Spanish, it is spoken by over 80% of all the people there. I am sure that the 6.3 million Paraguayans are happy that Guarani was included by the Googlers in charge of the console. Just FYI.

12:02 PM

THURSDAY, JANUARY 12, 2006

All your T-shirt are belong to us

POSTED BY RON AT 12:21 AM

Worldplay wrote:

Google: All your searches are belong to us!

which reminded me of something. It was at the height of the "All your base are belong to us" madness (and if you think that was weird you should take a look at this) when Doug held a little seminar to explain branding and marketing to us technodweebs over in engineering. The seminar was advertised with an incentive: everyone who attended would get "a special bonus prize" or something like that. The prize turned out to be a Google T-shirt with the slogan "All your search are belong to us" on the back. (Note that "search" has to be singular to remain true to the spirit of the thing.) We were admonished not to wear them in public.

Hm, I probably still have mine somewhere. I wonder how much I could get for it on eBay? ;-)

9 COMMENTS:

Cole Blak Pascal said...

I dun have a lot of cash but I'd give you my bike for that shirt. Seriously.

2:50 AM

Anonymous said...

I think the ??? guy should get a shirt just for that comment.

4:27 AM

Ryan said...

Ron doesn't like when someone doesn't speak or write correctly. I think that "dun" disqualifies you.  Ryan

Michael said...

The "All your base" phenomenon was certainly intriguing. My favorite sighting was in the index of Scott Meyers' Effective STL. It's on the second page of the index, between "algorithms" and "allocations". I think the indexer just slipped it in at the last minute.

11:51 AM

Doug said...

Yeah, that was one of the first non-standard logo shirts we made. I was surprised by the turnout for that Marketing 101 session and we quickly went through the allocated amount, pissing off some engineers who had been unable to attend. The whole point of the seminar was the power of marketing and having a collectible premium was intended to be part of the lesson. It worked pretty well. Btw, the first non-standard logo shirt was the one announcing the Yahoo deal. Susan put them together and they were given out at the party celebrating the signing. Jerry Yang was there, I think, but I was off on a family trip and missed the big event. I did manage to get a shirt though and have it stashed away somewhere with the rest of my collection. I'm off on another family trip, but if I get ambitious when I return, I'll post photos of all the t-shirts we made in the first few years. I tried to keep at least one of each. If nothing else, it should help future traders in Google logo gear establish provenance...

4:56 PM

Anonymous said...

Your 'you should take a look at this' link is one of the funniest things I have ever read. A cure for depression, indeed!  I rather suspect it would actually work, too.

11:16 PM

Ron said...

I rather suspect it would actually work, too.  It worked for me. (Reading the book, that is. I was laughing so hard that I couldn't be depressed!) ;-)

8:59 AM

Wordplay said...

You're kidding me.  You actually used that dumb phrase on a T-shirt?   I want royalties baby!

7:31 PM

sarahintampa.com said...

Thanks a lot for the Amazon link. Fabulous. Now Amazon thinks I like books about my *^#@. Ugh.

5:03 PM

THURSDAY, JANUARY 12, 2006

2001 in a nutshell

POSTED BY RON AT 11:00 AM

After I finished the translation console I was quietly (and, as far as I can recall, unofficially) transferred back to the ads group. Like I said, my memory of this time (early 2001) is pretty fuzzy, and I seem to remember that official groups had been abolished by then, but I went back through my old notebooks and from February through June I was working on day-to-day operations of AdWords, with an occasional tweak on the Translation Console (though it never really needed a whole lot of intervention).

Google was growing at a frantic pace even as the dotcom world was melting down all around us. We were hiring like crazy and space started to be a real problem. People were being rearranged and stuffed three and four to an office or cubicle. By early summer it was, for the first time, pretty much impossible to know everyone else who worked at the company. Google was not a small company any more. The transformation had happened with breathtaking speed.

It was a very exciting time for the company, but a pretty dull time for me, and to be honest I was not unhappy to have a little dull spell after my first six months. I had settled into a nice comfortable if not particularly challenging groove, and I was worrying less about making it to my one-year anniversary when I could finally stop this crazy commute.

Urs Hoelzle, the guy who had hired me who had been VP of engineering, stepped down (or maybe it was up?) from that position and became the first "Google Fellow". He was replaced by Wayne Rosing, a Silicon Valley legend who, among other things, led the (alas, ill-fated) development of the Apple Lisa. In March, Eric Schmidt was hired as CEO, and for a couple of months his office was two doors down from mine. (I think he actually was sharing an office himself for a while, space was that tight.) I never got to know either Wayne or Eric very well because they were extremely busy, and I knew I was going to be leaving the company soon and didn't really want to waste their time.

June finally rolled around and I had my second performance review the week after my one-year anniversary, where I told Ross I was giving my two weeks notice. Once more, to my great surprise, Ross seemed dismayed. It seemed to me that the company had been sending me pretty clear signals that it wanted me out, and by then I was only too happy to oblige. But no. They wanted me to stay, despite the fact that at least by my own self-assessment I was contributing very little by then. I shrugged my shoulders and said, OK, if you want me to stay I'll stay, but I can't do the commute any more. You have to let me work from LA.

They agreed.

Well, actually we compromised. Instead of coming up to Mountain View four days a week I'd only have to come up one day a week and telecommute the other four, which seemed reasonable. Some things really require face to face contact. Even my wife was OK with the new arrangement since it meant I would no longer be away from home overnight.

I was pretty sure that the new arrangment wouldn't last long because Wayne Rosing was an big fan of colocation, even if it meant stuffing people into cubicles like sardines (which was pretty much what was happening by then). They moved my desk (which consisted of a door on top of two yellow sawhorses) out into the hallway, which suited me fine. I actualy had more privacy out in the hallway than I would have had in an office at that point. At least I could lean back in my chair without physically bumping into another person.

In early June I was also contacted by the high-level manager at JPL who had made me promise to come back there after a year asking me to make good on that promise. I had some reservations about going back. Things had not been going terribly well for me there before I left (that's a whole 'nuther story which I might write about some day) and things didn't seem to have changed a whole lot while I had been away. Since I still had the Google job I felt like I had a little leverage, so I told him I'd come back if I got a promotion.

I have to digress a bit here and explain a little about the technical career ladder at JPL. There were at the time four rungs on that ladder: Associate, Staff, Senior and Principal. Climbing the first three rungs was not particularly difficult (I was a Senior when I left) but getting promoted to Principal was a Big Deal because once you became a Principal you automatically got the ultimate JPL fringe benefit: on-lab parking.

JPL is at once the smallest and largest NASA center. It has the largest number of employees and the smallest physical space. Parking has been a serious problem pretty much throughout the lab's history. Only the privileged few (Principals, very senior managers, and people who have worked there for twenty-five years or more) get to park on-lab. The rest have to park in two remote lots and hike in to work or take a shuttle bus.

Even among on-lab parkers there is a pecking order. The parking lots are color-coded. At the bottom of the hierarchy is the green parking pass, which lets you park in the least desirable spaces. Then there is the blue lot, which is centrally located next to the main plaza. Then there are the legendary red and silver passes, which give you access to the only covered parking spaces in all of JPL. There are only about forty of these space in total. Less than one person out of a hundred gets to park there.

The upshot of all this is that getting promoted to Principal is really really hard. It's the equivalent of getting tenure at a university or making partner at a law firm. You have to be approved by multiple committees, and finally, by the JPL Executive Council, and the number of slots is extremely limited. Many highly qualified people get passed over year after year.

My insisting on getting my Principal promotion as a condition of returning had the effect of delaying JPL's ability to make me an offer as it shepherded the paperwork through the system, which suited me just fine as my Google situation had gotten fairly cushy, at least for the time being, and I was more than happy to wait and vest in a few more stock options.

You'd think that working from home would be an ideal situation, but the fact of the matter is that it gets pretty lonely. Face time is really important to building relationships, and email is a very poor substitute. June turned into July and July turned into August. There was one delay after another at JPL. Funding was tight. They wanted to promote me, but couldn't find the money to pay my salary. And Google was starting to hint that my days there were, finally, numbered. I was beginning to worry that the return to JPL could fall through and I'd end up unemployed, which would be financially disastrous. Coming up with the cash to pay for exercising the stock options on top of my commuting expenses had left us with virtually no cash cushion. We were living paycheck to paycheck.

Then, at about 8 AM Pacific time on September 11, 2001, I got this email:

To: ...@google.com

Subject: Is NY Alive?

In case you aren't watching... one trade center collapsed.

The pentagon is on fire.

turn on your tv.

WTF? I turned on the TV. I stared at it for a while in utter disbelief at what I was seeing, then went downstairs where my wife was making coffee.

She could tell just by looking at me that something was up. "What's wrong?" she asked.

I said, "It's going to be a bad day."

15 COMMENTS:

Anonymous said...

Ron, Did you ever find out why they rejected your resignation (twice)?

3:23 PM

Ron said...

Did you ever find out why they rejected your resignation (twice)?  No, but stay tuned. The story is not quite done yet.

4:34 PM

There are statements being made constantly about "feeling" that days were numbered - but there is no illustration of exactly Why those feeling were valid and what "specifically they were based on - besides not getting certain anticpated reactions - were your expectations too high about how explicitely you should be reinforced that you were a valuable commodity.  If nothing was adversely said - it could have just been everyone being - justifiably - extremely busy with themselves to constantly reinforce the more advanced people. It may have been assumed that advanced people are not in need of great reassurance.  If better quality people were available - wouldn't they have hired and use them - with all the quality resumes they were getting??? ~~~~~~~~~~~~~~~~~~~~~~ Could there also be posts about any initial concerns about Click Fraud becoming a problem - and what proactively was done to help prevent it in AdWords - and any Politics?

7:48 PM

Haridas Dave said...

Ron, yours posts are very engrossing, hope you write more than your normal quota for a day. Waiting eagerly for whats next !!!

7:53 PM

Ralf said...

Just wanted to stop by and say: I read this all the time and i really like it. Keep up the good work! Now, back to my code....

1:36 AM

Isaack said...

What a cliffhanger, you left us with!

6:06 AM

Ben said...

I kept reading, and the moment I looked at the scrollbar and realize it's coming to an end, I screamed internally that I wanted more. Keep writing.   I'd also be interested to see how Google's been hinting to you that your days at google are numbered. what kinds of hints are they?

7:50 AM

Ron said...

I'd also be interested to see how Google's been hinting to you that your days at google are numbered. what kinds of hints are they?  Before June the main concrete indications were (as I mentioned in earlier posts) that I wasn't asked to do interviews any more, and that I wasn't given the customary token raise at my first performance review. On top of that was my own personal sense that I wasn't performing well or contributing much. After AdWords, most of the coding that needed to be done was in C++, and I just wasn't very proficient in it at the time. (To prepare for the performance review we had to fill out a form that included questions like, "List three areas where you need to improve." My answer for my second performance review was "1. C++ 2. C++ 3. C++".)  I don't know, maybe I was just being paranoid, and everything really was hunky dory. I'll probably never know.  After June, they were explicitly saying: Wayne wants everyone co-located, and so if you can't be here we will have to let you go before long, or words to that effect.  But like I said, stay tuned, there's more to the story.

8:47 AM

Ron said...

Oh, two more things:  1. Thanks to everyone for the kind words!  2. There was one other reason I thought my days were numbered as June approached: I knew I could no longer do the commute after June, and I wasn't expecting Google to allow me to telecommute full time because I knew that Wayne wanted everyone colocated. (That's why they were stuffing people in to the 'plex like sardines.)

8:51 AM

Anonymous said...

I always want to pat you on the back and tell you keep your chin up and things will get better. So far as your ending of this entry, we all remember where we were that day. I look forward to your future posts. Both of you have a comfortable style.

2:48 PM

Anonymous said...

Surprised you didn't just try to find some temp housing. During your time at Google apartment rent fell and there were many vacancies not far from the 'plex.  These posts have really been an eye-opener. It's good to see someone being frank and honest about what goes on within the company.

8:46 PM

Anonymous said...

hopefully, and more than likely googlers@google.com isnt a real distribution list. Otherwise, well - you can imagine

Anonymous said...

hopefully, and more than likely googlers@google.com isnt a real distribution list. Otherwise, well - you can imagine   Hey Anonymous, that address sounds like the real deal, or something close to it. I'd wager that most new tech companies have a list like that. But you should know that mailing list software for internal lists is usually set up to only allow subscribers to post. (As a side note, at the company where I worked at the time, we had an internal IRC channel that everyone used. As people logged in, we'd tell them personally...)

5:30 PM

Kunal said...

Hey - you work at JPL? Finally, a person to sympathize with about on-lab parking. We should do lunch sometime.

11:36 PM

Anonymous said...

 But you should know that mailing list software for internal lists is usually set up to only allow subscribers to post.   Yes - but any one can fake a from address. I guess either way it would be easy to spam the entire googlers group if that is the case.

6:48 AM

FRIDAY, JANUARY 13, 2006

The real surreal life

POSTED BY RON AT 1:22 PM

9/11 was the start of the strangest five years of my life. It ranged from subtly weird to overtly bizarre. Example: at about 10 AM, an hour or so after I'd first turned on the TV and the magnitude of what was happening was beginning to sink in, one of our next door neighbors called -- I swear to God this is true -- to complain that our cat was defecating in his flower beds. Much later we learned that this person was having serious problems with chemical dependency, but at the time we had no idea.

Traumatic events weren't new to me. In February of 1991 I came home one night and surprised a burglar who took a pot-shot at me with a .38 on his way out the window. The bullet missed my left knee by about a foot, punched through one of the interior walls, ricocheted off a heating duct, and embedded itself in the living room wall. The odd thing about that kind of experience, I have found, is that it takes a long time -- like days or weeks -- to become consciously aware of the extent to which you've been traumatized. Immediately after it happened my heart was beating pretty fast, but I didn't feel panicked or anything like that. I crawled over to the phone and called 9-1-1, told the cops what had happened, etc. etc. The panic came later, with the nightmares. It was several years before I could really get a good night's sleep again. But the really insidious effect of post-traumatic stress was that it really shortened my fuse. I had never been particularly easy to get along with, but after that night and for a very long time I would say things to people that in my more lucid moments I would be amazed at. It was almost as if I -- or at least a piece of me -- had turned into a different person over which the rest of me had no control, although at the time I just felt like me. It's a very weird and difficult to describe sensation. Jekyll and Hyde were not entirely a product of Robert Louis Stevenson's imagination.

9/11 did that to me again, and again I wasn't aware of it. (And in retrospect I expect that's what happened to my neighbor too, although he probably helped himself along with a couple of pills.) The thing I did that I regret most was arguing to remove the Arabic translations from the Google site. Fortunately, cooler heads prevailed and the translations weren't touched, but there were a couple of hours when taking them down really seemed to me like the right thing to do.

The days immediately following 9/11 were oddly peaceful in LA. There were no planes in the air and less traffic than usual on the roads, and the weather was gorgeous. I like to think that this might have been a preview of what life will be like in a few decades when the world really starts to run out of oil, but that's a topic for another day.

A few days after 9/11 my wife and I got an invitation from a wealthy acquaintance of ours to spend the day at their house. Actually, calling it a house doesn't even begin to do it justice. It's an estate. I have been there several times and only ever seen about half of it. He was a bond trader and the markets were all closed, so he had some time on his hands. I can remember hanging out in the jacuzzi with his trophy wife (who has since left him -- money really can't buy happiness) thinking to myself how surreal this situation was. Two thousand miles away in New York people were digging themselves out of the rubble and here I was sitting in a hot tub with two gorgeous women drinking a glass of very expensive wine. It seemed horribly self-indulgent on the one hand, but on the other I didn't see how sitting at home moping was going to help anyone in New York.

(Fast-forward briefly to June of last year. We were vacationing in Europe, and landed at London Heathrow about an hour after the subway bombs went off. We were really struck by how little overall impact it seemed to have on the city. Our hotel was in central London, and when we drove in to town it was deserted, but by evening the busses were running again and things seemed pretty much back to normal by the following day. I don't know whether it's rational or rationalizing, but it does seem to me that the best way to thumb our noses at the terrorists is to live our lives. It worked for London.)

When they finally reopened the airports I did a few more commutes, but it was completely untenable. What before had been merely horribly inconvenient was now a serious nightmare. Fortunately, I was flying out of Burbank instead of LAX, so things could have been even worse. I shudder to think.

I don't know how long Google might have kept me on if 9/11 hadn't happened, but it was obviously the last little nudge that Google needed to finally let me go. My tenure there ended at the end of September. At the same time the process of getting me re-hired at JPL came to a dead stop. October rolled around with no word, and for the first time in my life I was unemployed.

I don't want to be overly dramatic about it because JPL did come through just a few weeks later, but let me tell you, if you've never been out of work, count your blessings (and if you have, I know how you feel). I had been working more or less continuously since high school (I got my first job programming computers when I was a sophomore) and I never in a million years imagined that I would end up on the dole. (I actually went to the umemployment office and picked up the forms to file for unemployment benefits, but I'm happy to report that I never turned them in.) It felt as if life had become a game of musical chairs and I was in the wrong place when the music stopped.

I finally got the word that my promotion had been approved. I breathed about a dozen sighs of relief before I got up the nerve to ask The Question, the one that a year and half earlier was the beginning of the end of my Google career: what would I be working on?

The answer came back: they wanted me to work on -- wait for it! -- search engines!

Aaarrrgghhh!

Fortunately, working on search engines at JPL doesn't mean the same thing as working on search engines at Google. Working on search engines at Google means you're working on a search engine. Working on search engines at JPL means you're working on procuring a search engine, and that was actually a topic I knew something about: call Google, place an order. (They had just come out with their search appliance.) You don't want to know how many of your tax dollars went to pay me to help shepherd purchase orders through JPL's procurement system.

Which brings me to one of the very few bits of juicy Google inside information that I have to share. (I'm able to share it because I learned it as a Google customer and not a Google employee. Furthermore, Google has been aware of this for four years now, and if they haven't fixed this problem by now then they deserve to be outed.)

We got one of the very first Google Search Appliances to evaluate. As everyone by now surely knows, the thing works by crawling your organization's intranet just like www.google.com does. But because this was an evaluation copy we would have to send it back to Google when we were done.

To address the obvious security problem (there were files available on the JPL intranet that contained sensitive information) the search appliance had a self-destruct feature that could be invoked to erase all the files on the device's internal hard drive. The problem was that self-destruct was a little too thorough. It erased everything, including the operating system, making the device unbootable. On top of that, the box was physically sealed with rivets. This made it impossible for us to verify that the information had in fact been erased.

I had the very devil of a time explaining to Google why this was a problem and why we could not just take their word for it that the self-destruct feature did what they said it did. We could not send the box back without actually verifying that the data had really been erased. That meant either getting root access to the machine, or physically opening the box and extracting the hard drive. Google suggested that we run a magnet over the box, and it took me a long time to explain to them why that wasn't good enough. (Just in case you don't understand the problem, it's this: it is very easy to make it appear that the data is gone without actually erasing it. For example, just erasing the boot sector on the drive will make the disk unbootable without actually erasing any of the data on the disk. Erasing the directory will make it appear that the files are gone to a casual inspection, but in fact all the data is still there.)

Google didn't want to give us root access because they didn't want us to be able to access their sensitive information (like their code). The solution seemed obvious to me: self-destruct mode should erase everything, including the code, but leave the machine bootable with no root password. That they didn't do this indicated to me that Google might be starting to lose some of their technical edge.

We finally reached the following compromise: they would send a technician down who would witness our running self-destruct mode. After this the tech would open up the box, take out the hard drive and leave it with us, and take the rest of the appliance back up to Mountain View. This seemed fine.

The tech arrived and together he and I pushed the Big Red Button, as it were. After launching self-destruct mode the machine put up a screen saying "Self-destruct in progress..." or words to that effect, with the colorful Google trademark bouncing ball animation showing that us that the machine was still alive.

Half and hour later the machine was still self-destructing. I asked the tech how we would know when it was done. He said that it should have already finished, and he didn't know what was going on.

After another fifteen minutes or so he pulled the plug on the machine. He plugged it back in, and it booted right up.

So much for self-destruct mode.

By now over an hour had gone by and the tech and I were both starting to feel a little impatient, so the tech logged in as root and typed "rm -rf /*'. (For those of you who don't know unix, that is the command that erases all the files -- except that it doesn't really erase them, it just erases the directory.) When that was done he opened up the box, took out the hard drive, and handed it to me.

All the data, including all of Google's code, was still on it. But we had already spent nearly two hours on this (or several weeks depending on how you count) and I really didn't want to drag it out any longer, so I just thanked him and walked him and the box back to the visitor center.

But I was starting to have some real doubts about the future viability of the company.

41 COMMENTS:

Anonymous said...

Delightful, thanks Ron.

4:46 PM

Anonymous said...

well... what did you do with the code??? ;)

5:25 PM

Larry + Sergey said...

Another killer cliffhanger. Damed if this isn't the best blog on the whole stinking net.

5:48 PM

Anonymous said...

dd if=/dev/zero of=/dev/hda  Or better yet, plug the drive into another system running knoppix and do a few alternating writes with 0, 1, and rand.  Is anything stopping a customer from buying a unit and disassembling it?

6:28 PM

Andrew Hitchcock said...

the search appliance had a self-destruct feature  Oh :( . I read this line and was hoping it was a physical self-destruct. Perhaps in the next version...

6:34 PM

Dov said...

Amazing blog. I think it should be made into a thriller movie : )  Regarding this latest post about Google losing its edge, isn't a small incident like Google's refusal to change their appliance erase specifications a little too trivial to back up a "foreshadowing" statement like that?  Thanks for your story.

8:04 PM

A colleague walking to a 9 to 5 job in the World Trade Center at 8:43 a.m., on that 70 degree, bright, sunny Septemeber Morning - (just one block away from the office 17 minutes, I'll grab a cup of coffee to go)  then, all the pedestrians suddenly looking up the sky at that Low Cruising, Noisy Jet along lower Manhattan - minutes later crashing into the first Trade tower, people screaming and running out and all along the avenue...  then ultimately, dozens of office workers jumping out of windows many floors up with no other hope,  then the second crash ...  then the buildings collapsing...  the effect was a realization of how easily life and hard work can suddenly disappear, and the need to savor life's minutes and family more.

9:34 PM

Anonymous said...

Hmmm, quite a bind with the hard disk. Did anyone think about taking the hard disk out to the JPL carpark and smashing in with a hammer? Hard to recover data from that!

1:10 AM

Martin said...

"Damed if this isn't the best blog on the whole stinking net."  Sure is. Keep those updates coming!

3:37 AM

Anonymous said...

If Google already agreed that JPL keeps the HD, why did you guys insist on the the self-destruct? You could just take out the HD, send the HD-less appliance back to Google, and destroy the HD in whatever methods JPL deems fit without tying up a Google tech.

6:17 AM

Anonymous said...

hey, anonymous above me: did you read the thing about google wanting to be sure not to give the harddisk with their secret code to the customer, hence the sealing of the psysical box?

6:21 AM

Eater said...

writing zeros with dd isn't really sufficent. 'shred -v /dev/{s,h}da' would be more appropriate, writing multiple patterns of pseudo-random data to the disk. increase the iterations with '-n' to satisfy your level of paranoia :)

6:49 AM

pBay said...

Nice blog guys, continue.

7:44 AM

Matt Bland said...

Why don't they use two drives. One locked inside the machine with their code on it. And another removable drive with your search data on it. Then all you need to do is remove the data drive and send the application box back. It's completely obvious to me that would be the solution for the next revision. Or alternatively put a SATA, USB, FireWire or SCSI port on the machine for using an external storage device for the search data as an option in the setup stage.

8:14 AM

Anonymous said...

The non-functioning self-distruct and shoddy tech work aside, this is a very interesting problem.  It seems like most evaluation appliance boxes would have a similar delimma. How do you protect your data and give the customer verifiable control over their data?  The dual hard drive idea makes some sense, but how does the customer know that you haven't pulled some of their data to the other drive?  Putting your code on a ROM might work, but then you can't easily patch it and you would constrained by size.  If your appliance is in customer hands, you can never guarantee 100% secrecy. Even if you decrypt it as you run, at some point the code is unencrypted. All you can do is increase the difficulty level.

Brandon said...

Sounds to me like this's a time when a physical self-destruct would be helpful. Something which would physically destroy the HD and its contents, if not the entire machine.

Anonymous said...

Realistically from your perspective keeping the harddrive is the only solution available to you.  Let's say their self-destruct feature followed the procedure you advocate, and keeps a root login. How do you know their OS hasn't been modified to report to you that the data has been deleted when, in fact, it still exists?  Nothing!  You either have to physically keep the harddrive, trust Google, or remove the harddrive, put it in a computer you trust and wipe it from there.  Doug

Ron said...

why did you guys insist on the the self-destruct?  We didn't. That was Google's insistence.

9:01 AM

Ron said...

How do you know their OS hasn't been modified to report to you that the data has been deleted when, in fact, it still exists?  You always have to have a baseline level of trust or you quickly fall down a rabbit hole of paranoia. In the case of the self-destruct feature we had actual evidence up front that at least some people at Google "didn't get it", and that it was therefore plausible that the feature was not properly implemented, particularly since really erasing a hard drive properly is a decidedly non-trivial thing to do.  To modify the OS in the way you suggest would have required a significant covert effort on Google's part, one which, if discovered (which sooner or later it likely would have been), would cause significant damage to the company. At the very least it would have made a hash of their "don't be evil" slogan. So it would be a significant risk with very little potential benefit. It seemed safe to assume that Google would not put special effort into doing something that stupid. (I'll grant, though, that this is not always a safe assumption, c.f. the Sony rootkit.)  As an aside, I really believe that Google does do its best to live by "don't be evil". I think Larry and Sergey are idealists who in their heart of hearts really want to change the world for the better. (By way of contrast, I think Bill Gates is a ruthless businessman who just wants to maximize Microsoft's power, influence, and profit by whatever means he can get away with.) So I trust Google at least to try to do the right thing, but not necessarily always to succeed.  Is anything stopping a customer from buying a unit and disassembling it?  No, but this was an evaluation unit and did not belong to us. (We actually offered to buy it, but it was already an out of date model and Google didn't want to have to support it.) We later bought a box and someone at JPL did take it apart after the software license expired and it stopped working. It's just a generic rack-mount PC as far as we could tell.

9:26 AM

Rob L. said...

I think Bill Gates is a ruthless businessman who just wants to maximize Microsoft's power, influence, and profit by whatever means he can get away with  A common enough viewpoint, Ron, but in light of that what are your thoughts on the billions of dollars the Gateses putting towards world health through their foundation? The man is becoming or already is the biggest philanthropist in the history of humankind, isn't he? Is that nothing but a PR stunt?  Is it nothing compared to what we're going to see from Larry + Sergey?

10:38 AM

Ron said...

what are your thoughts on the billions of dollars the Gateses putting towards world health through their foundation?  I think it's completely irrelevant to the issue of whether or not Microsoft is trustworthy. I trust Google not because they are doing good things with their money, but because they made their money (and continue to make their money) by playing straight. Microsoft made their money playing crooked, and no amount of generosity can paper over that fact. (And I think that to the extent they're playing straight now it's not because they think it's the right thing to do but because they think they have no choice. I think they'd play crooked again in a minute if they thought they had half a chance of getting away with it.)  I also think that a person's generosity ought to be measured not by how much they give but by how much they have left after they give. A billion dollars is a lot of money, but if your net worth is tens of billions then giving away a billion here and there doesn't really call for tremendous sacrifice. To be fair, Bill is still a pretty young guy and I'm sure he's not done yet, but I'm not quite ready to nominate him for sainthood just because the numbers on the checks he writes are big enough to make most people's eyes bug out.

11:43 AM

SM Guy said...

The part about the being unemployed hit home.  Between Dec of 2002 and Sep of 2004 I was out of a job for a total of 18 months. I have a BS and MS in Computer Science (but I never wrote code for money, my expertise was elsewhere). 18 months is a mind boggling time to be out of work for someone that never even imagined he could be without a job. I still am shocked when I look at that number: 18 months.  I ran a small software company and we had to essentially shut down. So I had to fired my brother, cousin and 10 other employees (oh, that was fun) and tell 10 investors, their stock was worthless. And I got to watch every single penny I had to my name (in the high six figures) evaporate overnight because I had all my skin in the game.  Then I spent 8 months looking for a job. Got hired. My wife and I thought we could actually start to live a real life. Boy were we mistaken. 3 months later I was laid off from a well paying job and no severance pay. 10 months of sickening, mind-numbing agony followed. I sent out over 3,500 resumes over those two periods of being laid off (I am SERIOUS. three thousand five hundred). I contacted everyone I ever met (want to talk about eating every morsel of pride you ever had). It became utterly surreal. How many resumes can you actually send before it becomes ridiculous (and they were well written, with great cover letters)?   I had no Google stock or any other for that matter I could hope would pay off someday. I had cashed in every single investment, down to my 401K's to pay the bills at home. It got to the point that I couldn't even dream about buying myself a cup of coffee at Starbucks, I would feel too ashamed to spend that much on coffee. Every day unemployed meant walking around with a lump in my throat and a sickening burn in my belly wondering what the hell was wrong.  So even though Google was a rough ride, thank your lucky stars you have a nice job at JPL and got a year plus of options vested at Google. Don't get me wrong I am certain you earned it and deserve every bit of success you have. Just say a little prayer tonight thanking whatever god you pray to, if any. It ain't so bad my Xoogler friends, it ain't so bad.

2:05 PM

Anonymous said...

ron - I just stumbled upon this. great stuff! I am glad to read your comments about Larry & Sergey, but I am starting to feel a little uneasy about Google. They are getting too big and could wield too much power in the future, IMHO. But, all big things come tumbling down at some point, don't they?

2:13 PM

Anonymous said...

To the above commentator: Google is one of many search competitors. If you hate them so much buy a Yahoo search appliance instead.

4:13 PM

Anonymous said...

Ron,  I'm the anonymous lisp hacker.   Thanks for this amazing story. They are slipping; no true Unix hacker would think "rm" was sufficient to the task. This is a terrible sign.   I enjoy (and agree with your comments) on Bill and the value of his charity. You are such a very lucid and logical person; the kind who rubs folks the wrong way.  I'm sorry you suffered from the breakin/shooting and 9/11. Combine any of those with a tough job (and irritating managers) and a sensitive hacker will go nuts.

5:30 PM

SJ said...

Ahhh...even the mighty stumble if not fall occasionally.

8:12 PM

tats said...

Interesting as usual, thanks Ron.  Just quick fix:   Fast-forward briefly to June of last year ... about an hour after the subway bombs went off.  it was July wasnt it?

3:27 AM

jiv said...

yes, the london bombings (the first one at least) occured in july.  very interesting problem you describe there ron. rock on. =)

7:33 AM

melyviz said...

been reading this for a while and like it very much! keep it right up.  and just out of curiosity. were there any difficulties with google's legal department?

12:11 PM

Ron said...

were there any difficulties with google's legal department?  No. See http://xooglers.blogspot.com/2006/01/we-are-experiencing-technical.html, update 2.

1:02 PM

Anonymous said...

They are slipping; no true Unix hacker would think "rm" was sufficient to the task. This is a terrible sign.  Very few true Unix hackers want to be field support techs (or "sales engineers") either. Frankly, I'd be a lot more concerned if Google were wasting their really good engineers on this kind of task.

4:17 PM

The_Chef said...

Loved this story, sounds very similar to what happened once at EA with one of the companies it ended up devouring in the end. Almost same thing had happened. Turned out an intern was able to write up a quick script to hack the box, extract the info, burn it, and wipe the drive. So they ended up with the code anyway. Sneaky bastards those guys were  But EA was SOOOOOO ethical, right?

4:45 PM

Anonymous said...

Yo anonymous, who says Unix hackers don't want to be field support guys.  Yes, it is true that Unix hackers don't want to do that. But part of being a Sergey or a Larry is setting up a healthy, functioning organization that can solve that problem.   E.g. how does the NSA make sure they erase their disks? They have security gurus. They make policy. The non-gurus follow the policy. That ensures that secrets stay secret.  Google is apparently not functioning properly anymore. They have field support engineers making decisions about how to erase a disk. Or -- and this would be awful -- they've got policies that say that "rm -fr *" is good enough. Either way, things are screwed up.   If I was Larry or Sergey, I'd be trying to figure out what went wrong and fix things.

9:22 PM

Vivek said...

Hi ron and doug, I've been reading your blog for the past month or so, and its just fantastic....specially the last two ones, absolute cliffhangers. this probably is the best blog on the net. And I agree its not what you give, its what's left over that determines how generous you are. MS's FUD policy has earned itself lots of negative points everywhere.

7:41 AM

Anonymous said...

First off, if you really want to permanently destroy all data from a hard drive without any chance of any of it being recovered, the only solution is to destroy the HD. Magnets, dd, and even shred won't get rid of all possibility of recovering the data, due to magnetic signatures. Basically, if a bit on a hard drive is left in a 0 or 1 position for a long enough time, even if you change the value to something else, if you look at the analog signal of the value for that bit, it will appear just a bit higher than a normal 0, indiciating it used to be a 1 for a very long time. It's not very reliable, and theoretically if your filesystem constantly flipped 0's and 1's it would be much more secure against this analysis, but it's never done (as far as I know).  Secondly, how close were you to just begging for your old JPL job back, rather than holding out for the promotion? Is it something you would ever see yourself doing?

4:40 PM

Ron said...

how close were you to just begging for your old JPL job back, rather than holding out for the promotion?  That was never a serious consideration, but not because of any conscious decision on my part. It just worked out that way. Before 9/11 I had a cushy situation at Google with no deadline so I was quite happy to just wait it out. After 9/11 the whole JPL bureaucracy (well, the whole country actually) just ground to a halt for a while, so it probably would not have made a difference.

6:08 PM

Anonymous said...

Is it just me? "My tenure there ended"... as the entire description of the climactic event of the entire story? I know it is an unpleasant memory, but you are a writer now (and a good one), I think your audience deserves a little more.

10:28 PM

pBay said...

Nice blog, but sometimes it's look like pulp fiction.

8:43 AM

Ron said...

as the entire description of the climactic event of the entire story?  What makes you so sure that's the climactic event? ;-)

10:52 AM

Anonymous said...

FWIW. I worked with the google search appliances (both models) fairly extensively for a period of some months, and overall, I have to say they they were poor at best. There were all kinds of technical issues, like refresh times, a data submission protocol that was just bizarre and a search result security mechanism that only a neophyte would design. I lost my faith in Google's engineering at that point...

12:57 PM

Anonymous said...

Surely destroying a hard drive at JPL could involve a powerful robot probe prototype, a hypersonic wind tunnel, cryogenic liquids, and the test firing of a rocket motor?  please follow up and post video! :-)

5:06 PM

FRIDAY, JANUARY 20, 2006

Waiting for IPO

POSTED BY RON AT 11:17 AM

There's a really weird play by Samuel Beckett called Waiting for Godot in which the main characters basically flail around helplessly while they await the arrival of someone (or something) named Godot. It's one of those weird French things that no one really understands (a cynic might opine that this is because it actually makes no sense), but it made a pretty good metaphor for my life between 2001 and 2005. The big difference was, of course, that in the play Godot never arrives. (Sorry about the unannounced spoiler but believe me, you aren't missing a thing.)

In the late 1980's I bought some shares of a company called Amgen at about $13 a share, and watched it go up to $20, $30, $40... Woohoo! This stock market thing was pretty cool. I called my broker and put in a limit order to sell at $50 a share.

Amgen hit $49-7/8 (they were still doing fractions back then) before plunging back down. I finally bailed at $15. (Hm, I just looked at how the stock has done for the first time in over ten years. I should have hung on to it.)

That experience, coupled with horror stories I heard from Google colleagues about watching helplessly as their options sank underwater during the dotcom crash, made me determined that this time I would avoid the greed trap and just take the money and run. My wife and I had talked about what we would have to make in order to deem the year of nightmare commuting to have been worthwhile and we came up with a few numbers. There was the "it would be OK but just barely" number, the "that was definitely worth it" number, and the "holy cow, never in our wildest dreams would we have imagined it could do that well" number. That last number was today's equivalent (the stock split twice before the IPO) of $25 a share. Just before I left it was looking as if that number was not outside the realm of possibility.

I probably would have sold out then given half a chance if for no other reason than that I really wanted to put the whole Google experience behind me. But it was a private company, and I had no idea how to go about finding a buyer. I did a little investigating and was told that I couldn't even put a classified ad in the paper because that would constitute a public offering, which was illegal until the company was approved by the SEC for an IPO. So I waited. And waited. And waited.

In 2003 I decided to ask Google if, as a stockholder, I could see the latest financial reports. They were reluctant, but eventually agreed, but only on the condition that I would sign an NDA, which I did. I flew up to San Jose and was allowed to see the latest figures, but not to make copies of take any notes.

It should come as no surprise to anyone at this point that I liked what I saw. $25 a share was looking like not only a possibility but a likelihood. I ratcheted up my expectations a notch and started to allow myself to contemplate numbers like 40 and 50. (It's funny how your perceptions change. Numbers that once seemed beyond my wildest dreams would now seem like an unparalleled disaster.)

On this trip I also had a chance to visit my old colleagues and see how much the 'plex had grown. They were in multiple buildings now, and there was a little fleet of Segways for getting around between them. I got a very warm reception from everyone, which was a pleasant surprise. I was really worried that I'd not left on the best of terms. And a number of people even suggested that I come back, that the company could really use someone who knew what it was like in the good old days. Google had just acquired Applied Semantics down in Santa Monica, so I could actually work for them with only a slightly insane commute instead of a completely ridiculous one. But when I asked HR they came back and said that Google wasn't interested in having me come back. So I guess I probably had burned a bridge or two after all.

Which was unfortunate, because things were not going well for me at JPL. To really explain why I would have to dive too deeply into the whole JPL story, which is not really appropriate here. To put the situation in a nutshell, JPL, responding to pressure from Congress and the Bush Administration, was moving away from doing basic research. At the same time there was a greater emphasis within NASA on collaborative research involving large teams across multiple NASA centers, and being a team player was never one of my strongest skills. As a result it became harder and harder for me to get my research proposals funded.

I might have worked harder to adress the situation if not for two other things. First, I was really getting tired of playing the research game -- and it is a game. Success as a researcher ultimately boils down to how successful you are at getting proposals funded, and that ultimately depends on this bizarre mix of technology and politics. Because NASA research budgets were shrinking the competition was getting more intense. As a result, every CFP (call for proposals) would get more and more applications. Ostensibly, the review boards were looking for the highest quality proposals, but in fact what they were looking for were the proposals that, if funded, would get them in the least amount of trouble. If you have two proposals and one of them comes from CMU and the other one comes from Podunk U. it's a lot less likely that the president of Podunk is going to call their Congressman and complain if you send the money to CMU than the other way around.

Being at JPL was actually a significant advantage in that regard, because JPL, while not quite an 800 lb. gorilla in the robotics world probably weighs in at a solid 650. But there was another problem: in order to reduce the number of proposals and make it easier to weed them out, the requirements for proposals started to get more and more demanding. Back in the good old days you could dash of two or three pages sketching out an idea, send them to a program manager, and if the idea was a good one get a check in a matter of weeks. Now proposals were running twenty, thirty, forty pages, and most of those were budgets, schedules, legal documents, and other bogus crap that had nothing at all to do with whether or not the proposal had any technical merit, but which you had to have because if you didn't the proposal would just be summarily chucked. If you managed to get through all that, it would often happen that after all the proposals were submitted that the review process would drag on for many, many months, sometimes as long as a year or two. Sometimes the program would just be cancelled outright. And, finally, they started asking you to certify that if you got your funding (never mind that you had no idea if or when you would get it) that you would actually be available to do the work.

The upshot of all this was the the people who were getting their proposals funded were the ones who dedicated themselves to mastering the process of writing proposals, and who were willing to lie about their availability. It also helped if none of what you were proposing was too far out of the mainstream of fashion in your field. That was pretty much three strikes for me. My entire career had been built on advancing radical new ideas, and I have a record to show that I was pretty good at it. But there seemed to be no place for someone like me in the new NASA. So I cast my eyes towards the IPO and hung on as best I could.

And I wrote a screenplay.

I was reading Slashdot one day and saw a story about how Robert DeNiro's production company was looking for screenplays with a scientist as the main character. (Oddly enough, I can no longer seem to find that story either.) I thought to myself, hey, I know all about the drama of being a scientist, I can do this! So I bought myself a copy of Screenwriting 101 (actually it was "Secrets of Film Writing" by Tom Lazarus) and wrote up a draft.

When I go back and look at that first draft now it makes me cringe, but at the time I thought it was a shoe-in for next year's best screenplay Oscar. Little did I know. Hollywood has a well-deserved reputation for stomping big egos into little piles of quivering mush. But I'm getting ahead of myself again.

As everyone knows, Google filed for its IPO on April 30, 2004. You'd think that would have been a relief, but in fact it was the start of an incredibly nerve-wracking year of obsessing about what the stock price would be. The process seemed to drag on forever, and everything seemed to go wrong. The Playboy interview. The furor and confusion over the auction. The outrage over the $135 a share price, which eventually got rolled back to $85, which still seemed dangerously high at the time.

We ordinary shmoes weren't allowed to sell at the IPO. Our stock was restricted and could not be sold until 180 days after, which is common practice. The investors who put up the actual money for the company want to be able to cash out first. Google negotiated a special deal with the SEC that allowed the rank and file to sell some of their stock early, but it still struck me as terribly unfair that the senior managers were able to sell at the IPO while the rest of us had to wait.

Of course, having to wait worked out incredibly well for me, because by the time the restrictions were finally lifted in February of 2005 the stock price had more than doubled. If I had been able to sell when I wanted to I would have left a pretty significant amount of money on the table, even more than I actually have. Sometimes I look at the stock price and figure out how much money I would have had if I'd stayed at Google for four years and held on to all the stock. It's a mind-boggling number, one that I never in my wildest dreams imagined would ever be in reach.

I have no complaints, though. I'm not filthy rich, but I did make enough that I don't have to work to pay the bills any more (which is one of the reasons I have time to blog). I resigned from JPL in November of 2004.

Through the work I've done on my screenplay I've met a lot of people in the movie industry, though no one has bought my script yet. I spent a year trying unsuccessfully to launch an independnt film distribution company. While working on that, we hired a publicist who got me onto the red carpet at a number of Hollywood events. She pitched me as "the Google guy", which seemed to get the photographer's attention. Being on the red carpet is a really weird experience. The first time it's kind of cool, but after a while you start to realize that it's actually work, and pretty dull work at that. The reporters fire all kinds of inane questions at you and you have to come up with clever sounding answers on the fly. I was really, really bad at it. It gave me a renewed respect for celebrities. It's actually not easy at all looking good on camera, at least it wasn't for me.

So I've ended up in a really good place. I've been given a rare opportunity to figure out once again what I want to be when I grow up, and the truth is that I'm not entirely sure yet. I'm getting involved with a venture capital firm, hoping to learn more about how to run a business. But I have no idea if I'll still be doing that a year from now.

What strikes me most, looking back on the whole thing now, is the extent to which fate dragged me kicking and screaming the whole way to where I am now. I am not a particularly religious person, but sometimes I really wonder if there might not be a script writer behind the scenes. One of the reasons I'm taking so long to pick a new direction in life is that I feel a certain sense of responsibility to do something worthwhile in Act 2.

That's pretty much the end of my Google story, so I'll probably be signing off Xooglers for now. (I've been kind of rushing to get to the end because I'm actually going to be unavailable until mid-March.) Thanks to Doug for letting me crash his blog. And thanks to everyone who has been reading this and writing comments. There is no greater reward for a writer than to be read.

Some day I'll tell my JPL story. It's actually much more interesting :-)

35 COMMENTS:

Cole Blak Pascal said...

I made you a wallpaper Because I found the font that Google used. Very interesting blog. One of the few I actually read/subcribe to to see when updated.

12:11 PM

Bernardo said...

Aw, is this the end of Ron's story? Oh well. Thanks for the interesting and pleasurable read, and best of luck in your future endeavours. Do tell the JPL story sometime, though, and let us know where you post it.  Doug, I wonder how you feel about letting Ron steal the spotlight. Now we're all expecting some dramatic and interesting stories about Google branding...

12:13 PM

torh said...

We'll miss you and your stories.  Let's hope Doug has a lot of stories left, and that other ex-googlers will join in.

12:18 PM

Anonymous said...

The slashdot story must be this one: http://slashdot.org/articles/02/09/13/2131217.shtml?tid=97

12:29 PM

Ron said...

Yep, that's the one.

12:42 PM

Dov said...

Great read. Thanks for sharing

1:00 PM

Jonathan Ellis said...

I enjoyed your story. Thanks for taking the time to share it, and good luck.

1:06 PM

Bálint said...

All have been said... Thanks for the story and best regards!

Anonymous said...

Thanks for your story and please, tell JPL story as well. This blog has been absolutely fantastic, "best of the web" as someone wisely described.  Good luck for future, what ever it might be.

2:38 PM

RLT said...

Is the screenplay based on the JPL story? If not, maybe it should be...or the basis for a second one. I second Bernardo. Please post it, then drop by and let us know where to find it.  Also, I have the greatest admiration for your Act 2 intentions. Best of luck!

2:47 PM

Chris said...

Ron, it's been great reading your entries - this blog is one of the best stars in my feed list. Thanks for sharing!

3:18 PM

Anonymous said...

My thinking:  Had he stayed at Google, he's now be worth 20-30 million. Having left, he still made a million or two. Enough to retire (since that's what screenwriting amounts to).

3:34 PM

**** Based ONLY on the information in the Thread Topics , this appears to be a likely scenario... ****  1- The constantly expressed theme that Google was not satisfied with your productivity - and wanted to terminate the relationship  ________________  With networking, and the flood of quality resumes they were receiving, if there was others more qualified with proven experiences, they would likely have taken advantage of those opportunities to use them.  In one post, you had indicated that you were never easy to get along with - and in another there was an admission of sometimes being insulting when talking to others - and in another there was a concern about not being given any performance reviews,  So, what may have been occurring is valid concern that any attempt to commuicate would have be taken the wrong way or misinterpreted, so perhaps they decided to not risk it. Perhaps they felt your sensitively would adversly affect your productivity and further SCAR an already pressured atmosphere.  You pocessed important, rare skills, and had succeeded in the Major project you were given, which does provide a very, very important source of revenue for Google... the company may have been more willing to adapt to certain personality quirks, than they would have with an easily "replaceable" worker.  Ironically, if an insulting remark is made to someone that really hurts, it may prevent that person from ever being fully comfortable about interacting with the insulter again.  So, just as in a couple of posts you THANKED the readers for the KIND posted remarks,..... kind words do naturally seem to elicit more interaction.  Ease of Communication appeared to be the major problem - not work quality.  There was ALWAYS the option to be proactive, and politely and earnestly ask them - how could you improve, and to please access your weaknesses.   Ironically, there are some people who DO take that first step when they are "Worried".   If Fate - and the channels of communication ever open in the future, it would be beneficial to have a CALM and Intellectual discussion with those very same people on WHAT & WHY they felt -

5:07 PM

Kenneth Jr said...

I've been reading your blog entries almost concurrent with "The Google Story" and I have to say that your take is much more pragmatic and personal. Google is a great company, but it isn't everything to everyone. Will you continue posting your entries on this blog? (subscribed to ATOM feed)

7:03 PM

Anonymous said...

Ron, I've been following each one of your posts in this blog and I'd like to congratulate for sharing, and telling, a wonderful story. In the end, when you put it all together in retrospective, it was very well told.  Plus, your experience I think, will serve some of us who want to be part of the technology scene.

10:04 PM

Vivek said...

Ron, thanks for sharing a wonderfully well-written story. Too bad you are thinking of leaving Xooglers......your JPL story will be a bigger hit, I am sure. If your screenplay is anything like this, I am sure its going to be bought some time soon.  All the Best to you, wherever you go!!!

10:28 PM

Teemus said...

Beckett is Irish, so I'd hardly consider his play French ;)

3:50 AM

David Holmes said...

Thanks for sharing, Ron. If your screenplay is anywhere near as well written as your story here, then I'm sure we'll be hearing from you again.  I'll be looking forward to it.

6:33 AM

Anonymous said...

Thanks for telling your tale, Ron, it's been very entertaining and interesting. I'll add my voice to the growing list of people wanting to hear your JPL story - that ought to be really good!  Cheers, Peter (Cape Town, South Africa)

7:06 AM

Anonymous said...

Thanks Ron, best wishes for the future.

7:08 AM

Marcus said...

It's been a great read Ron. Thanks for sharing your story. It has been especially interesting and valuable to me as a computer science student to have an inside peek.  Best of luck with all your future endeavors!

8:53 AM

Ron said...

Beckett is Irish, so I'd hardly consider his play French   There's no reason an Irishman can't write a French play. See http://www.theatrehistory.com/french/beckett002.html:  "The utter simplicity of the play, in the histrionic sense, places it in the classical tradition of French playwriting."

Teemus said...

There's no reason an Irishman can't write a French play.  Ah, but tell that to the Irish ;)

11:14 AM

Anonymous said...

Ron,  I'm happy you are able to do what you want now. E.g. hack lisp all you want, make your own robots like Trevor Blackwell or write screenplays.  It is interesting that talented lisp hackers are so often loners who irritate the hell out of other people. Also, looks like you didn't so much burn your bridges, but blow them up [when you burn them, they might be easily repaired].  Best of luck with your screenplay. Just go easy on the schmalz, please!

12:28 AM

Anonymous said...

Hi Ron,  Thanks so much for your story. :) I was pretty much hooked all throughout..  All the best for the future  Terng

2:18 AM

Anonymous said...

am a nigerian based in nigeria and i read this blog all the time. I really enjoyed it. looking forward to reading bout your life. bet it would be as funny as this

7:30 AM

Anonymous said...

Thank you for the stories.

8:04 AM

Dumb Bacchus said...

Evey good story makes you forget whats on your head for a while... i do not have the ability to express it in words.  Ron i had a pleasent time reading all of your blogs. Thanks for writing

10:31 AM

Spiderman said...

Thank you Ron for your interesting posts. I've been reading your story with much interest and it mirrored to some extent my own story.   I hope that you will keep posting here once in a while.

2:01 PM

david said...

thanks for sharing

10:55 PM

Anonymous said...

Ron, Reading your narration was very nice and I was able to see Google through your eyes.I thought you would write more but it is disappointing that you have shortened it.I am eagerly waiting to read what you are going to write next

10:59 AM

Anonymous said...

Ron! A big thanks for all these stories. I read every piece and i must admit that it was gorgeous. Not only the behind-the-scene character of the thing but the way you built it up, the tesion and such things. I enjoyed reading it. You defintely have a story telling/writing ability, i wonder why you couldn't sold that screenplay yet:) I'm looking forward to reading that JPL story. Hope you'll find the thing that you'd most like to in the furure.

9:07 AM

Jack DeNeut said...

The original title of Beckett's play is En attendant Godot. The English version is a translation of the original French version.   Yes, Beckett was Irish. But Nabokov was Russian, and I wouldn't call Lolita a "famous Russian novel".

3:02 AM

Don said...

Ron, great writing to tell your story. I feel a real identification because of going through similar situations 20 years before. As you ponder what you will be when you grow up and work to get there, I would recommend that you look up your local Baha'i Community and ask them what they are doing with their lives and how they got where they are.

9:51 AM

amar said...

this was really interesting & well told. if you ever tell your jpl story i'd read it!

5:05 PM

FRIDAY, MARCH 03, 2006

(Un)Stuck in Peru

POSTED BY RON AT 5:43 AM

This isn't a Google-related post, but I thought I'd check in anyway because, as you will soon see, I have a bit of free time on my hands, and I thought you all might enjoy watching a little drama unfold in real time.

My wife and I are on the last leg of a trip to South America, which until now has gone very smoothly. The climax of the trip was an excursion to Machu Pichu, which I'll write about some other time. Suffice it to say that pictures can't even begin to do the place justice.

There are only two ways to get to Machu pichu. You can take the train from Cuzco. Or you can walk seventy miles or so along the Inca Trail. (We opted for the train.) Either way, you have to start in Cuzco. And when it's time to leave you have to leave from Cuzco.

At this altitude I was expecting Cuzco to be only a little village, but it is a real city with about half a million inhabitants, and a real airport with jetways and everything. At 11,000 feet or so, Cuzco is the world's highest city [Update: This turns out not to be true at all. See the comments.] (It is also the longest continually inhabited city in America, having been founded about 1000 years or so ago as the capital of the Inca empire.)

As I said, our trip so far has gone very smoothly. We've had no significant snafus, and very good luck with the weather. Yesterday at Machu Pichu we had a nice overcast that kept it cool for our hike up the Inca Trail to the Sun Gate overlooking the main ruins. Then the sun came out. It was a glorious day.

Today we were supposed to fly from Cuzco to Lima to catch an early morning flight to LA via Miami. But this morning our luck with the weather finally ran out. There's a storm system moving through, and Cuzco airport is closed because of low ceilings. And according to wunderground, it's not supposed to clear up until Sunday. So this could be the beginning of a very long day.

I've already looked into the possibility of getting out of here by driving. It's only an hour long flight, but I'm told that it's an 18 hour drive under ideal conditions. So we're hunkering down for an extended stay at the airport.

Wish us luck. We may be here a while.

Update at 7:40 PM Peru time: the sky cleared just long enough for us to catch a flight to Arequipa (on LAN airlines, appropriately enough) and from there to Lima, so it looks like we'll make it home on schedule. Quite a story to tell (including how I've been able to hook up to the internet) but not nearly enough time to do it justice right now. We have to pack for tomorrow's flight, get some food, and if we're really lucky, a little sleep. Fifteen hours on the road (so to speak) tomorrow.

Update #2, 9 PM California time on March 4: Made it home. Too beat to blog. Will try tomorrow.

10 COMMENTS:

Matthias said...

Not that it'll do you much good, but Patrice (a Swiss guy) is located in Lima. Small world etc. :-)

7:23 AM

LiQiuD said...

Well there are worse places to be stuck...at least you have internet access. :)  Good luck

9:58 AM

Alex Chiang said...

You don't want to drive. The roads between Cuzco and Lima are rather horrible, and you probably won't be ssving yourself much time anyhow.  Stay in Cuzco, check out some of the other nearby ruins like Sacsayhuaman, other museos, etc. and fly out on Sunday. Your quality of life will be much higher.  /ac

10:05 AM

Patrice said...

Probably too late now. But a bus drive with for example Cruz del Sur works fine. Not as pleasant as a flight but sufficient.  Also noticed that the weather wasn't as good yesterday here in Lima.  So enjoy Cusco and stay away from the boys around Plaza de Armas (many of them are dishonest and criminal).  There is an Inca museum in Cusco that I rather enjoyed. It's located just off the Plaza de Armas in a yellow building. They expect a tip for the tour but do take it - it's really worth it.

10:42 AM

mahlen said...

For what it's worth, Lhasa, Tibet, at 12,100 feet and estimated at 300,000 to 400,000 people, likely qualifies as a higher city than Cuzco.

1:41 PM

Ron said...

Lhasa, Tibet, at 12,100 feet and estimated at 300,000 to 400,000 people, likely qualifies as a higher city than Cuzco.  Right. I think Cuzco is the highest in the Western hemisphere. My brain is not firing on all cylinders just now.

4:46 PM

Andrew Hitchcock said...

I was in Peru this summer. The second time we flew into the Cusco airport, there was a transit strike. Were driven to our hotels in a military bus because no other buses were running and there were people marching in the street.

5:54 PM

Anonymous said...

I hope the sun was still up when you passed through Arequipa. One of my most vivid memories of Peru is the view of the volcano from the Arequipa airport on a similar stopover to yours.

8:24 AM

Anonymous said...

That was a very insightful story on the inner workings of google, as told by an ex-employee. Thank you.

8:37 AM

Anonymous said...

Cuzco is plenty high, but La Paz, Bolivia is higher with an elevation around 12,000 ft in the central city. The airport at La Paz is the highest in the world at 13,313ft.

1:47 PM

MONDAY, MARCH 06, 2006

Peroogle?

POSTED BY RON AT 3:25 PM

Doug asked me not to turn Xooglers into a travelogue, so I've been racking my brain to figure out some way to relate my Peru experience to Google. Other than the obvious, that but for Google I probably would not have had the time to make this trip, I couldn't come up with anything. (Well, there's a Peruvian Google: google.com.pe. But that's hardly news.) So I'll be posting the rest of my Peru story on my personal blog, and since I'm going to do that I'll probably just go ahead and turn it into a real travelogue, which means it will take me a while to write it up.

In the meantime, having taken the stage here, I feel obligated to come up with something Google-related to say, so here's a little anecdote:

While I was at Google they finished the first iteration of the human genome project. I was a member of the American Association of the Advancement of Science (AAAS) at the time, so I got their weekly journal, Science, in the mail. One issue came with an enormous fold-out poster showing a schematic diagram of the human genome. Of course, not being a biologist I couldn't make heads nor tails of it, but it looked cool, so I took it in to the office and taped it up on the wall outside my office.

A few days later someone stuck a post-it note on it showing the location of the "Google gene." A while later someone else put up a post-it showing the "hacker gene." After a week or two the whole thing was peppered with post-its. Some of them were pretty clever too, but unfortunately I can't remember any of them. (This was five years ago, and I've had a lot of Pisco sours in the last few days.)

When I went to revisit the 'plex in 2003 the poster was still on the wall where I had first put it up. It was almost completely covered in post-its. For all I know it's still there. Perhaps some Googler will post a few of the more interesting additions in the comments.

16 COMMENTS:

SM Guy said...

Wow this is now a Top 100 most boring blog. Zzzzzz.

5:30 PM

Blake6489 said...

cool,  looking forward to the travelog

5:54 PM

The Yacht Broker said...

Guys if'v got nothing to say, just leave it, don't make another stupid blog. ;-(

5:08 AM

Anonymous said...

Hehe I love the genome poster story. I wonder if its still there?

5:23 AM

David Holmes said...

Hey sm guy & e-globe... Don't like it here ? You may use the door silently, ie. without gratuitous nasty comments on the way out.  Fun anecdote that, I do wish someone from Google would show up and tell !  Oh, and agreeing with not turning this blog away from its original intent. Even if posts have to be few and far between while the whole story of a new xoogler is awaited, where else are Google fans gonna get such little inside stories ? :-)

5:49 AM

Peter said...

If anyone can tell me which building/floor/general location it's at I'll go take a look today and copy the good ones down. peter at peterd dot net

7:57 AM

Anonymous said...

Oh, I love that genome story! I love the way the computer community thinks. As a whole, they are quick-witted, ironic and/or totally off the wall. I love it. I'm so glad there are now more forums for their commentary than a few columns in computer magazines each month. I wish I knew where find more of their blogs.  I especially love this column. As far as I'm concerned, you can't beat having an interesting topic discussed in an entertaining manner. Please keep up the great work.  Rebecca

8:20 AM

Ron said...

If anyone can tell me which building/floor/general location it's at I'll go take a look today and copy the good ones down  It was outside the door of my old office, which was on the second floor of the original Googleplex building. (I think it's called Building 0 now, but I'm not sure.) It was room 2233, near the central stairway, in the wing opposite where Larry and Sergey's office used to be (or maybe still is?)  Here's a photo of what the view out the window looked like in case the room numbering scheme has changed:  http://www.flownet.com/ron/pictures/office-view.JPG

10:21 AM

Peter said...

I've actually never heard of building 0, but I'm pretty new. Most of my coworkers haven't either, they've been around for about 2 years. Can you point it out on a map for me?

8:44 PM

Ron said...

Can you point it out on a map for me?  http://www.flownet.com/ron/plex.jpg

12:55 AM

The Yacht Broker said...

To David Holmes- hey there's nothing nasty in my small comment. :-} In fact, I like Xoglers, thets why I'm here sometimes. But every time just trying to find some new info about Google(not Peru).:-)

8:34 AM

Ron said...

just trying to find some new info about Google(not Peru).:-)  I'm sorry if my posts are not living up to your standards. If you are not satisfied with the content on Xooglers I'm sure that Doug will cheerfully refund the price of admission.

12:54 PM

Anonymous said...

Well said, Ron.  And I would like to say I have found this blog quite enjoyable to read, and the small anecdotes like this one give it a friendly feel and are amusing and entertaining.

1:44 PM

Anonymous said...

Would you please stop posting now? I'm totally bored by your posts. All your knowledge is outdated and as you write in your posts, you didn't leave on good terms.

10:46 PM

gm said...

Would you please stop posting now? I'm totally bored by your posts. All your knowledge is outdated and as you write in your posts, you didn't leave on good terms.  All his knowledge is outdated? Perhaps what you're looking for is this. This is Xooglers, please try and know what you're talking about before you post. Thanks.  With the popularity come the trolls... ignore it. Thanks for keeping us updated on your travels Ron, I'll be watching your other blog...

7:55 PM

Anonymous said...

You guys have only a few days until April 1. My brain waves are tingling for a story..... -stefan

FRIDAY, MARCH 10, 2006

My two cents on China

POSTED BY RON AT 1:04 PM

A reporter from the New York Times was doing a story about Google and China, which made me sit down and think about where I stand on this issue.

Google's detractors argue that the company is being hypocrtical by saying that they will not be evil while at the same time doing business with the devil.

The problem with this critque IMO is that it conflates China with the Chinese government. They are not the same thing, any more than the United States is the same thing as the United State Government. The Executive Branch of our government has done some things that I would consider evil. If I were to follow the logic of Google's critics I suppose that to maintain my own ethical integrity I would have to renounce my citizenship and move to New Zealand. (To tell you the truth, there are times when that seems like a pretty tempting option.)

I also can't help but wonder how many of those who are labelling Google as hypocritical go to the trouble of maintaining their own moral purity by scrupulously avoiding the purchase of any product made in China.

16 COMMENTS:

Anonymous said...

I couldn't agree more, but actually for some different reasons. In the interst of full disclosure, I am a foogler (future googler, I've accepted an offer for after I graduate in June), and with a clear conscience regarding China for the following rationale.   For me, my reasons are based around potential harm versus benefit to individual Chinese citizens.  1. Google offers potential benefit to Chinese users in China. The way that Google is engaging China offers a pareto improvement over the status quo. Google is developing the first search engine located IN China that will actually inform Chinese citizens IF their search results are being censored "according to local law". Moreover, Google is still developing their Chinese language version of their regular search engine, providing improved resources to anyone managing to skirt the "great firewall". Lastly, I feel fairly confident that Google.cn is going to be able to provide more relevant and informative results than what baidu and other competitors in China had been able to provide.  2. Direct harm to individuals as a result of Google's actions is minimized, if not eliminated. Google is going out of its way to not provide services that would place it in a position of having to finger dissidents for the Chinese government. Google is thus not providing Chinese G-mail or blogging services.   Compare this to the actions of Yahoo, who has identified individuals at the request of the Chinese government. These individuals then WENT TO JAIL as a result of Yahoo's actions.  Then taking this one step further, compare this to Americans purchasing goods in China that are made by companies that face little or no environmental regulation. As the author of this post pointed out, this could be conceived as collaboration with the Chinese Government, but that doesn't really bother me all that much. What bothers me is that the lack of environmental regulation is proving disastrous to the lives of Chinese peasants in the countryside, who are dying of cancer in record numbers due to heavily contaminated rivers and groundwate. Regional Chinese officials frequently face incentives to maximize economic performance through industrial output, regardless of the environmental consequences.  If this sounds like a conspiracy theory to you, I'd recomend reading http://www.globalpolicy.org/socecon/envronmt/2005/0704pollutionchina.htm or googling for any number of other resources.  Here's the kicker: The demand for the products of the polluting industries is largely driven by the ravenous appetite of the US market. Ordinary people like you and me, those who work at Google, those who line up outside to protest, and most everyone else for that matter.  Civil liberties are to me certainly a subset of human rights, but I don't see them as universally trumping what I consider to be other human rights, like clean air, water and food that won't give you cancer.  Why do we seem to conceive of civil liberties as indivisible? An all or nothing game? We certainly don't hold this standard towards other human rights: The red cross only gets access to some portions of North Korea, rather than the entire famine-stricken countryside. Nobody accuses them of caving to Kim Jong-Il since they aren't given complete access.  I can't help but think that if Madison, Jefferson, and some of our other slave-holding founding fathers had championed environmental protection as part of the bill of rights (although this was obviously not on the radar at that point) that many would be singing a different tune.

2:36 PM

KenP said...

I guess that'd be a typical programmer's response...in binary.  You are painting a picture in black and white. Doesn't work for China, the US, NZ or Whistler's Mother.  When folks talk about moving out, I am always one who'll chip in for the ticket.   Regardless of side, these are difficult times. My father had his called The Great Depression and WWII. Today isn't a bed of roses either but if you don't like it then don't let the door...on your way out.

2:49 PM

////I also can't help but wonder how many of those who are labelling Google as hypocritical go to the trouble of maintaining their own moral purity by scrupulously avoiding the purchase of any product made in China   Whenever you TOTALLY STOP doing business with a * "COUNTRY" *....  ....you are hurting many innocent, powerless people who are attempting to earn a living to survive, and, who have absolutely no authority in deciding the policies of a regime.    How far is too far, how much loss should the average joe and his family experience???

3:21 PM

JoeChongq said...

I am not sure what to think about the China deal. There are too many things to consider to ever clearly decide which choice is better. People are never going to be happy as long as Google cooperates with China's government to censor results. Whether we like that or not, Google does need to follow the laws of governments where it operates just as companies from other countries must follow our laws when doing business in the US.  Anyway, if doing business in China is so bad why isn't the US government talking about boycotting the Beijing Olympics in 2008? That is going to bring a ton of income and exposure to the country. I am sure any company that is involved in providing services for that Olympics will be required to be very careful what they do and say over there.

3:33 PM

MissM said...

China, Japan, and Saudi Arabia pwn the United States, through our record breaking deficit. We don't dare boycott them, because of the economic catastrophe that would cause.

5:08 PM

P. Sternberg said...

Another thing that so many critics seem to forget: if you are unwilling to compromise with those in power, you will never affect change. The only alternative is to use force to overpower those in control. In most cases, this leads to tremendous human suffering, and often doesn't even do much to improve the situation.

6:41 PM

Anonymous said...

I would agree with Doug. It is quite hypocritical to say Google is doing business with the devil. Especially outrageous was the comment from Rep. Tom Lantos, D-San Mateo, who questioned that how Google executive could sleep at night. Well, Mr. Lantos, how could YOU sleep at night when the US government has been and is still engaged in prisoner abuse in Iraq, Afghanistan, and Guantanamo bay ? How could you sleep at night when you authorized President Bush to wage this immoral war that killed perhaps tens of thousands of innocent Iraqi people?

7:08 PM

Dan Harris said...

I am anything but an apologist for China, but I this whole Google in China thing has gotten out of hand. We are focusing on China now because the U.S. government wants us to focus on China now. China violates human rights, no doubt about it. But, it is slowly but surely improving its human rights record and that is occurring organically, not because of pressure from the West. What I find ridiculous is that China is getting so much heat when far worse human rights violators barely merit a mention. The Sudan, Saudi Arabia, Zimbabwe, Egypt, North Korea, Yemen. … The commenter who noted that every company is, at least to a certain extent, supporting a repressive regime by doing business with that regime is dead on. Google in China is getting the heat because both Google and China are newsworthy. One can sell computers to Egypt and nobody says a thing. www.chinalawblog.com

2:27 AM

ChinaLawBlog.com said...

2:28 AM

KenP said...

I can find sympathy and identify with almost everything said. Facts litter the responses and some might even be part of a valid dataset.  We've seemed to conclude that the world continues to make one step forward and two back. I wonder if my serf/peasant european ancestors would agree. We all want that binary result that makes for "Peace in our Time" and that isn't working any better than it did for Chamberlein.  The problem here isn't the mess the world is in. It is the self-proclaimed guru status associated with 'The Bobsey Twins build a search engine" and the mantra they created. Unique in our business world and so the media make their Beetlesque trek to the mountain top and come back unenlightened--not that one can enlighten the media. Does the world have a right to expect Google to resolve the world's problems? No. But Google did create an atmosphere that lets hope and aspiration converge on the boy genius pair and find them wanting.   'Do no harm' is a lovely idea but all of the above post point out that the world isn't a place that ever happens. The best ideas always end up with unintended consequences. And thats what is affecting Google.

4:36 AM

aJ said...

So you mean to say NZ is not evil? :P I know it's all because they have the Kiwis and US does not ;)  Ok, only kidding but I agree with your point. The government does not mean the country and not doing business with a country is not really a good decision.  Ps. No, I am not an american/NZ/chinese :)

7:41 AM

Anonymous said...

Ron, you misrepresent what the majority of Google's detractors on the China issue are saying. They are not complaining that Google has opened operations in China. They are not even complaining that Google is doing business with the Chinese government. And somehow you fail to mention the central theme to their complaints -- censorship?  That's lame. Of course it's a common tactic -- misrepresent people with whom you disagree, to undermine their arguments. Google's detractors are hypocrites that lack moral purity, because they have likely purchased goods manufactured in China.  Google claims a rather noble purpose [http://www.google.com/corporate/]: "Google's mission is to organize the world's information and make it universally accessible and useful."  Yet Google.cn filters many sites from search results and reorders the results. So if a citizen of China does a search on "Tiananmen Square" (or the Chinese equivalent), they will get results that, taken as a whole, misrepresent the events of 1989 and the people who took part in them.  So the Chinese government misrepresents their detractors. And you, Ron, misrepresent Google's detractors. No wonder why you don't have a problem with the deal.

8:51 PM

Ron said...

Ron, you misrepresent what the majority of Google's detractors on the China issue are saying.... you fail to mention the central theme to their complaints -- censorship  Well, I cloaked the argument in a metaphor: doing business with the devil. When you do business with the devil you have to do it on the devil's terms. Making all of the world's information fully accessible in China is simply not one of the available options. The choices are: make some of the world's information available, or make none of it available. Given the reality of the situation, I see making some of it available as a defensible choice even for a company striving not to do evil.  Censorship is evil, no question about it. But I don't see how Google taking its ball and bat and going home is supposed to make things any better.  BTW, the filters on google.cn are really easy to defeat, e.g.: http://images.google.cn/images?q=tienamen

5:54 PM

Anonymous said...

You all obviously know nothing about china.  You dont know anything about China unless you actually go there, speak the language, and know and understand the local problems and people that are in that country.   So don't sit back in your arm chair in a north american house and act like you know anything about China beyond the poor drivel which leaks through our news system every once in a while. And please refrain from spreading drivel - it does not make you an expert - and you sound stupid to anyone who actually KNOWS about china.   Cheers.

3:48 AM

Anonymous said...

And another thing. Before you start sqwalking like some poster child for free speech, why dont you think about this:   The news we get here is different from the news they get. Its also different from the news they get in every other place in the world.  Searching for some things will bring up results in Shanghai that you dont get in Boston.  So what.  They have 65 million people in the area that speak in Chinese, why do they give a rats ass about if your page shows up. They have chinese businesses. They read chinese. They want chinese sites. Chinese material. Chinese news papers because they read chinese. Not english.  But wait, there is more...  If you actually looked for  "Tiananmen Square" in Chinese, rather than english, you would get different results again.   And by the way, you can get CNN, and FOX and you can read about freedom of speech, and whatever you want in China.  If you want to make a difference in Asia you can start by arresting all of the American tourists who goto Thailand Bangkok to fuck childern.

4:02 AM

Anonymous said...

AND ANOTHER THING.  Next time you want to talk about  Tiananmen Square consider this,  The square is a place where people go to gather. They have celebrations. It is an important part of China. And it means something different to Chinese than Americans.   http://images.google.cn/images?svnum=10&hl=zh-CN&oq=Tiananmen&sa=X&oi=pinyin&resnum=0&ct=result&cd=1&q=%E5%A4%A9%E5%AE%89%E9%97%A8&spell=1  Stop looking for people to fight with. America has alot of domestic issues that can be solved before they need to worry about fixing other peoples problems.

4:12 AM

FRIDAY, MARCH 10, 2006

Speaking of trouble...

POSTED BY RON AT 1:16 PM

The bloom seems to be coming off the rose. After more than a year the naysayers are finally having their day.

Frankly, I have never had much respect for Google's detractors. Most criticism of Google has been based on ignorance and hysteria. The fact of the matter is, Google makes a kick-ass product, and they have used it to make a pile of money in the free market playing fair and square. I have nothing but the highest regard for them.

Nonetheless, it has always been a bit of a mystery to me why anyone bought Google's stock, let alone paid $470 a share for it not so long ago. Not that I'm complaining, mind you. But I am, and always have been, puzzled by people's willingness to pay these exhorbitant prices for Google. It's not because Google isn't a great company -- it is. But the stock has always seemed to me like a bad deal. Let me explain.

Suppose I wanted to sell you a share of my car. There are many reasons why you might want to buy. For example, I might let you drive it part of the time. Or maybe it's a limousine that I lease out, and I'd give you part of the income. Or maybe it's a collector's item and I'd give you part of the proceeds when I sell it.

But suppose I offered you the following deal: I'll sell you a share of my car, but you don't get to drive it. It generates income, but I won't share it with you, nor give you any say in what I do with the proceeds. I have no intention of ever selling it. I won't listen to anything you have to say about how the car ought to be used, and I won't tell you what I plan to do with it. In fact, buying a share of my car will provide you with no tangible benefits whatsoever. All it will do is give you bragging rights that you own a share of my car.

How many people do you think would accept a deal like that?

Well, if that car is Google, a lot.

Just as there are many reasons one might want to buy a share of a car, there are many reasons one might want to buy a share of a company. The company might pay dividends, or it might offer the potential of paying dividends in the future. If you buy a big enough share you can have a real say in how the company is run. Even if you don't have the wherewithall to buy a big enough share to let you run the company, your small share might be sought out by someone who does have the means to buy a controlling interest. Or being a shareholder might entitle you to certain special privileges. (We own some stock in a winery that gives a 30% discount to its shareholders. We've made back our original investments many times over on wine discounts alone. The stock also pays dividends.)

But with Google, none of those reasons apply. Google has explicitly stated that it will never pay dividends. They have explicitly said that they will offer no guidance to investors. Their stock structure is such that even if you bought every single publicly traded share of Google stock, Larry and Sergey would still control the company because their privately held stock has ten times as many votes as your publicly traded shares. So there is no hope that your shares will ever be of value to someone attempting a hostile takeover of Google. Such a thing is simply not possible, so no one will ever try.

There is really no difference at all between buying a share of Google and buying a share of my hypothetical car, which you never get to drive and over which you have no control and which will never put money in your pocket. The only difference is that the foolishness of the deal is more evident in the case of the car. In the case of a company as big and complicated as Google it is easier to convince yourself that there is some rational reason for the stock to have value. After all, there are a zillion people out there snapping up shares.

That's the same mentality that led people to buy shares of doctorkoop.com.

The big difference, of course, is that Google is making money. Lots of it. Theoretically the shareholders own part of that money. But in reality Larry and Sergey have all the say in how that money is spent, and the shareholders have none, and the company is structured in such a way that that can never change. It's a bizarre kind of ownership, no different at all from "owning" part of a car that you never get to drive and which provides you with no tangible benefit (and no hope of future tangible benefits) except the possibility of selling your share to the next sucker for more than you paid for it.

Why does anyone buy Google? I have no idea. But I'm damn glad that they do.

78 COMMENTS:

Anonymous said...

I'm the owner of fuckedgoogle.com, which you mentioned in your first sentence.  Couldn't agree with you more.

2:52 PM

ceejayoz said...

Honestly, how many people buy any stocks - Google or otherwise - with the intention of being able to change the direction of the company?  No small investor is ever going to have much ability to change the direction of a big company. I can't do a hostile takeover of General Electric, whether the company CEO has 51% of the stock or not.

3:33 PM

Ron said...

I can't do a hostile takeover of General Electric, whether the company CEO has 51% of the stock or not.  No, but some day someone else might. Just the possibility that someone might want to acquire a controlling interest in GE some day -- and that if they wanted to they could -- gives your stock value.  And while you wait for that happy day, your GE stock pays dividends.

5:10 PM

Anonymous said...

ceejayoz: think of a small investor as a voter in a political election. They can't take over the government with one vote, but they can use that one vote to support the "takeover bid" of the political party or candidate of their choice. A single vote is not a lot, but everything happens by the accumulation of single votes.  The small investor buys stocks as an investment, but the only value in Google common stock is that people still want to own it. Everything in a market derives it's value from the fact that people want to own it or control it in some way, but in general there is always something useful they can do with it once they've got it, *other* than selling it to the next sucker--some non-market value. If there isn't, the system is sustained purely by tacit agreement, which always collapses when someone realizes that the first person to get out wins, and everybody follows in a giant stampede. This is not a problem with normal stocks because even if everyone panics, you still own value even if the stock price goes to $0. If there is no other value.... Anyone who owns Google stock runs the risk of losing their investment if the demand collapses.  Things like this just go to show that "the greater fool" is a huge part of the way the stock market works. Not only are fundamentals optional, but value itself is optional. Value can apparently be created out of nothing (but hype) and sustained for indefinite periods of time.  I see potential for new business models in the area of finding the most efficient way to make the public believe that you can print money.... What does a certificate have to be backed by in order for fraud prosecutors to leave you alone? Hmm.... (Note: I'm not saying that Google has defrauded people, intends to defraud people, or ever intended to defraud people. In any case, I think suckers, erm, people who buy stocks, are responsible for their own decisions.)

6:16 PM

Anonymous said...

> In any case, I think suckers, erm, people who buy stocks, are responsible for their own decisions.)   Your 401K plan (it’s invested in stock market funds) is full of Google stocks. Why ? Becouse they are part of NASDAQ.  Stocks exchange was happy to sell thouse to you - becouse they earn commision on every transfer.

chl said...

You're not trying to be serious here, right?

7:08 AM

Anonymous said...

Your useing blogger.com OWNED by google so I guess you dont hate them that much since your useing blogger.com.

9:57 AM

Anonymous said...

Lots of people buy stock if they think it has a chance of going higher, even just a percent or two so they can makes some fast cash and get out. Its just gambling.

10:23 AM

Anonymous said...

Are you feeling a little embittered?

10:37 AM

Anonymous said...

I don't think anyone is embittered. It was mentioned a couple of times that Google is a great company. In the eyes of the author the stock just doesn't mean anything. It's like monopoly money.

10:53 AM

Anonymous said...

You're ignorning the simplist and most obvious reason people buy Google and most other stocks, which in the hope that the value of the stock will appreciate, the price will go up, and they will sell at a profit. While this can be "gambling" it isn't necessarily -- it depends on the psychology and strategy of the trader or investor. If I believe that Google will continue to grow and increase revenues over the coming years, it is entirely rational, and not at all "gambling" for me to invest in the stock in anticipation that the value of the stock will appreciate and the price will reflect this.   Also, I think probably the most common and unfortunate misunderstanding of the capital markets by the general public is the idea the price of a share of stock in any way reflects on whether that stock is "expensive". Whether or not a stock is expensive can only be determined by the relationship of price to earnings, not by stock price alone, and this is reflected in the P/E ratio. Also, it is crticially important to grasp the P/E is based not on price alone, but on EPS, which relates price to the number of shares outstanding (the float).  Another sense in which this can be understood is with stock splits. Google has essentially said they will not split the stock. What if they did split the stock? In your view, would the stock suddenly be 50% less expesnive? Well, of course not, because when the stock splits everyone gets 2 shares for each 1 they hold, and so the float is doubled. EPS and P/E stay exactly the same in a split, because the underlying value is unchanged. Now, it should be noted that usually when a stock splits, the price jumps on the news -- there are a variety of reasons for this, I won't get into.  You may still say Google is expensive, based on it's P/E of 67, but that's another matter!

10:59 AM

Anonymous said...

Google stock has the right amount of hype and speculation that makes it a good commodity to buy and sell.  *ironic*People are foolish enough to think that if the original people made so much money by selling their stock, they can also, just buy it a 400$ a piece and sell it at 800$ piece.*ironic*  It will certainly stabilize at some point and, while it won't be a crash per se, it will crash the finances of some foolish people in the public.  I think all this extreme hype might bite Google in the ass one day.

11:05 AM

Anonymous said...

This has to be one of the dumbest things I've ever read, unless it's a joke and then it's just unfunny.  Do you have any understanding of what a stock is, what investment is, and what speculation is?  If your car had the chance to magically appreciate in value, and I could sell my share in it for more then I paid, then yes it would be the same thing.  As it stands, you're just a retard.

11:14 AM

Anonymous said...

That's the entire problem with the stock market and the world economy at large.  If a company sold a million shares at 100 dollars each they raise 100 million dollars.  Now that 100 dollar share is suddenly worth 400 million dollars. The comany technically only has the 100 million hard cash it raised. The rest of that is some version of exotic matter called BS.  Some investors cash out at 400 dollars a share and they walk off with 100 dollars of the real money and 300 dollars of the BS.  Parent company goes bankrupt.. all shares are evaporated and any hard assets are sold for maybe 10% of the company if you're lucky. Debtors get the remaining money first.  Considering that most of the BS evaporated with the company there's the BS that left with investors that got out at the right time.  It would seem that this happens all the time and there must be a lot of BS in the stock markets.  Luckily all that BS exists as numbers in computers all over the world because if you tried to cash out every dollar there was there's not enough printed money to cover it. I doubt there's enough gold to cover it.

11:19 AM

paulwk said...

Comparing Google to a car is just a really bad comparison. Also, almost ALL stocks are overpriced really. I mean, no one know what the future performance of a company is going to be. Hell, Google might just turn out to be the greatest company ever, we just don't know, so whos to say that $400 is too much for a piece of the company. My opinion, yeah it's too expensive, but my friend, it sounds like you have a bone to pick.

11:20 AM

Anonymous said...

"There is really no difference at all between buying a share of Google and buying a share of my hypothetical car"  Expect that cars depreciate in value over time and earn no income, while google's revenues and profits have been growing consistently.   I'd delete this post, if possible, and try a new analogy.

11:25 AM

Anonymous said...

The US dollar is backed by nothing more tangible than faith in the US government.   It can no longer be directly exchanged with the government for something of value like gold.  Despite this the dollar does have value, and seems to share these qualities with google stock.

11:27 AM

Anonymous said...

How much for a share of your car then?

11:34 AM

Anonymous said...

I think you have to compare to BRK.A and Buffet whom the Google founders mentioned in their public offering. He offers no guidance or dividends, but people still buy that stock and have been rewarded amply for the past forty years. This is also why I think they will never split the stock.

11:35 AM

mylicon said...

Your car analogy is valid but you seem to be confusing stock shares with timeshares. If an investor wants a controlling interest in the company, then they become a venture capitalist. Small investors don't buy dozens of shares with the idea of having a controlling interest in the company. They buy them with the prospect that given an acceptable risk, they will claim a return on their investment either in dividents or share value. Google isn't the first or the last company to state they will not pay dividends but rather reinvest the capital into the company. If the stock split so share prices fell would this make you feel better? $1000 for 2 shares is the same as $1000 for 1000 shares. A 5% increase in share value for both would still yield the same return.

11:41 AM

Tarry said...

Hmmm all anonymous folks here,  Chicken? (Ok just kiddin)  97% of the money that we're talking about you is (as a study says) all gas and just 3% of it is money. So if everyone's gasballing why bash Google. Eventually when the dust and prices will settle down, then maybe people will get more realistic and value it's products.  These guys are busy doing good things on the web, (whore) fighting is just he part you have to get used to.

11:42 AM

Anonymous said...

are you stupid? no one is forcing anyone to buy their stock. either you buy it on their terms, or you don't. why should they relinquish any level of control? they don't have to - their stock sells itself, plain and simple. why complicate things? also, go take a basic high school level economics class

11:46 AM

Anonymous said...

It seems to me that a number of you have missed the point. The analogy between a google and some hypothetical car might not be as far off as you think.  Heck, if you've been following any sort of news, Google isn't growing at the same astounding rate it started with. It's slowing down.  When it boils down, google has about.. ohh.. I don't know. One salable product. Sure, they have search, search, blogging, mail, search, and more search, but the only thing that makes them any money is advertising (when did you last pay to send a message with gmail?). If you honestly believe that there is no saturation point for this kind of business model, I have a couple of investments you might be interested in...

11:53 AM

blogcruiser said...

lol, so true. The stock market has become nothing but a made up reality for people to make money for nothing (or steal from workers) and it is a bad idea all the way around.

11:55 AM

Anonymous said...

Someone said cars depreciate, but that depends on the car. Collectibles go up in value.  The only reason anyone is buying Google stock is they hope it will appreciate in value and then they will sell for a profit.  Stocks aren't really priced based on PE ratio, EPS or any other nonsense. It's all psychology - what people think it's worth.  If everyone believes it is valuable, then it is valuable as people start buying and driving up the price.   Now the opposite is starting to happen. Google is seen as deviating from their core business and searching for new revenue streams. People are questioning whether they can continually increase revenues, believe that they cannot, so are dumping the stock and forcing prices down.  It's all about investor confidence and that is falling a bit for Google at the moment.

11:59 AM

Anonymous said...

"If your car had the chance to magically appreciate in value, and I could sell my share in it for more then I paid, then yes it would be the same thing."  Until people realized that there is no reason to own stock in your car, then your stock will be worth nothing. I think that's the author's point. The value of your car is irrelevant, it has no impact on the value of the stock. The value of the stock is purely driven by hype, and once people recognize that the stock will become worthless.

12:01 PM

Anonymous said...

It's interesting to see people defending useless stock using terms like "magically appreciate".  Some stock has value outside of magic appreciation. Dividends, buybacks, buyouts... all of these transfer the actual value of the company to the stockholders who own that company. People are panicking over google's revenues without realizing that they have no stake in that revenue.  What is the connection between the stock price and the revenue then? Let me ask this, if 2006 was an amazing year for google, let's say they made a trillion dollars, what do you think would happen to the stock? I'm sure everyone will claim it will go up, but ask yourselves "why"? Since the stock has nothing at all to do with the revenue, claiming that the amazing windfall had anything to do with the increase of stock price is like claiming that the sun rises because I shave every morning.  In the end, these valueless stocks are nothing more than a popularity contest.

12:25 PM

speel said...

I wish I could work at Google so I can get all that free crap in the google food store. xD

12:29 PM

Anonymous said...

Before I even read your story I'm going to tell you that I might not make it through, or click on any of the links. Seriously, attaching links to random words in a story, giving them no specific context... is just idiotic.

12:40 PM

Anonymous said...

Google's stock price is predicated on the bigger idiot theory. People buy the stock, demand outstrips supply, the stock price increases, stockholders sell their stock for a profit to others who hope to enjoy the same benefits of irrational demand.  There is absolutely no other reason to buy Google's stock.

1:08 PM

Regular said...

the worst thing that could have been done is this entry being digg'd a lot of stupid anonymous diggers posting comments. Go back to Digg. Thanks.

1:13 PM

Anonymous said...

Google stock still has value even if it never pays dividends and does not give you any control in the way a company is run.   The stock has value because if another company buys Google you would get a percentage of the sale price based on the percentage of the company you owned.   You don't have to wait for that to happen to make money though, because you can sell your ownership to someone else who is willing to wait, and on and on. As the company grows, it can be potentially sold for more money. That, in turn, makes your stock worth more money.  Going back to your car analogy, I would buy stock in your car if I believed your cars value would increase. If your car is worth $1,000 today, and I can buy 10% for $100. If 5 years from now you sell your car for $2000 I get 10% of that, or $200. Whether or not I got to drive the car is irrelevant. I also believe that you are smart and will take car of the car so not having any control over it doesn't matter to me.

1:37 PM

Anonymous said...

The fact is, that our entire economy is based on the "bigger idiot theory".  Nothing is more based on that theory than the US dollar. The vallue of the dollar is backed by the fact that the USA promises that it will continue to be worth something. When we were on the gold standard, there was gold put away equal to the value of all the money in circulation. (Now there is just a powerful idea backing it. Truly, that is nothing to sneeze at.) But you could never go to Fort Knox and ask them for $27.53 worth of gold.  Google stock has value because it represents owning a piece of a company. The fact that it doesn't give anyone a say in the direction of the company, may actually be a comfort to some. (I certainly don't know how to run a major company.) It also means that each person who buys Google is putting trust in the current managers. Why is that so much more foolish than putting trust in a vast number of shareholders?

Anonymous said...

Special voting stock for a company whose mission has community responsibility is necessary -- look at Knight-Ridder for an example of the havoc that ensures when a newspaper company doesn't have that protection, and look at the New York Times for an example of the stability that comes from having that protection. Without it, we'd have a race to the bottom, every newspaper would be USA Today, and every search engine would be like the old Overture pay-for-play search engine.

Anonymous said...

I had always assumed that google entered the stock market (which I admit to knowing very little about), as a kind of parody. Sure, there will be money to be made, but I also have the feeling that the techies are trying to get another kind of payback against those driven by making money.

2:08 PM

Scring said...

Pfff, afraid of seing people like you so misunderstanding stock options, and saying google is bad only because they don't give dividende ...  Someone has previously said 'Do you have any understanding of what a stock is, what investment is, and what speculation is?'  I think you should have a look on this before thinking everyone is stupid to buy stock. And why having involving google, it is the same thing with many other companies ...  To conclude, really a ridiculous article, do not waste time to read it :(

2:26 PM

Ron said...

look at the New York Times  NYT pays a dividend. (In fact, NYT has been paying dividends for the last hundred and five years.)

2:29 PM

Anonymous said...

Believe the Lie

2:35 PM

Ron said...

Expect that cars depreciate in value over time and earn no income  Not so. Cars can produce income (e.g. rental cars, taxis, delivery trucks). Cars can appreciate in value. Companies can depreciate. There really is no substantive difference.  Cars tend to depreciate more than companies not because there is any inherent difference between a car and a company but simply because cars are mass-produced on a larger scale. But companies can be mass-produced too (they are called "franchises") and unless their production is carefully controlled to balance supply and demand, they will depreciate just like cars. Amway and Thomas Kinkade galleries are two good examples.

2:56 PM

Steve said...

You buy stock with the assumption that it will go up significantly. Its a risk/benefit analysis, you buy the stock at $470 and bet that it'll go up to $600. If your bet is right then you've made a good chunk of money. Dividends and discounts are not the only benefits of a stock, and often times are the least important. Are you sure you know how the stock market works? Some individual investors have seen returns of over 400% in their investment, that is why people are investing in it.

3:29 PM

mizzle said...

Larry and Sergey are very smart people as we know and since the beginning (after they got the $100,000 to get started) they have refused to give up control of their company--risking $25 million just to have their say. With that said, their stock still boomed even though the public knew about the inability to control the company. Investing is about taking risk.."mylicon" replied well to this..so if I could make a 500% profit in four years by doing absolutely nothing, I would love to do so.  In summary, Google had a good idea and useful product. For the same reason that the private donor gave Larry and Sergey $100,000 investors are buying stocks--they see potential and are willing to take a risk in hopes of making money.  I wrote an article on Google's success in my blog if you're interested, check it out:here

3:40 PM

Anonymous said...

Sorry to say it, but your car analogy is quite a weak one and you clearly do not understand the fundamental concepts of corporate finance. I will absolutley shed some light on it, if you like.   Nonetheless, keep in mind that it is the responsibility of company leadership to do what is the most beneficial for stockholders (dividend or no dividend) within the constructs set forth within the corporate charter, at the very least.   Furthermore, as a minority common stock holder (which I am assuming you are), you have no rights to directly affect the direction, intentions, or dealings of the company in which you have equity.

3:46 PM

Ron said...

you buy the stock at $470 and bet that it'll go up to $600  "Bet" being the operative word here in your theory of how the stock market works.  Some individual investors have seen returns of over 400%  Investors? You mean "gamblers" don't you?

3:46 PM

RevMike said...

I actually read the prospectus, and page 106 is interesting. Over time, the class B common stock will convert to class A common stock. So while the class A holders don't have much say now, they will in the future.  Interestingly, two of the major events for this conversion will be the deaths of Larry and Sergey. Upon their deaths other stock holders will be able to install their own board. Effectively, they've put prices on their own heads. I try to avoid any arrangement where someone else benefits from my death.

3:47 PM

Anonymous said...

The stock market may be ridiculous, but this has nothing to do with google specificallly. Your car analogy is god awful. I wish I could be more constructive, but you arise contempt in me.

4:04 PM

Anonymous said...

Ron notes:  "The NY Times pays a dividend"  Companies in growth businesses generally don't pay dividends, because they need every dollar they can get for investing back into the business. To pay a dividend is to say your company is boxed into an area with growth opportunities that are limited as compared to the amount of cash the business throws off.   Or else, you're trying to get institutions whose charter require dividend paying stocks to buy your stock.  Or else, as in the NY Times and Dow Jones cases, big stockholders are heirs to the founders, and need cash as  income, but want to hold on to stock to maintain control of the family business.  If Google paid a dividend, Wall Street would be calling for Eric's resignation, and with good reason, because the factors above are all not true. He'd be forced to underinvest in the business, or to borrow money to replace the dividend payout for investment, or do additional secondary offerings, all of which are bad ideas for Google in 2006.

Anonymous said...

If I am to understand what big 'G' has pulled off here, it is probably unprecedented in the history of the markets. I think everybody here has forgotten the basic concept of the capital markets- When you purchase stock in a company, be it 1 or 2000 shares you literally own a portion of the company. That means you have a vote on the leadership and direction of the company and most importantly: ACCOUNTABILITY OF THE BOARD AND LEADERSHIP to YOU as a stock holder. That is really what you are buying with a share, despite all the psychology of the marketplace.  Big 'G' has figured out how to effectively erase the accountability, and therefore erase the entire value of the stock! In fact, the stock is pure B.S. regardless of the shape of the company. The company in fact is irrelevent, there is a complete disconnect between the stock and the company and that is EXACTLY why we have sarbane-oxley- because this is NOT a good thing.  What a scam! I'm in the wrong biz. I should be President.

5:08 PM

Ron said...

Companies in growth businesses generally don't pay dividends  That's right, but they also generally don't have two-class stock either, and they generally provide investors with some guidance to help them decide how to value the stock.  The stock has value because if another company buys Google you would get a percentage of the sale price based on the percentage of the company you owned.  Yes, this is the merger/hostile-takeover scenario that I mentioned. However, this scenario is not possible in Google's case because it is not possible to buy control of the company by acquiring its class A stock. Control of Google can only be had by acquiring its class B stock, which is not publicly traded.

5:31 PM

Anonymous said...

Which winery are you referring to? I want to own it....

5:44 PM

Gus said...

There has never been a stock that somebody thinks is to expensive and someone else thinks is too cheap. That is what makes the market!  If you think Google is overpriced... short it. I'm guessing the people that shorted at $200 are still licking their wounds.  It may be overpriced... or it may be cheap. Nobody really knows, and the only opinions that matter are those that are willing to put there money on the line by buying or selling.  p.s. I spent 10 years making my living as a trader.

6:01 PM

Anonymous said...

Who is this idiot? A small minority of stocks offer dividends or special deals on wine. People buy those. This is the stupidest thing I've ever read on the internet. Seriously, this beats all the crap I've ever ready before. You're right, but not about GOOG, about nearly every stock. Why would you pick on one company like this.

6:57 PM

Christopher Constant said...

people buy stocks to sell them at a later date, when they raise in price. a 9 year old could have told you that.

7:44 PM

Anonymous said...

Who would want to buy google stock?  I did!   I bought 100 shares at $105 and sold them all at $400 Yahoo, Microsoft, and a bunch of other companies bought hundreds of thousands of shares. Look, a lot of people don't buy stock "because they have interest in the company" they buy it because they have interest in $ and buying stock in a particular company could possibly bring them more of it - but also possibly not. Tell you the truth, I would rather go to Atlantic City and put my $ on black than buy/sell stocks, its more fun. But when a company like google is going public, the hype alone is going to bring stock holders $.

8:30 PM

Anonymous said...

Most of a small, but signifcant, shareholders power lies not from the ability to sell to large investors - who may eventually accrue a controlling interest, but from their ability to rapidly dump stock and put a severe dent in the stock price.  For example if I owned 5% of Google, and decided one day that Larry and Sergey had lost the plot and to dump it all, I could easily cut in half - or more - the stock price. That's why even small investors have to be kept happy and how they wield some power.  Besides dividend, another source of future potential revenue for a shareholder is the theoritical prospect of the company owners wishing to re-privatise. So, if the owners wanted complete sole control again, and wanted to avoid the hassle of public filings, they would issue a buyback and the price can be said to be hthe market's guess at how much they'd be willing to pay to own the company outright again.  Of course Google is so large that they are obliged to do nearly all of the public filings anyway, and when the owners have a 10-to-1 control ratio, there really is no incentive for them to ever buyback. GOOG remains a terrible deal.

4:17 AM

Anonymous said...

While the OA raises interesting points, it is a bit simplistic.  Larry and Sergey have a very strong incentive to keep GOOG as high as possible. They still have a lot of shares, so their personal wealth is directly related to the share price. A high stock price will also enable them acquire other companies “on the cheap”. So if any of the OA arguments worried investors sufficiently to make Google’s share price wobble, you can bet L&S will take corrective action.  Concerning takeovers & class A/B stock. While the current structure forbids a hostile takeover, nothing can exclude a non-hostile take over. Of course, this seems extremely seems unlikely now. For an other take on the subject, see also mylicon's answer.  Concerning dividends. Google's non-dividend statement has to be taken with a grain of salt. Once Google becomes a mature company, and will not have any good opportunities to invest its profit (if it has some at that time) and it will redistribute them as dividends (or buy back its stock). Of course, it will take a long time for Google to become a mature company, but it *will* happen -- if Google does not go bankrupt before. This is analogous to Microsoft which paid its first dividend in *2003*.   P.S. If you read all posts to here, you are wasting your time. The only answers worth reading are from mylicon, Anonymous at 1:37 PM and RevMike.

6:31 AM

Anonymous said...

Ron, all the things you said about Google Share, apply equally to bills (bank notes). Except of course, that money is usually still owned by the government, so you can't even do with it what you want. You're only allowed to use it. Yet most humans spent their most valuable resource for those bills, their most valuable resource being, of course, their time which they spend at work to get more and more banknotes, in the hopes of trading it against something they think is more valuable to them (and do so hopefully, before inflation shrinks its' value).  The notable difference being, that the value of stock over the last few decades has one up, while a healthy dose of inflation is expected for moeny, and indeed every money used on this plent has seen more devaluation/inflation than stock.  Therefore many people prefer to trade their money for stocks, for the mere the "possibility of selling [the] share to the next sucker for more than [they] paid for it".

6:50 AM

Ron said...

Ron, all the things you said about Google Share, apply equally to bills (bank notes).  No, that turns out not to be true. Take look at:  http://wfhummel.cnchost.com/  and in particular:  http://wfhummel.cnchost.com/moneytaxesdebt.html  Dollars have value because they are what U.S. residents need to pay their taxes. This insures an ongoing demand for dollars, and hence their continued value. (So in essence, the value of the dollar is ultimately maintained by the force of the power of the U.S. government to put you in prison.)

Anonymous said...

Uh, are you kidding me? Many people buy GOOG because they speculate the price will go up.  Buy at $400 Price goes up to $470, sell Make $70 profit  Take your $70 to the bank, don't give a damn about dividends or control of the company.

Anonymous said...

Yeah, and there are these stupid things called "mutual funds," which not only don't pay dividends or offer any promise of benefitting from takeovers, but charge *fees* of their investors!  What kind of sap would ever want to buy into an arrangement like that one? Jeez....

1:19 PM

Anonymous said...

Most people here seem to be spending so much time slamming this post that they fail to see the validity of the question. Perhaps it should be rephrased:  Why do so many people feel Google's stock price will rise?  One of the closest comparisons I can come up with is Apple. Both companies have seen their stock price go up at least five-fold in 24 months. At least. Both stocks have large amounts of shares available, are high-profile, and are (somewhat) in the same market.  Now let's compare their bottom lines, products, and revenue sources.  Or, uh, let's not.   Google doesn't really have any except advertising. And it's finally becoming known just how overstated that revenue source is compared to its effectiveness.  A much, much better stock buy is Yahoo.  Definition of a bubble: an investor belief that defies reality to a point of moving stock prices up way more than reality supports.  NASDAQ closed Friday (3/10/2006) at 2262.04, almost 230 points higher than the close two years ealirer of 1964.15. That's a 15.27% gain. Problem is, it also is the anniversary of the best close ever (3/10/2000) of 5048.62.  This means it STILL is 55.19% lower than 6 years ago. Bubbles....  Until Google get some new revenue lines, they probably won't see their historic highs either.   Apple is a much better "bet" since they are ahead of their Intel switch promises, with an iBook being announced shortly in time for back-to-school purchaes... and a new video iPod eventually... since they are now into the video download business more than ever... oh, and there's that 30th anniversary comoing up in a couple of weeks.

1:27 PM

Anonymous said...

your car analogy is, let's just say, ridiculous: If I could sell your car share I just bought to John for a profit, that's good enough for me...and most investors.  No one is looking to "own" or rule Google forever; they're simply hoping to buy low and sell high.

9:50 PM

Anonymous said...

I think the point of your original post goes to "intrinsic value". It seems that Google has little intrinsic value for investors. The value of GOOG to investors is virtually entirely based on expectations. They expect the stock to go up, which obviously could happen, but begs the question "why?" P/E ratios have been a traditional metric for stocks, and create the expectation of rising stock price based on increased earnings. It's difficult to see how such metrics should apply to GOOG since the structure of company ownership clouds the concepts of enterprise and book value. Don't shareholders eventually have to ask what real benefit they derive from ownership? If they themselves don't ask, won't the pool of potential subsequent owners? Ponzi schemes offer an interesting comparison where "value" is derived totally from the expectation of a high return on investment without underlying financial ownership benefits. Eventually the music stops and everyone tries to sit down.  Your original post was both timely and insightful.

5:12 AM

Amit Sharma said...

Not that it matters with such a long discussion, but if you would care for a different point of view on this post please see http://zuberon.com/archives/21.

10:53 AM

Google Sucks said...

Good points except for one thing. Google does not make 'kick ass products'. Perhaps they did about 5 years ago but that was then, this is now.

2:20 PM

Anonymous said...

Many companies have no inherent value, outside of irrational value or worth. Google has some minimal inherent value (as does Yahoo), but it's nowhere near $300/share. It's a freakin' search engine company, and it only does that about half as well as a few years ago. 10 years from now, nobody will know who Google is in online search. I don't know who will take them over, but I do know it's inevitable that someone does (because history is a great teacher).

9:36 AM

Anonymous said...

Your logic is completely flawed. You act as if people buy a stock only for dividends. They buy it because they expect they value to rise. Cars depreciate with time: poor analogy too.

1:58 PM

Anonymous said...

It seems that 98% of the readers of this post have missed the point.  I propose that you demonstrate this point by selling Monopoly $100 bills, autographed by you. Sell them for $20 each on your web site.  Once a few people have bought them, you can convince more people to buy them by demonstrating that they could probably sell the bills at $30 each to investors would could sell the bills at $40 each. It's a giant pyramid scheme, must like some stocks.

2:14 PM

Anonymous said...

Even the Don agrees. Join the crusade, spread the word. Dont use google.com on 06/06/06.  dongoog.com

4:20 PM

Anonymous said...

It is true most the people that have responeded to this Blog don't have a clue as to what the author is getting at. As for why people buy stocks, the true investor is purchasing an asset. Something of value, but not only does it have value it also has the potential to generate income(dividends) annualy. Most people without enough money to really take ownership of ture assests have the small minded "gambling approach" short term investors looking to make a quick buck. Traders love you guys 'cause your in and out constantly. If you were to survey 100 of the richest people around they wouldn't buy into google at this point. And that has nothing to do with the quality of the company, it has to do with their value of the stock. What am I purchasing... "A hypothetical CAR" GET IT. it's not real.... it's value is perceeved and derived strictley from demand .

6:35 AM

zippy said...

Your thinking about dividends isn't quite right. Some companies, such as Berkshire Hathaway, choose not to issue dividends because they feel they can increase the value of the stock by reinvesting profits in their business.  The owners of Berkshire Hathaway shares benefit from this, because in the long run, it is Berkshire's ability to generate increasing revenue that causes the price of its stock to rise.  There's one other reason not to offer dividends - taxes. Dividends are taxed twice by the government, once as the company's profit, and once as revenue to the shareholder. When a company reinvests profits, it can potentially realize a greater benefit to stockholders by avoiding some of the tax penalty.  --Pat

4:57 PM

Anonymous said...

You stole your post from my post on CraigsList except I used my house as an analogy.  Thanks for giving me credit, dick.

1:22 AM

Ron said...

Thanks for giving me credit, dick.  A bit tricky for me to give you credit if I don't know who you are, don't you think?  (For the record, I have no idea what this anonymous poster is talking about. The only thing I've ever read on CraigsList is an ad.)

9:08 PM

Daddy Truth said...

I have a $ stake in google so my judgement could be clouded on the subject. Then again, maybe I did more research on this then some previous commentators.  Now check this...  Comments from the latest Analyst day steer to 100% in revenues growth year to year starting 2006 This makes 2.24 billion in revenues from 1.12 B a year earlier. Analyst exspectations are short of 2 B. that's ...ehrm... 10%, over 200 million dollars extra, in this quarter alone.  Google's stock moves since 1,5 years ago (has it just been that long?) were eventually based on the earnings that came , and are coming (?), through. (Check back in April)   IMPORTANT:Only in the short run is google's stock movement a popularity contest; missing/beating exspectations from banks are the MAIN movers of the market, as well as upgrades or downgrades from the same numbercrunchers , wich in this case, LUCKILY, don't get guidance) Why, you ask? This means that the quarterly outlook from the world's best stock model/analyst, isn't better then anyone elses that believes in Google's business model.  Google's stock price indeed has gone up fourfold over 2 years. This just means "the most visited website/15th largest supercomputer" 100 billion to borrow against/make deals. Wich it needed,being of course a Nasdaq growth company. Not just that. It needed it because it's the only worlwide name in internet. end of sentence.   Google is a media company. it delivers interactive content while showing advertising. Over 123 pageviews a month, for at least 69 million unique viewers each. Preselected for gender,income, and, most importantly affects, instantaneously when anyone enters an 'Adword' into the searchbox...  you can read the rest at my daddytruth.blogspot.com

7:52 PM

Anonymous said...

"Until people realized that there is no reason to own stock in your car, then your stock will be worth nothing. I think that's the author's point. The value of your car is irrelevant, it has no impact on the value of the stock. The value of the stock is purely driven by hype, and once people recognize that the stock will become worthless."  Three observations: 1.) Google, the company itself, is decently healthy. That is not the problem. 2.) Google, the company's stock, has been changing hands for a long time and is itself horribly overvalued. The problem is not that GOOG can't open a can; it's that GOOG is horribly overvalued and should be at least a fifth of what it is. If you buy Google stock right now, you are ignorant, insane, or foolish. 3.) Statements like yours and the author's basically tell me that you have no idea how stocks or our market works. In other words, please, please, please buy more stock! Someone like me will thank some lemming like you when they drill someone like you up the wrong end!

1:03 AM

ewangee said...

I'm with you on that score.   The whole stocks and shares thing is wierd, but then so a monetary system where the bank notes have no intrinsic value and the amout issued is in excess of the assets supporting them.

4:19 AM

Anonymous said...

Google may have good computers.And people who know how to gain information for people to search.  But information changes or is not avaluble to Google. For instanse in medicine.If someone has a medical problem don't assume that medicine has not advanced in that area.  Medicine is always advancing.You cannot go by old ways of medicine. It's just like technology it keeps getting better and changing.

5:38 PM

Anonymous said...

I'm the owner of Flogz.com (http://www.flogz.com). We've been noticing a hige increase in Google-related content on our site - as more and more sordid details come out day by day.  It's not just Google's silence and unwillingness to provide any transperency that makes it a horrible investment, but that many analysts are leading investors down the "primrose path" in terms of price targets - no matter how many lawsuits, or how much share dilution, or how ridiculously high the earnings targets are - for a long time they've been pumping this stock up like it's the "next Berkshire Hathaway" (which I've actually heard said, but can't quote the source).  Add to that the HUGE amount of shares that the google staff are dumping:  http://www.flogz.com/story/787/  and this could be the largest case of investment fraud the world has ever seen.

1:49 PM

Anonymous said...

lol, I made over $50,000 in a month buying there stocks

12:27 AM

MONDAY, MARCH 13, 2006

Google vs GOOG

POSTED BY RON AT 12:17 PM

My recent post on Google's stock price seems to have set the record for number of comments. Some of the comments indicate that some clarification is in order.

First, let me say that there is no question that Google, the company, is very valuable. Exactly how valuable is debatable, but they have a huge revenue stream and significant resources in terms of infrastructure and brain power, and so Google, inc. is clearly a very valuable asset. About this there is no question.

What I was questioning was not the value of the company, but specifically the value of Google's publicly traded (i.e. Class A common) stock. And the reason I was questioning it is because GOOG CMN A has three features that make it as far as I know unique among publicly traded stocks: 1) it pays no dividends and (if you believe the prospectus) never will, 2) ownership of all of the oustanding GOOG CMN A does not give you control of Google, inc. and 3) the company offers no guidance to its shareholders. (Berkshire Hathaway is similar in many respects, but the difference there is that BH's business model is much simpler, and it is much easier to determine what the value of the company's underlying assets are.)

Nominally, ownership of GOOG CMN A represents ownership of part of Google, inc. But the three features of the stock mentioned above make it a peculiar kind of ownership, a peculiarity that becomes evident if the conditions imposed on GOOG CMN A are translated into a different domain, like ownership of a car. These three features sever the usual ties between the stock and the underlying asset that the stock is supposed to represent, at least to a certain extent. Certainly there are differences between cars and companies, and reasoning by analogy is fraught with peril. But so far no one has pointed out any difference between cars and companies that I would consider significant.

Let me also say that I do not dispute the fact that GOOG CMN A has value. At the moment that value is $340 a share. Two months ago that value was $470 a share. By definition the value of a thing is what someone is willing to pay you for it. My question is whether that value is justified in some fundamental economic way, or if GOOG CMN A is just the latest example of a tulip craze.

15 COMMENTS:

Anonymous said...

Just to make sure the discussion doesn't end up like the last one, here is the only long-term rational explanation for the price I've seen:  When the founders die, all stock will have equal voting rights, so if you think the company will be around that long, there is fundamental value there.

2:42 PM

Eric Albert said...

I mostly agree with your comments about Google's stock, but I feel compelled to point out that the tulip craze wasn't actually a craze after all. I happened to write about it last week after a New York Times Magazine article piqued my interest, and it turns out that the evidence points to a rational market. It's just that most of the factors which explained why prices rocketed up are rarely mentioned.  Whether the same degree of rationality applies to Google's stock is of course another question altogether....

8:59 PM

Anonymous said...

The fact that no dividends are paid and that Google offers no guidance is not a characteristic of the Class of stock, rather are decisions made by the management at this point in time. S & G could decide to change this policy at any time and not run afoul of the convenant in the initial registration of the stock.

3:38 AM

Anonymous said...

Sorry, I meant "covenant" not "convenant"

3:41 AM

Miles Barr said...

I don't know the structure of Google's stock, or in particular L&S's stock, but it looks to me it's in their interest to keep the stock price up.  Beyond the ability to bring in debt and buy other companies on the cheap, L&S's value is tied to Google stock. If they want to turn that value into cash they have to sell that stock. Assuming they want to retain control as they do now, they'll sell more class A stock, which is the same that is currently out there. Hence it's in the interest of the people who control the company to keep the price of your class A stock up.  This seems enough to me to make the stock valuable, even though you have no direct control over the company. Or am I missing something blatantly obvious?

8:37 AM

Anonymous said...

Yeah I think the blatently obvious thing you're missing is that due to the complete lack of any other viable profitable dot.com stock at the time of Google's IPO, Google was able to write the rules so that only existing insiders and shareholders got anything, and anyone who bought the stock AFTER the IPO got fucked.  PT Barnum was right. But you know what? Even if everyone were to suddenly realize how bad they are fucked in this arrangement, it's too late- all the original founders and VCs have extracted about 19 billion dollars of profit from this machine.  Even if google stock went to zero tomorrow, not a single founder or VC would shed a tear or lose a second of sleep.  Guess what folks- you've been HAD.

4:35 PM

curiouscat said...

Berkshire Hathaway is similar in many respects. I do not believe Berkshire Hathaway is any easier to judge inheriently (in fact I believe it is harder). However Google has only been around a very short period of time which complicates valuation. Inheriently though I think Google's model is simpler to value than an insurance company with a bunch of full companies included.  Berkshire Hathaway is made much easier to value because you can trust Warren Buffett. But if you can't trust the complex financial dealings to be disclosed (which is most often the case) then the innate ease of valuing Berkshire Hathaway would be very difficult.  Keep up the great work.

7:18 PM

Shachar said...

Excuse me for "not getting it", but why is Google's stock any different than other dot coms?  In particular, why is the lack of guidance an issue?  If you buy stock for no financial benefit other than to eventually sell it at a higher price, then the subjective price is all that matters. What a company says about itself (guidance) may change the subjective, but does not, in my eyes, add any actual benefit to the stocks.  Someone explain the stock market to me?  Shachar

8:26 PM

Anonymous said...

Concerning point 1, are you aware that Microsoft was incorporated in 1981, and that its first dividend was handed out in 2003. I guess when Google says that they will never hand out dividends, their meaning of never is shorter then 22 years.

11:27 AM

Ron said...

why is the lack of guidance an issue?  Because without it you have no clue what Google's plans are, which gives you no basis for assessing whether you think they are going to grow. Sooner or later they will saturate the search advertising market. What then? Are they going to pursue print advertising? A web OS/office suite? The Google PC? Sergey's crazy idea du jour? (They've already branched out into space exploration.) Build hydrogen-powered cars? Stop trying to grow and start paying dividends?  Contrary to what many people seem to think, the stock market is not supposed to be a casino. It's supposed to be a market. Without guidance, GOOG CMN A is a pig in a poke, at least to a certain extent. I can understand wanting to roll the dice on an unknown (look at Acquicor). But at a PE of 70? You need some pretty serious growth to make that pay off, and if it were my money I'd want at least a clue as to how (or even IF) the company plans to achieve it.

12:24 PM

Ron said...

are you aware that Microsoft was incorporated in 1981, and that its first dividend was handed out in 2003  Yes, of course. I never said that Google's not paying dividends in and of itself was cause for concern. I never said that special voting stock in and of itself was cause for concern. (Google's failure to provide any investor guidance is, IMO, in and of itself cause for concern, though not necessarily a show stopper.)  It is these three things together that make me wonder what makes people value the stock as highly as they do. If any one of them were not the case then it would make more sense. If Google paid a dividend (like NYT does) then I could understand people putting up with no guidance and no voice. If there were no special voting stock then I could understand people putting up with (or perhaps even welcoming) no dividend. If there were guidance, if we knew what direction Google was heading, then I could understand putting up with everything else and buying the stock as a speculative play. At least we'd know what we were speculating on.  But none of those situations is the case here. With GOOG CMN A you get no dividend AND no voice AND no idea where the company is headed (AND, while we're at it, a stated policy of running the company for the benefit of their users rather than their customers or their shareholders). So why are people (still) paying three times more for Google (in terms of PE ratio) than they are for Yahoo or Microsoft?  Mind you, I am not complaining. My bank account is chock full of these people's dollars. But I really can't help but wonder why they were willing to pay so much for my stock.

1:53 PM

Anonymous said...

how odd, an anonymous poster said: "only existing insiders and shareholders got anything, and anyone who bought the stock AFTER the IPO got fucked."  you know, the IPO price was $85, I think it closed today just over $365, that's still close to 4 times IPO in no time at all. If you want, you can even look at its peak and people who sold then.  this person sounds more like a frustrated gambler who heard "on the street" that Google was skyrocketing so he/she wanted a piece of that action, got in at $400+ and then went "oops!", without realising the mistake was mostly his, partly the markets (for overvaluation) but really by no means Google's. The saddest thing about the person's - it would appear - uninformed view is that it seems characteristic of people who think that Google is taking money from shareholders. Yes shares were initially sold to the public and are still being issued and sold off at various prices, so some of the current "stock money" is going there. But much of that was in the past at a lower price and lets not forget Google's profits and besides, share issuing is a kind of loan anyway, and as all loans the terms vary.  Just lastly, I wanted to agree with a previous poster that the dividend thing is really a non-issue (apart from the fact that it might provide a type of indirect guidance which might be handy) and also to mention that a lot of companies have skyrocketed and looked ridiciulous at the time. Some of them crashed, others adjusted, in particular check out prices at MS and Dell for say last 15-20 years, although both are at a big low point at the moment, they still grew massively fast at one point and most of that stuck.  p.s. Guidance can excite short-term profiteering which is encouraged by most brokers (fee-related pay..) and so non-guidance was possibly meant to avoid that. Ooops...

1:28 PM

I-key Benney, CEO, Mscsrrr said...

Hello: Great eye-opening article. Thanks for the clarification. The Buddycom http://www.cashomatics.com

6:52 AM

Anonymous said...

"But so far no one has pointed out any difference between cars and companies that I would consider significant."  One is meant to Appreciate in value while another is built to Depreciate over time...

12:12 AM

Anonymous said...

The true value of a Google stock is zero, actually, less than zero. The questions is: does it really matter if it still makes you a multimillionaire? It speaks bundles about American values doesn't it?

FRIDAY, MARCH 24, 2006

Pardon me while I pick my jaw up off the floor

POSTED BY RON AT 9:26 PM

Just now I Googled for "mesothelioma lawyers". I clicked on an ad. Turns out that click cost someone $54.33. It's not just one crazy lawyer paying this much either. There are a staggering eighty-three ads that come up under those keywords! ("Tax attorney", at a mere $45.55 per click, gives you 124 ads! "Mortgages" at $34? 307 ads! And that's not even counting the top-of-the-screen ads.)

I didn't even have any idea what mesothelioma was before clicking on the ad. (Turns out it's a lung disease brought on by exposure to asbestos. No wonder there are lawyers involved.) I wonder, does my click count as click fraud?

Maybe GOOG CMN A isn't overpriced after all :-)

28 COMMENTS:

Anonymous said...

How do you think the advertiser feels about it?

11:46 PM

The Yacht Broker said...

I glad to see you guys back after 10 days, I thought  maybe something happend.:-/

2:55 AM

Anonymous said...

Was this a ruse to get "mesothelioma" ads served on your blog? Just 2 occurances of this magical word in your post and you already had those mesothelioma ads showing up.

6:01 AM

rafiq said...

I guess mesothelioma lawyers or attorneys in asbestos cases are willing to pay that much for the cost of advetising on adwords. They'll cover the costs on their clients bill, so I dont think they have a problem with paying those kind of prices.

6:39 AM

Anonymous said...

The reason they're willing to pay so much is that it's been lucrative to a constellation of individuals (lawyers, testing companies, x-ray readers, and people willing to claim they're victims), and it's been very slimey. Check out this story on NPR:  Silicosis Ruling Could Revamp Legal Landscape

10:30 AM

Rob said...

How do you find out the actual price someone has paid for an adwords ad? I checked the link to the tool in the article and it will give you a placement estimate based on your bid but I can't find the actual price anywhere?  -Rob

10:46 AM

Peter said...

"how to register a domain name" 27 bucks. Last time I checked, it only costs 20 max for a domain name for a year. (where 3 is the cheapest I've seen) So how do they make any money? for ever customer, they lose 7 bucks if it was from an Adsense referal.

11:32 AM

Anonymous said...

I hope your post bankrupts these lawyers. Frivolous lawsuits are a parasite on our economy.  So are peddlers of phentermine, viagra, and cialis by the way.

1:00 PM

Anonymous said...

 I hope your post bankrupts these lawyers. Frivolous lawsuits are a parasite on our economy.   No one is likely to get bankrupted, but if a few posts like these get some attention, they are going to be out a LOT of money. Even if only 20 people click the ad, be it from curiousity or contempt for the lawyers, that's over $1000 these firms are going to lose.

2:59 PM

Apparently, many people were suprised by this...That web page with the info made the front page of Digg yesterday.   just READ THEIR comments! :LOL  just READ THEIR comments! :LOL

6:53 PM

Andreas said...

You may know about working at Google, but you don't know much about Adwords. If someone bids $53 for a word, it doesn't mean they pay $53 for the click. The bid amt and the click price are totally different. Depending on the advertiser's CTR and many other factors, he could be paying $3 or less. I have Google Certification to manage Adwords. Some of my clients have some of the keywords in your list; they pay a small fraction of the bid amt. Poorly managed accts indeed pay high amts, but that's due to their errors.

2:28 PM

Matt said...

'I have a Google Certification to manage Adwords'   So you're feeding the machine then? Glad I've got my greasemonkey google ads remover  http://userscripts.org/scripts/show/2707

4:03 PM

Ron said...

you don't know much about Adwords  Guilty as charged. I went and signed up for an AdWords account. (I had one before -- in fact, I had the very first AdWords account, user id 1. But they changed their login system a long time ago.) It's true that the pricing scheme is much more complex than I thought. In fact, after half an hour of reading the documentation and playing around with the cost estimation tools I still don't understand exactly how it works.  Nonetheless, I'm still pretty surprised that someone would even be willing to pay that much for a single click, even if the amount they actually pay turns out to be less.

11:22 PM

Anonymous said...

Sell Structured Settlement 61.88 audio conference calling 61.54 800 conference calls 62.71 conference calling service 59.61   well? I'm sure you can go higher!

11:49 PM

Anonymous said...

Most of the high click amounts are only for ads that appear on google.com's search results. The lawyers learned a long time ago that it's not worth nearly that much to advertise on someone's blog or whatever. I wish whoever puts out that list of high paying keywords would mention that before they just encourage more people to start mesothelioma blogs.

6:58 AM

Chris Blake said...

These advertisers are not all paying the high click rate. The amount you mention might well be an indication of what a new advertiser might need to pay to get to the top. You can be sure that as you move down the list of ads on the search page, the lower the price paid. And also don't forget that ranking of the ads depends on the amount paid multiplied by the click-through rate. So it is likely that in many cases the guys on the number one ad spot with a well-worded ad may be paying less than several of the ads beneath them.

9:28 AM

Chris Blake said...

9:29 AM

Anonymous said...

Nice post. Mesothelioma makes me laugh, those lawyers must be making a killing. If you get bored, you can always go spend their money with a few clicks ;-)  Malc

12:31 PM

Anonymous said...

"Nice post. Mesothelioma makes me laugh,"  Tell that to Warren Zevon's family.

2:23 PM

Anonymous said...

As Bob asked about, being a newbee at all this...how does one find out the bidding price on Google?

2:16 AM

Anonymous said...

sorry..but you don't know or understand how this system works. first off...how did you get this high amount? through adwords estimator? if so, then it's more accurate than some "lists" on the net. secondly, there's a big difference between top bid and average bid. thirdly, as already mentioned, just because top bid pays that much, it doesn't mean most will pay that.

8:01 AM

BillyG said...

My friend lives right next door to one of the lead lawyers in this case. Believe me, he can afford it.  We are on the intercoastal, north of Chas., otherwise known as God's Country, but don't tell anybody lol.

12:47 PM

Anonymous said...

Just a ploy to get the mesothelioma on this website so people can click it and the xooglers can make some quick bucks! Not really something you'd expect from ex-GOOGs.....very unprofessional and unethical.

1:21 PM

Doug said...

To the anonymous post ahead of this one... It's a nice theory, but Ron doesn't get any of the proceeds from this blog. In fact, all AdSense earnings to date have been matched and donated to charity. I think I can safely say that neither Ron nor I are doing this for the money.

aaa said...

Now check yr adsense account it is banned by google

3:53 AM

Anonymous said...

Rob said...  How do you find out the actual price someone has paid for an adwords ad? I checked the link to the tool in the article and it will give you a placement estimate based on your bid but I can't find the actual price anywhere?  You can find some from: http://www.cwire.org/2006/03/23/updated-highest-paying-adsense-keywords/  Ron and others, note: The dollar amounts are CANADIAN DOLLARS!!!

4:31 PM

Anonymous said...

I make a living out of knowing which Adsense Ads have the highest payout per click. Sure you can buy a list of the top spends but this doesn't make any sense if the ads in positions #2, #3, and #4 etc. are paying a fraction of that.  This is why top paying keyword lists are a waste of time. 'Mesothelioma' being a prime example.  I use a nifty online appication that resolves these issues for me. Since I've been using it (started in late Jan) my revenue has increased 10-fold plus.  http://www.On1page.com/Adsense  Nothing wrong with monetizing a Blog or web site say I (and if it bankrupts lawyers so much the better).

10:32 AM

royashbrook said...

the money these people are paying is crazy. i wrote a little article on the latest high paying adword: austin dwi. i figured out i guess what is triggering so much searching on it. there's a law change. still, that's a nutty about of money to spend.

1:59 AM

MONDAY, APRIL 03, 2006

The Fool weighs in

POSTED BY RON AT 5:29 PM

The Motley Fool sez Google's Not Worth $109 Billion .

17 COMMENTS:

Anonymous said...

Fool contributor Stephen Ellis welcomes feedback at steve819@gmail.com

9:53 PM

SJ said...

The best way to appreciate what effect Google is having is to ignore the stock amrket. Stock price stopped reflecting the value of a company stopped being effective when...hmmm since day one.

11:41 PM

joe said...

^^ he didn't say Google sucks - in fact in the first paragraph he says they have "great services such as Google Maps and Gmail". But just because he uses a Gmail account doesn't mean he has to agree with the market cap that Google is worth $109 billion.

4:20 AM

Anonymous said...

After a week, that's all we get??

5:24 AM

Anonymous said...

I can go to the motley fool if I wanted to read the website. I come here for stories about Google back when you were a part of it.

8:09 AM

bhendryx said...

easy there, fellas. I'm sure he's a busy guy, and I'm also pretty sure you're not paying him to blog. You normally get what you pay for, but in this case, you've not paid a dime and in return have gotten a helluva story.

Anonymous said...

So... I take it you sold all your options at first opportunity?

11:30 AM

Anonymous said...

No offense, and I appreciate the prior posts a great deal... but this blog has gone from uber-interesting to tired-rehash in record time :(

5:03 PM

Anonymous said...

Bhendryx, I have to disagree with you. As someone who is deeply passionate about open source, there are plenty of great stuff you can get without paying for it.  And yes, I also come here to read about the inside stories from xooglers. Like others, I can go to fools or to some blog about Peru, if I was interested in that, I wont be coming here.  Yes, we are not paying them for it, but if they really want to turn this blog into a technorati level blog, they would be wise to heed the feedback from their loyal users. Open source projects like linux, sugarcrm, etc have become great only by listening to their user base, not by telling people that since they don’t pay for the software, they can f**k off. Same logic applies here.  Ray

12:00 PM

Anonymous said...

I like this blog a lot, but I think our "hosts" are likely in the middle of the extremes presented -- they have no interest in trying to make it a "technorati level" (if it does fine, whatever); however, they are also interested in feedback, but not to try to "keep customers."   My guess is they are guys with some busy lives. Some posts may not seem like much, but the juice ends up being in the comments. I do think that like all good things, there are ebbs and flows to it.

4:11 PM

Doug said...

Yeah... what 4:11PM Anonymous said is pretty close to true for me. I'm glad that people have found posts here interesting and I intend to keep adding to Xooglers as time allows, but it's purely avocational. I'm not trying to build an audience that I can monetize or to create a brand. So when it's a toss-up between finishing a post or riding bikes with my 7-year old, you'll find me pushing pedals, not my pen.   Ron is free to post what he likes and there certainly seems to be a lot of interest in GOOG as an investment. It's not of that much interest to me at this point, so I'm happy to let others debate the stock's merits. That being said, the discussion has moved far afield from the stated purpose of this blog, so it may make more sense for that conversation to relocate to a finance website.  In the meantime, I've got my computer back and I'm working on a post. It's nice to be back in the groove...

4:34 PM

Anonymous said...

..Gmail is great? Anyone who thinks that is a fool or drinkg too much of their own kool-aid.  Google has come up with some interesting web svcs. But besides the search engine, the overall implementation ad ease-of-use of their other svcs leaves much to be desired. Granted some of the more promising ones, Googlebase for one, are still in beta. Still you would think with so much brain pwer, they should do much better.  As for Gmail, it has 2GB going for it but the front end needs more thougjht..

10:05 PM

Anonymous said...

This blog is dead.

2:59 PM

rob said...

No-one is forcing anyone to read this... I assume most of you are rss-ing it? (and if not, why not?) If there's no new posts for a few days... or weeks... fair enough. Its not costing you any time. And if it doesn't say what you want it to say then... meh. Shift happens.  Keep up the blogging guys. Someone will read it.

1:06 AM

Anonymous said...

@rob,  Told you so.

3:07 PM

Fergus Macdonald said...

As a blog writer, reading this kind of comment really gets at my goat. These guys write a blog, not to feed your every whim and to be your personal little entertainer like a paid clown, but because they feel they have something to say. If you don't like it, or feel that it's not up to your high standards, then kindly don't read it and keep your comments to yourself. To complain about something which you get for free without asking is truly arrogant and i would be happier to not have that type of reader on my blog.  "Better to write for yourself and have no public, than to write for the public and have no self." Cyril Connolly (1903 - 1974)

9:17 PM

Fergus Macdonald said...

Forgot of an even more fitting quote that i put on my blog a few days ago...   “Wise men talk because they have something to say; fools, because they have to say something.”  Plato (427 – 347 BC)

9:28 PM

WEDNESDAY, APRIL 05, 2006

So long, and thanks for all the feedback

POSTED BY RON AT 5:57 PM

It would seem to me that it goes without saying that if your charter is to reminisce about a finite period of time in your life then sooner or later you are bound to run out of material. At that point you have to either branch out into other areas or call it a day. The scope of Xooglers always included "comment[ing] on the latest developments in search." That is, of course, open to a certain amount of interpretation, and talking about Google's market value arguably pushes pretty hard against the boundaries. Still, my first post on the topic generated a lively discussion -- 74 comments, more than any other post on Xooglers -- so continuing on that topic seemed defensible if not desirable.

Now, I will certainly grant that my last post was no paragon of erudition. Nonetheless, there is a long-standing and honorable tradition of short blog entries that do little more than point to items of interest elsewhere on the web. All in all, I thought it was (and is) an appropriate even if insubstantial entry, particularly in light of the fact that nothing else was being posted at all. I thought that something, even if just a link to an item of possible interest, would be better than nothing.

It seems I was wrong.

I must say I found the sense of entitlement conveyed by some of the responses to be quite shocking. Just on the off chance that it isn't clear, the only compensation that any of us get for posting to Xooglers is the fun of it. I don't know about Doug or Greg, but reading some of the comments posted in response to my last entry sucked a lot of the fun out of it for me.

(I can't help but wonder what you anonymous whiners were hoping to accomplish. This is a serious question: what were you thinking? I really want to know. What were you hoping to accomplish by saying "After a week, that's all we get??" Did you really think that would motivate me to write more?)

Still, I must confess that the whiners have a point (albeit made in a particularly ironic way): I'm fresh out of things to say about Google and search and anything even remotely on-topic for this blog, and it would probably be better if I just kept my mouth shut. So I'm signing off for the second time, and probably for good this time. Once again, to those of you who posted constructive comments, I thank you. You made it all worthwhile.

Hasta luego.

44 COMMENTS:

nick said...

I wouldn't normally do this, but after peeking at some of those comments, I wanted to say, in public, how much I've enjoyed reading this site. All the best in whatever you do next, and thank you for sharing your experience with the Google phenomenon. Cheers.

7:03 PM

Marcus Grimm said...

I agree with Nick... I've been reading this feed on Bloglines nearly since the blog went live and don't think I even ventured into the comments until now.  I see your point, though, Doug. You wanted to talk about a finite period in your life and you did. I for one greatly appreciated the insight. All the best to you and the other Xooglers.

7:24 PM

Jerry said...

I agree with Nick and Marcus. This blog was a great glimpse behind the scenes at a company that is unique and a bit enigmatic, and I found it fascinating reading. Thanks to all the Xooglers for writing.

7:35 PM

Carlos said...

I am really sorry to see you go! Xooglers was an everyday stop for me and incredibly entertaining to read. Sadly, the internet is filled with Trolls that think that on it, their desires and ideas are number one of all. I am from Brazil and will become a Googler this next Monday, and through your posts (specially the one about your GPA and GMAT)I caught a glimpse of how Google operates and its early days.   Sadly, I don´t recall reading how your tale on Google ended, so I´ll be wishing you change your mind eventually and give my (and others) curiosity a sense of closure. And even thought the Trolls were ignorant bullies, they too cherished your stories and texts.   Good luck and all the best!

7:55 PM

Anonymous said...

goodbye Ron

8:08 PM

PRab said...

Great blog. Very interesting to read about they inner workings of a company that has been doing a great job breaking the corporate mold form day one. I'm sad to see you leave, but hope that Doug will continue to post his messages. If this blog is shutting down completely, I wish you all the best of luck.

8:23 PM

Marcus said...

Thank you Ron. I am a Computer Science student currently doing an internship at Intel and this blog has always been high up in my blogroll.  When I first started reading I was in disbelief that I could actually hear a little about the inner workings of the Googleplex. Now, while other venues are opening windows on Google, I still have enjoyed all the nitty and gritty details in here. It adds some texture and reality to it all so that I have an easier time seeing where my Computer Science could lead me.  Cheers!

8:43 PM

Mary Catherine said...

Three cheers to Nick and those who agree with him, and nine cheers to the Xooglers who post here. I've been reading this religiously since I was directed to the site by one of my friends. I must say, it was highly addicting the first couple of days. I had so many posts to read! It was great. I've gotta say that I got this little spark of excitement when I'd see the rss feed tell me that there was a new post.  Sometimes topics just run dry. Like Marcus said, you did what you wanted to do and I also really enjoyed reading what all of you had (have?) to say.

8:44 PM

Ron said...

through your posts (specially the one about your GPA and GMAT)  I think you may be a little confused. I never posted anything about my GPA or GMAT. (I've never taken the GMAT).  Sadly, I don´t recall reading how your tale on Google ended  http://xooglers.blogspot.com/2006/01/waiting-for-ipo.html

8:52 PM

Ron said...

BTW, I'll still be posting to my own blog (http://rondam.blogspot.com/) where I'm a little less constrained on subject matter.

8:55 PM

Anonymous said...

Thank you! I had a blast, I wish those who posted stupid remarks would think a little before commenting... it makes me wonder how they function in society... but that is just me. Peace.

10:14 PM

Anonymous said...

Ron thank you for your posts. I was especially impressed with the quality of writing and the insight you guys provided. It's funny because I think a lot of people think working for Google would be the greatest thing in the world and make them happy, but it's funny how human nature never allows us to be 100% content. Anyway best of luck and don't let the trolls get to you. Let them go back to vandalizing wikis for fun.

10:31 PM

Anonymous said...

Ron,  Thank you for your great postings. Everyday look up to my rss aggregator in the hope that there will be a new xooglers post. I thought I would write as there are many readers who come for the stories but never end up posting. As one of those readers let me wish you all the best in your other endevours.  Cheers.

10:43 PM

Peter said...

11:51 PM

Jim said...

I am shocked, I am new to blogging and this is my favourite blog. Unfortunately I have not learned the habit of frequently posting responses to blog posts yet. I am sure that many people are in the same boat.

12:02 AM

Matthias said...

"I must say I found the sense of entitlement conveyed by some of the responses to be quite shocking."  Welcome to the internet :-). Some people always think you owe them something, particularly when they didn't pay for it in the first place. But these people a minority. A vocal minority, for sure - but still a minority.

12:54 AM

Soon said...

All things come to an end. Eventually. And at least there were some enlightening and entertaining posts along the way.

1:01 AM

Anonymous said...

Thanks for your input Ron,

3:57 AM

Anonymous said...

Thanks for the stories and sad to see you call it quits. If it was the negative comments that drove you away, you have to face reality. This is the way the internet works, and what makes it so great. In a public forum, there will always be opinions contrary to your own.  I hope Doug keeps posting and other Xooglers sign up to keep this blog alive.

4:01 AM

Anonymous said...

How annoying...  You've made fun, interesting and pleasant to read posts--and to let it get ruined by a few sad, ungrateful individuals is just outright annoying.  This blog, to be the best of my knowledge, is completely unique; there are thousands of blogs talking about the same thing, second-after-second, be it PHP, web standards or some YouTube video or other...No other blog does what Xooglers do, nor can they speak with any insight or from the perspective you guys do in respect to Google related issues.  Even when its a short post, or an infrequent one--it's a free 'service' you provide--I don't mind if you post on the hour every hour or once a year, the fact is I'll get to read a (probably humorous) post about something I never knew existed in something (Google) I use many, many times a day.  I hope to see you back sometime soon.  Thanks for all your work.

4:46 AM

Carlos said...

Ooops, sorry Ron, I mistook you for Doug! (Confusion explained) Anyway, I hope you do great after Xooglers and I´ll be sure to drop by your blog sometimes. Thanks for all the entries!

5:22 AM

Anonymous said...

Hey Ron. It was a great story you told, and I had great fun reading it. Very well written. Thanks

5:53 AM

Anonymous said...

Farewell, good, night, so-long, and may God Bless the United States of America.

7:06 AM

Anonymous said...

Er, just kind of expected to hear about why you all left. There seemed to be a sudden drop-off in the tale there... started with how you started, some early days working there, and then suddenly fast-forward several years to everyone's opinion about how the stock is overvalued.  Anonymous whiners are just reacting to the tease-factor, I think.

7:12 AM

David Holmes said...

Without going into any detail, others above have already done so : thanks for all your posts Ron, I very much appreciated reading them, and what a pity that a few (a single ? above ?) bugger(s) should ruin your feeling for them. Alas ! That is usually the way of things.  If you do sign on again, it will be a pleasure to read your prose once more.  All the best !

8:52 AM

David Holmes said...

(Sorry for re-posting immediately ; that "above ?" I wrote should not be misconstrued as the comment immediately above mine, and in fact the one I /was/ thinking about is probably genuine also, so please just ignore that).

8:55 AM

Step said...

The reference to Hitchhiker's Guide made me grin. Thanks for the posts.   It's rare that a blogger knows to call it quits on a subject, even though the lack of material should make it obvious.

9:38 AM

Dan Lewis said...

I enjoyed all you had to say, Ron. Sorry a bunch of trolls ruined it for you. Thanks for being here and sharing your insight.

10:30 AM

Ron said...

If it was the negative comments that drove you away  No, it wasn't that. It was, as I said, simply that I've run out of worthwhile Google-related things to say.  I've been hanging out on the net far too long to let a few jerks drive me away.

10:43 AM

Anonymous said...

Your last post really conveyed your sense of entitlement and more about Google culture than any of the previous.  Have you considered a de-programming? Just because you happened to work for a company that got lucky doesn't mean all present and future human contact will be characterized by doe eyed fawning.  I'll wait for the sequel on this thing., give it two years and most things google will be met with derision, what a bunch of whiners!

88Pro said...

Good bye Ron. I have always enjoyed reading your post. Its very sad to see you go.

11:51 PM

Anonymous said...

Ron, if you stop writing, you will help get those whiners what they want. Those whiners will never achieve anything significant and dont allow others to do so. So dont stop writing.

2:36 AM

Anonymous said...

Hi Ron,  I am one of the anonymous posters (the one with open source comparison), and first of all I would like to apologize to you as well as to Doug.  My intension of the feedback was neither to discourage and dishearten nor was it intended to make you stop blogging on the Xoogler. I have thoroughly enjoyed your posts (as well as Dougs). It is one of the very few blogs I follow regularly. As you have yourself noted, your first post created a good discussion, and I enjoyed both the posts and the discussions.  My point was that there are literally millions of blogs on the web on every conceivable topic. This particular one has been created and contributed to by Doug and you (and the rest) to reminisce about your days at Google. The topic, which personally intrigues me (as it does many others). That was the mandate (as was stated in the very first post by Dough) that attracted me to this particular corner of the net. If it were a personal blog about you or Doug (as there are thousands of others on the net), I most probably would not follow it. I would suspect that many feel the same way.   While I absolutely agree that some of the comments made were snide, and perhaps I, myself could have been a bit less irate in my tone, I think it was feedback meant to bring back the authors to their original promise.   I have no problem with the irregularity of the postings. I admire Doug for spending time with his daughter, rather than keeping the regular post here. However, when the articles are posted, they should adhere to the expectations that were set in the beginning (that this blog was about Google history, as created by the people working there). If that expectation has changed and instead this blog has become a personal one for keeping friends and family abreast, fair enough. It should be so stated.  I also, don’t believe that the comments were made out of sense of entitlement. Not to unnecessarily drag that particular point off topic, the posting of the blog is certainly a two way street. While money does not change hands, both authors and the readers get invaluable benefit out of it. The authors posts because they get emotional satisfaction of sharing knowledge. If you didn’t, you wouldn’t post here (or anywhere else). You post because you want people to read what your thoughts are (and they are admirable thought). That is indeed a form of compensation. Similarly, the readers don’t get paid to read either, yet we spend time to do so, because the past postings have been valuable and written entertainingly. And that is the compensation for the readers.  Again, if my comments have discouraged you from writing here, please except by apology. That is not what it was meant to do, as I have enjoyed your postings. You have shared your valuable experience and your writing style is very lively. My only request was to keep the postings aligned with the stated topic.  Ray

12:31 PM

Anonymous said...

12:33 PM

Roy said...

I'd like to add my own thanks to the list. I'm a computer science student interning at ESRI, and along with other youthful idealists, I have placed your blog high on my daily reading list.  Thanks most of all for the wit, insight, and general good humor your posts have carried. Cheers.

1:42 PM

No one really runs out of things to say ........Google is hyper-growing and redefining itself in the advent of the Web 2.0 evolution.   There is absolutely NOTHING Wrong with giving a perspective about the constant changes and the evolution of one of the most powerful and influential companies on Earth.  Having an insider's perspective from its early days is unique.  Just because some stranger ANONYMOUSLY writes insulting words, should not deter you from living your life as you choose, and engaging in your passions.  These people may have their own miserable demons that drive them.  Life is Too short,.... tommorrow is NEVER guaranteed for anyone.  Please continue to offer as many perspectives as you choose about the past and the present.    :LOL - just read some of the comments about SEARCH ENGINES WEB over the last few years.  Doesn't deter one bit :-)

3:35 PM

Jason Sparks said...

I’d also like to chime in and say I’m sorry for the juvenile behavior of some of the readership…  I am also guilty of not making as many positive comments as I perhaps should either (have to outweigh the assholes), and I think there are a few reasons we tend to do this. We tend to assume the writer knows he wrote a good article, when, in fact, usually the opposite is true. We tend to only go out of our way when we are wronged, this stems from the fact that we tend to downplay the good and overstate the bad… Anyway…  As a new blogger I now it is tough to keep the stream of great writing coming after the initial euphoria wears off and the rest of life’s demands beginning to assert their old demands. And I know the comments of which you speak definitely make it easier to walk away. Unfortunately, it will also have an effect on Doug and Greg and make future writers on this blog think twice about joining… Most unfortunate…  Thanks for the stories Ron, and good luck with your movie script J  Jason

10:38 AM

BionicBuddha said...

"Pushing pedals" with your 7 year old is ultimately going to add more value to your life than "pushing a pen" just for the sake of adding a post. A very well written and a great fram of reference for us all to reflect upon.   Besides 20 years from now, what will it really matter if the share price of Google on 03/27/2006 traded at $109 or $100 per share?      www.bionicbuddha.com