I've pondered that very question off and on for several years. I don't know what the "worst" thing would be but some places where I think there are clouds (pun intentionality uncertain) on their horizon - in no particular order:
1) Culture fail: leading to dithering and waste that swamps the ship. Until a recent few years, Google hired and operated like Larry and Sergey's late night bull session dream: hiring "people like them", creating outrageously extravagant perks (like grad school life on steroids with high pay), using the "20% free time" rule to implement a kind of AI search for business ops, etc. They over-hired. They lacked coherent focus. They became a perpetual ego-stroke for the boys at the top. They built an arrogant and self-validating culture. They developed a culture that is often snobbish beyond accomplishment. On and on. They even had a "uniquely theirs" house style for interior decorating. Historically those elements are a recipe for a company that at some point seizes up and becomes incapable of fighting in the markets (c.f., e.g., the collapse of SGI whose former campus, ironically, is mostly occupied by GOOG). Under Schmidt, et al. in recent years this appears to be starting to be reined in but I'm not convinced yet the professional management is making advances beyond shoring up results quarter by quarter - I don't see the new, customer-centric culture yet.
2) Collapse of advertising. Advertising can collapse out from under Google in at least three, plausible ways. First, some other content delivery medium comes to dominate as the place where effective advertising goes. Google can't be everywhere at once (not even Google) and, frankly, I think the big market for Internet advertising is going to look a lot more cable TV than web surfing. Second: a new market maker. As technological opportunities to place ads change, ad placement will drift away from what you can build on a search engine to more targeted placement. Right now it's cheap and easy to put some space in your content in Google's market - Google is a market maker taking a cut. As mass consumption of content condenses to a smaller number of content providers - providers better equipped to make sophisticated sales of their ad properties will emerege (e.g., e.g., content providers who do more than add links and declare a few keywords). New market makers will enter and take Google's margin on ads to $0 (in the limit). Third, on-line advertising as a paradigm may get severely shaken by its poor performance (for most advertisers) in bad economic times along with alternatives. An alternative I think likely to work, for someone, is the request-driven marketing paradigm. If I need to or want to buy X, typing search terms into a search engine seems a painfully indirect and horrible way to go about it. Instead, in any large market, I want domain-specific heuristics to connect me to the general search thing of "what consumers are saying" and connect me to the ad placement thing in the sense of domain-specific searching through ads. Ads on such a "decision engine" (!) should ultimately be based purely on actual sales commission, not CPMs (though even commissions will be a pretty small slice - perhaps bundled with on-line transaction handling service).
3) Misplaced capital investment. On the one hand, Google makes a plausible play to build their massive manufacturing plants for commodity computing (their newest generation "data centers"). On the other hand, if their closed, arrogant culture and over-reliance on ads causes them to fail to monetize those centers well they may wind up being left holding the bag where they have to sell of to someone who knows how to make money off of them. They seem well on their way to failing hard on those data centers by trying to monopolize what system software runs on them.
In general, you pretty much know that something is going to take them down, at least if history is any evidence. They wound up, a few years back, outrageously overcapitalized. Not just over-valued but actually over-capitalized. History shows that no management team in that situation - the extreme outlier in capitalization - is ever competent to invest the money flows wisely enough to keep it up.