Turns out the Google site: operator was broken.
Googler Adam Lasnik writes in the comments on John Battelle's Searchblog about Google's indexing problem:
"I work with Matt Cutts and other engineers in the Search Quality Team at Google. And yes, we noticed that lots of subdomains got indexed last week -- and sometimes listed in search results -- that shouldn't have been. Compounding the issue, our result count estimates in these contexts was MANY orders of magnitude off. For example, the one site that supposedly had 5.5 billion pages in the index actually had under 1/100,000th of that.So how did this happen? We pushed some corrupted data with our index. Once we diagnosed the problem, we started rolling the data back and pushed something better... and we've been putting in place checks so that this kind of thing doesn't happen again."
Meanwhile, Google devotees hope Googler Matt Cutts comes back from vacation soon.