You know, whatever the merits (and it sounds as if there were very few) of Google's presentation (or Pinchai's recent non-statement) might be, I would have expected the commentariat here at HN to display way more skepticism about language models as a replacement for search. If there's a crowd anywhere in the world large enough to leave 200+ comments on a post like this, you'd think thatg the one at HN would understan…
Probably the secret sauce to using LLMs for search and other forms of information retrieval is first taking a query and use traditional lookup to find likely information. Then aggregating this data, provide the aggregated data as context, append the query, and pass it to the LLM. Probably some post processing is a good idea also.
I used to do this with squad LLMs.