61–62 of 62 posts
How do you know the updated docs are correct? And what about the issue where doc training itself favors content near the beginning of the doc, but the salient facts might be near the end?
Earlier quoted context omitted.
Ironically, the common (used to be common?) trope about foundation models being picks and shovels and LLM wrappers having no values is probably backwards. ChatGPT is the most valuable AI product and it's just a wrapper around an underlying LLM that is not 10x better than the rest of the models.
I would not define ChatGPT as a wrapper, anything where you are doing actual training/learning and updating weights is by most persons definition, not a wrapper. Just injecting stuff into the context or using RAG is a wrapper because there are no weight updates anywhere.
So far, there is at most a few months' gap between state-of-the-art and commodity, if for no other reason than that other companies train on the output of the SOTA.
If I were a VC, I would invest in a wrapper (Cursor, Harvey, this idea, etc.) over a foundation model every day of the week.