51–60 of 354 posts
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.
Earlier quoted context omitted.
Ok - what's the ROI on the $10bn (++) that OpenAI have had? So far I reckon This isn't what VC's (or microsoft) dream of.
I think Azure OpenAI service is growing at 1000% per quarter according to last earnings call
Earlier quoted context omitted.
Search has definitely been dead since before LLMs, we just don't have a replacement yet.
For a while my replacement was “use google, add ‘reddit’ at the end.” Not sure how much longer that will work given even just this limited blackout impacted how effective that was lol
Earlier quoted context omitted.
GPT4 is really good at code and you can generally verify hallucination easily. The other good use cases are using LLM to turn natural language prompts into API calls to real data.
This is what I've been doing. GPT-4 to generate some data from some input, followed up by a 3.5T call to verify the output against the input to verify the content. You can feed the 3.5T output straight back into GPT-4 again and it will self-correct. Doing this a couple of times gives me 100% accuracy for my use case that involves some level of summarization and reasoning. Hallucinations are not as big of a deal at al…
A16z’s latest summary of the landscape was way more useful and relevant than this.
Earlier quoted context omitted.
Yep. We are in the very early innings of capital being deployed to all this.
Ok - what's the ROI on the $10bn (++) that OpenAI have had? So far I reckon This isn't what VC's (or microsoft) dream of.
I think its quite likely that OpenAI will make that money back and more, as both the industry leader and with the power of their brand (chatGPT).
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.
> that don't hallucinate “Hallucination” is part of thought. Solving a new problem requires hallucinating new, non existing, possible outcomes and solutions, to find one that will work. It seems that eliminating the ability to interpolate and extrapolate (hallucinations) would make intelligence impossible. It would eliminate creativity, tying together new concepts, creation, etc. Is the goal AI, or a nice database fr…
The latter given the kind of products that are currently being built with it. You don't want your code completion or news aggregator to hallucinate for the same reason you don't want your wrench to hallucinate, it's a tool.
And as for hallucinations, that's a PR friendly misnomer for "it made **** up". Using the same phrase doesn't mean it has functionally anything to do with the cognitive processes involved in human thought. In the same way a 'artificial' neural net is really a metaphorical neural net, it has very few things in common with biological neurons.
All our current approaches rely on dense matrix multiplications. These approaches necessitate a tremendous amount of communication bandwidth (and low latency collectives). This is extremely challenging and expensive to scale O(n^2.3).
The constraints of physics and finance make significantly larger models out of reach for now.
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.