21–30 of 70 posts
I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.
GPT-3.5 is good enough for many cases. I know Microsoft for their Bing chat implementation is using a pre-prosessor to select the most appropriate model to use.
I have a pretty decent understanding of how the current generation of LLMs work, and I too don't think we've achieved consciousness. But the prompts just sound so much like something you'd tell a human, or something you'd tell a host in Westworld.
As long as the template system of choice supports custom functions, I don't see the need for a custom language for this. Please correct me if there is something deeper that I am missing? (other than the specifics to messaging and roles, where I for one prefer to put these in separate files and then pass them as arguments, easier to manage, recombine, and reuse this way)
https://gist.github.com/verdverm/747b0b810dcc5518d699b14f0d0...
Open source will be behind always since it is harder to organize a large group of people and once it publishes it will be copied by big players, however if an organization like the original OpenAI gets formed that is truly non-for-profit and support the community, I believe it can beat big players. I think only Meta is the company that can support such an organization at the moment.
To counteract your point, OpenAI was a "truly non-for-profit"
I thought the point of Microsoft Guidance was that we could specify a grammar or regular expression that the output must match?
Stuff like "Make sure to use real numbers, not fractions." seems like such a weird and fragile way to write a program. I thought the point of Microsoft Guidance was that we could specify a grammar or regular expression that the output must match?
I think regarding: "Make sure to use real numbers, not fractions."
It would be simple using Guidance to create some kind of validation function, which guidance would invoke after it gets the response from the model. In this case you would check if the number is a real number or not. And if not, the model would invoke again with a different seed.
Assuming the model gives a real number, as expected, ~90% of the time, you would only see retries very infrequently, and you could be pretty confident that your output will match what you expect.
(again, I'm not associated with this project, I've only scratched the surface playing with it locally, so this is just speculation)
- Quality on task: For every task we tried, ChatGPT is still stronger than Vicuna on the task itself. MPT performed poorly on almost all tasks (perhaps we are using it wrong?), while Vicuna was often close to ChatGPT (sometimes very close, sometimes much worse as in the last example task above).
- Ease of use: It is much more painful to get ChatGPT to follow a specified output format, and thus it is harder to use it inside a program (without a human in the loop). Further, we always have to write regex parsers for the output (as opposed to Vicuna, where parsing a prompt with clear syntax is trivial).
- Efficiency: having the model locally means we can solve tasks in a single LLM run (guidance keeps the LLM state while the program is executing), which is faster and cheaper. This is particularly true when any substeps involve calling other APIs or functions (like search, terminal, etc), which always requires a new call to the OpenAI API. guidance also accelerates generation by not having the model generate the output structure tokens, which sometimes makes a big difference.
I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…
are we going to now invent a programming language that communicates with LLMs? I find that funny
Earlier quoted context omitted.
are we going to now invent a programming language that communicates with LLMs? I find that funny
Why do you find that funny?