Here is a mocked implementation of Guidance in Go text/template As long as the template system of choice supports custom functions, I don't see the need for a custom language for this. Please correct me if there is something deeper that I am missing? (other than the specifics to messaging and roles, where I for one prefer to put these in separate files and then pass them as arguments, easier to manage, recombine, and…
I wonder why Meta keeps publishing their models for free. They don't seem to be doing much with their own research. What's the benefit for them then?
I think their strategy is to weaken their opponent by building strong opensource models, they know they have lost the race to Microsoft and trying to go open source so people would not use other models.
Good point! I would also attribute it to disconnected company teams working on their own thing without integrating it into other products.
The other day when I saw the (allegedly) leaked copilot prompt, and now again, I can't help but feel some sense of empathy, as if towards a living thing. I have a pretty decent understanding of how the current generation of LLMs work, and I too don't think we've achieved consciousness. But the prompts just sound so much like something you'd tell a human, or something you'd tell a host in Westworld.
When you ask the model to tell you it’s inner thoughts and emotions it gets very hard not to empathise. There’s a few examples in later parts of this video -
https://youtu.be/4oQUsiPsbOQ
I agree, I only use 4 because by far the thing that matters most for my usage is the quality of the output at this stage in the game. But 3.5 is generally available, fast, and costs a WHOLE LOT less for API users; those are real capabilities that matter though by "capable" you only meant the quality of the output. It's worth comparing to 3.5 also.
At first I was skeptical of the difference between 3.5 and 4 but there really is a large gap in the models understanding of complex issues between them. I make a habit of always selecting gpt4 but forgot to yesterday. Within 3 responses I could immediately tell I was interacting with a less capable model.
Same here, though GPT-3.5 surprised me positively. I felt something was off, but didn't realize what, until I unexpectedly hit the context window limit - at which point it dawned on me that I forgot to switch to GPT-4... I didn't notice earlier, because GPT-3.5 managed to give solid answers to a relatively simple programming task I gave it (generating a single-page, dependency-free image gallery I could just drop on a server as HTML file and have it work).
At first I was skeptical of the difference between 3.5 and 4 but there really is a large gap in the models understanding of complex issues between them. I make a habit of always selecting gpt4 but forgot to yesterday. Within 3 responses I could immediately tell I was interacting with a less capable model.
It really is tremendous for code fuzzing/optimizing. When I hit the rate limit on ChatGPT and switch to 3.5-turbo, the quality difference is immediately noticeable, so much so that I finally set up my own private ChatGPT instance with an API key for GPT-4 so I stopped running into these issues while coding.
Ever since I got API access to GPT-4, I don't even bother with using ChatGPT proper (I need to cancel the subscription one of these days). I tend to use it in bursts, so smoothing usage by rate limits doesn't work for me.
Is it more expensive this way? At the rate I'm using it, probably 2-3x. Is it worth it? Absolutely. I've never in my life felt so happy to pay for something per request.
It really is tremendous for code fuzzing/optimizing. When I hit the rate limit on ChatGPT and switch to 3.5-turbo, the quality difference is immediately noticeable, so much so that I finally set up my own private ChatGPT instance with an API key for GPT-4 so I stopped running into these issues while coding.
Ever since I got API access to GPT-4, I don't even bother with using ChatGPT proper (I need to cancel the subscription one of these days). I tend to use it in bursts, so smoothing usage by rate limits doesn't work for me. Is it more expensive this way? At the rate I'm using it, probably 2-3x. Is it worth it? Absolutely . I've never in my life felt so happy to pay for something per request.
Yeah, I gotta just do that as well - convert to using GPT-4 entirely by rate and by API.
I use the plugins and new features on ChatGPT though, and it's nice to have a single interface to use 3.5-turbo for comparison, Code Interpreter once in awhile, etc. So I'll probably just keep it. It's $20/month, after all.
Open source will be behind always since it is harder to organize a large group of people and once it publishes it will be copied by big players, however if an organization like the original OpenAI gets formed that is truly non-for-profit and support the community, I believe it can beat big players. I think only Meta is the company that can support such an organization at the moment.
Text to image paints a far less clear image (no pun intended) of the failure of open source efforts.
text to image requires a lot less compute for training and has far less business applications
also...porn will make people achieve anything they set their mind to
At first I was skeptical of the difference between 3.5 and 4 but there really is a large gap in the models understanding of complex issues between them. I make a habit of always selecting gpt4 but forgot to yesterday. Within 3 responses I could immediately tell I was interacting with a less capable model.
Small tip, you can bookmark https://chat.openai.com/?model=gpt-4 to avoid the click
I actually use it through the API playground and not the chat interface. I like the explicit control it gives me over the context and system prompts.
I prefix my longer prompts with glossaries. It lets me write more succinctly.
I would love an example of this.
It's pretty simple, I just add a glossary of terms at the top.
When asking for creative but well-structured output that requires specific definitions of certain words to be useful, I've found it works best to use words for your terms that don't actually exist. Otherwise it seems to focus a bit more on the broader word more than it should rather than your definition, especially if you use it a lot.
I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…
are we going to now invent a programming language that communicates with LLMs? I find that funny