Hm, still don’t see the 32k context in the Playground
I think they never put 8k into the playground? Edit: I can call the api now with gpt-4-32k-0613
[dead]
Function calling and other API updates
151–160 of 174 posts
Re: Function calling and other API updates
#152I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind their APIs.
Thing is, as long as the field is growing in capabilities as fast as it is, there isn't going to be any kind of "democratizing" for an average person, or even average developer. Anything you or me can come up with to do with LLM, some company or startup will do better, and they'll have people working full-time to productize it.
Maybe it's FOMO and depression, but with $dayjob and family, I don't feel like there's any chance of doing anything useful with LLMs. Not when MS is about to integrate GPT-4 with Windows itself. Not when AI models scale superlinearly with amount of money you can throw at them. I mean, it's cool that some LLAMA model can run on a PC, provided it's beefy enough. I can afford one. Joe Random Startup Developing Shitty Fully Integrated SaaS Experience can afford 100 of them, plus an equivalent of 1000 of them in the cloud. Etc.
Yeah, I guess it is FOMO and depression.
Re: Function calling and other API updates
#153Re: Function calling and other API updates
#154I wonder how much of these changes are pushed by the local LLM shift we've seen recently. I would've expected them to totally focus on GPT-4 updates, but it's nice that we're getting 3.5 improvements. It's pretty clear that there's a large demand for much cheaper, if weaker LLMs. I'll need to test the "more reliable steerability via the system message" feature, but GPT-3.5's largely monotonic tone and lack of respons…
Describing functions to GPT still costs extra tokens, unfortunately.
Especially given they picked just about the most verbose way of doing it, second only to XML. While this is to be tested, given the examples they give, I somehow doubt that minified description with single-letter function names will perform as well as human-readable (verbose) schemas.
Re: Function calling and other API updates
#15516k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full article realm and that is already incredibly useful. Is there any open model with a similar context length? [I'm not talking about the dubious for long context fine-tuned LLaMA variants, I mean the real thing.]
Try looking into vector databases to solve that problem. You can chunk up a book and embed those partitions into a vector database. Then you can take a query and fuzzy match the most relevant documents in your vector database, then feed it back to open AI to resolve an answer. It's brilliant. Postgres has an extension to support indexing the vectors, and there are some other open source and turnkey solutions in the m…
Brilliant, but not the same. I think both approaches have their place and are not mutually exclusive.
Re: Function calling and other API updates
#156Anyone know how long it will take Azure to get this latest model in its OpenAI service?
"Unfortunately, the new GPT-3.5-turbo-16k model is not yet available on Azure OpenAI. So, we can't share much information with you regarding this. We don't have any ETA at this moment. Once we have anything will share it with you."
https://learn.microsoft.com/en-us/answers/questions/1305032/...
Re: Function calling and other API updates
#157Finally! I’ve been getting the shakes waiting for next OpenAI release. 16k context with 3.5-turbo is huge. It’ll make all those dime a dozen document driven assistants a lot more useful. I’m curious to see if people will figure out ways to hack functions to get more reliable structured JSON data out out of GPT without tons of examples, giving lots more context room to play with
Can you provide some examples of what document driven assistants you're referring to?
I'm assuming something like humata.ai
Re: Function calling and other API updates
#158Hm, still don’t see the 32k context in the Playground
Check under Mode > Complete. I see it listed there but not under Mode > Chat. Also try curl -X GET https://api.openai.com/v1/models \ -H "Authorization: Bearer your-api-key" It is listed there for me.
I've seen it on playground yesterday night (European time) and it disappeared by today
Re: Function calling and other API updates
#159GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news! The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel. Good creative prompts are long and detailed, and to get the best results you re…
> Good creative prompts are long and detailed They don't need to be tho. You can try shotgunning in (generate 100 titles about a novel around aliens, after the gen 'pick the one most likely to resonate to a X audience, explain why') Or you can let AI drive itself interactively (ask yourself 20 question about how to write creative alien stories, and answer yourself) Or you can process in spirals (generate a setting fo…
I'm seeing that same problem. Most of the blog posts in storybot.dev suffer that problem. They are too generic.
The only interesting ones have a lot of detail in its prompts.
Re: Function calling and other API updates
#160The big feature here is the function calls, as this is effectively a replacement for the "Tools" feature of Agents popularized by LangChain, except in theory much more efficient since it may not require an extra call to the API. In the case of LangChain which selects Tools and their functional outputs through JSON Markdown shennanigans (which often fails and causes ParsingErrors), this variant of ChatGPT appears to b…
There is also an alternative approach for running code with ChatGPT, the way Nekton( https://nekton.ai ) does it. It will use ChatGPT to generate typescript code code, and then just run it in the cloud. In the end you get similar result - AI generated automation, but you have an option to review what the code will actually do before running it.
While using Auto-GPT I realized that for most usecases, a simple script would have suited my needs better (faster, cheaper, deterministic). Then I realized those scripts can (a) be written by GPT, and (b) call into GPT!