141–150 of 174 posts
16k is tokens on 3.5 is amazing but they also complicated the pricing from a simple flat per 1k token to two separate fees for input and output … I think it’s better maybe but token based pricing is challenging to reason with and even more so to explain to customers
16k is tokens on 3.5 is amazing but they also complicated the pricing from a simple flat per 1k token to two separate fees for input and output … I think it’s better maybe but token based pricing is challenging to reason with and even more so to explain to customers
16k is tokens on 3.5 is amazing but they also complicated the pricing from a simple flat per 1k token to two separate fees for input and output … I think it’s better maybe but token based pricing is challenging to reason with and even more so to explain to customers
They should work towards making repetitive prompts be free or much cheaper because they don't have to generate it token by token and can be cached.
(I.e create a function that answer the user question in JavaScript, you can call llm(prompt, context) to process the data into natural language and search(query) to find data for the user) - and then you let langchainjs execute the output as apart of its loop
16k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full article realm and that is already incredibly useful. Is there any open model with a similar context length? [I'm not talking about the dubious for long context fine-tuned LLaMA variants, I mean the real thing.]
You can chunk up a book and embed those partitions into a vector database. Then you can take a query and fuzzy match the most relevant documents in your vector database, then feed it back to open AI to resolve an answer.
It's brilliant. Postgres has an extension to support indexing the vectors, and there are some other open source and turnkey solutions in the market as well.
> With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc). Yeah thanks for the heads up Sam.
This seems wildly dangerous. I wonder how hard it would be for a user to convince the GPT to run a query like `DROP TABLE ...`
I think a good mental security model might be - if you wouldn't expose your function as an unsecured endpoint on the web, then you probably shouldn't expose it to a LLM
The big feature here is the function calls, as this is effectively a replacement for the "Tools" feature of Agents popularized by LangChain, except in theory much more efficient since it may not require an extra call to the API. In the case of LangChain which selects Tools and their functional outputs through JSON Markdown shennanigans (which often fails and causes ParsingErrors), this variant of ChatGPT appears to b…
Anyone know how long it will take Azure to get this latest model in its OpenAI service?
Anyone know how they pick who to invite off the waitlist for GPT 4? I've been on there for a while. My project is open source and I wonder if that is getting me deprioritized.
> With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc). Yeah thanks for the heads up Sam.
Seems like a missed opportunity to make things a little more secure.