Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Function calling and other API updates

openai.com

Post reply on HN
Print view View original

81–90 of 174 posts

Re: Function calling and other API updates

#81

Post by ilaksh »

anotherpaulg wrote: post #68

I've been spending a lot of time figuring out how to make these GPT models successful at *editing* existing code. This is much more difficult than having them write brand new code. So far, gpt-4 has been significantly better at editing code than gpt-3.5-turbo. This is for two reasons (which I previously discussed here [1]): 1. GPT-4's bigger context window lets it understand and edit larger codebases. The new 16k win…

What I'm thinking is to give it a function called replaceAll that just replaces text, and another one called insertAfter. Maybe also replaceBetween.

Re: Function calling and other API updates

#82

Post by MuffinFlavored »

naiv wrote: post #70
MuffinFlavored wrote: post #47

Earlier quoted context omitted.

Ah, I missed this part. ChatGPT response after you tell it a function call exists, it can decide whether or not to call it. { "id": "chatcmpl-123", ... "choices": [{ "index": 0, "message": { "role": "assistant", "content": null, "function_call": { "name": "get_current_weather", "arguments": "{ \"location\": \"Boston, MA\"}" } }, "finish_reason": "function_call" }] } Gotcha. This makes it so that instead of English, g…

Every api should have its own validation so I don't even see this as a problem. What is returned from openai should be treated like any other user input.

> Every api should have its own validation so I don't even see this as a problem.

No.

I'm saying, little by little people will rely on OpenAI hypothetically for more and more.

How long until they are calling POST /credit/customer/bank/account and it just randomly goofs the ID/numbers?

A "human" may or may not have made that mistake, where an LLM will never be a 100% perfect trustable entity by design (aka, hallucinations).

Now you're just giving it a way to hallucinate into a JSON request body.

Re: Function calling and other API updates

#83

Post by ilaksh »

sashank_1509 wrote: post #18
bioemerl wrote: post #11

I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind their APIs.

I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even i…

I have been thinking about trying to do LoRA style fine tuning of Flacon-40b or Falcon-7b on RunPod. The new OpenAI 16k context and functions thinking made me lose the urge to get into that. Was questionable whether it could really write code consistently anyway even if very well fine tuned.

But at least that is something that can be attempted without $25k.

Re: Function calling and other API updates

#84

Post by ilaksh »

freediver wrote: post #78
egonschiele wrote: post #43

OpenAI continues to impress. Function calls will make working with JSON much easier, a current pain point. Dropping the price of embeddings and increasing context length means searching through your own content should become faster and more accurate. > $0.0015 per 1K input tokens and $0.002 per 1K output tokens, which equates to roughly 700 pages per dollar. This is such an incredible steal, especially when you consi…

> This is such an incredible steal, especially when you consider that no open source option comes close to GPT3.5. Orca comes close or is better. Good explainer here: https://www.youtube.com/watch?v=Dt_UNg7Mchg

But isn't Orca non-commercial? Or.. do people just ignore that part.

Re: Function calling and other API updates

#86

Post by anotherpaulg »

ilaksh wrote: post #81
anotherpaulg wrote: post #68

I've been spending a lot of time figuring out how to make these GPT models successful at *editing* existing code. This is much more difficult than having them write brand new code. So far, gpt-4 has been significantly better at editing code than gpt-3.5-turbo. This is for two reasons (which I previously discussed here [1]): 1. GPT-4's bigger context window lets it understand and edit larger codebases. The new 16k win…

What I'm thinking is to give it a function called replaceAll that just replaces text, and another one called insertAfter. Maybe also replaceBetween.

Absolutely! I will certainly be experimenting with the new functions as you suggest.

One thing which I haven't seen discussed elsewhere is the tension between the output format and the underlying task. When I ask GPT to use a simple, natural output format it does *better* at the actual code editing task. If I ask it to output using a more technical format like `diff -c` or heavily structured json formats... it "gets distracted" and does worse at the underlying coding request.

It writes worse code if you ask it to output edits in a terse, machine-readable diff format. It writes better code if you let it show you the code edits in a simple way.

For GPT3.5 this means I have to let it just type the whole source file back to me, with the edits included. GPT-4 is able to output a very simple diff-like format, but struggles with `diff -c`, etc.

So it will be interesting to see how the new function capabilities affect this tradeoff. Perhaps it is now "fluent" in function-json and so won't be "distracted" by that output formatting.

Re: Function calling and other API updates

#88

Post by ilaksh »

sergiotapia wrote: post #25

What are you thoughts here regarding functions: I have some data I can pass in CSV format to the context and ask a question against that data. "Who are my best customers?" and pass in a CSV of the top 100 customers. vs I create a function that returns my best customers and call a ChatGPT function. When would I use one or the other? The function call seems like it would be more accurate with better guardrails, but it…

Unless it's a small CSV then I would put it in sqlite or something and tell GPT-4 to write a query given the user's question. I have done that before and worked pretty well. Even worked fairly ok with gpt-3.5. You have to give it context like schema etc.

I was even able to get it to output custom-coded embedded Chart.js charts if requested by the user.

Re: Function calling and other API updates

#89

Post by alpark3 »

I wonder how much of these changes are pushed by the local LLM shift we've seen recently. I would've expected them to totally focus on GPT-4 updates, but it's nice that we're getting 3.5 improvements.

It's pretty clear that there's a large demand for much cheaper, if weaker LLMs. I'll need to test the "more reliable steerability via the system message" feature, but GPT-3.5's largely monotonic tone and lack of response to the system message was one of its largest weaknesses imo. I'm all for ggml and LLaMa, but there's almost zero need for me to invest in hardware/expensive GPUs (or /hour options) if 3.5 is this cheap. Only downsides I can see are data privacy and OpenAI's "safety" restrictions.

Function calls seem amazing, too. No need to use tokens commanding GPT about its ability to do function calls. I need to test it out though.

Re: Function calling and other API updates

#90

Post by zora_goron »

com2kid wrote: post #12

GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news! The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel. Good creative prompts are long and detailed, and to get the best results you re…

Definitely agree about prompts -- for MedQA [0] I ended up building up a prompt around 300 words long to get a collection of results I was aiming for. I'm still not sure about the best way to go about building a "stable" lengthy prompt that can maintain a predictable output even after adding to it; my approach was mainly via trial-and-error experimentation.

[0] https://labs.cactiml.com/medqa

Post reply on HN