Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

481–490 of 561 posts

Re: New models and developer products

#481

Post by williamcotton »

ChatGTP wrote: post #427
asaddhamani wrote: post #425

Earlier quoted context omitted.

Their products are incredible though. I’ve tried the alternatives and even Claude is not nearly as good as even ChatGPT. Claude gives an ethics lecture with every second reply, which costs me money each time and makes their product very difficult to (want to) embed.

What are you using it for? I want to know what people actually use these things for damn it !

Retrieving the non-metadata titles of 45,000 various PDF, docx, etc. without a bunch of rules/regexs that would fail half the time.

“Derp derp, hallucinations”.

Eh, no, not in practice, not when the entire context and document is provided and the tools are used correctly.

Re: New models and developer products

#482

Post by wokwokwok »

wizzard0 wrote: post #460
wokwokwok wrote: post #354

Earlier quoted context omitted.

Then you’re either not testing your prompts or doing something trivial. Remember: a good model with a good prompt will generate bad outputs sometimes. A bad model with a bad prompt will generate a good output sometimes. That is simply a fact with these non deterministic models. You have to do many iterations for each prompt to verify they are working correctly. > I’ve not had much problems moving between LLMs… If you…

> many iterations of each prompt BTW its much faster and cheaper to artive at a good prompt if you sample the model in deterministic mode (ie temperature=0) By default you have to guess if the difference is due to the prompt change or due to the dice roll, as you’ve noticed, but you don’t need to!

You should have a read of https://huggingface.co/blog/how-to-generate (the section on sampling, with regard to setting temperature to zero).

This is degenerate (greedy) behaviour, and not representative of the what the prompt will behave like at a higher temperature.

(At least, that’s my understanding; it’s a complex topic but broadly speaking there no specific reason, as far as I’m aware, to expect that a particular combination of params/prompt is representative of any other combination of params/prompt for the same model; it may be, but it may not. Certainly on models like GPT4 it is not, for reasons that are not clear to anyone. So… take care with your prompt testing. setting temperature to 0 is basically meaningless unless you expect to use a temperature of 0 in production. The results you get from your prompts at temp 0 are not generally reflective of the results you will get at temp > 0).

Re: New models and developer products

#483

Post by jillesvangurp »

zizee wrote: post #258

In people's experience with these sorts of tools, have they assisted with maintainance of codebases? This might be directly, or indirectly via more readable, bette organized code. The reason I ask is that these tools seem to excel in helping to write new code. In my experience I think there is an upper limit to the amount of code a single developer can maintain. Eventually you can't keep everything in your head, so m…

There's a nice code gpt plugin for intellij and vs code. Basically you can select some code and ask it to criticize it, refactor it, optimize it, find bugs in it, document it, explain it, etc. A larger context means that you can potentially fit your entire code base in that. Most people struggle to keep the details in their head of even a small code base.

The next level would be deeper integration with tools to ensure that whatever it changes, the tests still have to pass and the code still has to compile. Speaking of tests, writing those is another thing it can do. So, AI assisted salvaging of legacy code bases that would otherwise not be economical to deal with could become a thing.

What we can expect over the next years is a lot more AI assisted developer productivity. IMHO it will perform better on statically typed languages as those are simply easier to reason about for tools.

Re: New models and developer products

#484

Post by jpalomaki »

faeriechangling wrote: post #51
minimaxir wrote: post #4

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rath…

ChatGPT only costs a few dollars, but I'm also "paying" for the service by contributing training data to OpenAI.

Getting access to this type of interaction data with (mostly) humans must be quite valuable asset.

Re: New models and developer products

#485

Post by mirekrusin »

faeriechangling wrote: post #51
minimaxir wrote: post #4

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rath…

Obviously it is good strategy, surely created from GPT.

Re: New models and developer products

#486

Post by dri_ft »

icelancer wrote: post #473
nmfisher wrote: post #384

Earlier quoted context omitted.

I generally use it for boilerplate tasks like “here’s some code, write unit tests” or “here’s a JSON object, write a model class and parser function”. Claude is significantly faster, so even if it requires a couple more prompt iterations than GPT4, I still get the result I need earlier than with GPT4. GPT4 also recently developed this annoying tendency to only give you one or two examples of what you asked for, then…

> GPT4 also recently developed this annoying tendency to only give you one or two examples of what you asked for, then say “you can write the rest on your own based on this template”. I can’t overstate how annoying this was. The last model "update" has really ruined GPT-4 in this regard.

When was this? I noticed chatGPT becoming succinct almost to the point of being standoffish about a week or two ago. Probably exacerbated by my having some custom instructions to tame its prior prolixity.

Re: New models and developer products

#487

Post by reissbaker »

torginus wrote: post #446
openquery wrote: post #378

If I had no contact with society from the 29th of November 2022 (the day before ChatGPT was released according to Wikipedia) and came back today to see the OpenAI keynote I would have lost my mind. The progress and usefulness of these products is absolutely incredible.

I'm sorry, what breakthrough feature did we see here? - Code interpreter, function calling were already possible on any sufficiently advanced LLM that could follow instructions well enough to output tokens in a rigidly parseable format, which could then be fed into a parser, and its output fed back to the LLM. It was clunky to do with online APIs like ChatGPT, but still eminently possible. - Custom chatbots were easy…

The most exciting announcements for me were:

* GPT-4-128k. Sure, Claude exists, but it's closer to GPT-3.5 than 4 IMO. TBD how well 128k context works given that classic attention scales quadratically (so they're presumably using something else) but given how good OpenAI's models tend to be I'm willing to give them the benefit of the doubt.

* Pushed GPT-3.5 finetuning to 16k context (up from 4k when it was released this summer). IME 3.5 finetunes are very useful, very fast, very cheap replacements for specific specialized tasks over GPT-4, and easily outperform GPT-4 for the right kind of tasks. The 4k context limit was a bit of a bummer.

* New tts that to my ears sounds nearly equivalent to Eleven Labs or Play.ht, at one-tenth to one-twentieth the price (with zero monthly commitment). The Eleven Labs Discord is a bit of a bloodbath right now, most of the general chat is just people saying they're switching. (The Play.ht Discord is pretty dead most of the time anyway, so not much new since this morning.) I will say though that it's a bummer that the OpenAI tts doesn't have input streaming, only output streaming, so latency will likely be worse and you'll have to figure out some way to do chunking yourself which is fairly annoying, but for any kind of personalized use case (e.g. a bot talking to customers, as opposed to using pre-recorded snippets) a 10-20x price improvement is worth the extra pain and may be the difference between "neat prototype" and "shippable to production."

Plus, massive price drops for OpenAI's existing products across the board, along with a legal defense fund to protect OpenAI customers from getting sued for using OpenAI models. If you're building an "OpenAI wrapper startup," today was a very good day. If you're competing with OpenAI, though... Oof.

Re: New models and developer products

#488

Post by code_runner »

ChatGTP wrote: post #427
asaddhamani wrote: post #425

Earlier quoted context omitted.

Their products are incredible though. I’ve tried the alternatives and even Claude is not nearly as good as even ChatGPT. Claude gives an ethics lecture with every second reply, which costs me money each time and makes their product very difficult to (want to) embed.

What are you using it for? I want to know what people actually use these things for damn it !

Summarizing large documents. Finding relationships between two (or more) documents. Building a set of points bridging the gap between the documents. Correcting malformed text data.

Not everything is just data in a database or some structured format. Sometimes you have blobs of text from a user, or maybe you ran whisper on an audio/video file and now you just have a transcript blob… it’s never been easier to automate all of this stuff and get accurate results.

You can even have humans in the loop still to protect against hallucinations, or use one model to validate another (ask GPT to correct or flag issues with a whisper transcript)

Re: New models and developer products

#490

Post by nwoli »

dragonwriter wrote: post #348
nwoli wrote: post #229

Earlier quoted context omitted.

I guess the EO takes care of that in their eyes (outlawing open models). They’re probably right too

> I guess the EO takes care of that in their eyes (outlawing open models). The EO doesn't do anything even approximately like outlawing open models.

It basically bans models that can be finetuned to act as a biological weapon which is basically every model, llama, mistral etc
Post reply on HN