Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons after a Half-billion GPT Tokens

kenkantzer.com

Post reply on HN
Print view View original

101–110 of 179 posts

Re: Lessons after a Half-billion GPT Tokens

#101

Post by CuriouslyC »

robocat wrote: post #82
Me1000 wrote: post #78

Earlier quoted context omitted.

Interesting, Claude 3 Opus has been better than GPT4 for me. Mostly in that I find it does a better (and more importantly, more thorough) job of explaining things to me. For coding tasks (I'm not asking it to write code, but instead to explain topics/code/etc to me) I've found it tends to give much more nuanced answers. When I give it long text to converse about, I find Claude Opus tends to have a much deeper underst…

How much of this is just that one model responds better to the way you write prompts? Much like you working with Bob and opining that Bob is great, and me saying that I find Jack easier to work with.

It's not a style thing, Claude gets confused by poorly structured prompts. ChatGPT is a champ at understanding low information prompts, but with well written prompts Claude produces consistently better output.

Re: Lessons after a Half-billion GPT Tokens

#102

Post by sebastiennight »

CuriouslyC wrote: post #98
ein0p wrote: post #33

Same here: I’m subscribed to all three top dogs in LLM space, and routinely issue the same prompts to all three. It’s very one sided in favor of GPT4 which is stunning since it’s now a year old, although of course it received a couple of updates in that time. Also at least with my usage patterns hallucinations are rare, too. In comparison Claude will quite readily hallucinate plausible looking APIs that don’t exist w…

GPT4 is better at responding to malformed, uninformative or poorly structured prompts. If you don't structure large prompts intelligently Claude can get confused about what you're asking for. That being said, with well formed prompts, Claude Opus tends to produce better output than GPT4. Claude is also more flexible and will provide longer answers, while ChatGPT/GPT4 tend to always sort of sound like themselves and p…

> ChatGPT/GPT4 tend to always sort of sound like themselves

Yes I've found Claude to be capable of writing closer to the instructions in the prompt, whereas ChatGPT feels obligated to do the classic LLM end to each sentence, "comma, gerund, platitude", allowing us to easily recognize the text as a GPT output (see what I did there?)

Re: Lessons after a Half-billion GPT Tokens

#103

Post by egonschiele »

I have a personal writing app that uses the OpenAI models and this post is bang on. One of my learnings related to "Lesson 1: When it comes to prompts, less is more":

I was trying to build an intelligent search feature for my notes and asking ChatGPT to return structured JSON data. For example, I wanted to ask "give me all my notes that mention Haskell in the last 2 years that are marked as draft", and let Chat GPT figure out what to return. This only worked some of the time. Instead, I put my data in a SQLite database, sent ChatGPT the schema, and asked it to write a query to return what I wanted. That has worked much better.

Re: Lessons after a Half-billion GPT Tokens

#104

Post by eru »

gopher_space wrote: post #72
thisgoesnowhere wrote: post #11

The team I work on processes 5B+ tokens a month (and growing) and I'm the EM overseeing that. Here are my take aways 1. There are way too many premature abstractions. Langchain, as one of may examples, might be useful in the future but at the end of the day prompts are just a API call and it's easier to write standard code that treats LLM calls as a flaky API call rather than as a special thing. 2. Hallucinations are…

> Summarizing is pretty rock solid in my testing, but reasoning is really hard. Asking for analogies has been interesting and surprisingly useful.

Could you elaborate, please?

Re: Lessons after a Half-billion GPT Tokens

#105

Post by ein0p »

Me1000 wrote: post #97
robocat wrote: post #82

Earlier quoted context omitted.

How much of this is just that one model responds better to the way you write prompts? Much like you working with Bob and opining that Bob is great, and me saying that I find Jack easier to work with.

For the RAG example, I don’t think it’s the prompt so much. Or if it is, I’ve yet to find a way to get GPT4 to ever extrapolate well beyond the original source text. In other words, I think GPT4 was likely trained to ground the outputs on a provided input. But yeah, you’re right, it’s hard to know for sure. And of course all of these tests are just “vibes”. Another example of where Claude seems better than GPT4 is co…

I should mention that I do use a custom prompt with GPT4 for coding which tells it to write concise and elegant code and use Google’s coding style and when solving complex problems to explain the solution. It sometimes ignores the request about style, but the code it produces is pretty great. Rarely do I get any laziness or anything like that, and when I do I just tell it to fill things in and it does

Re: Lessons after a Half-billion GPT Tokens

#106

Post by waldrews »

waldrews wrote: post #48
msp26 wrote: post #36

> But the problem is even worse – we often ask GPT to give us back a list of JSON objects. Nothing complicated mind you: think, an array list of json tasks, where each task has a name and a label. > GPT really cannot give back more than 10 items. Trying to have it give you back 15 items? Maybe it does it 15% of the time. This is just a prompt issue. I've had it reliably return up to 200 items in correct order. The tr…

I've been telling it the user is from a culture where answering questions with incomplete list is offensive and insulting.

It's not even that crazy, since it got severely punished in RLHF for being offensive and insulting, but much less so for being incomplete. So it knows 'offensive and insulting' is a label for a strong negative preference. I'm just providing helpful 'factual' information about what would offend the user, not even giving extra orders that might trigger an anti-jailbreaking rule...

Re: Lessons after a Half-billion GPT Tokens

#107

Post by aubanel »

> I think in summary, a better approach would’ve been “You obviously know the 50 states, GPT, so just give me the full name of the state this pertains to, or Federal if this pertains to the US government.”

Why not really compare the two options, author? I would love to see the results!

Re: Lessons after a Half-billion GPT Tokens

#108

Post by fl0id »

chromanoid wrote: post #95

GPT is very cool, but I strongly disagree with the interpretation in these two paragraphs: I think in summary, a better approach would’ve been “You obviously know the 50 states, GPT, so just give me the full name of the state this pertains to, or Federal if this pertains to the US government.” Why is this crazy? Well, it’s crazy that GPT’s quality and generalization can improve when you’re more vague – this is a quin…

Well, or it is just memorizing mappings. Not like as in reproducing, but having vectors similar to mappings that it saw before.
Post reply on HN