51–60 of 83 posts
One of the weirdest side efects of 4o vs 4, was single character "hallucinations" where a completely correct answer would be wrong specifically by a single character I don't think I've seen anyone comment on it, but it was noticeable, specially when 4o was just released Has anyone noticed anything similar?
>In pre-training, we filter out(opens in a new window) information that we do not want our models to learn from or output, such as hate speech, adult content, sites that primarily aggregate personal information, and spam. Great so now the model would be unable to recognize this type of content, do not use it for moderation.
>In pre-training, we filter out(opens in a new window) information that we do not want our models to learn from or output, such as hate speech, adult content, sites that primarily aggregate personal information, and spam. Great so now the model would be unable to recognize this type of content, do not use it for moderation.
I think this is a strong conclusion to jump to. Maybe it's better at spotting content that needs to be moderated because it stands out more from what it's been trained on?
The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…
Earlier quoted context omitted.
But only if it could do it with reasonable accuracy. The problem is that AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.
Surely this is still a massive problem for any real world enterprise use case unless you throw a human in the loop (which kills the productivity benefit) or you stamp a massive disclaimer on the output
Actually, it doesn’t even need PDFs. It works just about as well if you just feed it PNGs of the pages. Crazy.
One of the weirdest side efects of 4o vs 4, was single character "hallucinations" where a completely correct answer would be wrong specifically by a single character I don't think I've seen anyone comment on it, but it was noticeable, specially when 4o was just released Has anyone noticed anything similar?
Earlier quoted context omitted.
> Take a loss on every sale and make up for it with volume! If you take a loss on every sale, it is impossible to make up for it with volume. The result will be a loss magnified by the volume.
It's a joke. Sadly, the origin is unknown, but it's a joke that's well over 10 years old.
The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…
My excitement is now tempered a bit. I just tried one of the too-big invoices with the new model. After successfully getting a little farther than 4o could do, it just went into an endless loop of repeating the same line item until it ran out of output tokens. So…not really an improvement!
Earlier quoted context omitted.
Yeah, to put these prices in perspective: when tokens get this cheap, $1M buys you more than a trillion output tokens. To earn appreciable revenue at this price, an LLM company needs to be regularly generating multiple internets worth of text. On the one hand, generating multiple internets of text seems outlandish. But on the other hand, we're now approaching the point where you can start building LLMs into software…
I don't expect organizations to need to generate 1T output tokens, but 1T input tokens is common. Consider developers at a large company running queries with their entire codebase as context. Or lawyers plugging in the entire tax code to ask questions about. Each of them running dozens of queries per day on multi-millions of context input, it's going to add up quick.
It looks like the vision costs the same for GPT-4o vs mini. Both start with 150x150px and if you click the (i) it says mini uses way more base tokens and way more tile tokens, it still costs the same...
It almost sounds shady... "it's 30x cheaper per token but you now need 30x more tokens per image"? Has anyone already validated this based on billed cost? running a batch myself to check EDIT: Ok so I captioned 500 images in "low resolution" mode with GPT-4o-mini Each one took approximately: "completion_tokens=84, prompt_tokens=2989, total_tokens=3073" Reported GPT-4o-mini cost is $0.25 Using GPT-4o this would cost m…
The price for using images as part of your prompt has indeed not changed between GPT-4o-mini and GPT-4o
Yet overall, captioning 500 images now costs me 5x less. This is because when I'm captioning an image, I'm providing both an image and a text prompt. The cost of using the image in the prompt stays the same, but the cost of the text dramatically dropped.