Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

151–160 of 426 posts

Re: OpenAI is too cheap to beat

#151

Post by BoorishBears »

nine_k wrote: post #128
kevlened wrote: post #108

Earlier quoted context omitted.

Their data retention policy on their APIs is 30 days, and it's not used for training [0]. In addition, qualifying use cases (likely the ones you mentioned) qualify for zero data retention for most endpoints. [0] - https://platform.openai.com/docs/models/how-we-use-your-data

In sensitive cases you do not think about the normal policy, you think about the worst case. You just can't afford a leak. Your local installation may be much better protected than a public service, by technology and by policy.

For years people have essentially made a living off FUD like "ignore the literal legal agreement and imagine all the worst case scenarios!!!" to justify absolutely farcical on-premise deployments of a lot of software, but AI is starting to ruin the grift.

There are some cases where you really can't afford to send Microsoft data for their OpenAI offering... but there are a lot more where some figurehead solidified their power by insisting the company build less secure versions of public offerings instead of letting their "gold" go to a 3rd party provider.

As AI starts to appear as a competitive advantage, and the SOTA of self-hosted lagging so ridiculously far behind, you're seeing that work less and less. Take Harvey.ai for example: it's a frankly non-functional product and still manages to spook top law firms with tech policies that have been entrenched for decades into paying money despite being OpenAI based on the simple chance they might get outcompeted otherwise.

Re: OpenAI is too cheap to beat

#153

Post by oceanplexian »

ldjkfkdsjnv wrote: post #136
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Completely wrong, the best AI will win. There is insane demand for better models.

Yep, quality over quantity. The difference between 99.9% accurate and 99.999% accurate can be ridiculously valuable in so many real world applications where people would apply LLMs.

Re: OpenAI is too cheap to beat

#154

Post by gbmatt »

ldjkfkdsjnv wrote: post #136
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Completely wrong, the best AI will win. There is insane demand for better models.

Only Big Tech (Microsoft,Google,Facebook) can crawl the web at scale because they own the major content companies and they severly throttle the competition's crawlers, and sometimes outright block them. I'm not saying it's impossible to get around, but it is certainly very difficult, and you could be thrown in prison for violating the CFAA.

Re: OpenAI is too cheap to beat

#156

Post by codedokode »

beauHD wrote: post #69

I signed up for OpenAI's ChatGPT tool, and entered a query, like 'What does the notation 1e100 mean?' (just to try it out). And then when displaying the output it would start outputting the reply in a slow way, like, it was dripfeeded to me, and I was like: 'what? surely this could be faster?' Maybe I'm missing something crucial here, but why does it dripfeed answers like this? Does it have to think really hard about…

Because it needs to do billions of arithmetic operations to generate a reply. Replying to questions is not an easy task.

Re: OpenAI is too cheap to beat

#157

Post by wkat4242 »

ldjkfkdsjnv wrote: post #136
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Completely wrong, the best AI will win. There is insane demand for better models.

The improvements seem to be leveling off already. GPT-4 isn't really worth the extra price to me. It's not that much better.

What I would really want though is an uncensored LLM. OpenAI is basically unusable now, most of its replies are like "I'm only a dumb AI and my lawyers don't want me to answer your question". Yes I work in cyber. But it's pretty insane now.

Re: OpenAI is too cheap to beat

#158

Post by AJRF »

I read this and think "That won't last long".

The pricing is too good to be true with you think about it rationally. If they raise prices they seem much, much less attractive than using AWS or Azure.

Amazon seem to have a much better business built around their Bedrock offering. And all their other tools are available there like SageMaker, ec2, integration with MLFlow, etc, etc.

I guess the same goes for Azure, if you are already using it it's much easier to just stick with whatever they are offering for LLM Ops.

OpenAI offering just models doesn't seem like it can last forever, and to compete with AWS or Azure at enterprise level they need to build all the things Amazon/MS have built.

The other side of that coin seems much more realistic.

Re: OpenAI is too cheap to beat

#159

Post by bugglebeetle »

wkat4242 wrote: post #157
ldjkfkdsjnv wrote: post #136

Earlier quoted context omitted.

Completely wrong, the best AI will win. There is insane demand for better models.

The improvements seem to be leveling off already. GPT-4 isn't really worth the extra price to me. It's not that much better. What I would really want though is an uncensored LLM. OpenAI is basically unusable now, most of its replies are like "I'm only a dumb AI and my lawyers don't want me to answer your question". Yes I work in cyber. But it's pretty insane now.

GPT-4, correctly prompted, is head and shoulders above everything for coding. All the text generation stuff and NLP tasks, it’s a toss-up.

Re: OpenAI is too cheap to beat

#160

Post by PaulHoule »

ldjkfkdsjnv wrote: post #136
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Completely wrong, the best AI will win. There is insane demand for better models.

Depends how you define quality. This paper reflects my own experience

https://arxiv.org/abs/2305.08377

and shows how LLM technology has a lot more to offer than "ChatGPT". The real takeaway is that by training LLMs with real training data (even with a "less powerful" model) you can get an error rate more than 10x less than you get with the "zero shot" model of asking ChatGPT to answer a question for you the same way that Mickey Mouse asked the broom to clean up for him in Fantasia. The "few-shot" approach of supplying a few examples in the attention window was a little better but not much.

The problem isn't something that will go away with a more powerful model because the problem has a lot to do with the intrinsic fuzziness of language.

People who are waiting for an exponentially more expensive ChatGPT-5 to save them will be pushing a bubble around under a rug endlessly while the grinds who formulate well-defined problems and make training sets will actually cross the finish line.

Remember that Moore's Law is over in the sense that transistors are not getting cheaper generation after generation, that is why the NVIDIA 40xx series is such a disappointment to most people. LLMs have some possibility of getting cheaper from a software perspective as we understand how they work and hardware can be better optimized to make the most of those transistors, but the driving force of the semiconductor revolution is spent unless people find some entirely different way to build chips.

But... people really want to be like Mickey in Fantasia and hope the grinds are going to make magic for them.

Post reply on HN