[flagged]
ChatGPT Pro
541–550 of 1001 posts
Re: ChatGPT Pro
#542[flagged]
Re: ChatGPT Pro
#543The price feels outrageous, but I think the unsaid truth of this is that they think o1 is good enough to replace employees. For example, if it's really as good at coding as they say, I could see this being a point where some people decide that a team of 5 devs with o1 pro can do the work of 6 or 7 devs without o1 pro.
I think the key is to have a strong goal. If the developer knows what they want but can't quite get there, even if it gives the wrong answer you can catch it. The use the resulting code to improve your productivity.
Last week when using jetpack compose(which is a react like framework). A cardinal sin in jetpack compose is to change a State variable in a composable based on non-user/UI action which the composable also mutates. This is easy enough to understand this for toy examples. But for more complex systems one can make this mistake. o1-preview made this mistake last week, and I caught it. On prompting it with the stacktrace it did not immediately catch it and recommended a solution that committed the same error. When I actually gave it the documentation on the issue it caught on and made the variable a userpreference instead. I used the userpreference code in my app instead of coding it by myself. It worked well.
Re: ChatGPT Pro
#544Earlier quoted context omitted.
Yes. The people who can use these tools to dramatically increase their capabilities and output without a significant drop in quality were already great engineers for which there was more demand than supply. That isn't going to change soon.
Ditto for other use cases, like writer and editor. There are a ton of people doing that work whom I don’t think are ever going to figure out how to use LLMs well. Like, 90% of them. And LLMs are nowhere near making the rest so much better that they can make up for that. They’re ok for Tom the Section Manager to hack together a department newsletter nobody reads, though, even if Tom is bad at using LLMs. They’re decen…
I disagree. By far, most of the code is created by perpetually replaced fresh juniors churning out garbage. Similarly, most of the writing is low-quality marketing copy churned out by low-paid people who may or may not have "marketing" in their job title.
Nah, if the last 10-20 years demonstrated something, it's that nothing needs to be any good, because a shitty simulacrum achieves almost the same effect but costs much less time and money to produce.
(Ironically, SOTA LLMs are already way better at writing than typical person writing stuff for money.)
Re: ChatGPT Pro
#545Earlier quoted context omitted.
I have several local models I hit up first (Mixtral, Llama), if I don’t like the results then I’ll give same prompt to Claude and GPT. Overall though it’s really just for reference and/or telling me about some standard library function I didn’t know of. Somewhat counterintuitively I spend way more time reading language documentation than I used to, as the LLM is mainly useful in pointing me to language features. Afte…
> But none of them are sufficient alone, you do need a “team” of them Given the sensitivity to parameters and prompts the models have, your "team" can just as easily be querying the same LLM multiple times with different system prompts.
Other factor is I use local LLM first because I don’t trust any of the companies to protect my data or software IP.
Re: ChatGPT Pro
#546I do wonder what effect this will have on furthering the divide between the "rich West" and the rest of the world. If everyone in the West has powerful AI and Agents to automate everything. Simply because we can afford it, but the rest of the world doesn't have access to it. What will that mean for everyone left behind?
Ai is no where near the level of leaving behind those that aren't using it. Especially not at the individual consumer level like this.
Even if its not making you smarter, AI is definitely making you more productive. That essentially means you get to outproduce poorer people, if not out-intellectualize them
Re: ChatGPT Pro
#547Earlier quoted context omitted.
Is compute that expensive? An H100 rents at about $2.50/hour, it's 80 hours of pure compute. Assuming 720 hours a month, 1/9 duty cycle around the clock, or 1/3 if we assume 8-hour work day. It's really intense, constant use. And I bet OpenAI spend less on operating their infra than the rate at which cloud providers rent it out.
Does an o1 query run on a singular H100, or on a plurality of H100s?
A single H100 has 80GB of memory, meaning that at FP16 you could roughly fit a 40B parameter model on it, or at FP4 quantisation you could fit a 160B parameter model on it. We don't know (I don't think) what quantisation OpenAI use, or how many parameters o1 is, but most likely...
...they probably quantise a bit, but not loads, as they don't want to sacrifice performance. FP8 seems like a possible middle ground. o1 is just a bunch of GPT-4o in a trenchcoat strung together with some advanced prompting. GPT-4o is theorised to be 200B parameters. If you wanted to run 5 parallel generation tasks at peak during the o1 inference process, that's 5x 200B, at FP8, or about 12 H100s. 12 H100s takes about one full rack of kit to run.
Re: ChatGPT Pro
#548OpenAI is racing against two clocks: the commoditization clock (how quickly open-source alternatives catch up) and the monetization clock (their need to generate substantial revenue to justify their valuation). The ultimate success of this strategy depends on what we might call the enterprise AI adoption curve - whether large organizations will prioritize the kind of integrated, reliable, and "safe" AI solutions Open…
The problem is that OpenAI don't really have the enterprise market at all. Their APIs are closer in that many companies are using them to power features in other software, primarily Microsoft, but they're not the ones providing end user value to enterprises with APIs. As for ChatGPT, it's a consumer tool, not an enterprise tool. It's not really integrated into an enterprises' existing toolset, it's not integrated int…
Is their valuation proposition self fulfilling: the more people pipe their queries to OpenAI, the more training data they have to get better?
Re: ChatGPT Pro
#549Why doesn't Pro include longer context windows? I'm a Plus member, and the biggest limitation I am running into by far is the maximum length of a context window. I'm having context fall out of scope throughout the conversion or not being able to give it a large document that I can then interrogate. So if I go from paying $20/month for 32,000 tokens, to $200/month for Pro, I expect something more akin to Enterprise's…
The longer the context the more backtracking it needs to do. It gets exponentially more expensive. You can increase it a little, but not enough to solve the problem. Instead you need to chunk your data and store it in a vector database so you can do semantic search and include only the bits that are most relevant in the context. LLM is a cool tool. You need to build around it. OpenAI should start shipping these other…
This isn't true.
Transformer architectures generally take quadratic time wrt sequence length, not exponential. Architectural innovations like flash attention also mitigate this somewhat.
Backtracking isn't involved, transformers are feedforward.
Google advertises support for 128k tokens, with 2M-token sequences available to folks who pay the big bucks: https://blog.google/technology/ai/google-gemini-next-generat...
Re: ChatGPT Pro
#550My friend found 2 chimney sweep businesses. One charges $569, the other charges $150. Plot twist: the same guy runs both. They do the same thing and the same crew shows up.
I used to work for a manufacturing company that did this. They offered a standard, premium, and "House Special Product". House special was 2x premium but the same product. They didn't even pretend it wasn't, they just said it was recommended and people bought it.