Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

501–510 of 562 posts

Re: GPT-4 API General Availability

#501

Post by lhl »

Gasp0de wrote: post #480
lhl wrote: post #446

Earlier quoted context omitted.

In terms of speed, we're talking about 140t/s for 7B models, and 40t/s for 33B models on a 3090/4090 now.[1] (1 token ~= 0.75 word) It's quite zippy. llama.cpp performs close on Nvidia GPUs now (but they don't have a handy chart) and you can get decent performance on 13B models on M1/M2 Macs. You can take a look at a list of evals here: https://llm-tracker.info/books/evals/page/list-of-evals - for general usage, I th…

Those GPUs are 1200$ and upwards. This is equivalent to 20,000,000 tokens on GPT-4. I don't think I will ever use this many tokens for my personal use.

I agree that everyone should do their own cost-benefit analysis, especially if they have to buy additional hardware (used RTX 3090s are ~$700 atm), but one important thing to note for those running the numbers is that all your tokens need to be resubmitted for every query. That means, that if you end up using the OpenAI API for long-running tasks like say a code assistant or pair programmer, with an avg of 4K tokens of context, you will pay $0.18/query, or hit $1200 at about 7000 queries. [1] At 100 queries a day, you'll hit that in just over 2 months. (Note, that is 28M tokens. In general tokens go much faster than you think. Even running a tiny subset of lm-eval against will use about 5M tokens.)

If people are mostly using their LLMs for specific tasks, then using cloud providers (Vast.ai and Runpod were cheapest last time I checked) can be cheaper than dedicated hardware, especially if your power costs are high. If you're needs are minimal, Google Colab offers a free tier with a GPU w/ 11GB of VRAM, so you can run 3B/7B quantized models easily.

There are reasons of course irrespective of cost to run your own model (offline access, fine-tuning/running task specific models, large context/other capabilities OpenAI doesn't provide (eg, you can run multi-modal open models now), privacy/PII, BCP/not being dependent on a single vendor, some commercial or other non-ToS allowed tasks, etc).

[1] https://gptforwork.com/tools/openai-chatgpt-api-pricing-calc...

Re: GPT-4 API General Availability

#502

Post by eli »

londons_explore wrote: post #259
eli wrote: post #255

Earlier quoted context omitted.

The former seems very believable. And I bet a lot of the fine tuned models that are active are still part of prototypes or experiments. I assume if you reach out they throw some credits at you

If it really was a tiny number of users, they would publically make a really good offer - for example: "Unfortunately, you will need to retune your models on top of GPT-4. OpenAI will do this for you for free, and refund all money you paid tuning your original model, and offer the new model for the same price as the original model." The extra trust gained by seeing another customer treated that way easily pays for a…

Yeah but that sets a precedent

Re: GPT-4 API General Availability

#503

Post by kaba0 »

ben_w wrote: post #486
kaba0 wrote: post #443

Earlier quoted context omitted.

> However, just because AGI has a good chance of making humanity extinct How? I mean surely it will lead humanity down some chaotic path, but I would fear climate catastrophe much much more than anything AI-related.

Imagine if you will that the companies responsible for the carbon emissions get themselves an AI, with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD. That's one of the better outcomes. A worse outcome is that an unrestricted AI helps walk a depressed and misanthropic teenager through the process of engineering airborne super-AIDS. Or that someone suffering from a schizophreni…

> with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD.

So what we have ongoing for half a century?

I honestly don’t see what changes here — super-human intelligence has limited benefits as it scales. Would you suddenly have more power in life, were you twice as smart? If so, we would have math professors as world leaders.

Life can’t be “won” by intelligence, that is only one factor, luck being a very significant other one. Also, if we want to predict the future with AIs we probably shouldn’t be looking at “one-on-one” interactions, as there is not much difference there compared to the status quo — a smart person with whatever motivation could easily do any of your mentioned scenarios. Hell, you couldn’t even tell the difference in theory if it happens through a text-only interface.

Also, it is naive to assume that many scientific breakthroughs are “blocked” by raw intelligence. Especially biology is massively data-limited, which won’t be any more available to an AI than to the researchers at hand, let alone that teenager.

The new dimension such a construct could open up is the complete loss of trust on the internet (which is again pretty close to where we stand today), which can have very profound effects indeed I’m not trying to diminish. But these sci-fi outcomes are just.. naive. It will be more of a newfound chaos with countless intelligent agents taking over the internet with different agendas - but their cumulative impact might very well move us back to closed forums/to the physical world. Which will definitely turn certain long-standing companies on its head. We will see, as this is basically already happening, we don’t need human-level intelligence, GPT’s output is more than enough.

Re: GPT-4 API General Availability

#504

Post by dontupvoteme »

esperent wrote: post #140
athyuttamre wrote: post #111

Earlier quoted context omitted.

(I'm an engineer at OpenAI) Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case? https://github.com/openai/openai-cookbook/blob/main/examples... We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have t…

It's kind of incredible how fast OpenAI (now also known as ClosedAI) is going through the enshittification process. Even Facebook took around a decade to reach this level. OpenAI has an amazing core product, but in the span of six months: * Went from an amazing and inspiring open company that even put "Open" in their name to a fully locked up commercial beast. * Non-existent customers support and all kinds of borderl…

They also no longer support data exports for many users (including myself) - at one point it worked but now it says you'll receive an email to download your data, which never arrives.

Re: GPT-4 API General Availability

#505

Post by rvcdbn »

tensor wrote: post #242
rvcdbn wrote: post #237

Earlier quoted context omitted.

We don’t actually know that it’s not legal. The copyrightability of model weights is an open legal question right now afaik.

It doesn't have to be copyrightable to be intellectual property.

I’m going to go out on a limb here and assume that you’re making this statement because it feels like they should have some intellectual property rights in this case. Independently of whether that feeling corresponds to legal reality (the original question) I would also encourage you to question the source of this feeling. I believe it is rooted in an ideology where information is restricted as property by default. This is a dangerous ideology that constantly threatens to encroach on intellectual freedom e.g. software patents, gene patents. We have a wonderful tradition in the US that information is free by default. It has been much eroded by this ideology but I believe freedom is still legally the default unless the information falls under the criteria of trademark, copyright or patent. I think it’s important to recognize how this ideology of non-freedom has perniciously warped people’s default expectation around information sharing.

Re: GPT-4 API General Availability

#506

Post by jabradoodle »

kaba0 wrote: post #503
ben_w wrote: post #486

Earlier quoted context omitted.

Imagine if you will that the companies responsible for the carbon emissions get themselves an AI, with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD. That's one of the better outcomes. A worse outcome is that an unrestricted AI helps walk a depressed and misanthropic teenager through the process of engineering airborne super-AIDS. Or that someone suffering from a schizophreni…

> with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD. So what we have ongoing for half a century? I honestly don’t see what changes here — super-human intelligence has limited benefits as it scales. Would you suddenly have more power in life, were you twice as smart? If so, we would have math professors as world leaders. Life can’t be “won” by intelligence, that is only one f…

> Life can’t be “won” by intelligence

Humans being the dominant life form on Earth may suggest otherwise.

> I honestly don’t see what changes here — super-human intelligence has limited benefits as it scales. Would you suddenly have more power in life, were you twice as smart? If so, we would have math professors as world leaders.

Intelligent humans by definition do not have super human intelligence.

Re: GPT-4 API General Availability

#507

Post by jabradoodle »

ftxbro wrote: post #369
bestcoder69 wrote: post #359

Earlier quoted context omitted.

https://github.com/search?q=repo%3Asimonw%2Fllm%20OPENAI_API... Let us know what you find.

I found a few github issues related to api key security and management. I'm not 100% sure of your point.

I see no open issues.

It's around 1k lines of python, audit the code if you care and rotate your keys.

Or don't use it.

Re: GPT-4 API General Availability

#508

Post by concordDance »

moffkalast wrote: post #461
concordDance wrote: post #430

Earlier quoted context omitted.

> Sam tells Congress that AI is so dangerous it will extinct humanity. Why? So Congress can license him and only his buddies. No, he says it because its true and concerning. However, just because AGI has a good chance of making humanity extinct does not mean we're anywhere close to making AIs that capable. LLMs seem like a dead end.

> its true and concerning > LLMs seem like a dead end These would seem contradictory. If you really think that both are true and Altman knows it, then you're saying he's a hype man lying for regulatory capture. And to some extent he definitely is overblowing the danger for his own gain. I really doubt they are a dead end though, we've barely started to explore what they can do. There's a lot more that can be extracte…

A dead end for human+ level AGI, they will still be useful.

Re: GPT-4 API General Availability

#509

Post by ben_w »

kaba0 wrote: post #503
ben_w wrote: post #486

Earlier quoted context omitted.

Imagine if you will that the companies responsible for the carbon emissions get themselves an AI, with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD. That's one of the better outcomes. A worse outcome is that an unrestricted AI helps walk a depressed and misanthropic teenager through the process of engineering airborne super-AIDS. Or that someone suffering from a schizophreni…

> with no restrictions, and task it to endlessly spew pro-carbon propaganda and anti-green FUD. So what we have ongoing for half a century? I honestly don’t see what changes here — super-human intelligence has limited benefits as it scales. Would you suddenly have more power in life, were you twice as smart? If so, we would have math professors as world leaders. Life can’t be “won” by intelligence, that is only one f…

> So what we have ongoing for half a century?

Except fully automated, cheaper, and with the capacity to fluently respond to each and every person who cares about the topic.

At GPT-4 prices, a billion words is only about 79800 USD.

> Life can’t be “won” by intelligence, that is only one factor, luck being a very significant other one.

It doesn't need to be the only factor, it just needs to be a factor. Luck in particular is the least helpful counterpoint, as it's not like only one person uses AI at any given moment.

> Especially biology is massively data-limited, which won’t be any more available to an AI than to the researchers at hand, let alone that teenager.

Indeed; I certainly hope this isn't as easy as copy-pasting bits of one of the many common cold virus strains with HIV.

But homebrew synbio and DNA alteration is already a thing.

Re: GPT-4 API General Availability

#510

Post by concordDance »

kaba0 wrote: post #443
concordDance wrote: post #430

Earlier quoted context omitted.

> Sam tells Congress that AI is so dangerous it will extinct humanity. Why? So Congress can license him and only his buddies. No, he says it because its true and concerning. However, just because AGI has a good chance of making humanity extinct does not mean we're anywhere close to making AIs that capable. LLMs seem like a dead end.

> However, just because AGI has a good chance of making humanity extinct How? I mean surely it will lead humanity down some chaotic path, but I would fear climate catastrophe much much more than anything AI-related.

Note that I said AGI there, not AI. The full AGI X-risk case is hundreds of pages, unsuitable for a hackernews discussion.

To oversimplify to the point of wrongness: Essentially how humans dominated our world, by being smarter.

Post reply on HN