Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

441–450 of 648 posts

Re: GPT-4 details leaked?

#441

Post by raydev »

kyledrake wrote: post #65
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

Unfortunately I've found the current OSS models to be vastly inferior to the OpenAI models. Would love to see someone actually get close to what they can do with GPT-3.5/4, except capable of running on commodity GPUs. What's the most impressive open model so far?

While OpenAI likely has some insights that open-source and closed-source competitors are lacking, OpenAI is mostly in the lead because they can burn absurd amounts of cash running an absurd amount of compute via their partnership with Microsoft.

Re: GPT-4 details leaked?

#442

Post by lossolo »

Vecr wrote: post #123
Llamamoe wrote: post #103

Earlier quoted context omitted.

How low can you get the memory and computational power requirements that way?

You can run that model (Wizard-30) on a computer with 64 gigabytes of RAM (or smaller, I don't know how tight you can cut it). You obviously want fast RAM and a good CPU, but you don't need a GPU.

You can also travel on a bike from NY to LA.

Re: GPT-4 details leaked?

#443

Post by typon »

Xeoncross wrote: post #359
michaelcampbell wrote: post #356

Earlier quoted context omitted.

> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. This is essentially the Capitalist Credo, expressed in practical vs theoretical terms.

> pressure the government into making it Not really, Capitalism is free-trade between two parties. The more government involvement you have the more it moves towards socialism or communism where the government controls trade. At least this was the historical meaning. These days Capitalism is being redefined to mean private (non-government) Communism. That is, power concentrated in the hands of a few.

I would recommend reading this entire page: https://plato.stanford.edu/entries/socialism/

But if you don't have the time for that, just read the intro paragraph.

Re: GPT-4 details leaked?

#444

Post by svnt »

agos wrote: post #138
svnt wrote: post #60

Earlier quoted context omitted.

Call it 100 million college-educated Americans. I don’t think 100,000 people can formulate working quantum computing algorithms in ten seconds. It is probably closer to 0.001%, but those people probably can’t describe 13th century medical technology very well.

the tricky part is that the rest of that million college-educated folks would answer "I don't know", something LLMs really struggle at

A product that mostly declines to be a product doesn’t seem like it would sell very well.

Re: GPT-4 details leaked?

#445

Post by kirill5pol »

lolinder wrote: post #354
juliensalinas wrote: post #100

Earlier quoted context omitted.

LLaMA 30B or 60B can be very impressive when correctly prompted. Deploying the 60B version is a challenge though and you might need to apply 4-bit quantization with something like https://github.com/PanQiWei/AutoGPTQ or https://github.com/qwopqwop200/GPTQ-for-LLaMa . Then you can improve the inference speed by using https://github.com/turboderp/exllama . If you prefer to use an "instruct" model à la ChatGPT (i.e. tha…

Just a reminder that LLaMA is not open—in order to use it legally you have to agree to Meta's terms, which currently means research use only. The versions circulating on torrents are essential pirated, and while I don't have an ethical problem with that at all you can't use it safely in a business. The open replacements for LLaMA have yet to reach 30B, let alone 65B.

That hasn’t been true for a while. Falcon 40B seemingly outperforms LLaMA 60B according to the OpenLLM leaderboard

https://huggingface.co/tiiuae/falcon-40b

Re: GPT-4 details leaked?

#446

Post by raydev »

ben_w wrote: post #236
wrasee wrote: post #228

Earlier quoted context omitted.

I don’t understand this. Won’t that hurt their progress on GPT-5?

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

More cynically, they think they've hit a wall and a projected GPT-5 won't be a huge or meaningful jump in performance, so they'd like everyone else to slow down too.

Re: GPT-4 details leaked?

#447

Post by dcow »

machina_ex_deus wrote: post #432
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

I'm tired of science as a religion. People treat it as some gospel, like if you check out some criterions you're suddenly "scientific" and instantly get a sense of validity and authority that you shouldn't logically get. I judge things as "what you can do", not "what can you predict". The only demonstration of knowledge and understanding is being able to do something. Not predict. Not "scientific method" and ridiculo…

You would love Thomas Kuhn.

Re: GPT-4 details leaked?

#448

Post by kirill5pol »

_nalply wrote: post #411
koheripbal wrote: post #137

Earlier quoted context omitted.

4-bit quantization removes a lot of the model's sophistication, and 60B parameters is still smaller than what GPT4 is using.

I remember to have read somewhere that GPT4 is not a single model but several models whose parameter counts are reported as a single sum. Perhaps quite doable but at lower speeds?

The article linked here talks about GPT4 being a mixture of experts, which is exactly what you’re describing

Re: GPT-4 details leaked?

#449

Post by _ea1k »

schappim wrote: post #97
Roark66 wrote: post #79

Earlier quoted context omitted.

Have you tried falcon 40b instruct? Also take into account that chatgpt likely has some preprompt and by talking to falcon or other OS models it's all in your hands. Furthermore, Not many people discuss the significance of proper output sampling. I myself used to just test open source models with the greedy decoding only. Who knows if they wouldn't even beat (not at all)OpenAI with some clever output sampling scheme.

Does anyone have a link / instructions on getting Falcon 40b to install on Apple Silicon? Apparently "Hugging Face" have some internal swift code that works (but it has not been released). I'm keen to see how it performs on a maxed out Mac Studio (with all that unified memory available).

I thought this too, but it sounds like the performance of the GPU ultimately holds it back. Maybe it is the software, but I have yet to see a test on Mac silicon that performed well.

Re: GPT-4 details leaked?

#450

Post by [deleted] »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

[deleted]
Post reply on HN