Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

411–420 of 648 posts

Re: GPT-4 details leaked?

#411

Post by _nalply »

koheripbal wrote: post #137
juliensalinas wrote: post #100

Earlier quoted context omitted.

LLaMA 30B or 60B can be very impressive when correctly prompted. Deploying the 60B version is a challenge though and you might need to apply 4-bit quantization with something like https://github.com/PanQiWei/AutoGPTQ or https://github.com/qwopqwop200/GPTQ-for-LLaMa . Then you can improve the inference speed by using https://github.com/turboderp/exllama . If you prefer to use an "instruct" model à la ChatGPT (i.e. tha…

4-bit quantization removes a lot of the model's sophistication, and 60B parameters is still smaller than what GPT4 is using.

I remember to have read somewhere that GPT4 is not a single model but several models whose parameter counts are reported as a single sum. Perhaps quite doable but at lower speeds?

Re: GPT-4 details leaked?

#412

Post by flangola7 »

2OEH8eoCRo0 wrote: post #390
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

> great news for open source. Yes, and great news for shills, bad actors, agitators, trolls, foreign intel, and propagandists. I'm impressed by the tech but terrified because for once I cannot conceive of what this means for the future. My guess is that this kills the open web and laws get passed which bury it.

Everybody is self-soothing with the idea that OpenAI's (frankly, half hearted) push for regulation is just mundane regulatory capture and profit seeking, and not the fact that it will, at best, absolutely destroy everything about the internet and technology that we've come to love and know. Should a 4chan torrent show up like LLaMA, with weights and code for a base GPT4-level model, modern society is done. Golden age over.

Re: GPT-4 details leaked?

#413

Post by rurp »

antupis wrote: post #92
KronisLV wrote: post #76

Earlier quoted context omitted.

> You can use the above without paying OpenAI. You don't even need a GPU. There are no license issues like with the facebook llama. I actually wrote about getting an LLM chatbot up and running a while ago: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... It's good that the technology and models are both available for free, and you don't even need a GPU for it. However, there are still large memory requ…

I bet that open models win in the end because porn. There is already very weird and vibrant community creating "waifus" and tinkering with these models.

Porn could definitely be a killer app for LLMs, and I strongly hope that open source models win out for a number of reasons, but I'm not sure it will happen. Microsoft and OpenAI have quickly gone to great lengths to try and limit the amount of output they find offensive, and those efforts have been fairly well received by the general public.

Despite the size of the regular online porn industry, it still gets strangled by payment processors. There is plenty of appetite out there for restricting porn in various ways.

Re: GPT-4 details leaked?

#414

Post by hyperthesis »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

I love this diatribe! But to be fair, this kind of engineering historically can at least lead to science.

Re: GPT-4 details leaked?

#415

Post by unshavedyak »

londons_explore wrote: post #378
jetrink wrote: post #310

Earlier quoted context omitted.

At a certain point though, models become good enough for particular tasks. Once that happens for whatever my application is, I don't care if OpenAI has a model that's twice as good on some metric, because it's overkill for my use-case. I'm going to be happy using a smaller, cheaper model from a competitor.

I think we're far from that point though. For the vast majority of use cases, I always wish that the answers could be more accurate. Sure - they might be 'good enough' to build a business on. But if a competitor builds their business on top of a more accurate model, their product will work better, and they will win the market.

Yea but the bench being discussed here is FOSS. Which for me, and many, translates to can i run something useful in my closet or on my phone. I've found LLaMA neat and yea, some FOSS models are getting decent - but they're a far cry from GPT4. I pay for GPT4, use it almost daily and that's my bench.

Yes, when i can run GPT4 in my closet, OpenAI will have GPT7 or w/e - but it doesn't change the fact that i have something useful running in my closed network and that opens up all kinds of data integration that i'm unwilling to ship to OpenAI. In that day i'll probably still use GPT7, but i'll _also_ have GPT4 running in my closet and integrating with a ton of things on my local network.

Re: GPT-4 details leaked?

#416

Post by gdubs »

Recently I was saying how much amazing stuff there is in retro computing. One thing that keeps coming to mind for me recently is just how visionary Thinking Machines Connection Machine supercomputer architecture was with its massive parallelism built in, with neural network applications being a key predicted use case at the time. That was so long ago!

Interesting to think about in comparison to the challenges today around parallelizing 'commodity' GPUs. Scare quotes because he A100 and H100 are pretty impressive machines in and of themselves.

Re: GPT-4 details leaked?

#417

Post by londons_explore »

renlo wrote: post #404
londons_explore wrote: post #367

Earlier quoted context omitted.

And the tweeter of the twitter thread paid the $1000, copied the useful info to twitter, and then did a credit card chargeback.

Seems he summarized it and didn't copy it

A summary isn't allowed under US copyright law. The copyright office calls them "condensations", and they are considered derivative works.

His use was likely not within US copyright law. "Effect of the use upon the potential market for or value of the copyrighted work" is one of four factors a judge should use to decide if fair use applies, and it is clear that publishing the main information from an article, information which is not available elsewhere, freely, severely degrades the market for the original.

Re: GPT-4 details leaked?

#418

Post by gmd63 »

mensetmanusman wrote: post #319
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

I think the colloquial understanding of "science" means studying how the world works. This is more like engineering: exploiting our understanding of how the world works to build new capabilities that we don't fully understand.

I don't think you would call it "science" for a bunch of single cellular organisms to cooperatively evolve a multicellular one. Similarly you wouldn't call it science when humans create digital lifeforms that require actual science to be done to understand how they work.

Re: GPT-4 details leaked?

#419

Post by mdp2021 »

ben_w wrote: post #276
namaria wrote: post #267

Earlier quoted context omitted.

And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publici…

> And safely here means non disruptive to established businesses Why would OpenAI care about that? No, it means safety, as in not giving out dangerous answers that get people killed.

> as in not giving out dangerous answers that get people killed

Literally taken, that is quite close to impossible.

It was news days ago of somebody who committed suicide after having an interaction with a bot about nuclear risks or similar.

To avoid that, the bot would have to be a high-ranking professional psychologist with an explicit purpose not to trigger destructive reactions.

And that would fail the nature of a "consultant", which is something "under best effort to speak the truth" - incompatible direction with "something reassuring".

Re: GPT-4 details leaked?

#420

Post by dwallin »

JumpCrisscross wrote: post #240
dwallin wrote: post #213

Earlier quoted context omitted.

Copyright does very little for individuals. Most benefits from the copyright system are accrued to large corporations.

> Most benefits from the copyright system are accrued to large corporations Citation fucking needed. Among those who study copyright and inequality, none suggest abandoning it [1][2]. Within the context of machine learning, one of the only pillars buttressing individuals against multi-trillion dollar corporations is copyright [3]. [1] https://journals.library.wustl.edu/lawreview/article/id/5108... [2] https://www.jst…

I never argued that copyright should be abolished, I was responding to a comment that essentially said that anyone who was opposed to copyright is doing so at the behest of large corporations at the expense of individuals, which is a really flawed and one sided way of looking at copyright laws, the first linked article you provided being a great example.

For the record, I can’t find a single large corporation pushing for the dissolution of copyright, and many clear examples of large corporations lobbying for increased copyright protections and terms. I thinks this puts the onus for providing evidence on those who would argue that corporations are completely failing to act in their own interest.

Post reply on HN