Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

131–140 of 648 posts

Re: GPT-4 details leaked?

#131

Post by Fiahil »

potatoman22 wrote: post #25

I wonder what the legal implications of them using SciHub and Libgen would be if that's true. I'd imagine OpenAI is big enough to make deals with publishers.

If that's true, then OpenAI has probably taken extreme protective measure to ensure the secret is well protected. Even if OpenAI is big enough to make deals, they probably did not spend several years making deals with all of them.

It's, however, very interesting to see if they fund efforts to massively (re)start books digitalisation.

Re: GPT-4 details leaked?

#132

Post by n1c »

errantmind wrote: post #120
mmahemoff wrote: post #96

I've been wondering how freemium services like Thread Reader still operate now that Twitter is charging prohibitive prices for API access and taking measures to prevent scraping. The cheapest API plan with read access is $100/month, which reads 10,000 tweets, so could only produce about 500 pages like this one on demand.

There was a post on HN recently with a workaround these apps are using. I don't have it handy but I'm sure you can find it if you look.

There's probably some interesting bits of info in yesterday's Nitter thread: https://news.ycombinator.com/item?id=36665406

Re: GPT-4 details leaked?

#133

Post by TeMPOraL »

DANmode wrote: post #46
DanAtC wrote: post #42

Earlier quoted context omitted.

I meant things like: - parameters - layers - "Mixture Of Experts" - tokens That's about as far as I made it

Parameters: In the context of AI and language models, parameters refer to the internal settings or variables that an AI model uses to make predictions or generate responses. Think of them as the knobs and switches that can be adjusted to fine-tune how the AI understands and generates language. These parameters are learned during the training process, where the AI model analyzes vast amounts of data to optimize its pe…

> "Mixture Of Experts": An "MoE" is an approach in AI that combines multiple specialized AI models, known as "experts," (...)

Wonder when that stopped being called just an "ensemble model", which is a term I recall from 10 years ago. Terminology churn?

Re: GPT-4 details leaked?

#134

Post by mahathu »

KronisLV wrote: post #112
antupis wrote: post #92

Earlier quoted context omitted.

I bet that open models win in the end because porn. There is already very weird and vibrant community creating "waifus" and tinkering with these models.

Huh, more power to those folks then, I guess. But I can easily imagine more conventional forms of entertainment, as well. Like a game of D&D that's narrated by the AI, or a text based adventure set in the Mass Effect universe, Lord of the Rings, Warhammer or any other fandom, really. Maybe like those old Choose Your Own Adventure games. I think some companies are also experimenting with characters in video games that…

The application in games I'm most excited about is commenters in FIFA career mode that don't have a limited set of prerecorded voice lines, and take your recent games, formation changes etc into account too, like real commentators would. The recent installments already do that to a small degree. Of course this would also easily open the doors to having multiple commentators/analysts to choose from, each with their individual "personalities". The technology for that is pretty much all already there right?

A game like Fallout/Elder Scrolls with AI generated NPCs and questlines would be so sick too if executed well.

Re: GPT-4 details leaked?

#135

Post by xeckr »

If this is true, then:

1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything?

2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

Re: GPT-4 details leaked?

#136

Post by TeMPOraL »

yosame wrote: post #75
anaganisk wrote: post #66

Earlier quoted context omitted.

Since 12 years?

Surely they'll find a use case before the sun burns out!

That's quite uncertain, given that Bitcoin is couple scaling rounds from literally sucking the sun dry to power the "proof of work" scheme.

Re: GPT-4 details leaked?

#137

Post by koheripbal »

juliensalinas wrote: post #100
kyledrake wrote: post #65

Earlier quoted context omitted.

Unfortunately I've found the current OSS models to be vastly inferior to the OpenAI models. Would love to see someone actually get close to what they can do with GPT-3.5/4, except capable of running on commodity GPUs. What's the most impressive open model so far?

LLaMA 30B or 60B can be very impressive when correctly prompted. Deploying the 60B version is a challenge though and you might need to apply 4-bit quantization with something like https://github.com/PanQiWei/AutoGPTQ or https://github.com/qwopqwop200/GPTQ-for-LLaMa . Then you can improve the inference speed by using https://github.com/turboderp/exllama . If you prefer to use an "instruct" model à la ChatGPT (i.e. tha…

4-bit quantization removes a lot of the model's sophistication, and 60B parameters is still smaller than what GPT4 is using.

Re: GPT-4 details leaked?

#138

Post by agos »

svnt wrote: post #60
tlb wrote: post #44

Earlier quoted context omitted.

What percentile rank among college-educated Americans would that correspond to? I'd guess that takes it out of the top 0.1%, but not the top 1%.

Call it 100 million college-educated Americans. I don’t think 100,000 people can formulate working quantum computing algorithms in ten seconds. It is probably closer to 0.001%, but those people probably can’t describe 13th century medical technology very well.

the tricky part is that the rest of that million college-educated folks would answer "I don't know", something LLMs really struggle at

Re: GPT-4 details leaked?

#139

Post by pyeri »

PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

In today's world, "Science equals Capitalism".

Or at least Science is allowed to progress and get funded only as long as it serves the interest of Capitalism.

Re: GPT-4 details leaked?

#140

Post by jackstraw14 »

lacrimacida wrote: post #122
jackstraw14 wrote: post #111

Earlier quoted context omitted.

> But what I find most interesting is that there is absolutely no taking of responsibility of any technological creations. I appreciate your willingness to talk about it, but to be honest it doesn't seem like it matters much what you, or any of us (not singling you out in particular), thinks about it, does it? It probably doesn't even matter who these people are who should take responsibility. This is one genie, like…

It actually does make a difference. The genie is out of the bottle partially but it depends a lot on what we allow it it be used on. If we sit idly and allow for ingesting all what’s written for instance, including whats currently written and let bros make derivative works for a quick buck then we mostly killed the writer’s incentive to write or publish. If we slow down and not allow ripping one another off it could…

> If we sit idly and allow for ingesting all what’s written for instance

It's already happened. What do you do now? For decades, it's happened regardless of the robots.txt so you have to assume it's all been ingested by it all, everywhere. What now?

Post reply on HN