Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hi everyone yes, I left OpenAI yesterday

twitter.com

Post reply on HN
Print view View original

421–430 of 472 posts

Re: Hi everyone yes, I left OpenAI yesterday

#421

Post by int_19h »

qeternity wrote: post #323
int_19h wrote: post #158

Earlier quoted context omitted.

I've heard claims like that 6 months ago. But so far nobody is even in the same ballpark. And not just freely distributed models, but proprietary ones backed by big money, as well. It really makes one wonder what kind of secret sauce OpenAI has. Surely it can't just be all that compute that Microsoft bought them, since Google could easily match that, and yet...

> But so far nobody is even in the same ballpark. Miqu is pretty good. Sure, it's a leak...but there's nothing special there. It's just a 70b llama2 finetune.

By the standards of other llama2 finetunes, sure. Compared to GPT-4, I stand by my previous assertion.

Re: Hi everyone yes, I left OpenAI yesterday

#422

Post by huytersd »

sjwhevvvvvsj wrote: post #414
huytersd wrote: post #375

Earlier quoted context omitted.

I’ll wait 5 seconds for the right code over 1 sec for bad code.

Yes but if a 7b LLM will give you the same “Hello World” as the 70b, and that’s literally all you need, using a bigger model is just burning energy for no reason at all.

The cost is fixed for me at least at this point so why would I choose the inferior version.

Re: Hi everyone yes, I left OpenAI yesterday

#423

Post by huytersd »

roody15 wrote: post #109
huytersd wrote: post #108

Earlier quoted context omitted.

But no one cares about 3.5. It’s an order of magnitude worse than 4. An order of magnitude is a lot harder to catch up with.

Yeah but for how long… at this rate I would expect some of the freely distributed models to hit gpt4 levels in as little as 3-6 months.

Order of magnitude means they’re going to take 20 times longer to get to the 4. So maybe on the order of 40-60 months from this point.

Re: Hi everyone yes, I left OpenAI yesterday

#424

Post by osigurdson »

15457345234 wrote: post #220
osigurdson wrote: post #138

Earlier quoted context omitted.

It lost a little of its cool factor. However, they provide a nearly essential service at this point. While it is easy to underestimate, I suspect this is already have a measurable impact on global GDP.

> I suspect this is already have a measurable impact on global GDP. Yeah putting people out of work on an industrial scale is probably gonna have a pretty big effect on global GDP

Is inefficiency the path to economic greatness and quality of life improvements? I suspect no.

Re: Hi everyone yes, I left OpenAI yesterday

#425

Post by chasd00 »

AnimalMuppet wrote: post #308
al_borland wrote: post #156

Earlier quoted context omitted.

I know things keep moving faster and faster, especially in this space, but GPT-4 is less than a year old. Claiming they are losing their luster, because they aren’t shaking the earth with new models every quarter, seems a little ridiculous. As the popularity has exploded, and ethical questions have become increasingly relevant, it is probably worth taking some time to nail certain aspects down before releasing everyt…

Perhaps GPT-4 is losing its luster because the more people actually use it, they go from "wow that's amazing" to "amazing, yes, but..."? And the "but" looms larger and larger with more time and more exposure? Note well: I haven't actually used it myself, so I'm speculating (guessing) rather than saying that this is how it is.

i got a feeling this is beginning to happen all over the place, I'm really curious to see where the hype train ends up at the end of this year.

Re: Hi everyone yes, I left OpenAI yesterday

#426

Post by sharadov »

johnnyanmac wrote: post #110
lyapunova wrote: post #60

Let me say, he's a great teacher! I took a CV class with him. He should teach more, and take it seriously. Being a popular AI influencer is not necessarily correlated with being a good researcher though. And I would argue there is a strong indication that it is negatively correlated with being a good business leader / founder. Here's to hoping he chills out and goes back to the sorely needed lost art of explaining co…

>He should teach more, and take it seriously. if only we compensated that knowledge properly. Youtube seems to come the closest, but Youtube educators also show how much time you have to spend attracting views instead of teaching expertise. > It makes you a target for offers and opportunities because of your name/influence, but not necessarily because of your underlying "best fit" That's unfortunately life in a nutsh…

I don't think he needs the money. I googled around and he's worth 50 million.

Re: Hi everyone yes, I left OpenAI yesterday

#427

Post by lolc »

IncreasePosts wrote: post #411
lolc wrote: post #390

Earlier quoted context omitted.

If we're going to be reductive, at least include the other main roadblock to a solution which is the current government of Israel.

That doesn't explain why deals weren't reached with the previous governments of Israel.

Sure it doesn't explain that. Would be nice if things were that easy wouldn't it?

Re: Hi everyone yes, I left OpenAI yesterday

#428

Post by sjwhevvvvvsj »

huytersd wrote: post #422
sjwhevvvvvsj wrote: post #414

Earlier quoted context omitted.

Yes but if a 7b LLM will give you the same “Hello World” as the 70b, and that’s literally all you need, using a bigger model is just burning energy for no reason at all.

The cost is fixed for me at least at this point so why would I choose the inferior version.

It’s not fixed whatsoever. Mistral 7B runs on a MacBook Air, and it’s free. Zero cost LLM, no network latency.

Re: Hi everyone yes, I left OpenAI yesterday

#429

Post by sjwhevvvvvsj »

ein0p wrote: post #417
sjwhevvvvvsj wrote: post #413

Earlier quoted context omitted.

So I think that’s a “your problem isn’t right for the tool” issue, not a “Mistral isn’t capable” issue.

It isn’t capable unless you have a very specialized task and carefully fine tune to solve just that task. GPT4 covers a lot of ground out of the box. The best model I’ve seen so far on the FOSS side, Mixtral MoE, is less capable than even GPT 3.5. I often submit my requests to both Mixtral and GPT4. If I’m problem solving (learning something, working with code, summarizing, working on my messaging) Mixtral is nearly…

Again, that’s precisely what I’m saying. A bounded task is best executed against the smallest possible model at the greatest possible speed. This is true for business factors ($$$) as well as environmental (smaller model -> less carbon).

LLM are not AGI, they are tools that have specific uses we are still discovering.

If you aren’t trying to optimize your accuracy to start with and just saying “I’ll run the most expensive thing and assume it is better” with zero evaluation you’re wasting money, time, and hurting the environment.

Also, I don’t even like running Mistral if I can avoid it - a lot of tasks can be done with a fine tune of BERT or DistilBERT. It takes more work but my custom BERT models way outperform GPT-4 on bounded tasks because I have highly curated training data.

Within specialized domains you just aren’t going to see GPT-4/5/6 performing on par with expert curated data.

Re: Hi everyone yes, I left OpenAI yesterday

#430

Post by Jensson »

kbelder wrote: post #406
altintx wrote: post #363

Earlier quoted context omitted.

NN were already a casual topic in my high school computer science class more than 20 years ago. I've always assumed they were already fairly common by that point. (~2000)

They were and they were in use, for instance in character recognition. They just hadn't had their breakout success yet.

Neural networks weren't the best models for character recognition, their breakout success was when they started being the best at recognize characters and other images which happened in the late 00's. OCR before then was really bad.

Might be hard to imagine today but back then OCR and image recognition was typically done with normal statistical regression models, and the neural networks they had then were worse than those.

Post reply on HN