Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things we learned about LLMs in 2024

simonwillison.net

Post reply on HN
Print view View original

191–200 of 615 posts

Re: Things we learned about LLMs in 2024

#191

Post by dartos »

jacobgkau wrote: post #173
ffsm8 wrote: post #158

Earlier quoted context omitted.

I agree, the core aspect of a ponzi scheme is that it redistributes the newly invested funds to previous investors, making it highly profitable to anyone joining early and incentivising early joiners to get new investors. This just doesn't hold true for open ai

Doesn't it hold true for investment in AI (or potentially any other industry that experiences a boom) in general? Anyone who bought in at the ground floor is now rich. Anyone who buys in now is incentivized to try and keep getting more people to buy in so their investment will give a return regardless of if actual value is being created.

If effect, kind of.

The money being invested does not go directly to investors.

It goes to the cost of R&D, which in turn increases the value of openai shares, then the early investors can sell those shares to realize those gains.

The difference between that and a ponzi is that the investment creates value which is reflected in the share price.

No value is created in a Ponzi scheme.

The actual dollar worth of the value generated is what people speculate on.

Re: Things we learned about LLMs in 2024

#192

Post by [deleted] »

jacobgkau wrote: post #173
ffsm8 wrote: post #158

Earlier quoted context omitted.

I agree, the core aspect of a ponzi scheme is that it redistributes the newly invested funds to previous investors, making it highly profitable to anyone joining early and incentivising early joiners to get new investors. This just doesn't hold true for open ai

Doesn't it hold true for investment in AI (or potentially any other industry that experiences a boom) in general? Anyone who bought in at the ground floor is now rich. Anyone who buys in now is incentivized to try and keep getting more people to buy in so their investment will give a return regardless of if actual value is being created.

[deleted]

Re: Things we learned about LLMs in 2024

#193

Post by justincormack »

com2kid wrote: post #93
sowbug wrote: post #84

Simon has mentioned in multiple articles how cool it is to use 64GB DRAM for GPU tasks on his MacBook. I agree it's cool, but I don't understand why it is remarkable. Is Apple doing something special with DRAM that other hardware manufacturers haven't figured out? Assuming data centers are hoovering up nearly all the world's RAM manufacturing capacity, how is Apple still managing to ship machines with DRAM that perfo…

Apple uses HBM, basically RAM on the same die as the CPU. It has a lot more memory bandwidth than typically PC dram, but still less than many GPUs. (Although the highest end macs have bandwidth that is in the same ballpark as GPUs)

Its not HBM, which GPUs tend to use, but it is on package and wider interface than other PCs

Re: Things we learned about LLMs in 2024

#194

Post by datadrivenangel »

fullstackchris wrote: post #171
datadrivenangel wrote: post #137

Earlier quoted context omitted.

I don't recall editing my message, but HN can be wonky sometimes. :) Nothing is truly exponential for long, but the logistic curve could be big enough to do almost anything if you get imaginative. Without new physics, there are still some places where we can do some amazing things with the equivalent of several trillion dollars of applied R&D, which AGI gets you.

Sure but think of the Higgs... how long that took for just _one_ particle. You think an AGI, or even an ASI is going to make an experimental effort like that go any bit faster? Dream on! It astounds me that people dont realize how much of this cutting edge science stuff literally does NOT happen overnight, and not even close to that; typically it takes on the order of decades!

Science takes decades, but there are many places where we could have more amazing things if we spent 10 times as much on applied R&D and manufacturing. It wouldn't happen overnight, but it will be transformative if people can get access to much more automated R&D. We've seen a proliferation in makers over the last few decades as access to information is easier, and with better tools individuals will be able to do even more.

My point being that even if Science ends today, we still have a lot more engineering we can benefit from.

Re: Things we learned about LLMs in 2024

#195

Post by dkkergoog »

nntwozz wrote: post #65

I think John Gruber summed it up nicely: https://daringfireball.net/2024/12/openai_unimaginable OpenAI’s board now stating “We once again need to raise more capital than we’d imagined” less than three months after raising another $6.6 billion at a valuation of $157 billion sounds alarmingly like a Ponzi scheme — an argument akin to “Trust us, we can maintain our lead, and all it will take is a never-ending stream of…

[dead]

Re: Things we learned about LLMs in 2024

#196

Post by torginus »

fullstackchris wrote: post #157
torginus wrote: post #152

Earlier quoted context omitted.

How good is the best model of your choice at doing architecture work for complex and nontrivial apps? I have been bashing my head against the wall over the course of the past few days trying to create my (quite complex) dream app. Most of LLM coding I've done involved in writing code to interface with already existing libs or services and the LLMs are great at that. I'm hung up on architecture questions that are uniq…

Don't wanna be that typical hackernews guy but I couldnt resist... if your app is "quite complex" there is probably a way or ways you can break it down into much simpler parts. Easier for you AND the LLM. It always comes back to architecture and composition ;)

I don't want to be mean, but that bit of eastern wisdom you dispensed sounds incredibly like what a management consultant would say.

Re: Things we learned about LLMs in 2024

#197

Post by refulgentis »

datadrivenangel wrote: post #122
lowercased wrote: post #55

Earlier quoted context omitted.

I have a 2023 mbp, and I get about 100-150 tok/sec locally with lmstudio.

Which models?

For context, I got M2 Max MBP, 64 GB shared RAM, bought it March 2023 for $5-6K.

  Llama 3.2 1.0B - 650 t/s
  Phi 3.5   3.8B - 60 t/s.
  Llama 3.1 8.0B - 37 t/s.
  Mixtral  14.0B - 24 t/s.
Full GPU acceleration, using llama.cpp, just like LM Studio.

Re: Things we learned about LLMs in 2024

#198

Post by dartos »

> There’s a flipside to this too: a lot of better informed people have sworn off LLMs entirely because they can’t see how anyone could benefit from a tool with so many flaws. The key skill in getting the most out of LLMs is learning to work with tech that is both inherently unreliable and incredibly powerful at the same time. This is a decidedly non-obvious skill to acquire!

I wish the author qualified this more. How does one develop that skill?

What makes LLMs so powerful on a day to day basis without a large RAG system around it?

Personally, I try LLMs every now and then, but haven’t seen any indication of their usefulness for my day to day outside of being a smarter auto complete.

Re: Things we learned about LLMs in 2024

#200

Post by datadrivenangel »

fullstackchris wrote: post #174
EMIRELADERO wrote: post #149

Earlier quoted context omitted.

What do you mean by this? Is there any fundamental property of intelligence, physicality, or the universe, that you think wouldn't let this work?

Not OP but yes. Electron size vs band gap, computing costs (in terms of electricity) any other raw materials needed for that energy, etc... sigh... its physics, always physics... what fundamental property of physics do you think would let a vertical take off in intelligence occur?

If you look at the rate of mathematical operations conducted, we're already going hard vertical. Physics and material limitations will slow that eventually as we reach a marginal return on converting the planet to computer chips, but we're in the singularity as proxy measured by mathematical operations.
Post reply on HN