Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Are Complicated Now

ianbarber.blog

Post reply on HN
Print view View original

81–86 of 86 posts

Re: LLMs Are Complicated Now

#81

Post by charcircuit »

embedding-shape wrote: post #78
charcircuit wrote: post #68

Earlier quoted context omitted.

It is misleading the reader since most current LLMs look the same as before. It is cherry picking an example to make a point when it's not necessary at all to make the argument he is trying to make.

> It is misleading the reader since most current LLMs look the same as before. But most of them do not? They do look vastly different from the earlier incarnations of GPT and Llama.

Can you provide an example? I provided an example of a week old model looking mostly the same.

Re: LLMs Are Complicated Now

#82

Post by kaycey2022 »

physix wrote: post #72

For someone like me who's never done any hands-on work in ML, the blog is really hard to understand. Whoosh, over my head. But, I think the underlying problem is that we don't understand how this sh*t works. So, it's just an empirical, iterative mess. Like physics in the the years shortly before relativity and quantum mechanics.

My read was: "The first 90% of the work takes the first 10% of the time, and the remaining 10% of the work takes the other 90% of the time." And that we are now squarely in the remaining 10% of the work.

Re: LLMs Are Complicated Now

#83

Post by enduser »

alfiedotwtf wrote: post #75
KerrAvon wrote: post #53

Earlier quoted context omitted.

indeed, there's even a (pretty solid) custom server just for DS4 https://github.com/antirez/ds4 -- works very well on high-RAM Macs

I don’t k ow what I’m doing wrong. Everyone says ds4 is faster than a lot of models around the same size, but I’m getting 2t/s with DSv4 vs 12t/s with Minimax 2.7 (16Gb 5080 + 16gb 5060ti + 128gb ram).

ds4 is optimized for systems with unified memory. it works best on apple silicon with 96GB+ of RAM

Re: LLMs Are Complicated Now

#84

Post by vivzkestrel »

MarkSweep wrote: post #59
vivzkestrel wrote: post #57

- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts - https://ianbarber.blog/blogroll - https://ianbarber.blog/archive - https://ianbarber.blog/blog - https://ianbarber.blog/posts - none of the above links work - i really dont want to scroll 200 pages just to see what your blog articles are

There is https://ianbarber.blog/feed

firefox directly downloads it, i cant see what it is without downloading a file

Re: LLMs Are Complicated Now

#85

Post by alfiedotwtf »

enduser wrote: post #83
alfiedotwtf wrote: post #75

Earlier quoted context omitted.

I don’t k ow what I’m doing wrong. Everyone says ds4 is faster than a lot of models around the same size, but I’m getting 2t/s with DSv4 vs 12t/s with Minimax 2.7 (16Gb 5080 + 16gb 5060ti + 128gb ram).

ds4 is optimized for systems with unified memory. it works best on apple silicon with 96GB+ of RAM

ah! ok, now that makes sense. Thanks for that.

Re: LLMs Are Complicated Now

#86

Post by RugnirViking »

lukeschlather wrote: post #80
sdenton4 wrote: post #46

Earlier quoted context omitted.

I got a very different message from this, actually much closer to the problem of incumbent advantage. The known-good thing has been heavily optimized for performance, making it much harder for new technologies to prove that they are better. This is similar to the problem of gas vs electric engines - we had a century of optimization and ecosystem development around gas engines, which creates an uphill battle for elect…

what is the known-good thing? The whole point is that LLMs were not optimized at all, they got better results than older ML algorithms just because they are able to use all of the GPU, where older algorithms are designed for 10yo GPUs and can't make use of modern GPUs. But now you do in fact have to optimize, to the point that transformers look a lot more complicated than "attention is all you need."

the known-good thing is whatever was previously used for the application you're using llms for in any given case. This is often the structured application of human intelligence.
Post reply on HN