Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S1: A $6 R1 competitor?

timkellogg.me

Post reply on HN
Print view View original

261–270 of 430 posts

Re: S1: A $6 R1 competitor?

#261

Post by palmotea »

101008 wrote: post #225
palmotea wrote: post #212

Earlier quoted context omitted.

> The intelligence that will be available to the average technically literate individual will be frightening. That's not the scary part. The scary part is the intelligence at scale that could be available to the average employer . Lots of us like to LARP that we're capitalists, but very few of us are. There's zero ideological or cultural framework in place to prioritize the well being of the general population over t…

I agree with you and I am scared. My problem is: if most people can't work, who is going to pay for the product/services created with IA? I get a lot of "IA will allow us to create SaaS in a weekend" and "IA will take engineers jobs", which I think they both may be true. But a lot of SaaS surive because engineers pay for them -- if engineer don't exist anymore, a lot of SaaS won't either. If you eat your potential cu…

> My problem is: if most people can't work, who is going to pay for the product/services created with IA?

A lot of those will probably go under, too. I think a lot of people are in for a rude awakening.

The only people our society and economy really values are the elite with ownership and control, and the people who get to eat and have comfort are those who provide things that are directly or indirectly valuable to that elite. AI will enable a game of musical chairs, with economic participants iteratively eliminated as the technology advances, until there are only a few left controlling vast resources and capabilities, to be harnessed for personal whims. The rest of us will be like rats in a city, scraping by on the margins, unwanted, out of sight, subsisting on scraps, perhaps subject to "pest control" regimes.

Re: S1: A $6 R1 competitor?

#262

Post by brookst »

gessha wrote: post #204
teruakohatu wrote: post #193

Earlier quoted context omitted.

> still have no real comprehensive understanding how the models work. We do understand how they work, we just have not optimised their usage. For example someone who has a good general understanding of how an ICE or EV car works. Even if the user interface is very unfamiliar, they can figure out how to drive any car within a couple of minutes. But that does not mean they can race a car, drift a car or drive a car on…

Your example is somewhat inadequate. We _fundamentally_ don’t understand how deep learning systems works in the sense that they are more or less black boxes that we train and evaluate. Innovations in ML are a whole bunch of wizards with big stacks of money changing “Hmm” to “Wait” and seeing what happens. Would a different sampler help you? I dunno, try it. Would a smaller dataset help? I dunno, try it. Would trainin…

Isn't that just scale? Even small LLMs have more parts than any car.

LLMs are more analogous to economics, psychology, politics -- it is possible there's a core science with explicability, but the systems are so complex that even defining the question is hard.

Re: S1: A $6 R1 competitor?

#263

Post by 3abiton »

ZeljkoS wrote: post #228
xg15 wrote: post #175

Earlier quoted context omitted.

I think the fact alone that distillation and quantization are techniques that can produce substantial improvements is a strong sign that we still have no real comprehensive understanding how the models work. If we had, there would be no reason to train a model with more parameters than are strictly necessary to represent the space's semantic structure. But then it should be impossible for distilled models with less p…

We have a partial understanding of why distillation works—it is explained by The Lottery Ticket Hypothesis ( https://arxiv.org/abs/1803.03635 ). But if I am understanding correctly, that doesn't mean you can train a smaller network from scratch. You need a lot of randomness in the initial large network, for some neurons to have "winning" states. Then you can distill those winning subsystems to a smaller network. Note…

So more 'mature' models might arise in the near future with less params and better benchmarks?

Re: S1: A $6 R1 competitor?

#265

Post by katzenversteher »

ascorbic wrote: post #185
mtrovo wrote: post #133

I found the discussion around inference scaling with the 'Wait' hack so surreal. The fact such an ingeniously simple method can impact performance makes me wonder how many low-hanging fruit we're still missing. So weird to think that improvements on a branch of computer science is boiling down to conjuring the right incantation words, how you even change your mindset to start thinking this way?

I've noticed that R1 says "Wait," a lot in its reasoning. I wonder if there's something inherently special in that token.

I bet a token like "sht!", "f*" or "damn!" would have the same or even stronger effect but the LLM creators would not like to have the users read them

Re: S1: A $6 R1 competitor?

#266

Post by lodovic »

katzenversteher wrote: post #265
ascorbic wrote: post #185

Earlier quoted context omitted.

I've noticed that R1 says "Wait," a lot in its reasoning. I wonder if there's something inherently special in that token.

I bet a token like "sh t!", "f* " or "damn!" would have the same or even stronger effect but the LLM creators would not like to have the users read them

I think you're onto something, however, as the training is done through on text and not actual thoughts, it may take some experimentation to find these stronger words.

Re: S1: A $6 R1 competitor?

#267

Post by Arn_Thor »

philipswood wrote: post #256
BobbyTables2 wrote: post #252

Earlier quoted context omitted.

May sound like a conspiracy theory, but NVIDIA and a whole lot of AI startups have a strong vested interest to not seek+publish such findings. If I don’t need a huge model and GPU, then AI is little more than an open source program running on an idle PC. I feel like AI was NVIDIA’s lifeboat as GPU mining waned. Don’t see anything after that in the near future.

I think NVIDIAs future is pretty bright. We're getting to the run-your-capable-LLM on-prem or at-home territory. Without DeepSeek (and hopefully its successors) I wouldn't really have a usecase for something like NVIDIAs Project Digits. https://www.nvidia.com/en-us/project-digits/

Except I can run R1 1.5b on a GPU-less and NPU-less Intel NUC from four-five years ago using half its cores and the reply speed is…functional.

As the models have gotten more efficient and distillation better the minimum viable hardware for really cooking with LLMs has gone from a 4090 to suddenly something a lot of people already probably own.

I definitely think a Digits box would be nice, but honestly I’m not sure I’ll need one.

Re: S1: A $6 R1 competitor?

#268

Post by lodovic »

immibis wrote: post #229
101008 wrote: post #225

Earlier quoted context omitted.

I agree with you and I am scared. My problem is: if most people can't work, who is going to pay for the product/services created with IA? I get a lot of "IA will allow us to create SaaS in a weekend" and "IA will take engineers jobs", which I think they both may be true. But a lot of SaaS surive because engineers pay for them -- if engineer don't exist anymore, a lot of SaaS won't either. If you eat your potential cu…

Those people will simply be surplus to requirements. They'll be left alone as long as they don't get in the way of the ruling class, and disposed of if they do. As usual in history.

That's a fallacy. You can't have an advanced economy with most people sitting on the side. Money needs to keep flowing. If all that remains of the economy consists of a few datacenters talking to each other, how can the ruling class profit off that?

Re: S1: A $6 R1 competitor?

#269

Post by umeshunni »

MR4D wrote: post #259
xg15 wrote: post #175

Earlier quoted context omitted.

I think the fact alone that distillation and quantization are techniques that can produce substantial improvements is a strong sign that we still have no real comprehensive understanding how the models work. If we had, there would be no reason to train a model with more parameters than are strictly necessary to represent the space's semantic structure. But then it should be impossible for distilled models with less p…

I like the analogy of compression, in that a distilled model of an LLM is like a JPEG of a photo. Pretty good, maybe very good, but still lossy. The question I hear you raising seems to be along the lines of, can we use a new compression method to get better resolution (reproducibility of the original) in a much smaller size.

> in that a distilled model of an LLM is like a JPEG of a photo

That's an interesting analogy, because I've always thought of the hidden states (and weights and biases) of an LLMs as a compressed version of the training data.

Re: S1: A $6 R1 competitor?

#270

Post by gorgoiler »

This feels just like telling a constraint satisfaction engine to backtrack and find a more optimal route through the graph. We saw this 25 years ago with engines like PROVERB doing directed backtracking, and with adversarial planning when automating competitive games.

Why would you control the inference at the token level? Wouldn’t the more obvious (and technically superior) place to control repeat analysis of the optimal path through the search space be in the inference engine itself?

Doing it by saying “Wait” feels like fixing dad’s laptop over a phone call. You’ll get there, but driving over and getting hands on is a more effective solution. Realistically, I know that getting “hands on” with the underlying inference architecture is way beyond my own technical ability. Maybe it’s not even feasible, like trying to fix a cold with brain surgery?

Post reply on HN