Understanding Reasoning LLMs

> it is (almost by definition) less of a problem than aggression which does cross the threshold

Unless you also take into account scale (how often the person does it or how many other people do it) and second-order effects (people who fall for the manipulation and spread it further or act on it). For this reason, I very much prefer people who insult me honestly and overtly, at least I know where I stand with them and at least other people are less likely to get influenced by them.

> I'm not sure this analogy is helpful

This is actually a very rare occasion when an analogy is helpful. As you point out, the emotional abuse can (often?) be worse. TBH when it "escalates" to being physical, it often is a good thing because it finally 1) gives the target/victim "permission" to ask for help 2) it makes it visible to casual observers, increasing the likelihood of intervention 3) it can leave physical evidence and is easily spotted by witnesses.

(I witnessed a whole bunch of bullying and attempts at bullying at school and one thing that remained constant is that people who fought back (retaliated) were left alone (eventually). It is also an age where physical violence is acceptable and serious injuries were rare (actually I don't recall a single one from fighting). This is why I always encourage people to fight back, not only is it effective but it teaches them individual agency instead of waiting for someone in a position of power to save them.)

> I can tell you why that doesn't work

I appreciate this datapoint (and the fact you are open to discussing it, unlike many mods). I agree that it's often hard to distinguish between mistake and malice. For example I reacted to the individual instance because of similar comments I ran into in the past but I didn't check if the same person is making fallacious arguments regularly or if it was a one-off.

But I also have experiences with good outcomes. One example stands out - a guy used a fallacy when arguing with me, i asked him to not do that, he did it again so i did it twice to him as well _while explaining why I am doing it_. He got angry at first, trying to call me out for doing something I told him not to do, but when I asked him to read it again and pointed out that the justification was right after my message with the fallacy (not post-hoc after being "called out"), he understood and stopped doing it himself. It was as if he wasn't really reading my messages at first but reversing the situation made him pay actual attention.

I think the key is that it was a small enough community that 1) the same people interacted with each other repeatedly and that 2) I explained the justification as part of the retaliation.

Point 1 Will never be possible at the scale of HN, though I would like to see algorithmic approaches to truth and trust instead of upvotes/downvotes which just boil down to agree/disagree. Point 2 can be applied anywhere and if mods decide to step in, it IMO is something they should take into account.

Anyway, thanks for the links, I don't have time to go through other people's arguments rn but I will save it for later as it is good to know this comes up from time to time and I am not completely crazy when I see something wrong with the standard threshold-based approach.

Oh and you didn't say it explicitly but I feel like you understand the difference between rules and right/wrong given your phrasing. That is a very nice thing to see if I am correct (though I have no doubt your phrasing was refined by years or trial and error as to what is effective). In general, I believe it should always be made clear that rules exist for practical reasons, not pretend they are some kind of codification of morality.

Well fuck... My comment was too long... and it doesn't get cached -___-

I'll come back and retype some of what I said but I need to do some other stuff right now. So I'll say that you're asking really good questions and I think you're mostly understanding things.

So give you very quick answers:

Yes, things are frozen. There's active/online learning but even that will not solve all the issues at hand.

Yes, we can put bounds. Causal models naturally do this but statistics is all about this too. Randomness is a measurement of uncertainty. Note that causal models are essentially perfect embeddings. Because if you've captured all causal relationships, you gain no more value from additional information, right?

Also note that we have to be very careful about assumptions. It is always important to uncover what assumptions have been made and what the implications are. This is useful in general problems solving and applies to anything in your life, not just AI/ML/coding. Unfortunately, assumptions are almost never explicitly stated, so you got to go hunting.

See how physics defines strong emergence and weak emergence. There are no known strongly emerging phenomena and we generally believe they do not exist. For weakly emerging, well it's rather naive to discuss this in the context of ML if we're dedicating so little time and effort to interpretation, right? That's kinda the point I was making previously about not being able to differentiate an emergent phenomena from not knowing we gave it information.

For the "getting better" it is about the spikes. See the first two figures and their captions in the response paper.

More parameters do help btw, but make sure you distinguish the difference between a problem being easier to solve and a problem not being solvable. The latter is rather hard to show. But the paper is providing strong evidence to the underlying issues being about the ease of problem solving rather than incapacity.

Proof is hard. There's nothing wrong with being empirical, but we need to understand that this is a crutch. It is evidence, not proof. We leaned on this because we needed to start somewhere. But as progress is made so too must all the metrics and evaluations. It gets exponentially harder to evaluate as progress is made.

I do not think it is best to put everyone in ML into the theory first and act like physicists. Rather we recognize the noise and do not lock out others from researching other ideas. The review process has been contaminated and we lost sight. I'd say that the problem is that we look at papers as if we are looking at products. But in reality, papers need to be designed with understanding the experimental framework. What question is being addressed, are variables being properly isolated, and do the results make a strong case for the conclusion? If we're benchmark chasing we aren't doing this and we're providing massive advantage to "gpu rich" as they can hyper-parameter tune their way to success. We're missing a lot of understanding because of this. You don't need state of the art to prove a hypothesis. Nor to make improvements on architectures or in our knowledge. Benchmarks are very lazy.

For information leakage, you can never remove the artist from the art, right? They always leave part of themselves. That's okay, but we must be aware of the fact so we can properly evaluate.

Take the passion, and dive deep. Don't worry about what others are doing, and pursue your interests. That won't make you successful in academia, but it is the necessary mindset of a researcher. Truth is no one knows where we're going and which rabbit holes are dead ends (or which look like dead ends but aren't). It is good to revisit because you table questions when learning, but then we forget to come back to them.

  > needs a well-built intuition not from a practical level, but from a theoretical level.

The magic is at the intersection. You need both and you cannot rely on only one. This is a downfall in the current ML framework and many things are black boxes only because no one has bothered to look.

When your hobbies can include things like jumping off mountains without parachutes? That's boring only for the people who secretly dream of being a spy.