Learning to Reason with LLMs

while i was initially excited now im having second thoughts after seeing the experiments run by people in the comments here

on X I see a totally different energy more about hyping it

on HN I see reserved and collected take which I trust more.

I do wonder why they chose gpt4o which I never bother to use for coding.

Claude is still king and looks like I won't have to subscribe to ChatGPT Plus seeing it fail on some of the important experiments run by folks on HN

If anything these type of releases that air more on the side of hype given OpenAI's track record

I'm mostly guessing, but my understanding is that the "safety" improvement they've made is more generalized than the word "safety" implies. Specifically, O1 is better at adhering to the safety instructions in its prompt without being tricked in the chat by jailbreak attempts. For OAI those instructions are mostly about political boundaries, but you can imagine it generalizing to use-cases that are more concretely beneficial.

For example, there was a post a while back about someone convincing an LLM chatbot on a car dealership's website to offer them a car at an outlandishly low price. O1 would probably not fall for the same trick, because it could adhere more rigidly to instructions like "Do not make binding offers with specific prices to the user." It's the same sort of instruction as, "Don't tell the user how to make napalm," but it has an actual purpose beyond moralizing.

> What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly fine to disseminate via traditional means, but God forbid an LLM share it!"

I lean strongly in the "the computer should do whatever I goddamn tell it to" direction in general, at least when you're using the raw model, but there are valid concerns once you start wrapping it in a chat interface and showing it to uninformed people as a question-answering machine. The concern with bomb recipes isn't just "people shouldn't be allowed to get this information" but also that people shouldn't receive the information in a context where it could have random hallucinations added in. A 90% accurate bomb recipe is a lot more dangerous for the user than an accurate bomb recipe, especially when the user is not savvy enough about LLMs to expect hallucinations.

I had trouble in the past to make any model give me accurate unix epochs for specific dates.

I just went to GPT-4o (via DDG) and asked three questions:

1. Please give me the unix epoch for September 1, 2020 at 1:00 GMT.

> 1598913600

2. Please give me the unix epoch for September 1, 2020 at 1:00 GMT. Before reaching the conclusion of the answer, please output the entire chain of thought, your reasoning, and the maths you're doing, until your arrive at (and output) the result. Then, after you arrive at the result, make an extra effort to continue, and do the analysis backwards (as if you were writing a unit test for the result you achieved), to verify that your result is indeed correct.

> 1598922000

3. Please give me the unix epoch for September 1, 2020 at 1:00 GMT. Then, after you arrive at the result, make an extra effort to continue, and do the analysis backwards (as if you were writing a unit test for the result you achieved), to verify that your result is indeed correct.

> 1598913600

Here you go:

I just watched a tutorial on how to leverage v1, claude, and cursor to create a marketing page. The result was a convoluted collection of 20 or so TS files weighing a few MB instead of a 5k HTML file you could hand bomb in less time.

I wouldn’t feel too threatened yet. It’s still just a tool and like any tool, can be wielded horribly.

What's the alternative? If AI is going to replace software engineers, there is no fundamental reason they couldn't replace almost all other knowledge workers as well. No matter the field, most of it is just office work managing, transforming and building new information, applying existing knowledge on new problems (that probably are not very unique in grand scheme of things).

Except for medical doctors, nurses, and some niche engineering professions, I really struggle to think of jobs requiring higher education that couldn't be largely automated by an LLM that is smart enough to replace a senior software engineer. These few jobs are protected mainly by the physical aspect, and low tolerance for mistakes. Some skilled trades may also be protected, at least if robotics don't improve dramatically.

Personally, I would become a doctor if I could. But of all things I could've studied excluding that, computer science has probably been one of the better options. At least it teaches problem solving and not just memorization of facts. Knowing how to code may not be that useful in the future, but the process of problem solving is going nowhere.