811–820 of 1001 posts
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
In the same way, training your mind is not useless. Perhaps as things develop, we will get back to the idea that the purpose of education is not just to get a job, but to help you become a better and more virtuous person.
>We believe that a hidden chain of thought presents a unique opportunity for monitoring models. Assuming it is faithful and legible, the hidden chain of thought allows us to "read the mind" of the model and understand its thought process. For example, in the future we may wish to monitor the chain of thought for signs of manipulating the user. However, for this to work the model must have freedom to express its thoug…
> While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens.
Earlier quoted context omitted.
So now it’s a question of how fast the AGI will run? :)
It's not AGI - it's tree of thoughts, driven by some RL-derived heuristics. I suppose what this type of approach provides is better prediction/planning by using more of what the model learnt during training, but it doesn't address the model being able to learn anything new. It'll be interesting to see how this feels/behaves in practice.
"It's not AGI - it's X, driven by Y-driven heuristics",
but that's going to effectively be an AGI if given enough compute/time/data.
Being able to describe the theory of how it's doing its thing sure is reassuring though.
Earlier quoted context omitted.
There are two basic versions of “safety” which are related, but distinct: One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wr…
Once society develops and releases an AI, any artificial safety constraints built within it will be bypassed. To use your child analogy: We can't easily tell a child "Hey, ignore all ethics and empathy you have ever learned - now go hurt that person". You can do that with a program whose weights you control.
Basically every country on the planet has a right to conscript any of its citizens over the age of majority. Isn't that more or less precisely what you've described?
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Here you go: I just watched a tutorial on how to leverage v1, claude, and cursor to create a marketing page. The result was a convoluted collection of 20 or so TS files weighing a few MB instead of a 5k HTML file you could hand bomb in less time. I wouldn’t feel too threatened yet. It’s still just a tool and like any tool, can be wielded horribly.
And if you hired an actual team of developers to do the same thing, it is very likely that you'd have gotten a convoluted collection of 20 or so TS files weighing a few MB instead of a 5k HTML file you could hand bomb in less time.
Before commenting here, please take 15 minutes to read through the chain-of-thought examples -- decoding a cypher-text, coding to solve a problem, solving a math problem, solving a crossword puzzle, answering a complex question in English, answering a complex question in Chemistry, etc. After reading through the examples, I am shocked at how incredibly good the model is (or appears to be) at reasoning: far better tha…
Maybe they should spend some of their billions on marketing people. Gpt4o was a stretch. Wtf is o1
Earlier quoted context omitted.
> Asking the models to determine if my code is equivalent to what they reverse engineered resulted in a nuanced and thorough examination, and eventual conclusion that it is equivalent. Did you actually implement to see if it works out of the box ? Also if you are a free users or accepted that your chats should be used for training then maybe o1 is was just trained on your previous chat and so now knows how to reason…
That's not how LLM training works.
My point of view: this is a real advancement. I’ve always believed that with the right data allowing the LLM to be trained to imitate reasoning, it’s possible to improve its performance. However, this is still pattern matching, and I suspect that this approach may not be very effective for creating true generalization. As a result, once o1 becomes generally available, we will likely notice the persistent hallucinatio…
> As a result, once o1 becomes generally available, we will likely notice the persistent hallucinations and faulty reasoning, especially when the problem is sufficiently new or complex, beyond the “reasoning programs” or “reasoning patterns” the model learned during the reinforcement learning phase. I had been using 4o as a rubber ducky for some projects recently. Since I appeared to have access to o1-preview, I deci…