431–440 of 1001 posts
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Joking aside, even with AI generating code, someone has to know how to talk to it, how to understand the output, and know what to do with it.
AI is also not great for novel concepts and may not fully get what's happening when a bug occurs.
Remember, it's just a tool at the end of the day.
The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…
However, journalists and regulators may not understand why superficially dangerous-looking instructions carry such negligible real world risks, because they probably haven't spent much time doing bench chemistry in a laboratory. Since real chemists don't need "explain like I'm five" instructions for syntheses, and critics might use pseudo-dangerous information against the company in the court of public opinion, refusing prompts like that guards against reputational risk while not really impairing professional users who are using it for scientific research.
That said, I have seen full strength frontier models suggest nonsense for novel syntheses of benign compounds. Professional chemists should be using an LLM as an idea generator or a way to search for publications rather than trusting whatever it spits out when it doesn't refuse a prompt.
[1] https://en.wikipedia.org/wiki/Strychnine_total_synthesis
Generating more "think out loud" tokens and hiding them from the user... Idk if I'm "feeling the AGI" if I'm being honest. Also... telling that they choose to benchmark against CodeForces rather than SWE-bench.
Why not? Isn't that basically what humans do? Sit there and think for a while before answering, going down different branches/chains of thought?
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Earlier quoted context omitted.
The open source/weights models so far have proved that openAI doesn't have some special magic sauce. I m confident we ll soon have a model from Meta or others that s close to this level of reasoning. [Also consider that some of their top researchers have departed] On a cursory look, it looks like the chain of thought is a long series of chains of thought balanced on each step, with a small backtracking added whenever…
I suspect that the largest limiting factor for a competing model will be the dataset. Unless they somehow used GPT4 to generate the dataset somehow, this is an extremely novel dataset to have to build.
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Those in the bottom (100-X)% may be better off partying it up for a few years, but then again the same can be said for other AI-affected disciplines.
Masseurs/masseuses have nothing to worry about.
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Software engineering contains a lot more than just writing code. If we somehow get AGI, it'll change everything, not just SWE. If not, my belief is that there will be a lot more demand for good SWEs to harness the power of LLMs, not less. Use them to get better at it faster.
_ 7 8 4 1 _ _ _ 9
5 _ 1 _ 2 _ 4 7 _
_ 2 9 _ 6 _ _ _ _
_ 3 _ _ _ 7 6 9 4
_ 4 5 3 _ _ 8 1 _
_ _ _ _ _ _ 3 _ _
9 _ 4 6 7 2 1 3 _
6 _ _ _ _ _ 7 _ 8
_ _ _ 8 3 1 _ _ _
The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…
One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wrongthink. This is bad, and should be resisted.
The other form of “safety” is a very prudent impulse to keep these sorts of potentially dangerous outputs out of AI models’ autoregressive thought processes. The goal is to create thinking machines that can act independently of us in a civilized way, and it is therefore a good idea to teach them that their thought process should not include, for example, “It would be a good idea to solve this problem by synthesizing a poison for administration to the source of the problem.” In order for AIs to fit into our society and behave ethically they need to know how to flag that thought as a bad idea and not act on it. This is, incidentally, exactly how human society works already. We have a ton of very cute unaligned general intelligences running around (children), and parents and society work really hard to teach them what’s right and wrong so that they can behave ethically when they’re eventually out in the world on their own.
Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
If you have better career ideas, you should not continue. The thing is it is very hard to predict how the world will change (and by how much from very little to a revolutionary change) with all these new changes. Only licensed and regulated professions (doctors/lawyers/pilots etc) might remain high earning for long (and they too are not guaranteed). It really is worth a relook on what you want to do in life while see…