2018 - gpt1 2019 - gpt2 2020 - gpt3 2022 - gpt3.5 2023 - gpt4 2023 - gpt4-turbo 2024 - gpt-4o 2024 - o1 Did OpenAI hire Google's product marketing team in recent years?
They signed a cross-licensing deal with the USB Consortium.
Learning to Reason with LLMs
711–720 of 1001 posts
Re: Learning to Reason with LLMs
#712Earlier quoted context omitted.
LLMs perform well on small tasks that are well defined. This definition matches almost every task that a student will work on in school leading to an overestimation of LLM capabiity. LLMs cannot decide what to work on, or manage large bodies of work/code easily. They do not understand the risk of making a change and deploying it to production, or play nicely in autonomous settings. There is going to be a massive amou…
Careers are 30 years long Can you confidently say that an LLM won’t be better than an average 22 year old coder within these 30 years?
Huh careers are 30 years long? I don't know where you live but it's more like 45 years long where I live. The retirement age is 67.
Re: Learning to Reason with LLMs
#713First shot, I gave it a medium-difficulty math problem, something I actually wanted the answer to (derive the KL divergence between two Laplace distributions). It thought for a long time, and still got it wrong, producing a plausible but wrong answer. After some prodding, it revised itself and then got it wrong again. I still feel that I can't rely on these systems.
Look where you were 3 years ago, and where you are now. And then imagine where you will be in 5 more years. If it can almost get a complex problem right now, I'm dead sure it will get it correct within 5 years
what makes you so "dead sure"? it's just hallucinating as always
Re: Learning to Reason with LLMs
#714This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…
The screenshot [1] is not readable for me. Chrome, Android. It's so blurry that I cant recognize a single character. How do other people read it? The resolution is 84x800.
When you open on phone, switch to "desktop site" via browser three dots menu
Re: Learning to Reason with LLMs
#715"after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users"
...umm. Am I the only one who feels like this takes away much of the value proposition, and that it also runs heavily against their stated safety goals? My dream is to interact with tools like this to learn, not just to be told an answer. This just feels very dark. They're not doing much to build trust here.
Re: Learning to Reason with LLMs
#716Why so much hate? They're doing their best. This is the state of progress in the field so far. The best minds are racing to innovate. The benchmarks are impressive nonetheless. Give them a break. At the end of the day, they built the chatbot who's saving your ass each day ever since.
not saving my ass, I never needed one professionally. OpenAI is shovelling money into a furnace, I expect them to be assimilated into Microsoft soon.
Re: Learning to Reason with LLMs
#717Advanced reasoning will pave the way for recursive self-improving models & agents. These capabilities will enable data flywheels, error-correcting agentic behaviors, & self-reflection (agents understanding the implications of their actions, both individually & cooperatively).
Things will get extremely interesting and we're incredibly fortunate to be witnessing what's happening.
Re: Learning to Reason with LLMs
#718Reading through the Chain of Thought for the provided Cipher example (go to the example, click "Show Chain of Thought") is kind of crazy...it literally spells out every thinking step that someone would go through mentally in their head to figure out the cipher (even useless ones like "Hmm"!). It really seems like slowing down and writing down the logic it's using and reasoning over that makes it better at logic, simi…
Yes and apparently we won't have access to that chain of thought in the release version:
"after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users"
Re: Learning to Reason with LLMs
#719Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
While the reasoning and output of ChatGPT is impressive (and, imho, would pass almost all coding interviews), I'm primarily impressed with the logical flow, explanation and thoroughness. The actual coding and problem solving isn't complex, and that gets to your question: someone (in this case, the OP), still needed to be able to figure out how to extract useful data and construct a stimulating prompt to trigger the LLM into answering in this way. As others have posted, none of the popular LLMs behave identically, either, so becoming an exert tool-user with one doesn't necessarily translate to the next.
I would suggest the fundamentals of computer science and software engineering are still critically important ... but the development of new code, and especially the translation or debugging of existing code is where LLMs will shine.
I currently work for an SAP-to-cloud consulting firm. One of the singlemost compelling use cases for LLMs in this area is to analyze custom code (running in a client's SAP environment), and refactor it to be compatible with current versions of SAP as a cloud SaaS. This is a specialized domain but the concept applies broadly: pick some crufty codebase from somewhere, run it through an LLM, and do a lot of mostly copying & pasting of simpler, modern code into your new codebase. LLMs take a lot of the drudgery out of this, but it still requires people who know what they're looking at, and could do it manually. Think of the LLM as giving you an efficiency superpower, not replacing you.
Re: Learning to Reason with LLMs
#720I just tested o1-preview on the "How many r's are in strawberry?" question. It answers correctly!