181–190 of 323 posts
"chain of thought" is just search, right? Wouldn't it make sense to tailor the search with heuristics relevant to the problem at hand?
No, it's not search. It's feeding the model's output back into itself.
Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
There is a weird intensity to the way they're hiding these chain of thought outputs though. I mean, to date I've not seen anything but carefully curated examples of it, and even those are rare (or rather there's only 1 that I'm aware of).
So we're at the stage where:
- You're paying for those intermediate tokens
- According to OpenAI they provide invaluable insight in how the model performs
- You're not going to be able to see them (ever?).
- Those thoughts can (apparently) not be constrained for 'compliance' (which could be anything from preventing harm to avoiding blatant racism to protecting OpenAI's bottom line)
- This is all based on hearsay from the people who did see those outputs and then hid it from everyone else.
You've got to be at least curious at this point, surely?
The whole competitive advantage from any company that sells a ML model through an API is that you can’t see how the sausage is made (you can’t see the model weights). In a way, with o1, openai is just extending “the model” to one meta level higher. I totally see why they don’t want to give this away — it’d be like if any other proprietary API gave you the debugging output to their codes you could easily reverse engin…
Earlier quoted context omitted.
Again, you're stretching definitions into meaninglessness. The way you are using "sampling" and "distribution" here applies to any system processing any information. Yes, humans as well. I can trivially define the entirety of all nerve impulses reaching and exiting your brain as a "distribution" in your usage of the term. And then all possible actions and experiences are just "sampling" that "distribution" as well. B…
No, causation isnt distribution sampling. And there's a difference between, say, an extrinsic description of a system and it's essential properties. Eg., you can describe a coin flip as a sampling from the space, {H,T} -- but insofar as we're talking about an actual coin, there's a causal mechanism -- and this description fails (eg., one can design a coin flipper to deterministically flip to heads). In the case of a…
There is nowhere that an LLM stores all possible outputs. Causality can trivially be represented by sampling by including the ordering of events, which you also implicitly did for LLMs. The coin is an arbitrary distinction, you are never just modeling a coin, just as an LLM is never just modeling a word. You are also modeling an environment, and that model would capture whatever you used to influence the coin toss.
You are fundamentally misunderstanding probability and randomness, and then using that misunderstanding to arbitrarily imply simplicity in the system you want to diminish, while failing to apply the same reasoning to any other.
If you are indeed an AI researcher, which I highly doubt without you providing actual credentials, then you would know that you are being imprecise and using that imprecision to sneak in unfounded assumptions.
Im pretty sure its just 4.0 but it re-prompts itself a few times before answering. It costs a lot more
Meanwhile folks have already found successful jailbreaks to expose the chain of thought / internal reasoning tokens.
Meanwhile folks have already found successful jailbreaks to expose the chain of thought / internal reasoning tokens.
CoT again is result of computing probabilities on tokens which happen to be reasoning steps. So those are subject to the same limitations as LLMs themselves. And OpenAI knows this because exactly CoT output is the dataset that's needed to train another model. The general euphoria around this advancement is misplaced.
Im pretty sure its just 4.0 but it re-prompts itself a few times before answering. It costs a lot more