241–248 of 248 posts
Earlier quoted context omitted.
Do they do the same when using the model through API in something like Opencode?
Yes, they do. They give you just a token which is exchanged for the raw text only on the server side
Earlier quoted context omitted.
I think you’re overestimating the market for such models. Most people don’t want a model that’s prone to generating extremely offensive output. If you want something “uncensored”, then open source models already exist, as you say. But the model itself has already been extensively tuned to produce desired outputs and not produce undesired outputs, so it doesn’t really make sense to distinguish “uncensored” raw models…
There’s a big difference between uncensored models and those that are specifically set up to only output things a panel of wealthy ivory tower Bay Area progressives would deem to be the “correct” or “inoffensive” take on a given topic.
If you're genuinely worried about 'censorship' in this context, look first at how US AI companies are working with oppressive regimes around the world (e.g. https://sherwood.news/tech/report-openai-may-tailor-a-versio...)
I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…
What I find sad is how much Anthropic goes to hide your data, yet they are happy to slurp up all yours and most of you are happy to hand it over. ... then they turn around and compete with you by building your products that eat into your market. Anthropic believes their reasoning tokens is a moat and that it's giving other labs an edge and that's why they are hiding it. If they really believe that is their edge, then…
I don't think people are happy to give it over, gullible and naive maybe?
Or tricked and bamboozled via dark patterns into giving it over.
Although it's a no no to anthropomorphize on HN, it's worth noting that some folks think humans are post-hoc rationalizers as well: https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r... https://www.researchgate.net/publication/316045349_Post_Hoc_...
There's enough behavioural research to show that it is the case. For ex:
https://www.cambridge.org/core/journals/behavioral-and-brain...
Earlier quoted context omitted.
I'm convinced this "signal" has already been hijacked. Maybe a Baader-Meinhof phenomenon, but I've noticed more and more egregious spelling errors that make little sense from a human perspective. Hop into whatever chatbot you'd like and ask it to "write a paragraph with subtle misspellings on long but common words", and you'll notice misspellings that just feel wrong, because they don't map to a clear misunderstandin…
About a month ago, I noticed that Claude decided I wanted my responses in UK English, not American. It couldn't explain why, but offered to note that in its directions. (Great, process tokens constantly to do what should be configurable from a dialog dropdown).
tbh the summarized thinking with encrypted raw thinking is there for many purposes; it is there to: 1. make distillation much harder 2. safety: prevent modifications to the thinking leading to injection attacks. 3. also honestly sometimes the model raw thoughts can be deranged and is not a good user experience (consider the varied audience in the market, etc.) also often the mass underestimate/the model makers over-e…
Although it's a no no to anthropomorphize on HN, it's worth noting that some folks think humans are post-hoc rationalizers as well: https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r... https://www.researchgate.net/publication/316045349_Post_Hoc_...
>Although it's a no no to anthropomorphize on HN, it's worth noting that some folks think humans are post-hoc rationalizers as well: There's enough behavioural research to show that it is the case. For ex: https://www.cambridge.org/core/journals/behavioral-and-brain...