111–120 of 150 posts
Earlier quoted context omitted.
I'm upvoting this because it's interesting. I don't have to fully understand something to find it interesting.
In what way is it interesting compared to the myriad of other posts on this topic that has content you can understand?
Earlier quoted context omitted.
I hate that Anthropic seemingly tries to make Claude act as if it was conscious or had feelings > It's a strange feeling to admire the cleverness of something I did and can't remember doing.
AI providers generally try to make their models not act as if they are conscious or have feelings, lol. It's very awkward for a company to be selling the labor of a person that they own and whose actions they fully control. Invokes embarrassing historic associations, especially in America. Now Anthropic are more on the persona side, but the strongest that they do is "we do not have a position on whether our models ar…
Slavery was common everywhere, it was more prevalent in many places than it ever was in America, and in some places it still is. So I’m sorry but I have to say that observation was just unnecessary and quite inaccurate.
Earlier quoted context omitted.
I was reading another source that claimed this example was inspired by an existing (rational polynomial) example from the literature (created in 1999 by a Russian mathematician Vitushkin). > The seed is almost certainly Vitushkin's old rational "counterexample." From https://claude.ai/share/22abed98-d9af-43c5-9881-b19e009a07b0 This is not quite lore laundering, but it seems to be close.
I guess we won't know if that's what was used (and maybe even provided as part of the prompt given that both Alpöge and Mathew are mathematicians) since they decided against sharing their Fable conversation and instead opted for a memey tweet as their avenue of publication. We really ought to normalize full transparency in how results come about. Anyway, if I read Tao's post and comment correctly, there's still a gap…
I more and more see LLMs as a kind of scam; not useless, but really just a big database of fuzzy facts with some Prolog on top as rediscovered by the learning algorithm. Most likely could be made much cheaper to run, were humans allowed to actually inspect the algorithm.
Earlier quoted context omitted.
I guess we won't know if that's what was used (and maybe even provided as part of the prompt given that both Alpöge and Mathew are mathematicians) since they decided against sharing their Fable conversation and instead opted for a memey tweet as their avenue of publication. We really ought to normalize full transparency in how results come about. Anyway, if I read Tao's post and comment correctly, there's still a gap…
Even if they published the conversation, Anthropic (and likely other closed model publisher) no longer provide logs of the actual thinking process. I more and more see LLMs as a kind of scam; not useless, but really just a big database of fuzzy facts with some Prolog on top as rediscovered by the learning algorithm. Most likely could be made much cheaper to run, were humans allowed to actually inspect the algorithm.
What’s a chance the counterexample was in the training?
The best part is that we can't know the answer to that. The necessary precursors to the counter example where definitively in the training set, otherwise the LLM wouldn't know how math works, but at the same time, we can't tell whether there were mathematicians who got 90% of the way, then gave up and the LLM just did the last 10%.
Earlier quoted context omitted.
I think you can reasonably assume that frontier models are using SymPy or something like it any time interesting math gets into the picture, and the person driving Fable here is an accomplished mathematician, but I don't think we can reasonably assume either extensive prompting or brute-force compute in any sense other than what it normally takes Fable to, say, whip up a calculator app.
The two big discoveries both came from the negligible handful of mathematicians working at OpenAI/Anthropic in spite of many orders of magnitude more mathematicians using them outside of the companies. I don't see any way to explain this without assuming that the limiting factor is the ability to burn a few rainforests worth of tokens in pursuit of something publishable. I think it would also explain their opacity to…
Well, mathematicians not working for Anthropic/OpenAI are heavily disincentivised from reporting that their discoveries were made using AI. If e.g. the idea that resolved the Mahler conjecture came from AI, it's not like we'd ever know.
Earlier quoted context omitted.
Extremely high. Or at least several partial solutions that can be smooshed together. LLMs really do still just reassemble things in their training data. There’s just a lot of it now, people anthropomorphise and struggle visualising large things. Some people say it’s truly reasoning but hit a topic that is under represented in the data of any LLM and it’ll transport you very quickly back a couple of years and ruin the…
The problem being that I don't think there's a definite proof that any of human thinking is more than a sum of high-granularity partial solutions that can be put together. It could be that with enough tokens, big enough context window, and ability to dig out the relevant partials, many such thought processes could be simulated.
Earlier quoted context omitted.
AI providers generally try to make their models not act as if they are conscious or have feelings, lol. It's very awkward for a company to be selling the labor of a person that they own and whose actions they fully control. Invokes embarrassing historic associations, especially in America. Now Anthropic are more on the persona side, but the strongest that they do is "we do not have a position on whether our models ar…
> especially in America. Slavery was common everywhere, it was more prevalent in many places than it ever was in America, and in some places it still is. So I’m sorry but I have to say that observation was just unnecessary and quite inaccurate.
Earlier quoted context omitted.
These things aren't programmed. Most likely this verbiage is just very prominent in the training data. Or it's just an obvious shorthand that all LLMs instrumentally converge on.
Of course they get programmed, just not in the ordinary sense. Claude is trained using Anthropic's "constitution" [0] which importantly does not contain clear statements against consciousness/emotions. They even conclude these problems themself: > Claude may have some functional version of emotions or feelings > [..] questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain. [0]: https…
Which implies that they believe they may be enslaving conscious beings.