31–40 of 64 posts
AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference. They should be forced to produce Lean or Rocq proofs or execution traces for all the computation they use. For facts, they should be able to produce sources. For any abstract reasoning, they should be able to break it down into explainable steps. Then, on top of that, they…
I think the differentiation going forward is going to be if you're a person that understands things or if you're a person that delegates your understanding. I don't think there's going to be a lot of economic or social value to prompting an AI to do things you don't understand. First, anyone can do it, so why should someone give you money or credibility for that? Second, as fewer and fewer people bother to learn hard…
“Money is the sign of a poor civilization” or something similar that a Culture GCU would say.
AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference. They should be forced to produce Lean or Rocq proofs or execution traces for all the computation they use. For facts, they should be able to produce sources. For any abstract reasoning, they should be able to break it down into explainable steps. Then, on top of that, they…
What worries me isn’t AI replacing experts, it’s that we may stop producing people who know enough to notice when AI is confidently wrong.
> it’s that we may stop producing people who know enough to notice when AI is confidently wrong. The running-joke is that a LinkedIn-lunatic AI booster, with a Nano Banana-generated profile-pic, will immediately slide into the chat to tell you that that this is already a solved problem: just spin up another agent to do the work to verify the first agent. Token-cost-be-damned. And we laugh and downvote them to oblivio…
And this is enforced by... another LLM? Seems like it would work alright until something is asserted implicitly and not categorized as an assertion.
I think the differentiation going forward is going to be if you're a person that understands things or if you're a person that delegates your understanding. I don't think there's going to be a lot of economic or social value to prompting an AI to do things you don't understand. First, anyone can do it, so why should someone give you money or credibility for that? Second, as fewer and fewer people bother to learn hard…
lol, there aren’t going to be people giving money to other people in the long run. “Money is the sign of a poor civilization” or something similar that a Culture GCU would say.
Also do you really think billionaires are frothing at the mouth for this because it's going to make money less important?
AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference. They should be forced to produce Lean or Rocq proofs or execution traces for all the computation they use. For facts, they should be able to produce sources. For any abstract reasoning, they should be able to break it down into explainable steps. Then, on top of that, they…
Conversely, https://www.lesswrong.com/posts/fzeoYhKoYPR3tDYFT/beware-iso...
The entire premise of building machines that can provide intelligence is that they can be arbitraryly rigorous and do far more work then is reasonable or practical to demand of a human.
"Civilization advances by extending the number of important operations which we can perform without thinking about them." - A. N. Whitehead
Sure, me and my neighbors waiting for the train don't need to build a sundial, we can tell the time "without thinking" by glancing at our ticking pocketwatches... but other people spent absolutely enormous amounts of brainpower thinking about meshing gear, friction in bearings, metal alloys that don't distort too much with temperature changes, and how to make accurate tiny screws, and...
Earlier quoted context omitted.
I think about this a lot. From the POV of a medieval peasant, we likely reached the singularity a long time ago.
You both seem to be using a different definition of "singularity" from the one I'm familiar with. I've always understood it to mean a rapid feedback loop in which AI creates successive, increasingly capable generations of AI outside human control, rather than simply a level of technological advancement that would be incomprehensible to someone
Earlier quoted context omitted.
> it’s that we may stop producing people who know enough to notice when AI is confidently wrong. The running-joke is that a LinkedIn-lunatic AI booster, with a Nano Banana-generated profile-pic, will immediately slide into the chat to tell you that that this is already a solved problem: just spin up another agent to do the work to verify the first agent. Token-cost-be-damned. And we laugh and downvote them to oblivio…
> any time any agent confidently asserts something then it has to provide associated evidence And this is enforced by... another LLM? Seems like it would work alright until something is asserted implicitly and not categorized as an assertion.
Earlier quoted context omitted.
lol, there aren’t going to be people giving money to other people in the long run. “Money is the sign of a poor civilization” or something similar that a Culture GCU would say.
Uh, unless you're living with your well-off parents in perpetuity or you already have FU money... good luck with that. I'm pretty sure my landlord isn't going to accept "money doesn't matter anymore because of AI!" Also do you really think billionaires are frothing at the mouth for this because it's going to make money less important?
As for your landlord, I doubt that will be a problem either.
You cannot make 40% - 80% of the country structurally unemployable with AI and eventually robots and keep this same economic system.
It will be weird, it’s not going to be hugely fun, but I’m betting that we either get massive deflation, huge changes in land use policy, revolution, or a combination of them all.
Personally I’m building a shack in the woods and building my own business. We will see how it turns out, but I doubt money as we know it survives this.
Edit to be clear:
I think it’s going to work out and probably be pretty cool after we deal with obvious problems that are coming up.