AI Is Breaking This Thing We Call Trust
terriblesoftware.org
AI Is Breaking This Thing We Call Trust
1–10 of 60 posts
Re: AI Is Breaking This Thing We Call Trust
#2I wonder when the people will get that intelligence is not only directed at the external, but only really starts when you look at the internal (joy, pleasure, traumas, taboos, awkwardness, abuse, trust etc.). Look up the word "interoception".
Re: AI Is Breaking This Thing We Call Trust
#3> AI Is Breaking This Thing We Call Trust
An increasingly large percentage of people being lying POS grifters is breaking trust.
AI just happens to be a tool that the laziest of that group of people leans on a lot to supercharge their grifts.
But our massive and growing trust problem predates and is way deeper than AI.
Re: AI Is Breaking This Thing We Call Trust
#4> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it.
This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some point in the near future, you will just trust that the software code works. Instead you’ll just debate the spec (the prompt). Your thinking can move one level up the system hierarchy. Everyone’s can. We’ve done it before, we’ll do it again, and this won’t even be the last time it happens. Let go of the anxiety and let it happen.
Re: AI Is Breaking This Thing We Call Trust
#5> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some po…
>This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level.
Just because something wasn't required at all levels or 100% it doesn't mean it wasn't required or it wasn't important though.
I trusted that the machine code worked because it was a mostly mechanical transformation, with little to zero non-determinism involved, and the design decisions for the high level code were still mine and understood. Not the same with an LLM, even when see just as an added level of abstraction.
Given some LLM produced code, I also trust the compiler will produce working machine code matching it. Doesn't mean I trust the LLM code in the same way.
Re: AI Is Breaking This Thing We Call Trust
#6> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some po…
You trust machine code because it was produced by a deterministic system that was written and tested thoroughly by other engineers.
AI systems are not the same. You can't guarantee deterministic output.
Also, code is still the best language we have to convey our intentions as engineers into function. Natural language is too imprecise and ambiguous.
Understanding the code allows us to understand the intention behind the code and identify bugs, plan and architect better. Inspecting prompts gives you a blinkered view of the system meaning you're more likely to make incorrect assumptions leading to serious bugs.
Re: AI Is Breaking This Thing We Call Trust
#7> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some po…
I'd argue the trust was always relative. Witness what happens when you are hired to untangle the buggy mess that was left by the previous "senior" engineer. Sure there was trust when they were in charge, but based on the code they wrote, they didn't actually understand anything they were doing.
Re: AI Is Breaking This Thing We Call Trust
#8[deleted]
Re: AI Is Breaking This Thing We Call Trust
#9> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some po…
You trust that those layers work
Unlike AI, those layers were traditionally compiled using deterministic tools. And generally came with a suite of other tools to trace and debug them. I don't trust AI in the same way at all.
Re: AI Is Breaking This Thing We Call Trust
#10> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it. This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some po…
You trust machine code because it was produced by a deterministic system that was written and tested thoroughly by other engineers. AI systems are not the same. You can't guarantee deterministic output. Also, code is still the best language we have to convey our intentions as engineers into function. Natural language is too imprecise and ambiguous. Understanding the code allows us to understand the intention behind t…
Right, it's a chain of trust between the layers. The link I choose next to me helps determine the quality of successive links, whether I inspect them or not.
When I buy a wooden shack from a carpenter, I'd never expect them to core-sample every plank to ensure it's good wood. That's because a good carpenter habitually buys wood from a good supplier, and will have the experience to detect other suspicious indicators.
Similarly, my coworker might not have verified the machine code, but I can be pretty confident they didn't vibe-install a compiler by Ken Thompson that puts backdoors into the binary.