They used to call this a computer virus.
Trusting-Trust Attack against an Entire Linux Distribution
31–40 of 65 posts
Re: Trusting-Trust Attack against an Entire Linux Distribution
#32Earlier quoted context omitted.
> Works until AI compromises a bunch of OSes. Just write a new OS. It's a weekend project to get enough groundwork that you can bootstrap a clean system from clean source code. > And wouldn't there be difficulty comparing binaries built from significantly different environments? Not really. Starting from stage 0, compile the compiler under test (stage 1), then use the compiled compiler to compile the compiler (stage…
Not sure if you're joking. How do you write an OS without these tools that might be compromised? It's the same problem.
Break expectations. Bootstrap it through an esoteric-enough system. Write an Uxn emulator in assembly targeting the cushy environment that UEFI has and you've got a system with graphics, a text editor, a spreadsheet editor, an assembler, games, and maybe even more. I have a Z80-powered email appliance that can be loaded with programs from a connected device. Whoever is breaking my trust in trust surely won't have planned for that.
Re: Trusting-Trust Attack against an Entire Linux Distribution
#33This paper can be boiled down to: If you have malware on your CI machine it can infect the artifacts it creates. In this paper the malware was a strip trojan, but it could have been just as well a normal piece of malware which starts a service and then waits for the final artifact to be generated after which it infects it.
Rebuilding strip from clean source doesn't clear it. The copy in the bootstrap seed modifies its replacement, and the replacement carries on from there. The provenance can still look normal.
The paper over complicates what's needed to maintain persistence between the different stages. It's not like the machine is turned off between them. You can simply keep a process running the whole time.
Re: Trusting-Trust Attack against an Entire Linux Distribution
#34The solution to this in the Orange Book (TCSEC) days in the 1980's-1990's was a system fully traceable from requirements to code, proven to embed a security policy, and analyzable and buildable from source locally by the customer using existing, trusted tools. Eventually, people added hashes for the code and data. So, your program that combines source files or checks dependencies would be fully specified in its succe…
> Most software isn't built that way. It can't be because the priorities of developers and customers work against it. The most significant such priority may be the costs of paying developers and of time to delivery. The dramatic reductions in those costs due to LLMs enable us to produce much more quantity and/or quality. Many complain now about quantity, so perhaps we are finally at a stage where we don't need much m…
It's very exciting to see the work on combining AI models with static analysis, test generation, formal proof, and refactoring. All of these suggests we might see high assurance (EAL6+) developed rapidly in the future. At least for combinations of well-understood concepts.
Re: Trusting-Trust Attack against an Entire Linux Distribution
#35[dead]
Re: Trusting-Trust Attack against an Entire Linux Distribution
#36Earlier quoted context omitted.
Not sure if you're joking. How do you write an OS without these tools that might be compromised? It's the same problem.
Break expectations. Bootstrap it through an esoteric-enough system. Write an Uxn emulator in assembly targeting the cushy environment that UEFI has and you've got a system with graphics, a text editor, a spreadsheet editor, an assembler, games, and maybe even more. I have a Z80-powered email appliance that can be loaded with programs from a connected device. Whoever is breaking my trust in trust surely won't have pla…
but now they will via delegation to an automated analyst/systems programmer.
get ready.
the effort required for a complete infiltration has been lowered a great deal.
Re: Trusting-Trust Attack against an Entire Linux Distribution
#37Earlier quoted context omitted.
Nothing about this uses AI in any way. I'm so tired of people injecting it into every conversation.
Is it a stretch to imagine a compromised LLM writing compromising code? I've been operating under the assumption that LLM code might be bad and not working, but I hadn't thought about it compromising a system.
huh?
Re: Trusting-Trust Attack against an Entire Linux Distribution
#38Earlier quoted context omitted.
> Works until AI compromises a bunch of OSes. Just write a new OS. It's a weekend project to get enough groundwork that you can bootstrap a clean system from clean source code. > And wouldn't there be difficulty comparing binaries built from significantly different environments? Not really. Starting from stage 0, compile the compiler under test (stage 1), then use the compiled compiler to compile the compiler (stage…
Not sure if you're joking. How do you write an OS without these tools that might be compromised? It's the same problem.
We have tons and tons of backups of clean Linux isos, compilers, etc. The idea that we are going to lose the ability to easily have an uncompromised system is a fairy tale told by the people pushing bootstrapable builds.
Re: Trusting-Trust Attack against an Entire Linux Distribution
#39Earlier quoted context omitted.
Works until AI compromises a bunch of OSes. And wouldn't there be difficulty comparing binaries built from significantly different environments? It sounds like some progress has been made in general for fixed identical builds, but isn't that also still a hard problem? I don't know enough low level C-level stuff about binary generation.
> Works until AI compromises a bunch of OSes. Just write a new OS. It's a weekend project to get enough groundwork that you can bootstrap a clean system from clean source code. > And wouldn't there be difficulty comparing binaries built from significantly different environments? Not really. Starting from stage 0, compile the compiler under test (stage 1), then use the compiled compiler to compile the compiler (stage…
[deleted]
Re: Trusting-Trust Attack against an Entire Linux Distribution
#40The solution to this in the Orange Book (TCSEC) days in the 1980's-1990's was a system fully traceable from requirements to code, proven to embed a security policy, and analyzable and buildable from source locally by the customer using existing, trusted tools. Eventually, people added hashes for the code and data. So, your program that combines source files or checks dependencies would be fully specified in its succe…
What happens when "security" includes a time component? A shocking number of modern systems depend on time, either in enough time having elapsed to prove something about the attacker, or in little enough time elapsing as a critical component of the system in question. That feels like it escapes the bounds of your definitions and is also somewhat unavoidable. Is that rectifiable somewhere?