Memory safety is necessary, not sufficient

If you restrict yourself to relaxed atomic loads and stores -- i.e., memory accesses that are atomic in the sense that they cannot tear, but don't have much in the way of ordering guarantees -- then you do get "one of the values that had been written to the memory in the past".

Aside from memory tearing, one other issue is that you generally want to be able to rematerialize loads (i.e., in the face of register pressure, you may want to turn one load of a value into multiple loads instead of loading it once, then spilling to the stack and reloading from the stack). But when the compiler rematerializes a load, then you don't get "one of the values written to memory". From the perspective of the original program it looks like you get some weird superposition of values.

So for example, there was recently a thread where someone had code that checked if a value was in range to safely coerce it directly to an enum then did so. But because of eager evaluation of an argument the unsafe cast happened first. From this the compiler reasoned that the variable was preconditionally range constrained to always be in range and it optimized out the in-range test (which itself was not unsafe code).

This is a classic C bug where someone implements an overflow check that itself can overflow, causing the branch for overflow to get optimized out. But at least in C the simpler syntax at least made it clear that the triggering code got executed first. The more complex rust syntax obscured that.

Rust has improved the situation by narrowing the cases where you can get into this trouble, but on the other hand it adds a lot of other complexity that contributes to faulty code (and a nearly mandatory packaging ecosystem which is a security nightmare-- it's the norm for even simple rust utilities to pull in a million lines of unauditable (just by bulk) third party code, including multiple HTTPS libraries).

As a result, I don't think it can be taken for granted that rust as a whole is an advancement in software integrity-- it may be, but it's something that ought to be formally studied. In some cases rust might be replacing memory safety bugs with an even greater number of other defects which, depending on the application, may be worse. (not everything is an internet exposed service where hacks are the only failure of consequence and where input really should be assumed to be intelligently adversarial.)

In any case, "break the rules and all bets are off" is an issue that likely will continue to exist in any performant language. Automatic code generation will generate stuff with awful performance unless an optimizer goes through and eliminates 'impossible cases', but optimization isn't possible unless the compiler can assume the rules are followed.