The Registry could have been a lot better.
It has no schema. Imagine if it had some sort of schema, declaring what sub-keys and values are allowed under each key. Imagine if the schema was self-documenting, with each key/value declaration in the schema had an associated description explaining what it was for.
Imagine if it had richer data types. For example, a "link" type, in which a value actually has the name of another key. If you try to delete the target, either it doesn't let you, or it sets the value to some sort of null value. (Basically, something like foreign keys in a relational database.) And an index to quickly find "back-references" (show me everything that points to this key.)
The registry should have included a database of installed packages/applications, and every key should have been marked with what package/application owns it, along with some sort of indexing to make it quick to find everything a package/application owns. Application uninstalls would have been a lot cleaner, and issues with apps leaving behind junk in the registry avoided.
Transactions: This was added in Windows Vista. But it could have been there from the beginning.
OTOH, remembering the registry was originally implemented in Windows 3.1, which had minimum system requirements of a 286 with 1MB of memory, maybe my suggestions above just wouldn't have been feasible.