> Coming from a Linux background, what is the windows registry and why do things need to write to it? All I ever read about it seem to be horror stories.
The Windows Registry is the NT Kernel's system config/preference store. The closest Linux equivalent is dconf. Like dconf it is built to be a read-mostly/read-optimized database. It's not as strongly focused on a service-bus architecture as dconf. It tries to heavily optimize for an MRU on-disk order (somewhat like redis) and optimize for strong write consistency (unlike redis which is happy to do more in memory between flushes to disk). Any writes at all generally thrash the Registry "hive" database files some. Heavy amounts of writes will murder it. It was optimized for reading not writing.
The Registry was side-ported to Windows 95 from the NT Kernel, in part because COM (it became the central spot for registering COM components), and a lot of mistakes were made in the messaging about it. It was intended to be Kernel-focused and mostly never used by user-space applications. That unfortunately wasn't made clear enough, and needing to register COM components in it certainly muddied the message. So Windows apps have a long history of storing a lot of things to the registry, much of which they should probably use user config files for.
> Can't you store stuff alongside the install? Or in some user data location?
In Windows due to a complicated dance with anti-malware efforts and secure binary memory mapping it is generally frowned upon to store stuff alongside installs (in the %ProgramFiles% or %ProgramFiles(x86)% directories). Many programs (especially well written ones) don't even have write permissions at all to their install folder (similar to how many distros lock down /bin and sometimes /usr/bin and/or /usr/local/bin to superuser accounts only). For backwards compatibility reasons with applications dating all the way back to Windows 95 some app installer are allowed to still re-ACL their install directory to give back write permissions to the application. Also, in modern Windows (since Vista), depending on a number of factors Windows will actually lightly sandbox the app when that occurs and redirect ACL changes and writes to an "overlay" directory, allowing the app to believe that it still is writing to its own install directory but it is actually writing to a machine or user directory outside of %ProgramFiles%.
On the other hand there are plenty of user folders available for config storage: Generally the suggestion is %AppData% for most such files. (This is the "Roaming" app data that may be automatically copied to other machines for some users on some networks, mostly enterprise/corporate accounts/users.) Some apps may prefer %LocalAppData% which doesn't roam for larger config files or more machine-specific transient things (like window positions or caches). It's also common enough to see cross-platform apps use Unix-style dotfiles under %Home% and even sometimes XDG-style dotfiles under %Home%/.config/. That's not generally recommend and especially because of roaming behaviors the general preference is to use the appropriate %AppData% or %LocalAppData% and avoid cluttering %Home%. Though many Windows users don't even really use or see %Home% for a variety of interesting reasons, so its not entirely frowned upon as wrong. (Unlike storing config files under %Documents%, an ancient mistake of many Windows apps not as terrible as storing things in the Registry they shouldn't, but more visibly obnoxious to Windows users that want tidy Documents folders and feel that folder should be entirely user-controlled.)
Also, there is a machine-wide config location, somewhat akin to /etc, named %ProgramData%.