I think calling it WSL2 was a mistake. The predecessor, WSL, "just worked" and it was more a less a linux experience for most practical purposes-- and certainly better than hoary old cygwin. This caused a lot of people to believe they could just transition to WSL2, lead on by the promise of an even more performant linux experience. The documentation didn't say anything about complications from attempting this, so a l…
WSL2 corrupting Git repositories and shell history
271–280 of 423 posts
Re: WSL2 corrupting Git repositories and shell history
#272Earlier quoted context omitted.
Oh you need to fucking disable Window's Defender though, or add exclusions for your Linux distro/WSL2 folder, because when installing "node_modules" it will attempt to do realtime scans for threat-protection which absolutely cripples the speed. https://www.cicoria.com/improving-windows-subsystem-for-linu... https://www.reddit.com/r/bashonubuntuonwindows/comments/eok7... "I noticed a significant performance difference…
This is nothing specific to WSL though, Windows has always had this problem. I expect there are many Windows developers who have no idea how much time they spend waiting for their code to be virus scanned every day.
Re: WSL2 corrupting Git repositories and shell history
#273Earlier quoted context omitted.
File system corruption is unforgivable mistake, but FS work is really hard to get right. Even stuff like VMware corrupts shared folders ... wherever they try to bypass a driver translation layer. Just Google for “vmware shared folder corruption”
Is this file system corruption? The article isn’t 100% clear it ‘only’ loses some data writes, but it also doesn’t explicitly say files not being written are affected, directory structures are corrupted, etc. So, to me, it looks like WSL2 not completely flushing writes to the underlying file system. Bad, but not as bad as file system corruption (which could lead to losing all data on the disk)
The first instance of a problem is:
[ 1.956835] JBD2: Invalid checksum recovering block 97441 in log
And that's corruption that leads to log replay failing, i.e. rejecting it because honoring the replay in the face of checksum errors could make things much worse. Subsequently mount fails: [ 21.151232] ERROR: MountExt4:1659: mount(/dev/sdb) failed 5
That's good because the purpose of journal replay is to make the file system consistent following a crash/power fail. And if the file system is dirty, replay is called for, but can't happen due to a corrupt journal, so now an fsck is required. i.e. it is in an inconsistent (you could say partly broken) state and needs repair.I haven't seen syslog/systemd journal for other cases to know if there's instances of ext4 log replay that succeeds, but with missing files. That's not file system corruption, even if it leads to an inconsistent state in a git repository (or even a database). But this is still concerning, because to get a situation where log replay is clean but files are missing suggests an entire transaction was just dropped. It never made it to stable media, and even the metadata was not partially written to the ext4 journal.
qemu-kvm has a (host) cache setting called "unsafe". Default is typically "none" or "write back". The unsafe mode can result in file system corruption if the host crashes or has a power failure. The guest's IO is faster with this mode, but the write ordering expected by the file system is not guaranteed if the host crashes. i.e. writes can hit stable media out of order. If the guest crashes, my experience has been that things are fine - subsequent log replay (in the guest) is successful, because the guest writes that made it to the host cache do make it to stable media by the same the guest reboots. The out of order writes don't matter... unless the host crashes, and then it's a big problem. The other qemu cache modes have rather different flush/fua policies that can still keep a guest file system consistent following a host crash. But they are slower.
So it makes me suspicious that for performance reasons, WSL2 might be using a possibly volatile host side caching policy. Merely for additional data point, it might be interesting to try to reproduce this problem using e.g. Btrfs for the guest file system. If write order is honored and flushed to stable media appropriate for default out of the box configuration of a VM, I'd expect Btrfs never complains, but might drop up to 30s of writes. But if there's out of order writes making it to stable media, Btrfs will also complain, I'd expect transid errors which are also a hallmark of drive firmware not consistently honoring flush/fua and then you get a badly timed crash. (And similar for ZFS for that matter - nothing is impervious to having its write order expectations blown up.)
Re: WSL2 corrupting Git repositories and shell history
#274Earlier quoted context omitted.
File system corruption is unforgivable mistake, but FS work is really hard to get right. Even stuff like VMware corrupts shared folders ... wherever they try to bypass a driver translation layer. Just Google for “vmware shared folder corruption”
Reminds me of an infuriating old bug in VirtualBox where it wouldn't notice the size of a file had changed. Devs were working in some Windows editor then using Git within VirtualBox to commit changes, resulting in inexplicable trailing nulls and garbage turning up in the repo Took quite some time to figure out what was causing it. "Everyone is somehow corrupting files except for me, wtf?" Magic filesystem translation…
I can't recall what made me start doing that, but now it's a habit.
Re: WSL2 corrupting Git repositories and shell history
#275WSL2 was promised as a good working extension for a Linux dev environment on windows by my peers. Sadly, a whole host of issues specific to my dev environment meant that it was useless as I’d spend more time fixing it than getting any valuable use out of it. More importantly, this became the very reason I switched to arch full time, and haven’t looked back since. I still hope that it becomes what it was promised to b…
Sounds like last time I tried Linux on the desktop.
Re: WSL2 corrupting Git repositories and shell history
#276> Feel like booting Linux on a separate disk because of these issues. It always seemed strange to me that people would rather use WSL than the real thing when Windows doesn't bring much advantage. What am I missing ?
Linux support for touchscreen is measly, and it's a really nice feature of many Windows laptops. IMO Windows GUI has the best out-of-box features, OSX second and Linux third. Now, if you like working from the terminal, that order is reversed.
Re: WSL2 corrupting Git repositories and shell history
#277WSL2 was promised as a good working extension for a Linux dev environment on windows by my peers. Sadly, a whole host of issues specific to my dev environment meant that it was useless as I’d spend more time fixing it than getting any valuable use out of it. More importantly, this became the very reason I switched to arch full time, and haven’t looked back since. I still hope that it becomes what it was promised to b…
Re: WSL2 corrupting Git repositories and shell history
#278WSL2 was promised as a good working extension for a Linux dev environment on windows by my peers. Sadly, a whole host of issues specific to my dev environment meant that it was useless as I’d spend more time fixing it than getting any valuable use out of it. More importantly, this became the very reason I switched to arch full time, and haven’t looked back since. I still hope that it becomes what it was promised to b…
Re: WSL2 corrupting Git repositories and shell history
#279Re: WSL2 corrupting Git repositories and shell history
#280The biggest issue across all these things is filesystems. There is simply no good way to share a filesystem between Windows and Linux that is both 1. fast 2. fully correct - i.e doesn't break some code.
SMB/NFS just never work super well because they are slow, or programs think they are local filesystems so things break (e.g VScode won't detect new files created on the linux side of a SMB mount without a manual refresh). Plenty of details on getting permissions right too. Not to mention extended attributes which differ slightly and break stuff. Network file systems in general are just too different to local FS's for programs to work seamlessly.
The 2 non-ideal solutions I've found are 1. keep all files in linux + edit over network using VSCode Remote or IntelliJ Remote support or 2. keep a copy of the dev workspace in windows, and use something like unison (or manual git) to sync.