Related tangent: Something I don't fully understand (perhaps I just haven't found a project that facilitates it) is why there isn't a Windows extension (driver?) that allows for native mounting of ext4 partitions within Windows explorer.
Surely, given the Linux native file system is open source, adding the ability to mount ext4 partitions natively should be something people can add to Windows?
I have seen third party applications with custom file explorers that can open ext4 partitions (in the same way you open a 7zip file) and I also understand you can mount linux partitions via WSL to access them via explorer - I would just love it if there was a way to natively mount ext4 in the Windows shell.
Up until now I have been using NTFS as my "universal" format - for drives shared between Windows, Linux and MacOS. I think exfat has recently gained support in all three platforms but it's not reliable for an internal hard drive (I think?).
> Related tangent: Something I don't fully understand (perhaps I just haven't found a project that facilitates it) is why there isn't a Windows extension (driver?) that allows for native mounting of ext4 partitions within Windows explorer.
Such projects have long existed, but I suspect there's a lack of interest.
It's a difficult technical problem with a narrow usage niche. You need somebody who wants both Windows and Linux, and wants to use the same physical disks. Today, why bother? You can use a NAS, cloud storage, virtualization, etc.
Wanting to dual-boot sometimes isn't that rare, surely. And local storage for documents remains popular too. It seems to reflect the reality that serious Linux development effort has been very server-focussed for a long time.
On the contrary I have the feeling dual boot is a thing of the late 90's-early 2000's when processors didn't come equiped with virtualization extensions and virtualization was slow as hell.
Having to reboot is so inconvenient, sooner or later ina dual boot setup you leave your non-preferred env colecting dusts and cobwebs.
Sure but the Linux audience is certainly not one that pays for such things. Like the Parent Poster explained, this is a hard and niche technical problem. Someone has to foot the bill for this, and apparently there is a "need" for it but no market to sell it.
> "Sure but the Linux audience is certainly not one that pays for such things."
Eh? People just pay for their stuff. The problem is there isn't really a market for OSes (as a consumer), period. Unless you build a PC yourself, which is not the norm, you don't usually even buy an OS.
This has nothing do with whether the linux audience pays for stuff or not...they pay fine. It is true that it is relatively niche even for linux folk to want to do this sort of thing, but that has more to do with laptop vs desktop form factors than the market. If it were a common need, I think it would be a common thing found on the market. In fact the existing ext4 stuff for Windows indeed used to be paid, but this was primarily for Windows users, Linux users have had no issue accessing ntfs drives for ages, and generally didn't really enjoy giving Windows access to their drives anyways, what with the market for viruses and whatnot...
Back when I was a desktop Linux user, a very sizeable portion of the other people I knew that used it weren’t “software engineers”. They were ‘power users’. I don’t think that my experience was all that atypical.
Well you'd need to fund the cost of a kernel driver signing certificate and get it through WHQL as well. That's just for a start. So this wouldn't be something you could build yourself.
There are corporate solutions (like the Paragon drivers), they just suck. Paragon tried merging this code into the Linux kernel, but the code was so bad and unmaintainable that it was rejected. The FOSS solution is using FUSE, but that requires a UNIX-like OS for easy porting. Unless Windows ditches the NT kernel altogether, I don't think you can expect native filesystem extensions beyond NFS support.
GNU philosophy is not about moneymaking or footing the bill. Sometimes a problem gets worse when you throw money at it.
If there was a Linux OS focusing on desktop/laptop users that offered a highly polished user experience - comparable to the best of MacOS and Windows - I would be very happy to pay for it.
I am probably in the minority of Linux users though.
I mean…using desktop Linux in the first place is already an incredibly harsh qualifier. macOS and even Windows (+ WSL) are no doubt serving the needs of many people that need a *nix environment for their work, and eating into the percentage of people that would otherwise push for Linux to be supported by their employer, or use it at home. That and the ubiquity of containerised workloads.
It’s really not outside the realm of reason for the subset of those people that want to use Windows for something other than playing video games is small enough for there to not be the base level of interest needed for this stuff to get off the ground.
Used to dual boot, but with the combo of steam/proton and VFIO passthrough + looking glass, it’s been really solid to just run both at the same time. File sharing between the two is pretty easy in that case, because it’s just the usual rsync/scp tools or SMB share.
Although, I’ve found I very rarely need the guest for gaming anymore, more so for opening the odd Photoshop file and similar tasks.
In 2010, used to dual boot. In 2014 or so, I switched to VFIO GPU/SSD passthrough and never booted to Windows again. In maybe 2019 or so, I pretty much stopped using passthrough even, though this is partly because I stopped playing a few games that proton didn't run (mostly because of e.g. Easy Anti Cheat, etc).
I don't even have my windows ssd installed in my tower any more, that's how little I expect to need it these days.
Virtualisation is really laggy if you have no GPU acceleration, there are ways to get it working but they either need two GPUs or are still slower than native
Not all dual boot systems are "main OS" plus "wintendo." Quite a lot of the ones I encounter on personal machines are, certainly, but people doing actual dev stuff on both is still a use case.
Linux has good NTFS drivers. If you want to dual boot and share files, you just make three partitions: Windows (NTFS), Linux (whatever the cool kids are using these days), and Shared Data (NTFS).
In my experience, exFAT support in Linux is worse than NTFS support (and NTFS is more likely to be installed out-of-the-box). If you also need to support macOS, then exFAT might be an option, but exFAT lacks journalling and doesn’t scale too well due to cluster size/count limits.
The niche surely ain't that narrow, especially as Linux-based devices like Raspberry Pis and Steam Decks grow in popularity; being able to ready EXT-formatted SD cards and such seems like it'd be valuable even from Windows.
Back in the day, and I mean like mid-late 90's, you could actually install the Linux kernel, files etc., straight onto a Windows FAT partition and boot into it.
Not exactly what was being asked for, but a weird old fact I'd forgotten about.
Sometime in the early 2000s, when microarchitectural features for virtualization were rare, I used coLinux in a working capacity. It has a design was amazing at how well it worked given the constrained size of the code changes. In short, it ran a copy of Linux in supervisor mode in addition to Windows (this is the "driver mode"). At the same time. Both in supervisor mode. As a giant coroutines yielding to one another.
I was really quite fond of colinux, out of assorted VM tech, colinux, cygwin and WSL1 and 2 that have provided me with enough of an X11+xterm+ssh rig to get work done comfortably on a windows machine I think colinux and WSL1 have been the least friction overall.
A friend of mine has been using this setup for a very long time do be able to write music and work on the same station.
I remember I was amazed at how solid the thing was.
Does anyone know if there's a relationship between colinux and how ms eventually ended up implementing wsl ?
Not so much. WSL1 was a too-audacious attempt to implement enough Linux-style system calls in Windows to run Linux software. Which it did surprisingly well, considering...but also, often strangely, and slowly.
I think the name is a bit ironic, since the whole idea is to avoid using any Linux code in Windows, but implementing system calls so that the non-Linux parts of a Linux distribution could pretend Windows was Linux. GNU/Windows-pretending-to-be-Linux, as it were.
WSL2 is virtualization-based, so it could be seen as more comparable to Docker's installation package as seen on non-Linux platforms, or Parallels: a somewhat elaborate attempt to improve the virtualization experience in interoperation with the host, e.g. setting up NAT, network file systems, or installing special drivers.
I don't think coLinux offers as much as it did, now that virtualization micro-architectural features are routinely available. At least one part of coLinux that took up quite a bit of its code was special drivers to interface with the host OS. This code would still be useful and has parallels in current approaches, but the decisive trick of toggling between two kernels running in supervisor mode has less room to be useful.
Also, you could only imagine the difficulty of identifying the origin of bugs in that set-up.
There was a distribution called WinLinux that did this, iirc. Installed like a normal piece of windows software. Was my first foray into unices when I was a young kid. There were some fun games in that distro!
It might be convenient, but it is also more complex than just being able to read file systems. External disks or partitions containing data in particular will / should always be fully encrypted nowadays, so you'd need interoperability of disk encryption schemes as well (dmcrypt, bitlocker). NAS can handle that transparently.
Now that the new NTFS3 driver is out in the wild there's also not much need. The number of people who boot windows and need to access an ext4 partition is tiny compared to the number of people booting Linux who need to access NTFS.
Don't use this. I once tried it and it changed the UUID of the Linux partition without any warning. Grub was unable to pick up the partition and boot, so I was stuck at grub rescue.
I haven't stubbed my toe on the former yet, but I agree it's not great. I'm mainly storing media, photos, etc. (relatively not compressible) so the latter doesn't matter much to me. I think that's probably a common use case for a drive you can read anywhere, so I still like ExFAT for that, generally speaking.
Writing filesystem drivers for Windows is extraordinarily difficult, more so than generic drivers and the result is hard to call GPL compatible because a signature which is basically not obtainable for individuals is required to load them.
exfat’s reliability “issues” are by design— it’s not journaled, to simplify implementation (and reduce resource use) on embedded devices and reduce wear on flash media.
It’s not a replacement for NTFS in applications where it’s already being used (large disks, anything where important data is being stored). Rather, it’s a modernized replacement for FAT32 for things like camera memory cards, eliminating some of FAT32’s limitations (like the file size limit) while remaining similarly lightweight.
There used to be an ext2 driver for NT, I think opensource even, but there simply had not been enough interest to keep it updated or built for newer versions
I installed Debian on WSL for the very first time a few weeks back. Windows explorer gives access to the Debian folder while it is running and no access otherwise. Had me scratching my head. I don't know much about virtualisation, but if that is the key, they could probably have a highly customised, small Linux kernel running all the time perhaps.
Pretty sure there isn’t an ext4 spec, it’s just the reference implementation. Microsoft could build something that is compatible, but keeping compatibility as the reference implementation changes may not be straightforward.
Ext4 was explicitly designed with maintaining backwards and (limited) forwards compatibility in mind precisely for the reason pointed out in my sibling comments.
Depending on the filesystem features you enable, you can even mount an ext4 filesystem with the ext2 kernel driver, and you can already mount an ext2 or ext3 filesystem with the ext4 kernel driver.
A large part of this compatibility is due to the bitmap of compatible, compatible read-only, and incompatible filesystem flags in the filesystem superblock.
For example, when ext4 got support for extents, that's an incompatible feature; you cannot mount an ext4 filesystem with extents with the ext3 kernel driver. This does not even require ext3 to know what an extent is because it's just indicated as an unknown (from the ext3 driver's perspective) incompatibility flag.
On the other hand, the implementation of sparse superblocks (keeping fewer copies of the superblock around to allow for more blocks to be used for file data on filesystems intended to house lots of large files) is merely a read-only compatibility bit; you can mount an ext4 filesystem using sparse superblocks with an ext2/ext3/ext4 driver that does not support them, in a read-only manner. This cannot be mounted read-write because the implementation may try to place a filesystem superblock where file data is supposed to be, which would lead to at best wasted space (negating the feature) or at worst limited filesystem corruption.
When feature changes that could break things are made to the ext4 implementation in the Linux kernel, they are always done so using these (in)compatibility flags, ensuring that only implementations that can entirely support the feature can mount them read-write, and that implementations whose ability to read would not be affected can still mount them read-only. This allows you to create future filesystems (using e.g. mke2fs(8)) and not turn those features on while doing so, if you want the filesystem to be used by other implementations. Something you already have to bear in mind if you're creating a filesystem using a modern mke2fs but intended to be read by ancient Linux kernels.
The counterpoint to this is that ext4 can't change dramatically enough to cause problems without breaking compatibility with older kernels or older filesystems.
Certain features (like case-insensitivity) already do break compatibility, as I found out the hard way trying to get GRUB to boot from an ext4 filesystem with said features enabled.
I've used ext2fsd in the past and it worked well. I don't think you could boot windows off of it but you can certainly start touching your Linux drive from there.
Microsoft didn't support ext because too few people were using it to justify the effort, yet somehow they went to great lengths and implemented the whole WSL, and... ~twice at that - for a possibly smaller user base?
Its `btrfs-rec inspect mount --pv=/dev/whatever` is a read-only FUSE implementation of btrfs that is more fault-tolerant (of corrupt filesystems) than the normal in-kernel btrfs driver (and even more tolerant than `btrfs rescue` or `btrfs recover`).
It's still missing a bit; RAID almost certainly doesn't work, and encryption is not implemented. But hopefully some folks will find it useful, or at least neat!
Oh, comparison with the existing https://github.com/adam900710/btrfs-fuse : (1) Again, mine has better fault tolerance, (2) but mine is read-only, (3) mine supports xattrs (TODO in Adam's), (4) mine supports separate inode address spaces for subvolumes (Adam's doesn't due to limitations in FUSE, mine works around this by lazily setting up separate mountpoints for each subvolume).
I intend for it to get incorporated in to btrfs-progs. In the mean-time: yeah, btrfs-progs-ng isn't a great name for the repo, but as I said, I isn't quite ready to publish it yet either, so the repo name wasn't at the top of my list. I originally had the repo named "btrfs-tools", but that's even more confusing (but for different reasons)!
(As a side note, I've had others fork software I maintain and call it "-ng", you learn to take it as a compliment :) )
First of all, I do intend for it to get incorporated in to upstream btrfs-progs.
Plus, for the most part, it's not a replacement for what's in btrfs-progs, it's additional tools:
Replace:
- `btrfs-rec inspect dump-trees` is a clone of `btrfs inspect-internal dump-tree`, but it isn't meant to replace it, I only implemented it so that I could diff the output to validate that my implementation agrees with the btrfs-progs implementation.
- `btrfs-rec inspect mount` is conceptually a replacement for `btrfs restore`.
- `btrfs-rec inspect rebuild-mappings` is a replacement for `btrfs rescue chunk-recover`
Additional:
- I don't believe that `btrfs-rec inspect rebuild-trees` has an equivalent.
- `btrfs-rec inspect list-nodes`, `btrfs-rec inspect ls-files`, and `btrfs-rec inspect ls-trees` have no equivalent in btrfs-progs.
- (for completeness) `btrfs-rec inspect spew-items` has no equivalent, but it's for debugging btrfs-rec itself.
Missing:
- `btrfs-rec` doesn't try to do anything with mounted filesystems, such as `btrfs balance`, `btrfs filesystem`, `btrfs scrub`, `btrfs send`/`btrfs receive`, ...
- `btrfs-rec` doesn't try to do anything with filesystem initialization, such as `btrfstune` or `mkfs.btrfs`.
- `btrfs-rec` doesn't try to do things with recovering the superblock, I've found `btrfs rescue super-recover` and `btrfs-select-super` to be adequate.
- In general, I didn't try to re-implement commands that already work well.
Yeah. That said, I've become disillusioned with FUSE over the course of writing this. FUSE is... not great, it's just what we're stuck with. Suggestions as to alternatives to FUSE would be welcome.
One life tip I've learned is that the Windwos Store's .msixbundle packages are just zip files that you can extract and usually will run fine.
Change the .msixbundle extension to .zip, and open with 7-zip. Inside the archive, open & extract the appropriate .msix file for your architecture. Now you have Windows Terminal Portable. (I've done this with other apps too)
To obtain a download link for a Windows Store package, you need to go to third party websites where you input the Windows Store URL, and it gives you the download link. Make sure the download link is from Microsoft domain.
But in Windows Terminal's case, the bundle is published on their Github so you can download directly. If you want the Cascadia Code font, you need to install it manually by double clicking on the font from the package.
The defacto website for downloading bundles (rg-adguard) is closed-source and hosted alongside questionable content, so I run an OSS version here: https://msft-store.tplant.com.au/
Haha wow that is so ridiculous it's awesome. Seeing both the Linux root and Windows root with Program Files living right next to usr, etc. is _wild_. Very cool hack!
I think getting around permissions would be an issue but when I was starting on Linux I remember there was a way to install to Fat32 inside a directory in parallel with Windows 95. The distro was called Phat, so the concept is not exactly new.
I think it was Ygdrassil, or maybe a free distro off of a CD. Great investment of time as it got me into the oilfield doing tech support. Running Rocky 8 on a P1 Gen4 now, so come a long way from my 486/Dx2.
I suppose you could store the chmod and group/owner in an NTFS alternative data stream, or reserve a special file on the file system that acts like the MFT.
WSL 1 was mostly like this, where Linux shared the file system with Windows, albeit in a subdirectory.
WSL 2 is a completely different beast, with Linux storing its file system in a virtual disk image. \\wsl$\ just gives you the illusion of being the same system, but that’s just a networked file system.
Speaking of Windows storage drivers, years ago, I managed to get Windows to boot and run entirely in RAM using some fairly obscure tools from the reboot.pro forums. I always wanted to have an immutable Windows system that I could manage like cattle. Unfortunately, setting up such a system was extremely complicated and a manual process. I am aware that Windows now has a driver called UWF that functions a bit like an overlay filesystem and can create ephemeral overlays, but it's not quite the same as booting and running in RAM. It's also only available in Enterprise unless you enable it via registry edits.
I did that for lulz few times in Linux when I didn't wanted to fuck up with some old server's boot when I was decommissioning it.
Just make a big file in /dev/shm, add it via loopback to LVM and migrate system partition on it, then run shred on drives without the annoying messages kernel screams when it has its storage destroyed underneath
JFYI, nowadays (if you have enough RAM) we have a good driver that can load Windows both as a Ramdisk and as Filedisk (using a VHD or RAW image) booting via grub4dos or grub4EFI, SVBUS:
It's good to see projects like this which basically confirm what we know about the modularity of the filesystem layer in Windows. In fact, the IFS API that enables this has been there since MS-DOS 4:
I was wondering about that too, specifically how ZwCreateFile and other file i/o functions would work. But if they're using a file system filter then it's not an issue. I'm curious where \\Device and \\DosDevices goes, considering Windows device files and their symlinks are there. I guess they just put them in under /?
My laptop is a small Windows + Arch (f2fs) partitions + a big btrfs "cold storage" partition. Some notes:
- "The btrfs driver for Windows is incredibly solid" is true... but (on NVMe) its also extremely slow. Like dramatically slower than NTFS, with or without compression.
- You can share data, but writing with winbtrfs will screw up the permissions on linux unless you get the mount options exactly right.
- Booting windows and ntfs conversion: there be dragons. The MS store isnt the only thing that breaks...
To me, this article goes a very long way to demonstrate how screwed up people's thinking is in the windows users world.
I mean, let's look at this objectively: the guy is amazed because he actually managed to boot windows off of a non MS-sanctioned software stack.
There's nothing amazing here: these things should be the standard, and it pretty much is the case in the open-source world.
And I certainly don't mean to diminish the technical achievement of the folks who stitched this together, quite the contrary, very nice feat.
But what I'm trying to say: instead of being amazed because they managed to pull it off, they should be demanding this kind of inter-operability between components of the steaming pile of shite that is windows in 2023.
Interesting, I would've expected more Windows software to just assume NTFS with some fallbacks for (ex)fat given how there haven't really been any other options for quite some time.
How stable is Btrfs these days? Remember reading years ago that it was bad about spontaneously corrupting itself for no apparent reason.
Curious if something similar is possible with ZFS.
IMO, it became stable somewhere between 2013 and 2016. I definitely haven't had any spontaneous corruption since at least 2016 (on multiple laptops and servers; btrfs volumes that mid-double-digit of terabytes)
That said,
- My BDB and SQLite databases tend to corrupt if my laptop battery dies and they're on a volume that's not on LUKS.
- `btrfs check` (fsck), `btrfs rescue`, and `btrfs restore` are IMO not up to snuff if you do encounter corruption (caused by something else, like a failing drive). (But I'm working on that https://git.lukeshu.com/btrfs-progs-ng/ !)
- `docker build` is weirdly slow on btrfs, I always set up a separate ext4 `/var/lib/docker` volume.
- I do RAID and encryption separately in MD or LVM and LUKS, so no comments from me about the stability of btrfs' built-in RAID cor encryption.
> `docker build` is weirdly slow on btrfs, I always set up a separate ext4 `/var/lib/docker` volume.
I'm currently doing this for a different reason. My docker builds started failing after some Ubuntu upgrade. I gave up and used ext4 for that docker directory to fix everything. https://serverfault.com/q/1127148
It unsettled me because now I'm not sure whether btrfs (or docker with btrfs) is production-ready.
Docker has 8 different storage drivers that it can use... that's a lot, so you know they don't all get equal attention. As much as I love btrfs, it's probably fair to say that the only Docker storage driver that receives adequate attention to be production-ready is "overlay2".
I haven't had a chance to dig in to it yet. Most of the time, Docker uses `overlayfs` to emulate COW, but on btrfs it can just use the filesystem's native COW. In my mind it's about equally likely that it's Docker's fault as it is btrfs's fault; that perhaps Docker's btrfs storage driver is doing something dumb.
For VMs and containers isn't it pretty common to use a subvol where CoW is disabled for their disk images? Same thing for ZFS with datasets that disabled CoW for things like that.
Huh. Looks like it doesn't! I knew that BTRFS did and saw that it was on the roadmap for bcachefs, and assumed it was something you could tweak in ZFS as well.
FWIW I've been using btrfs for the root partition for a decade without any (observed) corruption problems. This is across multiple installs of OpenSUSE Tumbleweed on laptops and PCs, HDDs and SSDs and most recently an SD card. Back in the day it needed manual intervention to rebalance but that is automated now. Also, OpenSUSE's update program automatically takes pre- and post- snapshots, and these snapshots have saved me a lot of headache from botched upgrades.
Same here. Never had to do anything, with the notable exception of it being somewhat annoying to recover when the disk is full (which is why I've started to keep subvolumes that I can quickly delete).
But I'm not running a RAID, which I believe is where almost all of the data corruption has happened.
There is no way to have btrfs caps insensitive. So some software (poorly coded) could have capitalization issues that would not appear in NTFS that would in btrfs. But other than that I believe you are right (barring utilities specifically designed to interact with FS attributes)
I've been running my Samba shares case sensitive for over a decade, only issue I've found is I can't rename "Foo" to "foo" without jumping through an extra hoop (add a letter or similar).
Perhaps if I ran some software off it I might find some more issues but, overall quite little. The difference in speed is immense though.
Well, there's ReFS as well. That experience suggests there are issues that the author might come up against following boot: Case insensitive lookup (NTFS has the option for case sensitive subdirectories but that probably doesn't matter), Mark of the Web/SQL Server (Alternate data streams support), UWP/Microsoft Store apps support (IIRC requires FS encryption support via a specific interface), DLT (Object IDs), getting DISM to work for updates... It's cool to get this far, full support will probably need a bit more work.
I enjoy it as ext4++. Would I trust it for RAID? Not particularly.
There seems to be a decent amount of foot-guns still and I would greatly prefer just using ZFS.
Granted, I do trust Synology using btrfs but they are also subject matter experts on the topic.
They are mistaken. In Synology DSM you create a "pool" which is LVM-based under the hood, then inside the pool you can add btrfs & ext4 "volumes" that coexist inside the pool. They would need to let you create a btrfs pool instead if they were doing btrfs on bare drives.
It's really matured, and although the RAID5/6 Write Hole still exists, I wouldn't reccomend a ZDEV over RAID10 either, which is what I believe offers the best tradeoffs for most users on both filesystems. Love the flexibility that BTRFS provides via eg. the balance comand, and Snapper is great for snapshot management.
Too bad the article doesn't talk about performance vs NTFS. Any quick and dirty comparative benchmark would have been nice. But then again, If BTRFS had been slower than NTFS it could have been due to a suboptimal (re)implementation.
I think nothing these days can perform worse than NTFS. I’ve been using Windows on my corporate issue laptop and anything that includes disk IO is a huge pain.
I can't find it right now, but I believe there was a talk from a Rust maintainer a while ago about the Rust installation being way slower than it should be, which was almost entirely a rant on Windows file system performance.
I chose the word because I remembered it being a bit rant-adjacent ("we're used to doing it this way, ... but windows does it that way"). But it's been a while since I watched it (not particularly attentively either, to be honest). That's just what stuck.
Linux has an upstream kernel driver for NTFS, you don't need fuse ntfs-3g any more (well, they have different quirks such as how UNIX permissions are emulated).
The readwrite driver is new, but it always had an older readonly driver too, which would be fine for benchmarking.
On Windows, I can enumerate a directory that accidentally got millions of files perfectly fine without the system itself breaking and with results being returned incrementally.
On Linux/ext4, it takes an eternity for an enumeration to even start returning results and the entire OS gets stuck on IO while doing so.
Hmm. I thought the problem was that NTFS is not particularly fast at file creation. Generally not a problem, except for the folks who wrote the npm tooling which generates 10,000 files in the blink of an eye.
I benchmarked both with a cross platform program. btrfs is massively slower on windows, especially with random IO (but also with sequential). Its even more disparate with compression turned on both.
I cant remember the linux results... not that it matters, as I avoid NTFS writes on linux like the plague, even with the new kernel driver. Just this year, I tried trusting it for a little while, and linux ntfs killed an entire backup HDD by corrupting the partition table :/.
This is really cool and speaks to the modularity of the Windows file system stack. I love projects that customize Windows (working against its closed-source nature).
There’s an OpenZFS port to Windows[0]. I wonder if my hopes of having ZFS on Windows (including the boot drive, because I would love to be able to snapshot and rollback) would actually be possible.
Having used both VSS and ZFS fairly heavily I much prefer the ZFS experience. It seems more predictable and reliable in fulfilling its function.
I am pretty strongly pro-Windows NT. I think there’s a ton of good design and implementation in it. (The parts of NT that came from Win32 are the more irritating parts.)
VSS isn’t wholly unreasonable. The design seems pretty well thought-out. I particularly like the VSS providers that can interface with hardware-based snapshot mechanisms and VSS writers to allow software to quiesce IO. It’s all pretty neat, but it seems to fail to take snapshots “mysteriously” pretty often in practice. Since I can’t see the source I don’t have much chance of actually figuring out what’s going wrong either.
I also like that VSS is clearly distinct from volume management and the filesystem itself. The layer violation inherent in the design of ZFS seems a bit clunky to me. In use, though, it has done very well for me and I haven’t bumped up against limitations re: volume management and redundancy being tied to the filesystem too much (probably because I’ve always come at it with the expectation that my volumes are pretty much decided at deployment time and migration means starting a fresh zpool and starting over).
Btrfs still has no native encryption right? There is Dislocker that lets you open Windows partions encryption with Bitlocker. I switched from btrfs to zfs because Ubuntu has it naively integrated into the install. There was some issues about it recently about possible removal or some tools because badly maintained though.
So if I want everything encrypted this is not helping me.
Opening LUKS from Windows project is dead I think. But I am fine this way I basically have not booted up Windows up for gaming in months. And this one way sharing with Dislocker is enough for me. I think btrfs and zfs are rather a hasse on personal desktop systems, you just loose lots of space for snapshots you not really need, well I needed them on OpenSuse tumblweed because it was unstable but I since switched to Ubuntu LTS and I do not need them.
In Linux terms, think about what would happen if you tried to use ACL (or fsattr or whatever non standard stuff) on a filesystem not supporting ACL: it would throw weird errors, and you may not properly handle them if you expected all filesystems to support ACLs.
User land generally doesn’t care about file system specifics. Even in your ACL, example, opening a file would succeed or fail.
I suspect this is much more likely to be a secure environment issue. Windows store wants to prevent piracy, there is no way the environment can be trusted because the boot loader wasn’t signed by Microsoft, and therefore may be modified to help people pirate software.
The C:\Program Files\WindowsApps (or the equivalent X:\WindowsApps on other disks) needs some very specific ACL settings or the UWP sandbox will throw fits and refuse to start.
A more sensible approach would be to have a separate data volume to store important stuff on and leave it as BtrFS while letting Windows and Windows software run off NTFS.
But, then, what’d be the fun of having a computer you don’t need to fix?
I've always thought filesystem compatibility between OSes should be a priority. Moreover, I've never understood why Microsoft developed ReFS instead of adopting ZFS.
More likely Microsoft doesn't want its OS features to be controlled by a third party. The issues with using ZFS in Linux stem from the GPL. Neither Windows nor MacOS are affected.
Yeah, so the semantics of the filesystem APIs on Windows is very different, so you're not going to easily port filesystem code to Windows from other operating systems. ZFS is CDDL and that won't fly on Linux. Linux filesystem code is GPL2 and that won't fly on macOS. Licensing is a bitch.
This is insane and I love/hate it. Besides the MS Store stuff, which I try to avoid anyway as, in my experience, trying to launch them on Windows via SSH doesn't always work because of permissions, it's interesting as this opens up the ability to possibly dual boot windows from a subvolume without separate partitions/drives.
Regardless of how cool this is, which I truly think, won't this "break" tons of apps and tools? NTFS and (V)FAT(32) arent't case sensitive and I have seen tons of projects which will fopen file.xy regardless if the file was created as File.xy or fIle.xy or FILE.xy.... (you get the picture).
NTFS is case-sensitive internally and you can have case-sensitive files or directories. Back when WSL1 was the only version, I tried to compile some C++ stuff with MSVC. I ran `git clone` in WSL but cloned to the Windows partition. Visual Studio complained that EXAMPLE.H could not be found, even though it was there, except it looked like "example.h" in Explorer. (WSL2 does not use this feature, since cross-OS file access is done via network shares, and Linux files live in a virtual disk image.)
If the BRTFS driver on Windows would implement setting ACLs I think that the Windows Store apps would work fine. The reason the probably fail is because those apps run within a Sandbox with limited permissions and need the right ACLs to run.
You could then even hide all Linux directories of you wanted.
I think it's the "from the Microsoft Store" part that's going wrong, and it just so happens the author noticed it on AMD-related store installed stuff.
I gave up on running windows because of the difficulty of sharing files with linux. So now I run only Linux. Some kind of native support for btrfs in windows would be great!
I wouldn’t be so sure, Windows is famously annoying to goof with the boot loader on from a running install. You usually need to boot from another system to fix/reinstall it (at least, you did last time I needed to bring back the Windows bootloader on a borked dual boot).
Although, since UWP doesn’t work it’s a moot point, that means you probably can’t run Windows Update.
Unless it supports the transactional filesystem API, Windows Update won't even work in the first place. ReFS boot in fact is affected by the same issue at this time.
> I am sure this will break the second time you run Windows Update.
Then you are confidently wrong, because Windows Update rarely (if ever) touches the bootloader, while this is the main innovation of the WinBTRFS project.
Until the day it touches it and replaces your selection from the UEFI firmware. Or repartitions the recovery partition. Or decides to move the recovery partition from the beginning of the disk to the end of the disk. All stuff I have personally seen Windows Update do.
I am sure your custom bootloader will not be overwritten. All types of hell can happen, borking the entire installation -- and your bootloader will still be perfectly fine.
There won't be any reference, as touching other OS partitions never happened for real.
For the rest, there could be references, but it's just lingering legends now: yes, in the past, it could happen BUT only for the Windows partitions (recovery 2700, efi EF00, msr 0701 c: 0700) and only if your partition was too small for a specific update. Then windows would do some careful resizing, in the worst case creating another partition out of its C: to avoid touching the rest. So it would't break anything, just like how ntfsresize works on linux!
For the selection from the UEFI, it was only if you used the default bootx64.efi and tried to repair the boot (windows detected this is not normal and would replace it by something to chainload bootmgfw.efi with)
Anyway, since Windows 10 it doesn't happen, and I never had any problem at all with 11.
Just use the recommended partition size (or more, I regularly make my EFI 4Gb), name your bootloader something else than bootx64.efi and it will be fine.
The most windows might do is to reorder your UEFI boot entries (then go to settings and you can boot another system, same in recovery: just select your linux UKI or grub EFI)
It forces you to understand how Windows functions on unexpected conditions and that knowledge can be extrapolated to other situations where something on Windows fails for no apparent reason and you just know it because you saw it before.
Do not underestimate the power of arcane knowledge.
Why keep different filesystems outside of specific performance needs? I've experimented with weirder things, like Linux running on NTFS with the new kernel driver.
Assuming subvolumes work you could dual boot and avoid having to partition your disk, which would allow you share the space on a drive freely between multiple OSes. That would be especially nice in places where drives and space are a premium, like a Steam Deck.
Actually I have been looking for just this for several years, even as recently as a few weeks ago I was looking, but only found some commercial btrfs driver. The idea of unified storage is really nice, you don't have to decide how much each OS gets, and with btrfs you can just plop in another drive and extend onto it to transparently increase storage capacity. On top of that, tings like duplicating steam game assets would probably be a big win.
During my transition period from Windows to Linux, I stored the majority of my games on my NTFS drive because it was much bigger and I wanted them available on Windows in case they didn't work on Proton. I could play them from Linux with ntfs-3g but some of them would just refuse to launch unless I moved them a non-NTFS drive. Maybe they would've worked in this scenario? Hard to say.
But TBH its like this on linux too. BTRFS is a feature heavy CoW filesystem like ZFS, not a "light," intentionally low overhead fs like ext4 or f2fs (which is what I benched on linux).
Surely, given the Linux native file system is open source, adding the ability to mount ext4 partitions natively should be something people can add to Windows?
I have seen third party applications with custom file explorers that can open ext4 partitions (in the same way you open a 7zip file) and I also understand you can mount linux partitions via WSL to access them via explorer - I would just love it if there was a way to natively mount ext4 in the Windows shell.
Up until now I have been using NTFS as my "universal" format - for drives shared between Windows, Linux and MacOS. I think exfat has recently gained support in all three platforms but it's not reliable for an internal hard drive (I think?).