Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why does Thunderbird need to support maildir?

I just never understood this use case.



A very traditional way to run a mail system is to have the mailserver store the user's mail in their home directory, typically in either maildir or mbox format. Mail clients can then read mail directly from the same place the mail server stores it, rather than having to maintain their own copy. This is a very handy optimization and a lot of "traditional" (read: TUI) mail clients fully support and, indeed, were primarily designed to be used this way. Common TUI mail clients tend to assume a local maildir or mbox, and accessing mail boxes via IMAP is a feature bolted on later, sometimes a bit awkwardly.

A major advantage is that it allows all of the mail tools you might use to operate on the same working copy of your inbox, saving a lot of time synchronizing things. This was a lot more important historically when a typical Unix user would probably have a few different tools they used for mail, like a full-on TUI mailbox viewer and a couple of CLI tools.

I don't find it very compelling that Thunderbird needs to add this functionality, because that's a pretty old-fashioned way of running mail that you probably won't see outside of technical universities and other institutions with a very "traditional" approach to their systems. That's the use-case, though, and I do see that if you are in an environment where your mail is already present in your home directory, or you use multiple mail clients, it would be annoying that Thunderbird can't operate this way when a lot of its contemporaries can.

It's practically more of an issue, though, because Thunderbird kind of supports it, it's just incomplete. That makes it feel less like a feature Thunderbird doesn't have and more like a bug.


One nice thing about maildir is that eg command line grep just works.


Also reading maildirs it's much faster than an mbox file.


Yes, if you have a good enough filesystem, that doesn't mind reading thousands of tiny files. Or housing tens of thousands of files in a directory.

(I suspect that wasn't always the case. Especially over NFS or so?)


Yes, over NFS it's slow, but nowadays everyone has local devices or they just login via SSH or virtualize whole desktops.

Still, if you have over 100k emails, mbox will be slow to parse for obvious reasons as it has to parse every line with often having mboxes as huge as 500MB if not more. Your FS and VFS layers will do a far better task on reading the first lines of thousands of files. In such case your mail client will parse the headers to generate the inbox layout in a much faster way.


> Still, if you have over 100k emails, mbox will be slow to parse for obvious reasons as it has to parse every line with often having mboxes as huge as 500MB if not more.

No, you don't. You store in your metadata database what the offset of the message in the mbox file is. If you don't have such a metadata database, then you're going to be very slow no matter what the storage is.


Load a GB sized mbox into mail(1) and then say that again.

Mbox it's just a literal

cat mail/*.msgs >> mbox

setup.

Reading will be slow, and file locking will be hell.

Unix should've switched to maildir long ago. Since 1999 storage and inodes are not a problem any more.


> Load a GB sized mbox into mail(1) and then say that again.

In other words, use mbox with an application that doesn't have its own message metadata database and you get problems. But if you're designing your own email client in the 21st century, then you're going to want to build your own message metadata database.


This thread is very strange with people talking about mail formats and methods of handling email that haven't seen serious use since 1990

I mean, I studied Comp Sci at university in 1997 and we were given IMAP credentials


I used maildir with mutt in the 2010s.


You and 100 other people globally


You are very wrong, lots of people saves down email in order to be sure.

You can use Maildir with IMAP. mbsync will do it fine. Dito with SMTP and msmtp.


if thunderbird doesn't support maildir, either you can't store your email in a maildir, or you can't use thunderbird. maildir has some real advantages, and so does thunderbird, and it would be nice to be able to combine them


I have used maildir for a long time, but I just run a local IMAP server to do this... (dovecot)

IMO this is a superior way.


that's an excellent point: imap is a standard protocol for decoupling your mail client from your mailbox format


Fetchmail on my laptop fetches messages from the IMAP server and procmail + postfix delivers them (after some mangling) to the maildir in my home directory. So whatever mail client I use needs to be able to read that.


Maildir is probably the best way to handle multiple readers one mail source, ie, reading your mail from two+ locations, two+ operating systems, whatever.

IMAP just kludges, and you end up with situations like I have, where thunderbird frequently reports that one mailbox is 'locked' because thunderbird crashed while accessing it several years ago and who knows how to get gmail to forget about imap 'locks'.


> thunderbird frequently reports that one mailbox is 'locked' because thunderbird crashed while accessing it several years ago

That's a lockfile left over on the local filesystem. That's caused by Thunderbird having two threads open on the same folder (i.e. you ran 'update all folders' and then 'compact folders' and they're overlapping)

If you open TB, let it sit for a while, then do file>offline>sync now - then let it complete - make sure all the folders are selected for local download (if this is applicable or viable for your use case)

then wait a while

then do 'compact folders'

it shouldn't happen again

You might also want to go into the 'account settings' and turn 'maximum number of server connections to cache' down to 2 - some IMAP servers get really bothered if you open too many connections to them. It's meant to autoadjust but it doesn't always. 2 lets you have one connection always open to the inbox for push mail and one available for working on other folders.


> then wait a while

Yeah, that was underselling it a little, it took over an hour to download all 60,000 emails.

Time will tell as to whether it happens again, but regardless, thanks for the pointers.


Yeah TB behaves a lot nicer when you have a 100% complete snapshot of the IMAP account locally; a lot of things in it seem to work a lot faster and more predictably when all the folders are fully synced.

I'd run that 'offline->download/sync' every week or so. It'll automatically sync every folder you open, but running that syncs them all, and if you're an organisation fiend like me you have like 100 folders.

In addition you can take a manual backup of your email now by just zipping up the thunderbird profile folder. Always useful.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: