A Maildir directory (often named Maildir) usually has three subdirectories named tmp, new, and cur.2
Sam Varshavchik, the author of the Courier Mail Server and other software, defined the Maildir++ extension45 to the Maildir format to support subfolders and mail quotas. Maildir++ directories contain subdirectories with names that start with a '.' (dot) which are also Maildir++ folders. The extension complies with the original Maildir specification, which allows for subdirectories in addition to tmp, new and cur.
A mail delivery agent is a program that delivers an email message into a Maildir. The mail delivery agent creates a new file with a unique filename in the tmp directory.678 At the time of its invention guaranteeing unique filenames efficiently was difficult. The original qmail9 algorithm for unique names was:
By 2000, the qmail author recommended in an updated specification10 to append the value of a per-process counter to the PID, whose value should be incremented after each delivery. The rate-limiting recommendation to "wait two seconds" was dropped.
By 2003, the recommendations had been further amended to require that instead of the PID and counter, the middle part of the filename should be created by "concatenating enough of the following strings to guarantee uniqueness" even in the face of multiple simultaneous deliveries to the same maildir from one or more processes:11
This 2003 algorithm was criticised12 in 2006 as being unnecessarily complex by Timo Sirainen, the creator of Dovecot.
As of November 2023, qmail author Daniel Bernstein had made no further changes to the 2003 filename generation recommendations.13 On modern POSIX systems, temporary files can be safely created with the mkstemp C library function.
The delivery process stores the message in the maildir by creating and writing to tmp/uniquefilename, and then moving this file to new/uniquefilename. The moving can be done using rename, which is atomic in many systems.14 Alternatively, it can be done by hard-linking the file to new and then unlinking the file from tmp. Any leftover file will eventually be deleted. This sequence guarantees that a maildir-reading program will not see a partially written message. There can be multiple programs reading a maildir at the same time. They range from mail user agents (MUAs), which access the server's file system directly, through Internet Message Access Protocol or Post Office Protocol servers acting on behalf of remote MUAs, to utilities such as biff and rsync, which may or may not be aware of the maildir structure. Readers should never look in tmp.
When a cognizant maildir-reading process (either a POP or IMAP server, or a mail user agent acting locally) finds messages in the new directory, it must move them to cur. It is just a means to notify the user "you have X new messages".15 This moving needs to be done using the atomic filesystem rename(), as the alternative link-then-unlink technique is non-atomic and may result in duplicated messages. An informational suffix is appended to filenames at this stage. It consists of a colon (to separate the unique part of the filename from the actual information), a "2", a comma and various flags. The "2" specifies the version of the information that follows the comma. "2" is the only currently officially specified version, "1" being an experimental version. The specification defines flags that show whether the message has been read, deleted and so on: the initial (capital) letter of "Passed", "Replied", "Seen", "Trashed", "Draft", and "Flagged".16 Applications often choose to supplement this very limited set of flags, for example notmuch17 offers flag synchronization in addition to arbitrary user-defined flags,18 while Dovecot uses lowercase letters to match 26 IMAP keywords,19 which may include keywords such as $MDNSent or user-defined flags.
Although Maildir was intended to allow lockless usage, in practice some software that uses Maildirs also uses locks, such as Dovecot.20
The Maildir standard can only be implemented on systems that accept colons in filenames. 21
Systems that don't allow colons in filenames (this includes Microsoft Windows and some configurations of Novell Storage Services) can use a non-standard alternative separator, such as ";" or "-". It is often trivial to patch free and open-source software to use a different separator.22
As there is currently no agreement on what character this alternative separator should be, there can be interoperability difficulties between different Maildir-supporting programs on these systems. However, not all Maildir-related software needs to know what the separator character is, because not all Maildir-related software needs to be able to read or modify the flags of a message ("read", "replied to" etc.); software that merely delivers to a Maildir or archives old messages from it based only on date, should work no matter what separator is in use. If only the MUA needs to read or modify message flags, and only one MUA is used, then non-standard alternative separators may be used without interoperability problems.
Bernstein, Daniel J. (1995). "maildir(5)". Archived from the original on 1997-10-12. Retrieved 2018-11-23. /wiki/Daniel_J._Bernstein ↩
Blum, Richard (2001). Postfix. Sams Publishing. ISBN 978-0-672-32114-6. 978-0-672-32114-6 ↩
Varshavchik, Sam (2009). "maildir(5)". Archived from the original on 2024-04-17. Retrieved 2024-08-09. http://www.courier-mta.org/maildir.html ↩
Varshavchik, Sam (2011). "Maildir++". Archived from the original on 2024-05-29. Retrieved 2024-08-09. http://www.courier-mta.org/imap/README.maildirquota.html ↩
Bernstein., Daniel J. (c. 2000) [First published 2000 or earlier]. "Using maildir format". Archived from the original on 2000-09-02. Retrieved 2018-11-23. /wiki/Daniel_J._Bernstein ↩
Dovecot Wiki: maildir format https://doc.dovecot.org/admin_manual/mailbox_formats/maildir/ ↩
Bernstein., Daniel J. (2003) [The earliest version of this document was first published in 2000 or earlier]. "Using maildir format". Archived from the original on 2003-04-01. Retrieved 2018-11-23. /wiki/Daniel_J._Bernstein ↩
Sirainen, Timo. "Maildir Mailbox Format: Mail Delivery'". Archived from the original on 2024-06-24. Retrieved 2024-08-09. All this trouble is rather pointless. Only the first step is what really guarantees that the mails won't get overwritten, the rest just sounds nice. Even though they might catch a problem once in a while, they give no guaranteed protection and will just as easily pass duplicate filenames through and overwrite existing mails.¶ Step 2 is pointless because there's a race condition between steps 2 and 3. PID/host combination by itself should already guarantee that it never finds such a file. If it does, something's broken and the stat() check won't help since another process might be doing the same thing at the same time, and you end up writing to the same file in tmp/, causing the mail to get corrupted.¶ In step 4 the link() would fail if an identical file already existed in the Maildir, right? Wrong. The file may already have been moved to cur/ directory, and since it may contain any number of flags by then you can't check with a simple stat() anymore if it exists or not.¶ Step 2 was pointed out to be useful if clock had moved backwards. However, this doesn't give any actual safety guarantees because an identical base filename could already exist in cur/. Besides if the system was just rebooted, the file in tmp/ could probably be even overwritten safely (assuming it wasn't already link()ed to new/).¶ So really, all that's important in not getting mails overwritten in your Maildir is step 1: Always create filenames that are guaranteed to be unique. Forget about the 2 second waits and such that the Qmail's man page talks about. /wiki/Timo_Sirainen ↩
"Wayback Machine snapshots of cr.yp.to/proto/maildir.html". Internet Archive. 2023. Retrieved 2023-11-23. https://web.archive.org/web/20230503185934/cr.yp.to/proto/maildir.html ↩
"rename". The Open Group. 2013. Retrieved 23 July 2016. That specification requires that the action of the function be atomic. http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html ↩
Sam Varshavchik (25 July 2016). "Management of maildir structures". courier-users (Mailing list). Retrieved 26 July 2016. http://www.mail-archive.com/courier-users@lists.sourceforge.net/msg38512.html ↩
"Notmuch mail system homepage". notmuchmail.org. Retrieved 2019-06-22. https://notmuchmail.org/ ↩
"notmuch 0.38.3 documentation". notmuch-config. Retrieved 2024-04-17. https://notmuchmail.org/doc/latest/man1/notmuch-config.html#nmconfig-maildir.synchronize_flags ↩
Sirainen, Timo. "Maildir Mailbox Format: Locking". Archived from the original on 2024-06-24. Retrieved 2024-08-09. /wiki/Timo_Sirainen ↩
"mailbox — Manipulate mailboxes in various formats". Python documentation. Retrieved 2023-06-19. https://docs.python.org/3/library/mailbox.html ↩
mutt maildir support: workaround for filesystems that don't accept colons https://marc.info/?l=mutt-users&m=131644437917477&w=2 ↩
"aerc - the world's best email client homepage". aerc-mail.org. https://aerc-mail.org/ ↩
"Maildir in Thunderbird". mozilla.org. Retrieved 2020-12-06. https://support.mozilla.org/en-US/kb/maildir-thunderbird ↩