Edmund A. Hajim  School of Engineering and Applied Sciences

HSEAS Mail Spools

The Mail Spool

The system mail spool is a disk area reserved for inbound and outbound mail. It is designed to contain transient information, and is not intended for long-term storage of mail.

Ideally, your mail arrives, you read it, and then dispose of it, either by deleting unwanted messages, or by refiling messages you wish to keep into mail folders. Even if you receive large attachments, they should require a large amount of space only briefly.

If your mail spool space usage is large, several bad things can happen.

Scope of the Problem

Mail spool usage (or mis-usage) is a problem for almost all mail servers. Some sites/ISPs place disk quotas on the spool area - any given user can use only so much space. If a user exceeds that quota, no more mail can be received for that person.

Some sites/ISPs clean out mail spools daily - any message older than (e.g.) 30 days is simply discarded, which minimized the growth of any given spool file.

We have avoided these rather draconian mesaures and have instead relied on our users having a sense of community, of responsibility to the other members of HSEAS. This has allowed those who need to receive several large attachments the freedom to do so. But that freedom also places a responsibility on that user - to clean out that spool as soon as is possible (and not just when it is convenient to do so).

However, relying on a "sense of community" is proving to not be so effective anymore, and we may have to move towards using an automated cleanup, in which a program runs daily to excise any messages older than N days (e.g., N may be 15 or 30) from the mail spool.

However, in the interim, the following information is relevant.

Mail Client Types

There are several types of mail client programs (a mail client program is a program you use to read/send mail).

POP Mail Problems

POP mail is very nice - lean and usually quick. If you always read your mail on the same desktop computer, then POP mail is probably the best choice.

If you are using POP mail service for reading your mail, then setup/configuration becomes critical. It is important that you do not select "leave mail on server". Some POP clients will nominally allow you to say "leave mail on the server for "N" days, and then delete". Unfortunately, this rarely works, and the mail is left on the server - in the spool - in perpetuity. Worse - once you've seen a message with a POP mail client and left it on the server, you cannot go back and see it again with that same desktop system. This is how a lot of mail gets stranded on the server, using considerable spool space that should be reserved for incoming/outgoing mail.

We've noticed that there are a number of people who read their mail from multiple locations, e.g., an office in Hopeman, an office in CSB, and an office in the Med. Center or LLE. These people intentionally set their mail client to "leave mail on the server" so they can get to the mail from any of their several desktop systems. Unfortunately, they also never clean out their mail spool file, and it just keeps getting bigger and bigger.

This has a huge impact on the mail server (and all users) and has a huge effect on the individual's ability to read their mail. When you check your mail with a POP mail client, it talks to the POP service on the mail server. The POP server first has to make a complete copy of your mail spool file - which means you need at least that much free space on the server in the spool. A large file also takes considerable time (and a number of CPU cycles) to copy - and this slows the system down. Then your mail client has to scan through the copy of the spool file, looking for mail messages you haven't read on your specific desktop computer. This takes considerable time and effort also. While this is going on, the POP service is also checking your real spool file; if it changes (i.e., new mail has arrived) then it has to make a new working copy of the spool file, and pick up where it left off with its scanning. While these operations are usually fairly quick and have only a small effect on the system, large spool files take considerable resources and considerable time - and have a substantial effect on the system., and the network.

In the worst-case situations, a single large mail spool file can cripple the system - filling the disk so that no one can receive mail, and even so that there is no workspace for POP, and other users cannot even read their mail (no space to make a working copy of the spool file).

It is imperative that all users keep their mail spool as clean as possible. This is true for all users (not just the POP mail users). It does however tend to be more of a problem for POP mail users.

Corrective Action

If you currently use POP mail, and need to read mail from multiple locations, you should consider switching to using IMAP service. If you wish to do this, send mail to problem to discuss what needs to be done, and also to see if IMAP is the right solution for you.

If you currently use IMAP mail, or one of the UNIX mail clients, you should examine your spool file (also known as your Inbox) and delete messages you no longer want or need, and then refile (transfer) the remaining messages into folders. In the case of IMAP clients, you can choose to transfer messages into folders local to your desktop host, or into mail folders in your home directory on the home directory server. In the former case, the messages are only available to you when you are using that particular desktop computer. In the latter instance, the message are available to you on any IMAP mail client, or any UNIX mail client that you run within HSEAS. In the case of the UNIX mailers, you can refile messages into appropriate folders within your home directory.

Messages refiled into mail folders in your home directory will be available to you using any UNIX mail client run on a HSEAS (including departmental hosts) UNIX system. They will also be avaialble to you if you use the HSEAS WebMail interface.

If you use POP mail, and have a large mail spool, and are not able to see a large number of messages, then you may need to do the following:

  1. Make sure your POP mail client is configured so that it does not leave mail on the server. If you need to read your mail in multiple locations, you may need to either switch to IMAP service or start using a UNIX mailer after completing this cleanup.
  2. Exit your POP mail client
  3. Manually clean up your mail spool. This is not a lot of fun, which should provide incentive for not letting your spool file get in such a state again.

To manually clean up your mail spool, you have several choices. In all cases, you need to have exited all and any mail clients prior to starting the cleanup procedure. This is very important, unless you (a) want to lose mail and (b) wish to have your mail spool file corrupted so that it can longer be read as mail nor appended to - so you can't receive mail.

Choice #1: UNIX or POP mail users

Login to an appropriate UNIX host in your department (e.g., BME: Bionic, CHE: Oz, ECE: Nebula, Galaxy, Val2, ME: Isaac, Optics: Moe). Then start a UNIX text-based mail program: pine, elm, mutt or Mail. You'll need to know something about the basic operation and use of the mail program prior to using it in this step. Use that mail program to scan the list of messages in your spool file (you may be surprised - some people we've worked with have found they have messages from as long as 10 years ago still in their spool file), and delete the messages you no longer wish to keep, and then refile the remaining into appropraite mail folders. This should result in a clean and empty spool file. Note that the host you choose needs to have sufficient free space in the mail program's work area (typically in /var/tmp) to perform all of its actions - usually needing at least as much space free as is in your spool file - plus a little more.

Choice #2: Any mail user.

This is currently somewhat of an experimental method. Again, you first need to login to an appropriate UNIX host in your department. We've written a small script (mbcleaner). Mbleaner will examine the mail file/folder you specify and removes any messages older that a certain number of and archives them in an alternate folder (your Mail folders are assumed to be in a Mail directory in your home directory; the archiving directory is by default a subdirectory "Pack" in your home directory). Optionally, it will compress the archived mail to save space in your home directory. While this is handy and quick, it too has limitations. First, it will not completely clean out your mail spool file, unless the mssages are all older than the specified date. Second, it needs workspace (in /var/tmp) somewhat larger than the amount of space used by your spool file. Third, mbcleaner currently only works reliably on Linux and Sun Solaris systems. You should NOT use mbcleaner on SunOS or SGI IRIX systems. Fourth, mbcleaner does not get rid of any mail, nor refile it in a structured way. It simply takes all messages from the specified mail folder older than a certain number of days and moves them to an archive file.

Choice #3: Any mail user, but especially POP mail users

This choice requires you have access to a mail client that can do IMAP mail. Most POP mail clients can also do IMAP mail. Some UNIX mailers, even text-based client programs like Pine, can also do IMAP mail, but there is no advantage to using one of those clients for this work, and probably some disadvantages. Stick with a mail client like Thunderbird (Windows, Macintosh and Linux) or AppleMail (Macintosh only) our recommended desktop mail clients - Outlook is just so problematic and vulnerable to exploits and viruses, that we really strongly recommend that you not use Outlook or Outlook Express.

This method is similar to choice #1, but you are using a GUI, desktop-based mail client (e.g., Thunderbird) rather than a UNIX mail to scan your mail and then properly dispose of it (i.e., discarding the messages you no longer want/need, and refiling - transferring from Inbox to an alternate folder - all other messages,

Choice #4: All users

Periodically, if there are space problems in the mail spool, the CNG (Computing and Networking Group - the HSEAS systems staff) will take action. Typically we run a small program that moves your entire mail spool into your home directory's mail folder directory, compresses it, and sends you email telling you what we have done and how you can access your accumulated mail. This is unappealing for several reasons. First, we're not a maid service; it is your responsibility to clean up after yourself. If we are cleaning up after you, then we are not doing the work we should be doing. It has been recommended that we charge for this type of service. Second, it is not a real solution - it merely shifts the problem to the home directory areas, which are also often quite full. Third, it is really not a cleanup - no junk is discarded, and no messages are refiled/transferred into a logical structure - your set of mail folders - that works for you.

Last modifed: Thursday, 07-Apr-2011 09:24:39 EDT