fdm ============================================================================

1	Introduction
2	Installation
3	Quick start
4	The configuration file
4.1	Macros
4.2	Including other files
5	Invoking fdm
5.1	Command line arguments
5.2	Running from cron
5.3	The lock file
5.4	Testing and debugging
6	Fetching mail
6.1	POP3 and POP3S
6.2	IMAP and IMAPS
6.3	IMAP over a pipe or via ssh
6.4	Stdin and local mail
6.5 	From maildirs
6.6	Using NNTP
7	Defining actions
7.1	Drop and keep
7.2	Maildirs
7.3	Mboxes
7.4	SMTP
7.5	Write, pipe and append
7.6 	Rewriting mail
8	Filtering mail
8.1	Nesting rules
8.2	The all condition
8.3	Matched and unmatched
8.4	Matching a regexp
8.5	Matching bracket expressions
8.6	Matching by age or size
8.7	Using a shell command
8.8	Attachments
8.9	Tagging
9	Setting options
10	Archiving and searching mail
11	Using fdm behind a proxy
12	Bug reports and queries
13	Frequently asked questions

Introduction ===================================================================

fdm is a program to fetch mail and deliver it in various ways depending on as
user-supplied ruleset. Mail may be fetched from stdin, IMAP or POP3 servers, or
from local maildirs, and filtered based on whether it matches a regexp, its
size or age, or the output of a shell command. It can be rewritten by an
external process, dropped, left on the server or delivered into maildirs,
mboxes, to a file or pipe, or any combination.

fdm is designed to be lightweight but powerful, with a compact but clear
configuration syntax. It is primarily designed for single-user uses but may
also be configured to deliver mail in a multi-user setup. In this case, it uses
privilege separation to minimise the amount of code running as the root user.

Installation ===================================================================

Installing fdm is simple. Download the source tarball, and run:

	$ tar -zxvf fdm-?.?.tar.gz
	$ cd fdm-?.?.tar.gz
	$ make

Then run 'make install' to install fdm to the default locations under
/usr/local. The PREFIX environment variable may be set to specify an
alternative installation location:

	$ export PREFIX=/opt		# defaults to /usr/local
	$ sudo make install

If being run as root, fdm requires a user named "_fdm" to exist. It will drop
privileges to this user and its primary group. The user may be added on
OpenBSD with, for example:

	# useradd -u 999 -s /bin/nologin -d /var/empty -g=uid _fdm

It is not necessary to add a user if fdm is always started by a non-root user.

Quick start ====================================================================

A simple ~/.fdm.conf file for a single user fetching from POP3, POP3S and IMAP
accounts and delivering to one maildir may look similar to:

 	# Set the maximum size of mail.
	set maximum-size 128M

	# An action to save to the maildir ~/mail/inbox.
	action "inbox" maildir "%h/mail/inbox"

	# An action to drop unwanted mail.
	action "drop" drop

	# Accounts: POP3, POP3S and IMAP. Note the double escaping of the '\'
	# character in the password. If the port is omitted, the default
	# ("pop3", "pop3s", "imap" or "imaps" in the services(5) db) is used.
	account "pop3" pop3 server "my.pop3.server"
		user "my-username" pass "my-password-with-a-\\-in-it"
	account "pop3s" pop3 server "pop.googlemail.com" port 995
		user "my-account@gmail.com" pass "my-password"
	# If the 'folder "my-folder"' argument is omitted, fdm will fetch mail
	# from the inbox.
	account "imap" imap server "my.imap.server"
		user "my-username" pass "my-password" folder "my-folder"

	# Discard mail from Bob Idiot. Note that the regexp is an extended
	# regexp, and case-insensitive by default.
	match "From:.*bob@idiot\\.net" action "drop"

	# Match all other mail and deliver using the 'inbox' action.
	match all action "inbox"

A simple initial configuration file without filtering, perhaps to replace
fetchmail or getmail delivering to maildrop, may look similar to:

 	# Set the maximum size of mail.
	set maximum-size 128M

	# Action to pipe directly to maildrop.
	action "maildrop" pipe "/usr/local/bin/maildrop"

	# Account definitions.
	account ....

	# Send all mail to maildrop.
	match all action "maildrop"

To run fdm every half hour from cron, add something like this:

*/30    *       *       *       *       /usr/local/bin/fdm -l fetch

See the fdm.conf(5) man page or the rest of this manual for more detail of the
configuration file format.

The configuration file =========================================================

fdm is controlled by its configuration file. It first searches for a .fdm.conf
file in the invoking user's home directory. If that fails, fdm attempts to use
/etc/fdm.conf. The configuration file may also be specified using the '-f'
command line option, see the section on that subject below.

This section gives an overview of the configuration file syntax. Further
details of syntax, and specific keywords, are covered in later sections.

The configuration file has the following general rules:

- Keywords are specified as unadorned lowercase words: match, action, all.
- Strings are enclosed in double quotes (") or single quotes ('). In double
  quoted strings, double quotes may be included by escaping them using the
  backslash character (\). Backslashes must also be escaped ("\\") - this
  applies to all such strings, including regexps and passwords. The special
  sequence '\t' is replaced by a tab character. In single quoted strings no
  escaping is necessary, but it is not possible to include a literal ' or a
  tab character.
- Comments are prefixed by the hash character (#) and continue to the end of
  the line.
- Whitespace is largely ignored. Lines may generally be split, concatenated
  or indented as preferred.
- Lists are usually specified as 'singular item' or 'plural { item item }', for
  example: 'user "nicholas"', 'users { "nicholas" "bob" }'.
- Regexps are specified as normal strings without additional adornment other
  than the "s (not wrapped in /s). All regexps are extended regexps and
  case insensitive by default.

Definition/option lines generally follow the following basic form:

	<keyword> <name or command> <parameters>

Example lines that may appear in a configuration file are:

	# This is a comment.

	set lock-types flock

	account "stdin" disabled stdin

	action "strip-full-disclosure"
		rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'"

	match "^X-Mailing-List:.*linux-kernel@vger.kernel.org" in headers
		or "^(To:|Cc:):.*@vger.kernel.org" in headers
		action "linux-kernel"

Including other files ----------------------------------------------------------

The fdm configuration may be split into several files. Additional files may
be referenced using the 'include' keyword:

	include "my-include-file.conf"

	include	"/etc/fdm.d/shared-conf-1.conf"

Macros -------------------------------------------------------------------------

Macros may be defined and used in the configuration file. fdm makes a
distinction between macros which may hold a number (numeric macros) and
those that hold a string (string macros). Numeric macros are prefixed with
the percentage sign (%) and string by the dollar sign ($). Macros are
defined using the equals operator (=):

	%nummacro = 123

	$strmacro = "a string"

Macros may then be referenced in either a standalone fashion anywhere a string
or number is expected, depending on the type of macro:

	$myfile = "a-file"
	include $myfile

	%theage = 12
	match age < %theage action "old-mail"

Or embedded in a string by enclosing the macro name in {}s:

	$myfile2 = "a-file2"
	include "/etc/${myfile}"

	%anum = 57
	include "/etc/file-number-%{anum}"

Invoking fdm ===================================================================

fdm may be invoked manually from the command line or regularly using a program
such as cron(8).

Command line arguments ---------------------------------------------------------

The fdm binary has the following synopsis:

fdm [-klmnv] [-f conffile] [-u user] [-a account] [-x account] [fetch | poll]

The meaning of the flags are covered in the fdm(1) man page, but a brief
description is given below. The flags are also mentioned at relevant points
in the rest of this document.

Flag	Meaning
-k 	Keep all mail (do not delete it from the server). This is useful for
	testing delivery rules without risking mailing ending up permanently
	in the wrong place.
-l	Log to syslog(3) using the 'mail' facility rather than outputting to
	stderr.
-m	Ignore the lock file.
-n	Run a syntax check on the configuration file and exit without fetching
	any mail.
-v	Print verbose debugging output. This option may be specified multiple
	times for increasing levels of verbosity. Useful levels are -vv to
	display the result of parsing the configuration file, and -vvvv to copy
	all traffic to and from POP3 or IMAP servers to stdout (note that -l
	disables this behaviour).
-f conffile
	Specify the path of the configuration file.
-u user
	Use 'user' as the default user for delivering mail when started as
	root.
-a account
	Process only accounts with a name matching the given pattern. Note that
	fnmatch(3) wildcards may be used to match multiple accounts with one
	option, and that the option may be specified multiple times.
-x account
	Process all accounts except those that match the given pattern. Again,
	fnmatch(3) wildcards may be used, and the -x option may be specified
	multiple times.

If -n is not specified, the flags must be followed by one of the keywords
'fetch' or 'poll'. The 'fetch' keyword will fetch and deliver mail, the 'poll'
keyword print an indication of how many mails are present in each account.
'fetch' or 'poll' may be abbreviated.

Examples:

	$ fdm -v poll

	$ fdm -vvnf /etc/my-fdm.conf

	$ fdm -lm -a pop3\* fetch

	$ fdm -x stdinacct fetch

	# fdm -u nicholas -vv f

Running from cron --------------------------------------------------------------

To fetch mail regularly, fdm must be run from cron. This line in a crontab(5)
will run fdm every 30 minutes:

*/30   *      *      *      *      /usr/local/bin/fdm -l fetch

The '-l' option sends fdm's output to syslog(3) rather than having cron mail
it. To keep a closer eye, adding '-v' options and removing '-l' will have
debugging output mailed by cron, or, using a line such as:

*/30   *      *      *      *      fdm -vvvv fetch >>/home/user/.fdm.log 2>&1

Will append extremely verbose fdm output to the ~/.fdm.log file. Note that this
log file can become pretty large, so another cronjob may be required to remove
it occasionally!

The lock file ------------------------------------------------------------------

fdm makes use of a lock file to prevent two instances running simultaneously.
By default, this lock file is .fdm.lock in the home directory of the user who
runs fdm, or /var/db/fdm.lock for root. This default may be overridden in
the configuration file with the 'set lock-file' command:

	set lock-file "/path/to/my/lock-file"

Or disabled altogether by being set to the empty string:

	set lock-file ""

The '-m' command line option may be used to force fdm to ignore the lock file
and run regardless of its existence and without attempting to create it.

Testing and debugging ----------------------------------------------------------

fdm has some features to assist with testing and debugging a ruleset:

The '-n' command line option. This is particularly useful in conjunction with
'-vv', for example:

	$ cat test.conf
	account "pop3" pop3 server "s" user "u" pass "p"
	action "rw" rewrite "sed 's/\\(Subject:.*\\)\\[XYZ\\]/\1/'"
	action "mbox" mbox "%h/INBOX"
	match all actions { "rw" "mbox" }
	$ fdm -vvnf test.conf
	version is: fdm 0.6 (20061204-1433)
	starting at: Tue Dec  5 15:45:41 2006
	user is: nicholas, home is: /home2/nicholas
	loading configuration from test.conf
	added account: name=pop3 fetch=pop3 server "s" port pop3 user "u"
	added action: name=rw deliver=rewrite "sed 's/\(Subject:.*\)\[XYZ\]/1/'"
	added action: name=mbox deliver=mbox "%h/INBOX"
	finished file test.conf
	added rule: actions="rw" "mbox" matches=all
	configuration loaded
	locking using: flock
	headers are: "to" "cc"
	domains are: "yelena"
	using tmp directory: /tmp

Looking at the output, the parsed strings used by fdm can be seen, and it is
possible to spot that an escape character has been missed in the command.

If '-vvvv' is used, fdm will print all data sent to and received from remote
servers to stdout. Note that this is disabled if the '-l' option is given, and
includes passwords, usernames and hostnames unmodified. The 'fdm-sanitize'
script provided with fdm may be used to remove passwords and usernames from
this output, either while it is being collected:

	fdm -vvvv -a testacct f 2>&1|./fdm-sanitize|tee my-output

Or afterwards:
	
	./fdm-sanitize <vvvv-output >my-output

Since fdm fetches multiple simultaneously, which may mix intersperse debugging
output, it is recommended to fetch each account seperately if running the output
through fdm-sanitize. If this is not done, it may not be able to detect all
username/password occurances.

The '-k' command line option (and the 'keep' keywords on actions and accounts,
covered later) prevent fdm from deleting any mail after delivery. This may be
used to perform any number of test deliveries without risk of losing mail.

Fetching mail ==================================================================

fdm fetches mail from a set of 'accounts', defined using the 'account'
keyword. Each account has a name, a type, a number of account specific
parameters and a couple of optional flags. The general form is:

	account <name> [<users>] [disabled] <type> [<parameters>] [keep]

The <name> item is a string by which the account is referred in filtering
rules, log output and for the '-a' and '-x' command line options.

The <users> portion specifies the default users to use when delivering mail
fetched from this account as root. It has the same syntax as discussed in
detail in the section below on defining actions.

If the optional 'disabled' keyword is present, fdm ignores the account unless
it is specified on the command line using the '-a' flag.

The optional 'keep' keyword instructs fdm to keep all mail from this account
(not delete it from the server) regardless of the result of the filtering
rules.

The <type> item may be one of: 'pop3', 'pop3s', 'imap', 'imaps', 'stdin',
'maildir' or 'maildirs'.

POP3 and POP3S -----------------------------------------------------------------

Mail may be fetched from a POP3 account. A POP3 account is defined by
specifying the following parameters: the server host and optionally port, and
the user name and password. If the port is not specified, the default port
('pop3' in the services(5) database) is used. Examples of such an account
definition are:

	account "pop3acct" pop3 server "pop.isp.com" user "bob" pass "pass"

	account "gmx" pop3 server "pop.gmx.net" port 110 user "jim" pass "pass"

	account "acct" pop3 server "10.0.0.1" port "pop3"
		user "nicholas" pass "my-password" keep

	account "lycos" disabled pop3 server $localserver port 10110
		user "bob@lycos.co.uk" pass "password"

Note that the server string is enclosed in double quotes even if it is an IP,
and don't forget to escape any " and \ characters in passwords!

POP3S is specified in exactly the same way, except using the 'pop3s' keyword
for the type, and the default port is 'pop3s' rather than 'pop3':

	account "pop3sacct" pop3s server "pop.isp.com" user "bob" pass "pass"

IMAP and IMAPS -----------------------------------------------------------------

IMAP and IMAPS accounts are defined using exactly the same syntax as for POP3
and POP3S, aside from using the 'imap' or 'imaps' keywords and that the default
port is 'imap' or 'imaps'. There is also an additional, optional 'folder'
option to specify the folder from which mail should be fetched. If omitted,
the folder defaults to the inbox.

Note that with IMAP and IMAPS, mail is still removed from the server unless the
'keep' option is given, or the '-k' command line option used.

Examples of IMAP and IMAPS accounts include:

	account "imapacct" imap server "imap.server.ca" user "bob" pass "pass"

	account "oldimap" disabled imaps server "192.168.0.1" port 10993
		user "nicholas" pass "pass" folder "Saved"

	account "workspam" disabled imap server "my-work.ath.cx"
		user "Nicholas" pass "pass" folder "Junk"

IMAP over a pipe or ssh --------------------------------------------------------

Mail may be fetched using IMAP via a pipe. This is particularly useful for
fetching mail over ssh using public keys.

An IMAP user and password may be supplied, but fdm will only use them if
required. If the connection is preauthenticated, the user and password are
unnecessary.

Communication takes place via the pipe program's stdin and stdout. If any
output is found on stderr, fdm will print it (or log it with '-l').

Examples are:

	account "imapssh" imap pipe "ssh jim@myhost /usr/local/libexec/imapd"
	
	account "imapssh2" imap pipe "/usr/bin/whatever" user "bob" pass "bah"

Stdin and local mail -----------------------------------------------------------

fdm may be configured to fetch mail from stdin, by specifying an account of
type 'stdin', for example:

	account "stdin" disabled stdin

This is most useful to have fdm behave as a mail delivery agent. To configure
it for single-user use with sendmail, the simplest method it to add:

	"|/usr/local/bin/fdm -m -a stdin fetch"

To the user's ~/.forward file (including the double quotes). Note the use of
'-m' to prevent stdin delivery from interfering with any normal cronjob, and
'-a' to specify that only the disabled "stdin" account should be fetched.

From maildirs ------------------------------------------------------------------

Fetching from maildirs allows fdm to be used to filter mail on the local
machine. This is covered more detail in the later section on archiving and
searching.

Maildir accounts are specified as follows:

	account "mymaildir" maildir "/path/to/dir"

	account "mymaildirs" maildirs { "/path/to/dir1" "/path/to/dir2" }

Shell glob wildcards may be included in the path names to match multiple
maildirs, but every directory found must be a valid maildir.

Using NNTP ---------------------------------------------------------------------

fdm can fetch news messages from a news server using NNTP. News accounts are
specified like so:

	account "news1" nntp server "news.server.sk" port 119
		group "comp.unix.bsd.openbsd.misc" cache "%h/.fdm.cache/%s"

	account "mynews" nntp server "my.news.server" port "nntp"
		groups { "alt.test" "alt.humor.best-of-usenet" }
		cache "%h/.fdm.cache"

The cache is a file used to store details of the last article fetched. If only
one group is supplied in the account definition, %s tokens are replaced by the
name of the group in the cache path. If multiple groups are provided, %s tokens
are just removed.

Note that whether a message is kept or deleted is irrelevent to NNTP, articles
are always left on the server. The index and message-id of the last article
is recorded in the cache file so that older articles are skipped when the a
newsgroup is again fetched.

For rules to match based on group name, the %s token may again be used. This
can ensure articles are matched purely on the group they are fetched on (trying
to do this using headers is unreliable with cross-posted articles). For example:

match all account "news" {
	match string "%s" to "comp.lang.c" action "news-%s"
	match string "%s" to "comp.std.c" action "news-%s"
	match all action "drop"
}

Defining actions ===============================================================

An action is a particular command to execute on a mail when it matches a
filtering rule (see the next section on filtering mail). Actions are named,
similar to accounts, and have a similar form:

	action <name> [<users>] <type> <parameters>

The <users> item may be either:

- the keyword 'user' followed by a single username string or uid, such as:

	user "nicholas"

	user 1000

- the keyword 'users' followed by a list of users in {}s, for example:

	users { 1001 "nicholas" }

- the keyword 'from-headers', that will attempt to ascertain a set of users
  from the mail headers (see the 'domains' and 'headers' options in the
  section on setting options).

If users are specified, the action will be run once for each user, with fdm
changing to that user before executing the action. Note that fdm will execute
the action once for each user even when not started as root, but will not be
able to change to the user. The user keyword is primarily of use in multiuser
configurations. If users are present on an action, they override any specified
by the account defintion.

If running as root and no user is specified on either the action or on the
filtering rule (see the section on filtering below), the default user is
used, see the '-u' command line option and the 'default-user' option in the
setting options section

Drop and keep actions ----------------------------------------------------------

The simplest actions are the 'drop' and 'keep' actions. They have no parameters
and are specified like this:

	action "mydropaction" drop

	action "mykeepaction" keep

The 'drop' action arranges for mail to be dropped when rule evaluation is
complete. Note that using 'drop' does not stop further evaluation if the
filtering rule contains a 'continue' keyword, and it may be overridden by a
'keep' option on the account or by the '-k' flag on the command line.

The 'keep' action is similar to 'drop', but it arranges for the mail to be
kept once rule evaluation is complete, rather than dropped.

Maildirs ----------------------------------------------------------------------

Mails may be saved to a maildir through a 'maildir' action, defined like so:

	action "mymaildiraction" maildir "/path/to/maildir"

In the path, the following special tokens are replaced:

Token	Replaced with
%a	The name of the account this mail was fetched from.
%s	A string specific to the type of account the mail was fetched from.
	For maildirs, it is the basename of the maildir, for IMAP and POP3 it
	is the server hostname as specified in the account definition. For
	NNTP it is the name of the group.
%h	The delivery user's home directory.
%n	The delivery user's uid.
%t	The name of the action.
%u	The delivery user's username.
%H	The current hour (00-23).
%M	The current minute (00-59).
%S	The current second (00-59).
%d	The current day of the month (00-31).
%m	The current month (01-12).
%y	The current year.
%W	The current day of the week (0-6, Sunday is 0).
%Y	The current day of the year (000-365).
%Q	The current quarter (1-4).
%1-%9	The result of any bracket expressions in the last regexp (see later
	section on regexps).
%%	Replaced with a literal %.

If the maildir does not exist, it is created.

Mboxes -------------------------------------------------------------------------

An action to deliver to an mbox is defined in the same way as for a maildir:

	action "mymboxaction" mbox "/path/to/mbox"

The same % tokens are replaced in the path. If the mbox does not exist, it
is created. Mboxes may optionally be gzip compressed by adding the 'compress'
keyword:

	action "mymboxaction" mbox "/path/to/mbox" compress

fdm will append .gz to the mbox path (if it is not already present) and append
compressed data. If the mbox exists but is not already compressed, uncompressed
data will be appended.

SMTP ---------------------------------------------------------------------------

An action may be defined to pass mail on over SMTP. The server host must be
specified and optionally the port and a string to pass to the server with
the RCPT TO command. If the port is not specified it defaults to "smtp".
Examples include:

	action "mysmtpaction" smtp server "smtp.server"

	action "mysmtpaction" smtp server "smtp.server" port 587

	action "mysmtpaction" smtp server "smtp.server" port "submission"

	action "mysmtpaction" smtp server "smtp.server" to "me@somewhere"

Write, pipe and append ---------------------------------------------------------

Actions may be defined to write or append a mail to a file, or to pipe it to
a shell command. The append action appends to and write overwrites the file.
% tokens are replaced in the file or command as for maildir and mbox actions.
Examples are:

	action "mywriteaction" write "/tmp/file"

	action "myappendaction" append "/tmp/file"

	action "mypipeaction" pipe "cat > /dev/null"

Rewriting mail -----------------------------------------------------------------

Mail may be altered by passing it to a rewrite action. This is similar to
the pipe action, but the output of the shell command to stdout is reread by fdm
and saved as a new mail. This is useful for such things as passing mail
through a spam filter or removing or altering headers with sed. Note that
rewrite only makes sense on filtering rules where the continue keyword is
specified, or where multiple actions are used (see the next section for details
of this). Possible rewrite action definitions are:

	action "myspamaction" rewrite "bmf -p"

	action "mysedaction" rewrite "sed 's/x/y/'"

Filtering mail =================================================================

Mail is filtered by defining a set of filtering rules. These rules tie together
mail fetched from an account and passed to one or more actions. Rules are
evaluated from top-to-bottom of the file, and evaluation stops at the first
matching rule (unless the continue keyword is specified).

The general form of a filtering rule is:

	match <conditions> [<accounts>] [<users>] <actions> [continue]

The optional <users> item is specified as for an action definition. If users
are specified on a filtering (match) rule, they override any specified on the
action or account.

The <conditions> item is set of conditions against which the match may be
specified, each condition returns true or false. Conditions are described in
the next few sections. Aside from the 'all' condition, which is a special case,
conditions may be chained as an expression using 'and' and 'or', in which case
they are evaluated from left to right at the same precedence, or prepended with
'not' to invert their outcome.

The optional <accounts> item is a list of accounts for which the rule provided
applies. It is specified as either a single account ('account "name"') or a list
of accounts ('accounts { "name1" "name2" }'). fnmatch(3) wildcards may also be
used.

The <actions> item is a list of actions to execute when this rule matches.
It is in the same list format: 'action "name"' or "actions { "name1" "name2" }'.

If a rule with the 'continue' keyword matches, evaluation does not stop after
the actions are executed, instead subsequent rules are matched.

Nesting rules ------------------------------------------------------------------

Filtering rules may be nested by using the special form:

	match <conditions> [<accounts>] {
		match ...
	}

If the conditions on the outer rule match, the inner rules are evaluated. If
none of the inner rules match (or they all specify the 'continue' keyword)
evaluation continues outside to rules following the nested rule, otherwise it
stops.

The all condition --------------------------------------------------------------

The all condition matches all mail. It may not be used as part of an expression
with the 'and', 'or' and 'not' operators.

Examples include:

	match all action "default"

	match all rewrite "rewaction" continue

Matched and unmatched ----------------------------------------------------------

The matched and unmatched conditions are used to match mail that has matched
or has not matched previous rules and been passed on with the 'continue'
keyword. For example,

	match all action "act1" continue
	# This rule will match only mails that also matched the first.
	match matched action "act2"
	# This rule will match only mails that matched neither of the first two.
	match unmatched action "act3"

Matching a regexp --------------------------------------------------------------

Matching against a regexp is the most common form of condition. It takes the
following syntax:

	[case] <regexp> [in headers|in body]

The 'case' keyword instructs fdm to match the regexp case sensitively rather
than the default of case insensitivity. The 'in headers' or 'in body' keywords
in struct fdm to search only the headers or body of each mail, the default is
to match the regexp against the entire mail. The regexp itself is an extended
regexp specified as a simple string, but care must be taken to escape \s and
"s properly.

Examples include:

	match "^From:.*bob@bobland\\.bob" in headers account "pop3" action "act"

	match ".*YAHOO.*BOOTER.*" in body action "junk"

Matching bracket expressions ---------------------------------------------------

The results of any bracket expressions within the last regexp are remembered,
and may be made use of using the 'string' condition, or used to construct an
action name, maildir or mbox path, etc. The bracket expressions may be
substituted using the %1 to %9 tokens. For example,

	match regexp "^From:.*([a-z]*)@domain" in headers action "all" continue
	match string "%1" to "bob" action "bobmail"

Or:

	match regexp "^From:.*([a-z]*)@domain" in headers action "all" continue
	match all action "%1mail"

This is particularly useful in combination with nested rules (see later):
bracket expressions in regexp on the outer rule may be compared on inner
rules.

Matching by age or size --------------------------------------------------------

Mail may be matched based on its age, or its size. An age condition is
specified as follows:

	age [<|>] <age> [hours|minutes|seconds|days|months|years]

If '<' is used, mail is matched if it is older than the specified age. If '>',
if it is younger. The <age> item may be a simple number of seconds, or suffixed
with a unit. Examples are:

	match age < 3 months actions { "act1" "act2" }

	match age > 100 hours action "tooold"

The size condition is similar:

	size [<|>] <size>

Where <size> is a simple number in bytes, or suffixed directly (no space) with
'K', 'M' or 'G' to specify a size in kilobytes, megabytes or gigabytes, such
as:

	match size < 1K action "small"

	match size > 2G action "whoa"

Using a shell command ----------------------------------------------------------

Mail may be matched using the result of a shell command. This condition follows
the form:

	[exec|pipe] <command> returns (<return code>, <stdout regexp>)

If 'exec' is used, the command is executed. If 'pipe', the mail is piped to the
command's stdin. The <command> is a simple string. % tokens are replaced as
normal, with the exception of %t.

Any of the <return code> or <stdout regexp> or both may be specified. The
<return code> is a simple number which is compared against the return code from
the command, the <stdout regexp> is a regexp that is matched case insensitively
against each line output by the command on stdout. Any output on stderr is
logged by fdm, so 2>&1 must be included in the command in order to apply the
regexp to it too.

Examples:

	match exec "true" (0, ) action "act"

	match not pipe "grep Bob" (1, ) action "act"

	match pipe "myprogram" (, "failed") actions { "act1" "act2" }

	match exec "blah" (12, "^Out") action "meep"

Attachments---------------------------------------------------------------------

There are five conditions available to filter based on the size, quantity, type
and name of attachments. Thay all specified prefixed with the 'attachment'
keyword. Two compare the overall number of attachments:

The 'attachment count' conditions matches if the number of attachments is
equal to, not equal to, less than or greater than the specified number:

	match attachment count == 0 action "action"

	match attachment count != 10 action "action"

	match attachment count < 2 action "action"

	match attachment count > 7 action "action"

The 'attachment total-size' condition is similar, but compares the total
size of all the attachments in a mail:

	match attachment total-size < 4096 kilobytes action "action"

	match attachment total-size > 1M action "action"

There are also three conditions which matches if any individual attachment
fulfils the condition: 'any-size' to match if any attachment is less than or
greater than the given size, and 'any-type' and 'any-name' which compare the
attachment MIME type and name attribute (if any) using fnmatch(3):

	match attachment any-size < 2K action "action"

	match attachment any-type "*/pdf" action "action"

	match attachment any-name "*.doc" action "action"

Tagging ------------------------------------------------------------------------

Mails may be assigned one of more tags using a special form of filtering
rule with the following syntax:

	match <conditions> [<accounts>] tag <tag>

The string <tag> is attached to the mail. A mail may have multiple tags.

The tag may be tested for using the 'tagged' condition:

	match tagged "mytag" action "a"

	match tagged "ohno" and size >1K action "drop"

Setting options ================================================================

fdm has a number of options that control its behaviour. These are defined
using the set command:

	set <option> [<value>]

The possible options are:

- maximum-size <size>

This specifies the maximum size of mail that fdm will accept. fdm uses /tmp
(by default) to save mails, it is recommended that this be less than the
size of the /tmp partition (less than half the size if using rewrite). If
mail over this size is encountered, fdm will abort with an error, without
deleting the mail from the server (unless 'delete-oversized' is set).

- delete-oversized

If this option is set, fdm will delete oversized mail from the server rather
than leaving it for the user to sort out.

- allow-multiple

This option makes fdm ignore the lock file, similar to the '-m' command line
option.

- lock-file <path>

This option specifies string which should be the path of the file to use as the
lock file. For example:

	set lock-file "/tmp/my-lock-file"

- default-user <user>

This specifies the default user to use if run as root and no users are specified
on the action or filtering rule. This option may be overriden with the '-u'
flag on the command line.

- lock-types [fcntl] [flock] [dotlock]

These specify the type of locking to use when writing to mboxes. fcntl and
flock locking is mutually exclusive, but dotlock may be used with either.
Some NFS servers do not support fcntl. The default is flock only. Example:

	set lock-types fcntl dotlock

- [domain|domains] <domains>

This specifies a domain or list of domains to use when searching for users in
mail headers with the 'from-headers' keyword. Examples include:

	set domain "xyz.ath.cx"

	set domains { "abc.ca" "def.co.uk" }

The default is the hostname of the local machine.

- [header|headers] <headers>

This specifies a header or list of headers to search with the 'from-headers'
keyword, such as:

	set header { "to" "cc" }

	set headers "x-envelope-to"

The default is "to" "cc".

- proxy <url>

This specifies a URL to proxy outgoing connections through. See the section
on proxying below.

- unmatched-mail [drop|keep]

This option controls how fdm should deal with mail that reaches the end of
the ruleset (doesn't match any rules, or only rules with the 'continue'
keyword). If 'drop' is specified, such mail is dropped. If 'keep', it is
kept. The default is to keep the mail, and issue a warning.

In addition to these options, some environment variables may be used to control
fdm. If TMPDIR is present, its value will be used instead of /tmp for saving
temporary files.

- purge-after <message count>

The 'purge-after' option instructs fdm to attempt to purge deleted mail after
the specified number of mails has been fetched. This is useful to limit the
number of mails refetched on the next run if the connection fails. It can have
a large effect on performance, particularly if the message count is set to a
low number.

Note that for POP3, purging deleted mail involves connecting and reconnecting
from the server; some POP3 servers refuse reconnections if too many are made too
quickly, so this option should be used with care.

Archiving and searching mail ===================================================

As fdm can fetch from maildirs, it can be used to filter mail from one maildir
into another, for example to archive older mail (with drop) or to search for
mail matching a particular pattern (with keep). The 'age' condition, and the
%s and time/date % tokens are particularly useful for archiving. For example,
to archive all mail older than 30 days by quarter, something like this may
suffice (obviously, the account restriction can be dropped if being used in
a single-purpose configuration file):

	account "archive" disabled maildir "source-maildir"
	action "archive" mbox "%sq%Q"
	match age > 30 days account "archive" action "archive"
	match all account "archive" action "keep"

Then, fdm may be run to move the mail:

	fdm -vaarchive fetch

To search mail, similar rules may be used, but all mail should be kept, in this
example by marking the account with 'keep' so that mail is kept no matter what
rules it matches:

	account "search" disabled maildir "source-maildir" keep
	action "found" mbox "search-results"
	match "^From.*Bob" in headers account "search" action "found"
	match all account "search" action "keep"

All mail matching the regexp will be copied to the target mbox. There are
several other ways to write this ruleset and achieve the same end.

Using fdm behind a proxy =======================================================

fdm may be used behind a proxy by specifying the proxy URL using the 'proxy'
option. HTTP and SOCKS5 proxies are supported:

	set proxy "http://proxy.server/"

	set proxy "http://proxy.server:port/"

	set proxy "socks5://proxy.server/"

	set proxy "socks5://proxy.server:port/"

	set proxy "socks5://user@proxy.server/"

	set proxy "socks5://user:pass@proxy.server/"

Authentication is not support for HTTP proxies.

Bug reports and queries ========================================================

Bug reports, queries, suggestions and code are best sent by email to:

	nicm@users.sf.net

Bug reports may also be registered on Sourceforge, but they are likely to be
dealt with much faster by email, particularly as it is easier to elicit
further information if it is required.

Frequently asked questions =====================================================

Why? ---------------------------------------------------------------------------

Two main reasons:

1. I didn't like the existing tools. I'm not going to do my fetchmail rant
   since getmail's Charles Cazabon has done his own in greater detail here:

	http://pyropus.ca/software/getmail/faq.html#faq-about-why

   I'll also spare you my procmail rant, but suffice to say the syntax is
   abysmal. getmail and maildrop I was less unhappy with overall, but I was
   not really comfortable with their configuration file formats. Plus getmail
   always seemed very slow. And there is reason 2...

2. I disliked the fact that as a home user with a relatively simple setup I
   had to have five programs to deal with mail: sendmail, fetchmail, procmail,
   archivemail, mutt; all with different, variously broken configuration file
   syntaxes, and weird quirks. I now have three programs: sendmail, fdm and
   mutt, and fewer weird configurations to learn and potential problems.

How do I write regexps? --------------------------------------------------------

See the re_format(7) man page. It is online here (for OpenBSD):

	http://www.openbsd.org/cgi-bin/man.cgi?re_format

There are also a number of books and probably websites on the subject.

Keep doesn't work with gmail! --------------------------------------------------

This is because gmail is broken, see:

	http://pyropus.ca/software/getmail/faq.html#faq-notabug-gmail-bug

Gmail also consistently gives false sizes when asked using the POP3 LIST
command, which fdm will warn about.

Why doesn't fdm run as a daemon? -----------------------------------------------

Because that is what cron is for: it comes as standard with all sensible
operating systems, and by using it I avoid a lot of code to deal with signals,
configuration reload, extra configuration options and general other crap
related to running all the time. Daemonising is for servers, for stuff that
needs to run periodically, use cron.

Why does fdm fork child processes when not running as root? --------------------

Because one design is much cleaner and easier to work with than two. And it
has a negligible practical effect on performance in any case.

Can fdm get rid of that crap in []s added to subjects on some mailing lists? ---

Yes! Rewrite the mail using sed to remove the crap, for example:

action "full-disclosure" maildir "%h/mail/full-disclosure"
action "strip-full-disclosure"
	rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'"

match "^List-Id:.*<full-disclosure\\.lists\\.grok\\.org\\.uk>" in headers
	actions { "strip-full-disclosure" "full-disclosure" }

================================================================================
$Id: MANUAL,v 1.38 2007/01/25 17:42:57 nicm Exp $

