Postfix ETRN Howto

-------------------------------------------------------------------------------

Purpose of the Postfix fast ETRN service

The SMTP ETRN command was designed for sites that have intermittent Internet
connectivity. With ETRN, a site can tell the mail server of its provider to
"Please deliver all my mail now". The SMTP server searches the queue for mail
to the customer, and delivers that mail by connecting to the customer's SMTP
server. The mail is not delivered via the connection that was used for sending
ETRN.

Postfix versions before 1.0 (also known as version 20010228) implemented the
ETRN command in an inefficient manner: they simply attempted to deliver all
queued mail. This is slow on mail servers that queue mail for many customers.

As of version 1.0, Postfix has a fast ETRN implementation that does not require
Postfix to examine every queue file. Instead, Postfix maintains a record of
what queue files contain mail for destinations that are configured for ETRN
service. ETRN service is no longer available for domains that aren't configured
for the service.

This document provides information on the following topics:

  * Using the Postfix fast ETRN service
  * How Postfix fast ETRN works
  * Postfix fast ETRN service limitations
  * Configuring the Postfix fast ETRN service
  * Configuring a domain for ETRN service only
  * Testing the Postfix fast ETRN service

Other documents with information on this subject:

  * flush(8), flush service implementation

Using the Postfix fast ETRN service

The following is an example SMTP session that shows how an SMTP client requests
the ETRN service. Client commands are shown in bold font.

    220 my.server.tld ESMTP Postfix
    helo my.client.tld
    250 Ok
    etrn some.customer.domain
    250 Queuing started
    quit
    221 Bye

As mentioned in the introduction, the mail is delivered by connecting to the
customer's SMTP server; it is not sent over the connection that was used to
send the ETRN command.

The Postfix operator can request delivery for a specific customer by using the
command "sendmail -qRdestination" and, with Postfix version 1.1 and later,
"postqueue -sdestination".

How Postfix fast ETRN works

When a Postfix delivery agent decides that mail must be delivered later, it
sends the destination domain name and the queue file name to the flush(8)
daemon which maintains per-destination logfiles with file names of queued mail.
These logfiles are kept below $queue_directory/flush. Per-destination logfiles
are maintained only for destinations that are listed with the
$fast_flush_domains parameter and that have syntactically valid domain names.

    Postfix                        Postfix               One logfile
    delivery -(domain, queue ID)->  flush  -(queue ID)-> per eligible
     agent                         daemon                   domain

When Postfix receives a request to "deliver mail for a domain now", the flush
(8) daemon moves all deferred queue files that are listed for that domain to
the incoming queue, and requests that the queue manager deliver them. In order
to force delivery, the queue manager temporarily ignores the lists of
undeliverable destinations: the volatile in-memory list of dead domains, and
the list of message delivery transports specified with the defer_transports
configuration parameter.

Postfix fast ETRN service limitations

The design of the flush(8) server and of the flush queue introduce a few
limitations that should not be an issue unless you want to turn on fast ETRN
service for every possible destination.

  * The flush(8) daemon maintains per-destination logfiles with queue file
    names. When a request to "deliver mail now" arrives, Postfix will attempt
    to deliver all recipients in the queue files that have mail for the
    destination in question. This does not perform well when queue files have
    recipients in many different domains.

  * The flush(8) daemon maintains per-destination logfiles only for
    destinations listed with $fast_flush_domains. With other destinations it
    not possible to trigger delivery with "sendmail -qRdestination" or, with
    Postfix version 1.1 and later, "postqueue -sdestination".

  * Up to and including early versions of Postfix version 2.1, the "fast flush"
    service may not deliver some messages if the request to "deliver mail now"
    is received while a deferred queue scan is already in progress. The reason
    is that the queue manager does not ignore the volatile in-memory list of
    dead domains, and the list of message delivery transports specified with
    the defer_transports configuration parameter.

Configuring the Postfix fast ETRN service

The behavior of the flush(8) daemon is controlled by parameters in the main.cf
configuration file.

By default, Postfix "fast ETRN" service is available only for destinations that
Postfix is willing to relay mail to:

    /etc/postfix/main.cf:
        fast_flush_domains = $relay_domains
        smtpd_etrn_restrictions = permit_mynetworks, reject

Notes:

  * The relay_domains parameter specifies what destinations Postfix will relay
    to. For destinations that are not eligible for the "fast ETRN" service,
    Postfix replies with an error message.

  * The smtpd_etrn_restrictions parameter limits what clients may execute the
    ETRN command. By default, any client has permission.

To enable "fast ETRN" for some other destination, specify:

    /etc/postfix/main.cf:
        fast_flush_domains = $relay_domains, some.other.domain

To disable "fast ETRN", so that Postfix rejects all ETRN requests and so that
it maintains no per-destination logfiles, specify:

    /etc/postfix/main.cf:
        fast_flush_domains =

Configuring a domain for ETRN service only

While an "ETRN" customer is off-line, Postfix will make spontaneous attempts to
deliver mail to it. These attempts are separated in time by increasing time
intervals, ranging from $minimal_backoff_time to $maximal_backoff_time, and
should not be a problem unless a lot of mail is queued.

To prevent Postfix from making spontaneous delivery attempts you can configure
Postfix to always defer mail for the "ETRN" customer. Mail is delivered only
after the ETRN command or with "sendmail -q", with "sendmail -qRdomain", or
with "postqueue -sdomain"(Postfix version 1.1 and later only),

In the example below we configure an "etrn-only" delivery transport which is
simply a duplicate of the "smtp" and "relay" mail delivery transports. The only
difference is that mail destined for this delivery transport is deferred as
soon as it arrives.

     1 /etc/postfix/master.cf:
     2   # =============================================================
     3   # service type  private unpriv  chroot  wakeup  maxproc command
     4   #               (yes)   (yes)   (yes)   (never) (100)
     5   # =============================================================
     6   smtp      unix  -       -       n       -       -       smtp
     7   relay     unix  -       -       n       -       -       smtp
     8   etrn-only unix  -       -       n       -       -       smtp
     9
    10 /etc/postfix/main.cf:
    11   relay_domains = customer.tld ...other domains...
    12   defer_transports = etrn-only
    13   transport_maps = hash:/etc/postfix/transport
    14
    15 /etc/postfix/transport:
    16   customer.tld     etrn-only:[mailhost.customer.tld]

Translation:

  * Line 8: The "etrn-only" mail delivery service is a copy of the "smtp" and
    "relay" service.

  * Line 11: Don't forget to authorize relaying for this customer, either via
    relay_domains or with the permit_mx_backup feature.

  * Line 12: The "etrn-only" mail delivery service is configured so that
    spontaneous mail delivery is disabled.

  * Lines 13-16: Mail for the customer is given to the "etrn-only" mail
    delivery service.

  * Line 16: The "[mailhost.customer.tld]" turns off MX record lookups; you
    must specify this if your Postfix server is the primary MX host for the
    customer's domain.

Testing the Postfix fast ETRN service

By default, "fast ETRN" service is enabled for all domains that match
$relay_domains. If you run Postfix with "fast ETRN" service for the very first
time, you need to run "sendmail -q" once in order to populate the per-site
deferred mail logfiles. If you omit this step, no harm is done. The logfiles
will eventually become populated as Postfix routinely attempts to deliver
delayed mail, but that will take a couple hours. After the "sendmail -q"
command has completed all delivery attempts (this can take a while), you're
ready to test the "fast ETRN" service.

To test the "fast ETRN" service, telnet to the Postfix SMTP server from a
client that is allowed to execute ETRN commands (by default, that's every
client), and type the commands shown in boldface:

    220 my.server.tld ESMTP Postfix
    helo my.client.tld
    250 Ok
    etrn some.customer.domain
    250 Queuing started

where "some.customer.domain" is the name of a domain that has a non-empty
logfile somewhere under $queue_directory/flush.

In the maillog file, you should immediately see a couple of logfile records, as
evidence that the queue manager has opened queue files:

    Oct  2 10:51:19 myhostname postfix/qmgr[51999]: 682E8440A4:
        from=<whatever>, size=12345, nrcpt=1 (queue active)
    Oct  2 10:51:19 myhostname postfix/qmgr[51999]: 02249440B7:
        from=<whatever>, size=4711, nrcpt=1 (queue active)

What happens next depends on whether the destination is reachable. If it's not
reachable, the mail queue IDs will be added back to the some.customer.domain
logfile under $queue_directory/flush.

Repeat the exercise with some other destination that your server is willing to
relay to (any domain listed in $relay_domains), but that has no mail queued.
The text in bold face stands for the commands that you type:

    220 my.server.tld ESMTP Postfix
    helo my.client.tld
    250 Ok
    etrn some.other.customer.domain
    250 Queuing started

This time, the "ETRN"" command should trigger NO mail deliveries at all. If
this triggers delivery of all mail, then you used the wrong domain name, or
"fast ETRN" service is turned off.

Finally, repeat the exercise with a destination that your mail server is not
willing to relay to. It does not matter if your server has mail queued for that
destination.

    220 my.server.tld ESMTP Postfix
    helo my.client.tld
    250 Ok
    etrn not.a.customer.domain
    459 <not.a.customer.domain>: service unavailable

In this case, Postfix should reject the request as shown above.

