Tutorial on Sendmail

*** Note: ALWAYS start Sendmail with an absolute pathname, or it may not function properly due to relativity.


REQUIREMENTS for Sendmail:

- The FULLY QUALIFIED name of the machine running sendmail has to be in DNS or /etc/hosts
- When sending straight to a domain (user@somedomain.com), the MX (mail exchanger) record is being looked up.
An example is: (partial output of "dig MX somedomain.com")

;; ANSWER SECTION:
somedomain.com. 86374 IN MX 10 mail.somedomain.com.

This means that all mail for somedomain.com will be routed to machine mail.somedomain.com.
10 is the cost value. The lower it is, the higher the preference of this particular MX record.
If there are 2+ servers with the same cost value, DNS will rotate them using round robin.
If there are 2+ servers with different cost values, the one with the lowest value will get the mail, unless it is down.
If all MX servers are down, the message is queued for lated delivery.


INSTALLATION:

1. Download the latest non-beta version from ftp.sendmail.org (or www.sendmail.org).
2. gzip -dc sendmail* | tar xvf -
3. cd sendmail*
4. Read README, RELEASE_NOTES (optional), FAQ (optional), and doc/op/op.ps (optional)
5. cd sendmail
6. ./makesendmail -m <-- make sure our system is supported, don't write anything
or
6. ./makesendmail -n <-- same, however create ../obj.*/
7. ./makesendmail <-- this compiles sendmail (need compiler -- gcc, etc)
8. cd ../obj.*/sendmail
9. ./sendmail -d0.1 -bt < /dev/null <-- check for errors (and for NEWDB)
10. make install <-- if everything looks good, install the new binaries

If NEWDB is not found, download the Berkeley Database from www.sleepycat.com, install it, and recompile sendmail

(optional) Installing Berkeley DB:
- unzip + untar
- cd db-*
- cd build_unix
- ../dist/configure (or, CC=gcc ../dist/configure)
- make
- make install

Then unzip + untar sendmail again, and start from step 6:

6. ./makesendmail -m -I/usr/local/BerkeleyDB.4.0/include -L/usr/local/BerkeleyDB.4.0/lib
or
6. ./makesendmail -n -I/usr/local/BerkeleyDB.4.0/include -L/usr/local/BerkeleyDB.4.0/lib
7. ./makesendmail
8. cd ../obj.*/sendmail
9. ./sendmail -d0.1 -bt < /dev/null <-- now you should see NEWDB in the list
10. make install <-- if everything looks good, install the new binaries

* Sendmail is installed by default at /usr/lib/sendmail.

*** If upgrading from a previous version of Sendmail, make sure the mail queue is empty before switching binaries (mailq).

* Command line options:

-ba use ARPAnet protocols
-bD run as a daemon -- don't use fork
-bd run as a daemon command equivalent: smtpd
-bH purge persistent host status command equivalent: purgestat
-bh print persistent host status command equivalent: hoststat
-bi rebuild alias database command equivalent: newaliases
-bm be a mail sender
-bp print the mail queue command equivalent: mailq
-bt test mode: resolve addresses only
-bv verify: don't collect or deliver

* The default configuration file (when running as a daemon) is /etc/mail/sendmail.cf
* The default pid file is /var/run/sendmail.pid (/etc/sendmail.pid on older versions)

* Troubleshooting by manually sending mail:
- /usr/lib/sendmail -v <recipient> <-- verbose
- /usr/lib/sendmail -d <recipient> <-- debug (even more verbose)


*** Alias database -- used for forwarding mail, setting up simple mailing lists, etc:
(the default alias file is /etc/aliases (symlink to /etc/mail/aliases).
After editing, run "newaliases" to activate the changes)

* Majordomo or GNU's Mailman are generally more intelligent solutions for managing mailing lists.

- Typical alias example -- mail for "webmaster" will be sent to "tweety" (note that "tweety" may be another alias):
webmaster: tweety

- Mail sent to "webmaster" will go to remote user "tweety@yahoo.com":
webmaster: tweety@yahoo.com

- Create a simple list:
accountants: tweety, sylvester, grandma

- Combine two existing lists into a third one:
list1: tweety, sylvester
list2: grandma, root
list3: list1, list2

- If you don't want "tweety" to be run against the alias database again:
list1: \tweety, sylvester

- Create an owner for a mailing list. That user will receive error messages (bounces, etc):
list1: tweety, sylvester
owner-list1: postmaster

- Set the same owner for all existing mailing lists:
owner-owner: postmaster <-- general rule
owner-list1: tweety <-- exceptions

- Send messages to a file:
webmaster: /export/home/tweety/messages

- Send messages to a program:
webmaster: |/full_path/name_of_program

- Use an include file for managing messages. It can contain usernames, filenames, or programs:
webmaster: :include:/full_path/filename

* Aliases can be kept in sync across a network via the use of NIS, NIS+, or LDAP
* The "postmaster" alias is always required.
* For preprocessing (filtering) mail, you should consider procmail.


*** SECURITY:

* Sendmail by default constructs the following environment variables for external programs:
- agent variable set to "sendmail"
- TZ
- ISP
- SYSTYPE

For setting additional variables ("path" for example), an "E" variable can be used in sendmail.cf:
Epath=/usr/bin:/bin


* Simple security check: telnet to port 25 and type "debug":
- If you get "500 command unrecognized" -- server is not vulnerable.
- If you get "200 Debug set" -- server is vulnerable. Then, you can type "showq" to list the mail queue, etc.

Other commands you can use when connected to port 25:
- vrfy <username> -- checks if the user exists.
- expn (short for expand) -- expand the name of a mailing list and see the recipients.
- check "P" lines in sendmail.cf - deliery agent (P=/bin/mail). If P=<non_standard_agent> (like /tmp/mail) --> suspicious.
- securing .forward files -- change "P=/bin/sh" to "P=/usr/lib/smrsh", and set "A=smrsh". Then, only stuff residing
in /var/adm/sm.bin/ will be executed. If the program is not found there, the message bounces.
- have /etc/, /etc/mail/, and /etc/mail/* owned by root and writeable only by root.
- be careful with permissions of the include files, if those are used.
- when sending e-mail to a program, you should be careful of things like setuid and setgid as well as write permissions.
- check the alias file (/etc/aliases by default) to make sure that no mail is being sent to programs.
- protect the mail queue directory -- /var/, /var/spool/, and /var/spool/mqueue/ should be writable only by root.
- examine sendmail.cf for "Ct" flag. Any user listed here can hide their username using "sendmail -f <new_name> ..."
Also, the "T" flag has the same effect. These users should be in the "Trusted Users" section of the config file.
Also, the "Ft" flag has the same effect, however it simply lists an external file containing trusted users.
- you can adjust the log verbosity of sendmail by modifying the LogLevel setting.
- you can edit the PostmasterCopy option in sendmail.cf -- send every bounced message to a specific user.
- restrict the list of people able to use "mailq" -- add "restrictmailq" to PrivacyOptions.
- restrict who can process the queue -- add "restrictqrun" to PrivacyOptions.
- prevent mail being sent to special files (devices, etc) -- set "SafeFileEnvironment=/"
- secure temporary files -- set "TempFileMod=0600".


*** THE QUEUE:

* Sendmail does all operations relative to the queue directory. Therefore, relative paths, etc, have to match.
- change the default queue directory -- set the QueueDirectory option in sendmail.cf
- ordinary users should not even be allowed to enter the queue directory.
- 2 types of files in the queue directory -- df<queue_ID> (mail contents), and qf<queue_ID> (headers, etc).
- process the queue manually -- "/usr/lib/sendmail -q".
- while processing the queue, sendmail renames the "qf" file to a "tf" file, edits it, and renames it back to "qf".
- when a message is sent to multiple recepients, an "xf" file is also created -- contains delivery results.

- Start sendmail with "/usr/lib/sendmail -bd -q15m" -- run as a daemon, process the queue every 15 minutes.
- see how sendmail processes the queue -- "/usr/lib/sendmail -v -q" (verbose)


*** LOGGING AND STATISTICS:

* Sendmail send various information to syslog (config /etc/syslog.conf). Log messages are identified by a number.
The lower the error number, the higher the error. Here is a table (info=11+, critical=1):
1 Log_Crit and Log_Alert
2-8 Log_Notice
9-10 Log_Info
11+ Log_Debug

* By default, sendmail logs its activities via syslog in /var/log/syslog.
- Configure sendmail to produce statistics -- add to sendmail.cf: "O StatusFile=/etc/mail/sendmail.st".
This file has to exist before sendmail can use it -- "touch /etc/mail/sendmail.st".
- To stop collecting statistics, remove the option or rename the file.
- To start over with statistics, empty the file -- "cp /dev/null /etc/mail/sendmail.st".

* The sendmail daemon responds to the following kill signals:
SIGINT -- perform an orderly shutdown: unlocks queue, cancels deliveries, resets identity, exists with EX_OK.
SIGHUP -- reread sendmail.cf, and the PID will change.
SIGUSR1 -- dump the current running state to the syslog daemon.

- Log every transaction with sendmail ("-X") -- "/usr/lib/sendmail -X /var/adm/sendmail_log"


*** OPTIONS:

"O QueueDirectory=/var/spool/mqueue" = "OQ/var/spool/mqueue"

* List of required options:

Option Name Shortcut Type Description
==================================================
==================================================
========
Queue Directory Q String Location of the queue
------------------------------------------------------------------------------------------------------------
Timeout r String Set various timeouts
------------------------------------------------------------------------------------------------------------
Delivery Mode d Char Set delivery mode
------------------------------------------------------------------------------------------------------------
TempFileMode F Octal Permissions for temp files (0600-0666)
------------------------------------------------------------------------------------------------------------
DefaultUser u String Default delivery agent identity (user:group)
------------------------------------------------------------------------------------------------------------
LogLevel L Number Set logging level
------------------------------------------------------------------------------------------------------------
OldStyleHeaders o Boolean Allow spaces in recipient list (default: comma)
------------------------------------------------------------------------------------------------------------
BlankSub B Char Replaces a space with a dot (blah com => blah.com)
------------------------------------------------------------------------------------------------------------


* List of non-required but interesting options:
O AliasFile=/path/filename -- multiple alias files can be used.
O AllowBogusHELO (Yes/No) -- turn on or off DNS lookup of the sender's hostname.
O AutoRebuildAliases (Yes/No) -- automatically rebuild the alias database when the alias file is updated.
O CheckAliases (Yes/No) -- make sure the final alias name is a valid e-mail address.
O MaxDaemonChildren (number) -- how many child processes will sendmail be allowed to spawn.
O MaxMessageSize (maxsize) -- any e-mail messages over this size will bounce.
O MeToo (Yes/No) -- when the owner of a mailing list sends a message to the list, should he/she receive it too?
O SmtpGreetingsMessage (text/variables) -- self explanatory =)
O StatusFile (/path/filename) -- the full path to the file containing all statistics.
O TimeZoneSpec (zone) -- use a different time zone for sendmail.

* See a list of options -- "/usr/lib/sendmail -d37.1 -bt < /dev/null"


*** DATABASE FILES:

* Types of database files (listed are compile-time flags for enabling different database formats):
Auto_NIS_Aliases -- searches the NIS server for aliases
LDAPMAP -- support for LDAP databases
MAP_REGEX -- search the database using regular expressions
NDBM -- support for NDBM databases
NEWDB -- support for hash or btree (must have Berkeley database in advance)
NIS -- support for NIS
NISPLUS -- support for NIS+
PH_MAP -- support for a CCSO phonebook database
UDB_DEFAULT_SPEC -- specifies the default path used for the user database
USERDB -- support for the user database


** The User Database (USERDB):
- usable only if compiled with USERDB or NEWDB.
- applied after processing the alias file and no match has been found.
- it can be applied to either incoming or outgoing messages.
- to convert a flat ASCII file to database file, we use "makemap".

* user database records are similar to aliases, with the addition of one of two keywords:
- maildrop -- used for incoming mail:
"tweety:maildrop sylvester" in USERDB is equivalent to "tweety: sylvester" in the alias file.
"tweety:maildrop sylvester,grandma" or "tweety:maildrop sylvester,grandma@yahoo.com" -- creates a mailing list.
- mailname -- used for outgoing mail:
"tweety:mailname sylvester@yahoo.com" -- when tweety sends mail, the sender will appear as sylvester@yahoo.com

!!! Difference between an alias entry and a user database entry:
- aliases can point to other aliases. They can be processed over and over before the final destination is reached.
- user database entries MUST point to final addresses (either local users, or fully qualified e-mail addresses).

* USERDB is recommended only if such simple tasks are needed. All aliases (incoming and outgoing) are kept in a single file.


** The Access Database:
- accept or reject e-mails based on source or destination.
- typically a hash or btree database and usually resides under /etc/mail as access_db.
- each line in the ASCII file contains two fields -- address field and action field.
- address field: user, e-mail, source IP, partial network address, domain, "To:", "From:", "Connect:".
- action field: DISCARD (drop), OK (accept), REJECT (bounce), RELAY (pass through), [ERROR:[dsn:]] code text - return RFC 821

Examples:
nobody@here.com REJECT -- any mail to or from nobody@here.com will be rejected with an error message.
From:spammer@here.com DISCARD -- any mail from spammer@here.com will be dropped. No error will be sent back.
To:spammer@here.com OK -- any mail to spammer@here.com will be sent.
yahoo.com OK -- any mail to or from yahoo.com will be accepted.
10.1.1 RELAY -- if mail is coming from 10.1.1.*, we will act as a relay host.

* Names in the access database apply to inbound mail only. To make this work for outbound mail, "blacklist_recipients" needs
to be added to sendmail.cf.


** The virtusertable:
- similar to the alias file, but you can define mail by partial domain names.
- don't forget to add an MX record in your DNS server for each of the virtual domains.
- you can set a "catch all" address for a domain -- will accept mail for any non-existant user in the domain.
- each entry in the virtusertable has two fields -- virtual domain name and delivery address.
- virtual domain name -- complete address (tweety@somedomain.com), or partial address (@somedomain.com).

Examples:
tweety@somedomain.com sylvester -- any mail for tweety@somedomain.com will be locally delivered to sylvester.
@somedomain.com sylvester -- "catch all" for somedomain.com -- will be sent locally to sylvester.
tweety@thisdomain.com tweety@yahoo.com -- simple mail routing to an external (valid) address.


** The genericstable -- for sending out mail -- rewrites the sender's e-mail address:
- has two fields: local username and the name it is rewritten to.
- Example:
tweety sylvester@somedomain.com -- outgoing mail from tweety will appear from sylvester@somedomain.com


** makemap -- convert ASCII text files to database format.
- usage: makemap database_type database_name < input_file

* Options:
-d permit duplicate entries in the database
-f allow uppercase characters in the database
-N append a null character to the end of each key
-o append the new entries to an existing database
-r overwrite duplicate entries
-v run in verbose mode

* General rules:
- you should not allow duplicate entries or null values to the end of each key
- if you don't use the -o option, the existing database is overwritten by a new one
- you can use -o and -r together to append to an existing database while overwriting existing entries

* Example:
makemap { hash | btree | dbm } virtusertable < virtusertable
- makemap will append the appropriate filename extensions


!!! Choose one method (alias file, database, etc) and stick with it! It makes things cleaner.


*** CLASSES:

* In addition to databases, sendmail also has classes, defined in sendmail.cf.

- Class T: trusted users -- any user listed there can masquerade as someone else.
This class should be kept to an absolute minimum.

- Class R: relay-domains class -- list of remote domains allowed to relay via this server.
This class should be used sparingly because remote users should use their ISP's mail relay
instead of your server. To set up relay domains, create/edit /etc/mail/relay-domains. Add
all the domains that you want to relay for. Then SIGHUP sendmail and it will pick up the change.
- The relay-domains rule in /etc/mail/sendmail.cf is the following:
FR-o /etc/mail/relay-domains
- all your virtual domains should be listed in /etc/mail/relay-domains

- Class w: list of domains for which we will deliver mail locally. All virtual domains have to be listed here.

* Examples for class w:

- default setup:
Cw localhost

- one domain added:
Cw localhost
Cw somedomain.com

- when multiple domains are server, using a file is better:
Fw-o /etc/mail/sendmail.cw

* Remember to SIGHUP sendmail after you make changes to this file
* By default, sendmail will only accept messages for domains that it currently belongs to.
It determines those domains from /etc/hosts. A fully qualified name for the local machine must exist there.


*** SPAM:

- spam can originate from within your network, or can be sent to you, or can be relayed by you to other networks.
- starting with version 8.9 of sendmail, mail relaying is denied for everyone except local users.
- the easiest way to configure sendmail against spam is to setup a relay-domains file.
- you can use the access database (described above) to reject e-mail from domains you know as spammers.
- you can also subscribe to a blackhole list, and they will search the spammers for you. However, then you don't have control.
This can be accomplished by adding the following to sendmail.mc (this file will be described later):
FEATURE(`dnsbl`,`server_name`)
- you can filter incoming mail by using procmail as a local delivery agent instead of the default /usr/lib/mail.local

* For more information on spamming and how to prevent it, visit http://mail-abuse.org/.


*** HEADERS:

* Headers are information that sendmail appends to the e-mail message. All header lines begin with a capital H.
* RFC 822 requires the two headers -- "From:" and "Received:".
- "From:" means who sent the message.
- Each machine that relays a message adds its own "Received:" header to the message. This can be used to determine the path.

* Other headers:
- "Full Name:" -- displays the full name of the sender. This data comes from /etc/passwd.
- "Date:" -- shows the time and date the message originated. It is used by timeout counters on the MTAs.
- "Message-ID:" -- used to uniquely identify each message.
- "Reply-To:" -- used when you want the recepient to repond to a different e-mail address.


*** MACROS (variables):

* Macros can be defined ones and then reused multiple times. Most of them have predefined names and meaning.
* In sendmail, all macros begin with a capital D.
- Example: "DSmailhost.somedomain.com" tells sendmail to send all outgoing mail to mailhost.somedomain.com (relay).

* Some common variables and their meaning:
- n -- identity of the error message sender.
- v -- version of sendmail currently running.
- w -- the short hostname.
- j -- the fully qualified hostname.
- m -- the domain name.
- k -- the UUCP nodename.
- b -- date in RFC1123 format.
- _ -- identification information.
- opMode -- current operating mode (sendmail v8.7 and greater).

* To see all defined macros and their values, use the following command:
/usr/lib/sendmail -d35.9 -bt < /dev/null | more
* After defining a macro, you can use it by placing a dollar sign in front of its name.


*** DEBUGGING:

* To internally debug sendmail, you can use -v (verbose) and -d (debug) switches.
* -d needs some options in the form of "category.level".
* The default lever is 1, and it will not show some tolerable errors/warnings.
* Example: /usr/lib/sendmail -v -d11.1 <email_address> < /dev/null

* Debugging categories and descriptions:
-d0.1 -- prints version information.
-d0.4 -- our name and alias.
-d0.15 -- dump delivery agents.
-d0.20 -- print network address of each interface
-d3.1 -- print the load average
-d4.80 -- trace enough space
-d6.1 -- show failed mail
-d11.1 -- trace delivery
-d11.2 -- show the uid/gid running as during delivery
-d13.5 -- show addresses that we should not send to
-d22.36 -- show each token
-d25.1 -- trace "sendtolist"
-d27.1 -- trace aliasing
-d27.9 -- show uid/gid changes with :include: reads
-d27.20 -- show how an alias will be looked up in a map
-d31.2 -- trace processing of headers
-d35.9 -- macro values defined
-d37.1 -- trace settings of Options
-d37.8 -- trace adding of words to a class
-d38.20 -- trace map lookups
-d40.1 -- trace processing of the queue


*** SENDMAIL.MC:

* This is a raw configuration file (kind of cryptic) used to generate sendmail.cf.

* Things to know about sendmail.cf before you can build it:
- a line like this shows support for a feature:
##### $Id: virtusertable.m4,v 8.16 1999/07/22 17:55:36 gshapiro Exp $ #####
- each line starts with a special character

* List of special characters:
- V -- define a configuration file version (sendmail v8.6 and newer)
- M -- define a mail delivery agent
- D -- define a macro (variable)
- R -- define a rewriting rule
- S -- declare a rule set start
- C -- define a class macro
- F -- define a class macro from file
- O -- define an option
- H -- define a header
- P -- define delivery priorities
- T -- declare trusted users
- K -- declare a keyed database
- E -- define an environment variable
- L -- include extended load average support

* Some letters (V) should appear only once. Others (R, H, etc) will appear multiple times.
* Comments are entered with the usual hash symbol #.

* Every sendmail.mc file should contain the following four items:
- OSTYPE -- required -- defines support for various operating systems.
- MAILER -- required -- sendmail uses local or remote delivery agents for mail delivery.
- DOMAIN -- recommended -- the local domain (may be used to simplify configuration).
- FEATURE -- recommented -- Smrsh, genericstable, virtusertable, masquerading are all features.

* Simple steps for creating a new sendmail.cf file:
cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf.orig
cp /etc/mail/sendmail.mc /usr/lib/mail/cf/
cd /usr/lib/mail/cf
/usr/ccs/bin/m4 /usr/lib/mail/m4/cf.m4 sendmail.mc > sendmail.cf
cd /etc/mail
cp /usr/lib/mail/cf/sendmail.cf .
- Then HUP sendmail so it can reread its configuration.


*** RULESETS:

* For information on rulesets (and more!), consult the bat book (O'Reilly's "Sendmail").