|Allegro CL version 9.0|
Moderate update since 9.0 release.
This document contains the following sections:1.0 The IMAP interface
imap is a client-server protocol for processing electronic mail boxes. imap is the successor to the pop protocol. It is not an upward compatible successor. The main focus of this document is the imap protocol. Only one small section describes the functions in the pop interface.
The imap interface is based on the Imap4rev1 protocol described in rfc2060. Where this document is describing the actions of the imap commands it should be considered a secondary source of information about those commands and rfc2060 should be considered the primary source. (We have not given a link to rfc2060 or other rfc's mentioned in this document because such links tend to become stale. To see them, use a web search site such as Google (tm).)
The advantages of imap over pop are:
The symbols defining operators and other objects in this interface are
in the net.post-office package. The functionality is
in various modules. The IMAP/POP message retrieval functionality is in
the :imap module. The functionality for sending messages is in
the :smtp module. MIME support is in the :mime
module. Symbols in the net.mail package are in the
:rfc2822 module. The :mime module is loaded
automatically when the
:smtp module is loaded. (Note it is not an error to require a
module that is already loaded.) Sample requires are:
(require :imap) ;; For message retrieval (require :smtp) ;; for sending messages. MIME module is ;; loaded as well (require :mime) ;; for MIME functionality (does not load :smtp) (require :rfc2822) ;; for net.mail symbols
The net.mail package and associated symbols are loaded by the :rfc2822 module (load with (require :rfc2822)). It is described in section Section 6.0 The net.mail interface for parsing and validating email addresses.
Mailboxes are repositories for messages. Mailboxes are named by Lisp strings. The mailbox "inbox" always exists and it is the mailbox in which new messages are stored. New mailboxes can be created. They can have simple names, like "foo" or they can have hierarchical names (like "clients/california/widgetco"). After connecting to an imap server you can determine what string of characters you must use between simple names to create a hierarchical name (in this example "/" was the separator character).
Each mailbox has an associated unique number called its uidvalidity. This number won't change as long as imap is the only program used to manipulate the mailbox. In fact if you see that the number has changed then that means that some other program has done something to the mailbox that destroyed the information that imap had been keeping about the mailbox. In particular you can't now retrieve messages by their unique ids that you had used before.
Messages in a mailbox can be denoted in one of two ways: message sequence number or unique id.
The message sequence number is the normal way. The messages in a mailbox are numbered from 1 to N where N is the number of messages in the mailbox. There are never any gaps in the sequence numbers. If you tell imap to delete messages 3, 4, and 5 then it will return a value telling you that it has deleted messages 3, 3, and 3. This is because when you deleted message 3, message 4 became the new message 3 just before it was deleted and then message 5 became message 3 just before it was deleted.
A unique id of a message is a number associated with a message that is unique only within a mailbox. As long as the uidvalidity value of a mailbox doesn't change, the unique ids used in deleted messages will never be reused for new messages.
A flag is a symbol denoting that a message or mailbox has a certain property. We use keywords in Lisp to denote flags. There are two kinds of flags - System and User flags. System flags begin with the backslash character, which is an unfortunate design decision since that means that in Lisp we have to remember to use two backslashes (e.g. :\\deleted). A subset of the flags can be stored permanently in the mailbox with the messages. When a connection is made to an imap server it will return the list of flags and permanent flags (and these are stored in the mailbox server object returned for access by the program). If the list of permanent flags includes :\\* then the program can create its own flag names (not beginning with a backslash) and can store them permanently in messages.
Some of the important system flags are:
Use the function make-imap-connection to connect to the imap server of a host machine. close-connection closes the connection. with-imap-connection is a macro that connects and then closes the connection after executing body forms.
These functions work on mailboxes as a whole. The mailbox argument to the functions is the object returned by make-imap-connection. If a return value isn't specified for a function then the return value isn't important - if something goes wrong an error will be signaled.
The functions are select-mailbox, create-mailbox, delete-mailbox, and rename-mailbox.
The function mailbox-list returns information about a mailbox.
The following functions work with the messages in the currently selected mailbox. The mailbox argument is the object returned by make-imap-connection. The messages argument is either a number (denoting a single message), or is the list (:seq N M) denoting messages N through M, or is a list of numbers and :seq forms denoting the messages specified in the list.
The mailbox object contains information about the imap server it's connected to as well as the currently selected mailbox. This information can potentially be updated each time a request is made to the imap server. The following functions access values from the mailbox object.
When using fetch-parts to access letters, you must specify the parts of the messages in which you are interested. There are a wide variety of specifiers, some redundant and overlapping, described in the imap specification in rfc2060. We will describe the most common ones here. The specification is always a string. It may specify more than one thing by the use of parentheses in the string, e.g. "(flags envelope)".
The most common specifiers are:
The result of a fetch-parts is a data structure containing all of the requested information. The fetch-field function is then used to extract the particular information for the particular message.
The imap server is able, using search-mailbox, to search for messages matching a search expression. A search-expression is a predicate (described below), or one of these forms:
A predicate is
We show an example of using this interface.
The mailbox object, the value of
mb, will be used
in subsequent examples.
user(2): (setq mb (make-imap-connection "mailmachine.franz.com" :user "myacct" :password "mypasswd")) #<mailbox::imap-mailbox @ #x2064ca4a>
The value of
mb is the mailbox object returned by
Section 1.11.1 Connect to the imap server on the machine holding the email.
user(3): (select-mailbox mb "inbox") t
The value of
mb is the mailbox object returned by
Section 1.11.1 Connect to the imap server on the machine holding the email.
user(4): (mailbox-message-count mb) 7
There are seven messages at the moment. Fetch the whole 4th
message. We could evaluate
(fetch-letter mb 4) here
instead and then not have to call fetch-field later.
user(5): (setq body (fetch-parts mb 4 "body")) ((4 ("BODY" "Return-Path: <email@example.com> Received: from tiger.franz.com (jkf@tiger [22.214.171.124]) by tiger.franz.com (8.8.7/8.8.7) with SMTP id LAA20261 for <firstname.lastname@example.org>; Mon, 13 Sep 1999 11:36:26 -0700 Date: Mon, 13 Sep 1999 11:36:26 -0700 From: jkf mail tester <email@example.com> Message-Id: <199909131836.LAA20261@tiger.franz.com> message number 5 ")))
The value was returned inside a data structure designed to hold information about one or more messages. In order to extract the particular information we want we use fetch-field:
user(6): (fetch-field 4 "body" body) "Return-Path: <firstname.lastname@example.org> Received: from tiger.franz.com (jkf@tiger [126.96.36.199]) by tiger.franz.com (8.8.7/8.8.7) with SMTP id LAA20261 for <email@example.com>; Mon, 13 Sep 1999 11:36:26 -0700 Date: Mon, 13 Sep 1999 11:36:26 -0700 From: jkf mail tester <firstname.lastname@example.org> Message-Id: <199909131836.LAA20261@tiger.franz.com> message number 5 "
We use the search function to find all the messages containing the word blitzfig. It turns out there is only one. We then extract the contents of that message.
user(7): (search-mailbox mb '(:text "blitzfig")) (7) user(8): (fetch-field 7 "body" (fetch-letter mb 7 "body")) "Return-Path: <email@example.com> Received: from main.verada.com (main.verada.com [188.8.131.52]) by tiger.franz.com (8.8.7/8.8.7) with ESMTP id NAA20541 for <firstname.lastname@example.org>; Mon, 13 Sep 1999 13:37:24 -0700 Received: from main.verada.com (IDENT:jkf@localhost [127.0.0.1]) by main.verada.com (8.9.3/8.9.3) with ESMTP id NAA06121 for <email@example.com>; Mon, 13 Sep 1999 13:36:54 -0700 Message-Id: <199909132036.NAA06121@main.verada.com> To: firstname.lastname@example.org Subject: s test Date: Mon, 13 Sep 1999 13:36:54 -0700 From: jkf <email@example.com> secret word: blitzfig ok? "
We've been using message sequence numbers up to now. They are the simplest to use but if you're concerned with keeping track of messages when deletions are being done then using unique id's is useful. Here we do the above search example using uids:
user(9): (search-mailbox mb '(:text "blitzfig") :uid t) (68) user(10): (fetch-field 68 "body" (fetch-letter mb 68 "body" :uid t) :uid t) "Return-Path: <firstname.lastname@example.org> Received: from main.verada.com (main.verada.com [184.108.40.206]) by tiger.franz.com (8.8.7/8.8.7) with ESMTP id NAA20541 for <email@example.com>; Mon, 13 Sep 1999 13:37:24 -0700 Received: from main.verada.com (IDENT:jkf@localhost [127.0.0.1]) by main.verada.com (8.9.3/8.9.3) with ESMTP id NAA06121 for <firstname.lastname@example.org>; Mon, 13 Sep 1999 13:36:54 -0700 Message-Id: <199909132036.NAA06121@main.verada.com> To: email@example.com Subject: s test Date: Mon, 13 Sep 1999 13:36:54 -0700 From: jkf <firstname.lastname@example.org> secret word: blitzfig ok? "
We'll delete that letter with the secret word. Note that after we have deleted that one, only six messages are left in the mailbox.
user(11): (delete-letter mb 68 :uid t) (7) user(12): (mailbox-message-count mb) 6
Now we assume that a bit of time has passed and we want to see if any new messages have been delivered into the mailbox. In order to find out we have to send a command to the imap server since it will only notify us of new messages when it responds to a command. Since we have nothing to ask the imap server to do we issue the noop command, which does nothing on the server.
user(13): (noop mb) nil user(14): (mailbox-message-count mb) 7
The server told us that there are now 7 messages in the inbox, one more than before. Next we create a new mailbox, copy the messages from the inbox to the new mailbox and then delete them from the inbox. Note how we use the :seq form to specify a sequence of messages.
user(15): (create-mailbox mb "tempbox") t user(18): (let ((count (mailbox-message-count mb))) (copy-to-mailbox mb `(:seq 1 ,count) "tempbox") (delete-letter mb `(:seq 1 ,count))) (1 1 1 1 1 1 1) user(19): (mailbox-message-count mb) 0
When we're done there are 0 messages in the currently selected mailbox, which is inbox. We now select the maibox we just created and see that the messages are there.
user(22): (select-mailbox mb "tempbox") t user(23): (mailbox-message-count mb) 7
Finally we shut down the connection. Note that imap servers will automatically shut down a connection that's been idle for too long (usually around 10 minutes). When that happens, the next time the client tries to use an imap function to access the mailbox an error will occur. There is nothing that can be done to revive the connection however it is important to call close-imap-connection on the lisp side in order to free up the resources still in use for the now dead connection.
user(24): (close-connection mb) t
The pop protocol is a very simple means for retrieving messages from a single mailbox. The functions in the interface are:
When an unexpected event occurs a condition is signaled. This applies to both the imap and pop interfaces. There are two classes of conditions signaled by this package:
Instances of both of these condition classes have these slots in addition to the standard condition slots:
|identifier||po-condition-identifier||keyword describing the kind of condition being signaled. See the table below for the possible values.|
|server-string||po-condition-server-string||If the condition was created because of a message sent from the mailbox server then this is that message.|
The meaning of the identifier value is as follows
|:problem||po-condition||The server has responded with a warning message. The most likely warning is that the mailbox can only be opened in read-only mode because another process is using it.|
|:unknown-ok||po-condition||The server has sent an informative message that we don't understand. It's probably safe to ignore this.|
|:unknown-untagged||po-condition||The server has sent an informative message that we don't understand. It's probably safe to ignore this.|
|:error-response||po-error||The server cannot execute the requested command.|
|:syntax-error||po-error||The arguments to a function in this package are malformed.|
|:unexpected||po-error||The server has responded in a way we don't understand and which prevents us from continuing|
|:server-shutdown-connection||po-error||The connection to the server has been broken. This usually occurs when the connection has been idle for too long and the server intentionally disconnects. Just before this condition is signaled we close down the socket connection to free up the socket resource on our side. When this condition is signaled the user program should not use the mailbox object again (even to call close-connection on it).|
|:timeout||po-error||The server did not respond quickly enough. The timeout value is set in the call to make-imap-connection.|
|:response-too-large||po-error||The value returned by a command is too large to fit in a lisp array. When this occurs you should close the connection and reopen it since the imap/pop interface code has gotten out of sync with the imap/pop server.|
Allegro CL supports constructing MIME (Multipurpose Internet Mail Extensions) compliant email messages. The Allegro CL MIME API, combined with the send-letter and/or send-smtp functions make it easy to construct simple or complex MIME messages.
A full explanation of MIME is beyond the scope of this documention. MIME is defined and specified in RFC 2045 (http://www.faqs.org/rfcs/rfc2045.html). However, some of the basic concepts are described here to facilitate immediate experimentation and use.
If you want a simplified interface for sending emails with attachments, please see send-letter.
MIME functionality is in the :mime module. The :mime module is loaded automatically when the :smtp module is loaded (the :smtp module has functionality for sending messages). Use one of these require forms to load the MIME module:
(require :mime) ;; for MIME functionality (does not load :smtp) (require :smtp) ;; for sending messages. MIME module is ;; loaded as well
MIME messages can be composed of multiple pieces. In Allegro CL, these pieces are referred to as parts. Messages have at least one part. Messages that contain more than one part are called multipart messages.
There are two types of parts, multipart-parts and non-multipart parts:
The interface contains the following classes and associated operators:
mime-part: the superclass of mime part objects. (Currently, the only user-visible subclass is
mime-part-constructed: the class of user-visible mime parts.
(require :smtp) ;; Note: this loads the :mime module as well (use-package :net.post-office) ;; Construct a simple part. cl-user(168): (setf p1 (make-mime-part :text "This is a simple single-part message with text contents")) RETURNS #<net.post-office::mime-part-constructed @ #x71c849f2> ;; Let's see what it looks like when rendered cl-user(169): (mime-part-writer p1) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit This is a simple single-part message with text contents RETURNS nil ;; As you can see, the headers contain only information specified by ;; the MIME specification. If you want to make a top level part with ;; fuller headers, you can do something like the following: cl-user(170): (setf p1 (make-mime-part :text "This is a simple message with more headers." :headers '(("From" . "Test User <email@example.com>") ("To" . "Recipient <firstname.lastname@example.org>") ("Subject" . "This is a test email")))) RETURNS #<net.post-office::mime-part-constructed @ #x71c87ab2> cl-user(171): (mime-part-writer p1) MIME-Version: 1.0 From: Test User <email@example.com> To: Recipient <firstname.lastname@example.org> Subject: This is a test email Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit This is a simple message with more headers. RETURNS nil ;; Or you can use send-letter which will add in these headers for you ;; using a simpler interface. cl-user(172): (setf p1 (make-mime-part :text "Simple again")) RETURNS #<net.post-office::mime-part-constructed @ #x71c89d9a> ;; send-letter will accept a mime-part as the message data and it will ;; fill in the headers based on the arguments you supply. ;; Replace "mail-server" and the sender/recipient addresses with ones ;; suitable for your environment. cl-user(173): (send-letter "mail-server" "email@example.com" "firstname.lastname@example.org" p1 :subject "Just testing") ;; You can specify the content-type if the default is not suitable. cl-user(176): (setf p1 (make-mime-part :text "<html><h3>Hello!</h3></html>" :content-type "text/html")) RETURNS #<net.post-office::mime-part-constructed @ #x71c9671a> cl-user(177): (mime-part-writer p1) MIME-Version: 1.0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit <html><h3>Hello!</h3></html> nil ;; Now let's construct a simple multipart message. The first part ;; will be introductory text and the second part will be a file ;; attachment. ;; Make the text part cl-user(178): (setf p1 (make-mime-part :text "Here is the file you requested")) RETURNS #<net.post-office::mime-part-constructed @ #x71c989f2> ;; Make the file attachment part, specifying the file we want to use. ;; You should use a small file for testing purposes. cl-user(179): (setf p2 (make-mime-part :file "/tmp/quickref.pdf")) RETURNS #<net.post-office::mime-part-constructed @ #x71c9a38a> ;; And finally we need a top level part to contain these two ;; individual parts cl-user(180): (setf p0 (make-mime-part :subparts (list p1 p2))) RETURNS #<net.post-office::mime-part-constructed @ #x71c9e34a> ;; Let's see what that gives us. If the size of the file attachment ;; is large, then you will generate a large amount of data. cl-user(181): (mime-part-writer p0) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------_4299707ac57b86bae36a01272cde43d0" This is a multi-part message in MIME format. - ------------_4299707ac57b86bae36a01272cde43d0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Here is the file you requested - ------------_4299707ac57b86bae36a01272cde43d0 Content-Type: application/pdf Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="quickref.pdf" JVBERi0xLjIKJcfsj6IKNiAwIG9iago8PC9MZW5ndGggNyAwIFIvRmlsdGVyIC9GbGF0ZURl [ ... many lines deleted ... ] NzMgL1Jvb3QgMSAwIFIgL0luZm8gMiAwIFIKPj4Kc3RhcnR4cmVmCjY5MTMzCiUlRU9GCg== - ------------_4299707ac57b86bae36a01272cde43d0-- RETURNS nil ;; When looking at the rendered output (above), you can see that you ;; generated a multipart/mixed message. You can also see that the ;; content-type of the second part was determined to be ;; "application/pdf". This was based on the filename (which ended ;; with .pdf). If no suitable guess can be made, ;; "application/octet-stream" will be used. ;; You can also see that base64 encoding was used. Base64 encoding is ;; always used for file attachments unless the content-type is ;; specified as a text-type. ;; Here is a more complex multipart message structured like so: ;; top-level (multipart/mixed): ;; part1: (multipart/alternative) ;; part1: (text/html) ;; part2: (text/plain) ;; part2: (application/pdf) ;; ;; This is a common arrangement for a message that includes both html ;; and plain text versions of the message text (so that both HTML and ;; non-HTML email clients can show whichever version works best). The ;; message also contains a file attachment. cl-user(187): (setf html (make-mime-part :text "<html><font color=red>You have won big prizes!</font><html>" :content-type "text/html")) RETURNS #<net.post-office::mime-part-constructed @ #x71cbc4da> cl-user(188): (setf plain (make-mime-part :text "You have won big prizes!")) RETURNS #<net.post-office::mime-part-constructed @ #x71cbde1a> cl-user(191): (setf file (make-mime-part :file "/tmp/quickref.pdf")) RETURNS #<net.post-office::mime-part-constructed @ #x71cc0a5a> cl-user(192): (setf alternatives-container (make-mime-part :content-type "multipart/alternative" :subparts (list html plain))) RETURNS #<net.post-office::mime-part-constructed @ #x71cc5d5a> cl-user(193): (setf top (make-mime-part :subparts (list alternatives-container file))) RETURNS #<net.post-office::mime-part-constructed @ #x71cc7e92> ;; Fill in the To, From and Subject headers and send it off. cl-user(194): (send-letter "mail-server" "email@example.com" "firstname.lastname@example.org" top :subject "Test email")
With the SMTP interface, a Lisp program can contact a mail server and send electronic mail.
smtp module is not loaded automatically when
imap is. To load SMTP functionality into a
running image, evaluate:
The interface contains these functions:
The net.mail interface allows parsing and cursory validation of
portions of email addresses. Symbols in the interface are in the
net.mail package. The interface is loaded with the
:rfc2822 module with
(require :rfc2822). The
interface implements part of RFC2822 (that link is
to a URL outside the Allegro CL documentation).
The parse-email-address function parses an email address string. The valid-email-domain-p function provides information on whether a string naming a domain appears valid (this function is most useful in identifying obviously invalid strings since seemingly valid strings may still fail to accept email for any number of reasons). extract-email-addresses parses a string representing an email header and returns information about the email addresses found.
An email address consists of a username part, followed by @, followed by a domain name.
Arguments: string &key require-domain require-dotted-domain
Parses an email address string and, if the address has valid syntax,
returns two values: the local part of the address and the domain
part of the address. If address is invalid, the single value
nil is returned. The keyword arguments provide some
control over what is or is not considered invalid.
The parser is RFC2822 compliant except:
require-domain, which defaults to
t, controls whether or not domainless email addresses
(i.e., addresses without the @domain part) are accepted.
t, controls whether or not non-dotted domain parts will be accepted. When
nil, a single-component domain part will be accepted (e.g., "com"). If true, then the domain part of the email address must have at least two dotted components (e.g., "franz.com" or "mymachine.franz.com"), else
(require :rfc2822) (use-package :net.mail) (parse-email-address "email@example.com") returns "support" "franz.com" (parse-email-address "david;firstname.lastname@example.org") returns nil ;; ';' is not allowed in email addresses. (parse-email-address "support") returns nil (parse-email-address "support" :require-domain nil) returns "support" nil
domain should be a string (such as the second return value of parse-email-address). This function returns information on whether or not the DNS configuration for domain is configured properly for Internet email reception.
The possible return values are:
nil: this means that the DNS records for domain are not properly configured for Internet email.
:unknown: this means that no information was successfully collected. No conclusion can be drawn.
t: this means that domain has DNS records that are suitable for Internet email reception. However, this does not necessarily mean that email delivery will succeed (it could fail for reasons unrelated to the validity of the domain).
This function is more useful for its negative response (
nil) than any other response. If it returns
nil, it means that no standard mail transfer agent
would be able to locate the mail server for the domain. As noted
above, a non-
nil value does not guarantee
that the mail will be delivered.
(require :rfc2822) (use-package :net.mail) (valid-email-domain-p "franz.com") returns t ;; Under normal circumstances (valid-email-domain-p "xnosuchdomainx.com") returns nil ;; at least when this document was written as ;; "xnosuchdomainx.com" is a currently nonexistent ;; domain name. (valid-email-domain-p "nosuchdomain.com") returns t ;; under normal circumstances as at the time ;; this document was written, "nosuchdomain.com" ;; does exist and has a DNS MX record.
Arguments: string &key start end require-domain errorp compact
extract-email-addresses parses string and returns a list of entries describing the email addresses and display names found. This function is suitable for use on RFC2822-compliant email headers such as the To:, From:, and Cc: headers. Compliant folded lines are acceptable.
start and end specify the subsequence of string to operate on. start defaults to 0 and end defaults to the length of the string.
require-domain, which defaults
t, controls whether or not domainless
email addresses (i.e., addresses without the @domain part) are
errorp, which defaults
t, controls whether or not to signal an
error if there is a syntax error or other problem during parsing. If
nil is returned if there is a problem during
compact: when true,
causes extract-email-addresses to return its
results as a list of
user@domain strings (the
@domain part may not exist if
the require-domain keyword arg
extract-email-addresses returns a list of
group lists, depending on the contents of
mailbox list has the following form:
(:mailbox display-name user-part domain-part)
display-name may be
no display name was found. If
domain-part may be
nil if no domain part was found.
group list has the following form:
(:group display-name mailbox-list)
mailbox-list will be a list of
mailbox lists. If no mailboxes were supplied,
mailbox-list may be
;; Simple folded list of addresses which display names. cl-user(186): (extract-email-addresses "Tech Support <email@example.com>, Sales Department <firstname.lastname@example.org>") returns ((:mailbox "Tech Support" "support" "franz.com") (:mailbox "Sales Department" "sales" "franz.com")) ;; A group commonly seen in emails. There are no mailboxes in the group. cl-user(187): (extract-email-addresses "Undisclosed Recipients:;") returns ((:group "Undisclosed Recipients" nil)) ;; A variety of legal formats: cl-user(188): (extract-email-addresses "email@example.com (Cron Daemon), Mailing List: Bill Johnson <firstname.lastname@example.org>, email@example.com ;") returns ((:mailbox "Cron Daemon" "root" "example.com") (:group "Mailing List" ((:mailbox "Bill Johnson" "customer1" "example.com") (:mailbox nil "customer2" "example.com")))) ;; Use of the COMPACT keyword argument: cl-user(13): (extract-email-addresses "firstname.lastname@example.org (Cron Daemon), Mailing List: Bill Johnson <email@example.com>, firstname.lastname@example.org ;" :compact t) returns ("email@example.com" "firstname.lastname@example.org" "email@example.com") cl-user(11): (extract-email-addresses "Undisclosed Recipients:;" :compact t) returns nil
Copyright (c) 1998-2019, Franz Inc. Oakland, CA., USA. All rights reserved.
This page was not revised from the 8.2 page.
|Allegro CL version 9.0|
Moderate update since 9.0 release.