| Allegro CL version 8.2 Significant update since 8.2 release. 8.1 version |
This document contains the following sections:
1.0 Introduction and backgroundSockets are a mechanism for interprocess communication designed at U.C. Berkeley for use in their version of Unix. Sockets have been added to many other versions of Unix and there is an implementation of sockets for Windows called Winsock. This document describes the Allegro interface to sockets. This interface works on Unix and on Windows.
Symbols naming objects in the socket utility are in the
acl-socket
package. It has the nickname
socket
.
The socket module is not included in all versions of Allegro CL. If it is present, it is (by default) included in a development image (one built with the include-devel-env argument to build-lisp-image specified true). To load the socket module if it is not present in an image, evaluate
(require :sock)
Note that runtime images cannot include the development environment
(so include-devel-env must be specified nil
when a runtime image is being built). If the
socket module is needed, it must be loaded when the image is
built. See runtime.htm,
building-images.htm and
delivery.htm for more information.
Allegro CL supports Internet Protocol version 6 sockets (IPv6
sockets). As part of this support, several new functions have been
added and several functions have been modified. The new functions are
ipv6, get-ip-interfaces, ipaddrp, ipaddr-equalp, ipv6-address-p, and (added in a June,
s006 update) dotted-address-p. The modified functions are
dotted-to-ipaddr,
dns-query, lookup-hostname, make-socket, and send-to. There is also a new
variable *ipv6*
.
The feature :ipv6
is added
to the *features*
list to indicate IPv6 support.
Because of apparent bugs in Mac OS X 64-bit, certain IPv6 functionality may be restricted or unusable. In particular:
Throughout the socket documentation, we make use of the term IP address. But what exactly is an IP address? Unless further clarified in the context in which it is used, an IP address is either an unsigned 32-bit integer or an ipv6 address structure.
The function socket:ipaddrp returns true when passed an IP adress. That function can be used to identify an object as an IP address. (Unsigned 32 bit integers obviously have other uses that representing IP addresses. The function simply determines whether the type and form of its argument is suitable as an IP address.)
There are three independent characteristics of sockets:
type | Valid values: A A |
address family | Valid values: In order to send to another socket the socket must have a name.
An
On Unix, port numbers less than 1024 can only be allocated by a
process with the user id of root. A
Note that the current version of the socket interface on Windows
(Winsock, version 1.1), does not support the
|
format | Valid values: This isn't a property of the Unix
socket implementation but is instead something we've added for the Common Lisp
implementation since a Lisp stream is either binary (supports |
Note on bivalent format:
Starting in release 5.0.1, the bivalent format is accepted for stream sockets. Bivalent means that the stream will accept text and binary stream functions. That is, you can write-byte or write-char, read-byte or read-char. A bivalent stream is useful in the http protocol (used between web browsers and web servers) since in that protocol the header data is sent in text format and the body can be in binary data (image files, for example).
Internally a bivalent socket stream is configured like a binary socket stream with 8 bit bytes. Character position is not maintained.
Bivalent socket streams have very efficient read-sequence and
write-sequence implementations (as long as the sequence is either a
vector of element-type character
, (unsigned-byte
8)
or (signed-byte 8)
).
Bivalent socket streams also support the chunking protocol found in http/1.1. This protocol allows the sender to signal end of file without closing down the stream.
Stream sockets have a fourth characteristic called connect,
with a value :active
or
:passive
. In order to use stream sockets you have
to set up a link between two of them. That link is called a
connection. You set up a connection in this way:
(setq s-a (make-socket :connect :passive :local-port port-b))
(setq s-b (make-socket :remote-host "machine-a" :remote-port port-b))
(setq str-a (accept-connection s-a))
s-b
to send messages to machine A.
Note that steps 2 and 3 can occur in either order.
Note the asymmetry: a passive socket is not a Lisp stream (you can't do read and write to it). An active socket is a Lisp stream.
When accept-connection
is called on a
passive socket, it does not return until a connection is made to the
passive socket. The value
accept-connection
returns is a
stream.
As long as the passive socket is not closed, new connections can still be made to the port of that socket.
An active socket can be used for only one connection. Once that connection has been made, the socket should be closed and a new active socket created.
Host naming conventions: this package supports three conventions for naming a host:
hostname | A string using the domain naming convention, e.g. "ftp.franz.com". The domain naming system is case-insensitive. |
dotted |
A string which is the printed representation of the numeric address:
e.g. "192.132.95.84" . We also support the non
standard Berkeley extensions to this format for class A addresses:
"23.3" (which is the same as
"23.0.0.3" ) and class B addresses
"128.1.3" (which is the same as
"128.1.0.3" ). IPv6 colon hex format, e.g.,
"fe80::209:5bff:fe8e:61c1", is also supported. See dotted-to-ipaddr.
|
ipaddr | An unsigned 32-bit number, representing the IPv4 address in the native byte order for the host. Thus 192.132.95.84 is 192*2^24 + 132*2^16 + 95*2^8 + 84 = 3229900628. |
IPv6 | An IPv6 address structure. |
The variables defined by the interface are:
Please provide the value of this variable when asking for technical support with sockets as it tells us whether you have the latest version.
This variable controls whether the socket printing code converts the ip address of a socket into a hostname. This is usually what you want, however this can be a slow process (taking up to a minute to accomplish). The default value for this variable is
t
. See the full description for a discussion of the causes of the possible slowdown when the value ist
.
Specifies the default value of the ipv6 keyword argument to lookup-hostname and make-socket.
The first table shows general functions defined by the interface and the second shows accessors.
Function | Arguments | Notes (follow function link for full description) |
accept-connection | (sock passive-socket) &key wait |
Generic function. Establishes a connection. If wait is nil and no connection is pending, returns nil and does nothing further. If wait is true (the default),
waits until a connection is established. When a connection is established, returns the
stream that communicates with the socket. |
dotted-to-ipaddr | dotted &key errorp |
Function. Converts a string like "192.132.95.84" or
similar format to an unsigned 32-bit IP address.
IPv6 "colon hex" address notation, including the %scopeid extension is also supported as is IPv4-mapped IPv6 address notation (::ffff:w.x.y.z). |
dotted-address-p | object |
Function. Returns true if its argument is a string in dotted IP address form. |
get-ip-interfaces | Function. Returns a list of conses of interface id's and names. | |
ipaddr-to-dotted | ipaddr &key values |
Function. Convert a 32-bit unsigned IP address, ipaddr ,
to a string in dotted form.
This function works on IPv6 address structures as well.
|
ipaddr-equalp | add1 add2
&key compare-scope-id |
Function. Returns true if its two internet address arguments match. |
ipaddr-to-hostname | ipaddr | Function. Returns, as a string, the hostname of the machine with address ipaddr. ipaddr should be a 32-bit IP address or an IPv6 address structure or IPv6 colon hex strings. |
ipaddrp | object |
Function. Returns true if its argument is an IP address. |
ipv6 | internet-socket |
Generic function. Returns true if its argument is an IPv6 socket. |
ipv6-address-p | object |
Function. Returns true if its argument is an IPv6 address structure. |
lookup-hostname | hostname | Given a string naming a host, a 32-bit IP address, a string in dotted form, or a IPv6 address structure or IPv6 colon hex strings, return the 32-bit IP address for the host. |
lookup-port | portname
protocol |
Function. Finds the port number using the symbolic name and the protocol. |
make-socket | &key type format address-family
connect eol ipv6 scope-id
&allow-other-keys |
Function. See the full description for details. |
with-pending-connect | &body body |
Macro. See the full description for details. |
receive-from | (sock datagram-socket) size &key
buffer extract |
Generic function. This is used to read from a datagram socket. |
send-to |
sock &key
remote-host remote-port ipv6 scope-id
|
Generic function with methods for internet-datagram-sockets and file-datagram-sockets |
set-socket-options | sock &key |
Generic function for modifying existing sockets. |
shutdown | sock &key direction |
Generic function that closes down the specified half of the bidirectional socket connection. |
socket-control | stream &key
output-chunking output-chunking-eof input-chunking |
This function modifies the state of the socket stream, controlling input and output chunking. |
socket-os-fd | sock |
Generic function. Return the operating system file descriptor associated with this socket. |
These functions retrieve slot values from socket instances. The values of these slots are set when the socket is created.
Function | Arguments | Notes (follow function link for full description) |
remote-host | socket |
Generic function. Returns an IP address. |
local-host | socket |
Generic function. Returns an IP address. |
local-port | socket |
All are generic functions. All return the values of the particular attribute for socket. Note: Both internet stream and internet datagram sockets use 16-bit port numbers. Note that stream (tcp) port N is totally distinct from datagram (udp) port N. |
remote-filename | socket |
|
local-filename | socket |
|
remote-port | socket |
|
socket-address-family | socket |
|
socket-connect | socket |
|
socket-format | socket |
|
socket-type | socket |
|
ipv6 | internet-socket |
When errors are raised by the socket interface, Lisp conditions are signaled. This section describes those conditions.
A condition
is a CLOS class and thus fits into
the hierarchy of CLOS classes. The condition socket-error
is a subclass of the condition
stream-error
.
socket-error
is the
superclass for all socket related errors.
See More on
cl:stream-error in errors.htm.
socket-error
denotes operating system
detected socket errors. It has the following slots:
Name | Reader function | What |
excl::identifier |
stream-error-identifier | Symbol denoting this error (see table below) |
excl::code |
stream-error-code | Operating system dependent error code (if any) |
excl::action |
stream-error-action | String describing the operation in progress when the error occurred |
Handling socket error is difficult because the error returned in exceptional situations can depend on the operating system and the address of the other side of the connection. For example, attempting to make a connection to a machine that is down may result in a "Connection Timed Out" or a "Host Unreachable" error, or maybe something else on certain systems.
The error codes assigned to socket errors vary from operating
system to operating system. We translate a large set of the common
error codes from a machine dependent number to a symbol which we call
the identifier
to make it easier for you to
write portable code. Condition handling code should check the
identifier field (using stream-error-identifier) If the
identifier value is :unknown
then this is not a
common socket error and the operating system dependent code value of
the condition must be used.
Possible identifier
values and their meanings:
Identifier | Meaning |
:address-in-use |
Local socket address already in use |
:address-not-available |
Local socket address not available |
:network-down |
Network is down |
:network-reset |
Network has been reset |
:connection-aborted |
Connection aborted |
:connection-reset |
Connection reset by peer |
:no-buffer-space |
No buffer space |
:shutdown |
Connection shut down |
:connection-timed-out |
Connection timed out |
:connection-refused |
Connection refused |
:host-down |
Host is down |
:host-unreachable |
Host is unreachable |
:protocol-not-available |
Protocol not available |
:unknown |
Unknown error |
An SMP (symmetric multiprocessing) Lisp runs one of more OS threads which can use more than one processor (see multiprocessing.htm). When a socket is opened in one SMP process, do not close it from another process.
If you have a process which is reading from a socket and that socket fails so the reading process hangs, you can wake the process up using a process interrupt. So assuming hung-process is the reading process and hung-socket is the socket, the following form executed in another process breaks the read (with an error) and closes the socket:
(mp:process-interrupt hung-process 'close hung-socket)
You can also wrap the read in a (catch 'dead-partner
...)
and the from another process do:
(mp:process-run-function hung-process #'(lambda () (throw 'dead-partner t)))
Create an active stream socket connection to a socket that just prints characters to
whomever connects to it. After connecting, read the first five characters and print them
out. USER(1): (let ((s (make-socket :remote-host "vapor" :remote-port "chargen"))) (dotimes (i 5) (print (read-char s))) (close s)) #\space #\! #\" #\# #\$ |
Sending a message from frisky to vapor: on vapor: USER(1): (print (read (accept-connection (make-socket :connect :passive :local-port 9933)))) .. this hangs ... on frisky: USER(1): (let ((s (make-socket :remote-host "vapor" :remote-port 9933))) (format s "Secret-message~%") (close s)) Then you see on vapor: Secret-message Secret-message USER(2): A flaw in this example is that on vapor we've left the socket and the stream open and we lost track of the objects to close them. So, while concise, this is not a good programming style. Another problem with this example is that when we created the port on vapor we used a
specific port number (9933). This means our program will fail if port 9933 is already in
use. If possible, it is best to let the system choose a port number (this is done by not
specifying a : |
If we just want to send a simple message then datagrams might be more appropriate (although the program must guarantee that the message made it because datagram communication is unreliable). on vapor: user(2): (setq s (make-socket :type :datagram :local-port 9999)) #<text datagram socket waiting for connection at */9999 @ #x20664e82> user(3): on frisky: user(10): (setq x (make-socket :type :datagram)) #<text datagram socket waiting for connection at */45602 @ #x20717fb2> user(11): (send-to x "foo-the-bar" 11 :remote-host "vapor" :remote-port 9999) 11 user(12): on vapor: user(3): (receive-from s 100 :extract t) |
Allegro CL supports Secure Socket layers as described in this section.
See
also aserve/aserve.html,
which describes Webserver support in Allegro CL. Using any https
feature in aserve triggers loading of the :ssl
module (i.e. evaluates (require :ssl)
).
Starting with a patch released in June, 2016, Allegro CL loads OpenSSL
libraries dynamically when the :ssl
module is
loaded. Also loaded is the aclssl
or aclissl
shared library (which depends on the
type of Allegro CL being run). Prior to the patch, the
OpenSSL library was linked into the aclsslaclissl library (except on
Windows, which has always worked like all platforms now work). This
made installation and usage easier but meant that a patch and an
update was required whenever a new version of the OpenSSL library was
released. Now, newly installed versions of OpenSSL libraries will be
used when installed on your computer without (usually) requiring an
Allegro CL update. However, it is now the user's responsibility to
ensure that OpenSSL libraries are available, properly updated, and
findable.
This documentation has been updated to reflect behavior assuming the new SSL patch has been installed. See sys:update-allegro for information on getting patches.
Users must themselves download and install OpenSSL libraries for their computers, and make sure these are updated when necessary. On some platforms, such as the Mac and some Linux machines, this is often done automatically. Further, Users must ensure that Allegro CL knows where to find the OpenSSL libraries. On some platforms, there are standard locations where Allegro CL will look. On all platforms, environment variables or Lisp variables can be used to specify the library location.
Here is how to set things up for using OpenSSL:
(require :ssl)
in Lisp. If it works,
you are done. Allegro CL will always use the latest OpenSSL library
installed on your machine.
(require :ssl)
. If the library cannot
be found, you must set the appropriate environment variable (always
prior to starting Allegro CL).
*ssl-library-names*
.
In all cases, if (require :ssl)
loads the SSL fasl
file and the aclssl/aclissl library and loads the OpenSSL library
without error, you are ready to use SSL.
The function openssl-version returns information about the version of the OpenSSL library which is loaded and the version used to build the aclssl/acl1ssl library.
Most Allegro CL platforms support SSL. All that do
have :ssl-support
included on
the *features*
list.
The SSL functionality is in the ssl module. To
ensure it is loaded, evaluate (require
:ssl)
. Calling either of the two SSL functions, make-ssl-client-stream and
make-ssl-server-stream, automatically
loads that module.
If you are including the SSL facility in an application intended for
delivery, be sure to include the :ssl module by adding the
keyword :ssl
to the list which is the value of
the input-files argument to generate-application. If
the copy-shared-libraries argument to that
function is false, then the aclssl/aclissl shared
library must be copied explicitly from the Allegro CL installation
directory to the generated application directory. When the generated
application is run, it must be possible to locate the the OpenSSL
shared libraries in the same way that is described above.
Our aclssl and aclissl shared libraries depend on the OpenSSL version specified in Required versions of OpenSSL needed for the :ssl module in the release-notes. SSL library releases often have lettered suffixes corresponding to fixes, and you should always be running the latest, patched version. For some operating systems, such as RHEL/CentOS, Fedora and Ubuntu, patches usually come out quickly after vulnerabilities are found, and the fixed versions may and usually do appear older than the versions available from http://openssl.org. For example, when this was written, at which time the current OpenSSL version is 1.0.1t, CentOS 6.8 had a patched 1.0.1e, which had the same fixes for vulnerabilities as 1.0.1t from http://openssl.org.
If your application uses foreign libraries that use OpenSSL, those foreign libraries must be compatible with a version of OpenSSL compatible with our :ssl module. Otherwise, our :ssl module cannot be used.
In this section, we give the exact steps on the various platforms for finding and loading the OpenSSL library.
There are two library files. Their names (without the file extensions)
are in a list which is the value of *ssl-library-names*
. The initial value of that
variable is the standard library names on each platform (so the
initial value may be different on different platforms). It is usually
not necessary to change the value of that variable. The system will
not look for libraries with names other than specified by that
variable.
Here are the steps for finding the libraries on the various supported platforms:
PATH
is set to
modify the search used by the system loader).
*ssl-library-names*
should be a list of two items, each a string nameing a file with an
extension. Search the directories in the PATH environment variable
for a directory containing the first file.
After the OpenSSL library files are loaded,
load aclssl
or aclissl
(depending on what version of Allegro CL is running).
LD_LIBRARY_PATH
is set to
modify the search used by the system loader).
*ssl-library-names*
should be a list of two items, each a string or a list of strings. If
the element is a string, it should name a file complete with
extension. Load that file. If the value is a list, it should be a list
of filenames with extensions. Try to load each until one
succeeds. LD_LIBRARY_PATH
is set to modify the
search used by the system loader.
After the OpenSSL library files are loaded,
load aclssl
or aclissl
(depending on what version of Allegro CL is running).
Simply load aclssl
or aclissl
and assume that the system loader will find the OpenSSL libraries.
The system loader looks in the directory specified in
DYLD_LIBRARY_PATH (as well as other places) on the Mac and in
LD_LIBRARY_PATH (as well as other places) on Freebsd, and Solaris.
In 1994 Netscape Corporation designed the Secure Socket Layer (SSL) protocol to provide a means of safely and securely transporting private data (such as credit card numbers) between a Web Browser and a Web Server. Rather than tie SSL to the http protocol, Netscape wrote it as a protocol for making any TCP/IP connection secure.
At the end of 1994 version 2 of SSL was introduced and this was the first version shipped with a commercial web browser (Netscape Navigator (r)). In 1995 version 3 of SSL was introduced. At that point an international standards organization (IETF) took over work on SSL and introduced Transport Layer Security (or TLS) protocol (which is based on SSL but has a different handshake protocol). The IETF introduced TLS version 1.0 in 1999.
Allegro CL, starting in release 6.0, provides an interface that supports SSL version 2, SSL version 3 and TLS version 1. When we use the name SSL, we mean SSL or TLS.
A secure TCP connection exists between two processes when both agree on the following:
These three items are determined via negotiation when the connection is made and the first data is to be sent.
In an SSL connection, one side is the client and the other side is the server. In the http environment, the web browser is the client and the web server is the server.
When a secure connection is started, the client starts the negotation by telling the server all the possible ways that it can communicate securely. The server then chooses one of the possible ways and informs the client.
Then the server sends its certificate and possibly other certificates if they are needed to prove that its certificate can be trusted. The important item in the certificate is the public key for the server. The client will use this public key to encrypt a random value which will be used by both the client and server to create the keys needed for the cipher chosen for data transmission.
In theory a certificate isn't necessary if both the client and server side support a key exchange algorithm that can generate a public key on the fly. The SSL libraries we use do not have this capability, thus you must always supply a server certificate.
Once both sides know the keys the other side will use to transmit, the secure data transmission can occur.
The SSL protocol also permits each side of the connection to declare who they are. This is done by the exchange of certificates. The server must send a certificate describing itself to the client. The server can request that the client send a certificate to the server (although in the use of SSL on the web this is never done).
A certificate is a digital document that stores information about an entity in such a way that it can be verified to be true. The primary use of certificates is to store the public key that can be used to send encrypted messages to the entity.
In the SSL protocol certificates have two uses:
Strictly speaking a certificate isn't required for SSL communication if both sides support a certain key exchange protocol. The OpenSSL libraries we use do not support this protocol thus whenever you create a server SSL stream you must supply a certificate (if you don't have your own we supply one in <Allegro directory>/examples/ssl/server.pem that you can use).
While certificates support authentication, the SSL protocol doesn't require that you take advantage of this facility.
A certificate contains the following:
A certificate is a combination of text and binary data and in order to make it easy to transport certificates they are usually encoded in a form called PEM which turns them into a sequence of printable characters.
When a web browser connects to a site via SSL (which is caused by the use of the 'https:' at the beginning of the url), it checks three things about the certificate:
https://www.foo.com/whatever
then the certificate
must be for www.foo.com
. The convention used is to
store the name of the server machine in the CommonName slot of the
Subject Identifier field of the certificate.
If all three tests pass then the web browser silently accepts the certificate and does a secure web page access. If any of the tests fail then the web browser notifies the user and waits for a response. Each browser displays the failure differently. For example, the Microsoft Internet Explorer (r) shows which of the three tests passed and which failed while the Netscape Navigator (r) just says that it received an invalid certificate. In both cases the person using the web browser is given the option of continuing with the web access. Transmission will still be secure if it is elected to continue. The only issue in doubt is the authenticity of the web server.
The SSL implementation includes certificate revocation list (CRL) support. CRL checking is controlled by the crl-check and crl-file keyword arguments to make-ssl-client-stream and make-ssl-server-stream.
If you enable CRL checking, you must supply a proper PEM-encoded CRL, even if it contains zero revocations. If you do not supply a CRL, peer verification will never succeed.
The following operators, variable and class comprise the SSL API. make-ssl-client-stream and
make-ssl-server-stream
create the streams that are used for communication. All symbols naming
these objects are in the acl-socket
(nicknamed
socket
) package.
x509-certificate
(class)
The following function, variable, and classes (naming conditions) are
related to loading and using OpenSSL libraries. The conditions are
signaled when there are problems loading the OpenSSL library. All
these symbols are in the excl
package.
The file <Allegro directory>/examples/ssl/server.pem is a sample certificate and private key file. You can use this file when starting the server side of an SSL connection. The AllegroServe facility uses SSL. It is described in aserve/aserve.html.
Copyright (c) 1998-2016, Franz Inc. Oakland, CA., USA. All rights reserved.
This page was not revised from the 8.1 page.
Created 2010.1.21.
| Allegro CL version 8.2 Significant update since 8.2 release. 8.1 version |