A discussion of the encapsulating streams feature in Allegro CL

The Allegro CL encapsulating streams facility was added in release 6.1 but not documented until the Allegro CL 6.1 documentation update. In this entry, we discuss the facility and one of the examples. The full story is in stream encapsulations in streams.htm

Stream encapsulation is a method of customizing stream behavior which is a kind of filtering approach: data flows through various streams which have been attached end-to-end, and those data are processed and possibly transformed at each stream stage. Because encapsulation is more modular, each component stream can be simpler and, therefore, more widely applicable.

Here are some definitions:

Encapsulation, encapsulator, encapsulate, encapsulatee: An encapsulation is the attachment of more than one stream of any class in a chain, beginning with the i/o device and ending with the program. A stream X is said to encapsulate a stream Y when stream Y appears as stream X's input-handle or output-handle. In this case, stream X is the encapsulator and stream Y is the encapsulated stream or the encapsulatee.

Input direction and output direction: The input direction has data moving from the i/o device to the program. The output direction has data moving from the program to the i/o device. These are obvious definitions, but are needed as reminders of the definitions or inner and outer, given next, which may be less intuitive.

Inner and outer: The innermost encapsulator is the stream which is closest to the program. The outermost encapsulator is the stream which encapsulates the closest stream to the i/o device (note that the closest stream to the i/o device is not an encapsulator). This naming may seem counterintuitive, since an encapsulator is inside (inner with respect to) the stream it encapsulates. Normal usage would have an encapsulator outside its encapsulatee, but viewing the whole process, inner more usefully describes closer to the program and further from the i/o device.

There are several slots in Allegro CL simple-streams which must be exposed in order for encapsulations to be allowed. We will not go into them here (the descriptions are here). The names of these slots are not (typically) exported. They will be used in the example.

When chaining streams for encapsulation, you have to be sure that they fit together properly. Different streams in Allegro CL do buffering differently. For example, single-channel-simple-streams and dual-channel-simple-streams are octet-oriented, that is, they employ octet buffers internally, while string streams are character-oriented streams, which means that their internal buffers are strings. Only octet streams can be connected to external devices. These differences affect encapsulations.

If we label character-oriented streams with the letter C, and octet-oriented streams with the letter O, then the following kinds of encapsulation configurations can be done:

Simple unencapsulated string stream:

  program - C - string

Simple unencapsulated octet stream, externally connected:

  program - O - i/o

Simple unencapsulated octet stream, internally connected:

  prog - O - octet-buffer

Encapsulations on internally connected string streams:

  prog - C - C - ... - C - string

Encapsulations on internally connected octet streams:

  prog - C - C - ... - C - O - ... - O - buffer

Encapsulations on externally connected octet streams:

  prog - C - C - ... - C - O - ... - O - i/o

In other words, a character oriented stream may encapsulate any kind of stream, and an octet-oriented stream may be encapsulated by any kind of stream, but an octet-oriented stream cannot encapsulate a character-oriented stream.

Encapsulation example

There are fully described examples in Examples of stream encapsulations. We discuss the first example from there in this document and also here. The example shows a string-stream which uses two buffers and which thus allows bidirectional communication on dual-channel encapsulatees.

Rot13 is a simple translation technique used many times to shield internet readers from potentially offensive text. The simple rule is that alphabetic characters are shifted by exactly 13 characters in the alphabet, whichever way is possible. Because there are exactly 26 characters in the English alphabet, two such shifts will reproduce the original result. Thus the original text is easy to get, but it takes a conscious act on the part of the reader to read such text.

This encapsulation example uses a bidirectional string stream with two buffers, and is intended to encapsulate another dual-buffer stream (either another encapsulating dual-buffer string stream or a dual-channel stream).

Here is what an example run looks like:

cl-user(1): :cl rot13b.cl
; Fast rot13b.fasl
cl-user(2): (setq xxx (make-instance 
                       'rot13-bidirectional-stream 
                       :base-stream *terminal-io*))
#<rot13-bidirectional-stream "^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@" @ #x716fa842>
cl-user(3): (format xxx "hello")
uryyb
nil
cl-user(4): (format xxx "uryyb")
hello
nil
cl-user(5): (read-line xxx)
The quick brown fox jumped over the lazy dog.
"Gur dhvpx oebja sbk whzcrq bire gur ynml qbt."
nil
cl-user(6): (read-line xxx)
Gur dhvpx oebja sbk whzcrq bire gur ynml qbt.
"The quick brown fox jumped over the lazy dog."
nil
cl-user(7): 

Note that the encapsulated stream being used above is *terminal-io*, which is the same as *standard-input* in this example transcript. This naturally causes the listener to wait after each (read-line xxx) for the input, just as a (read-line *terminal-io*) or else a (read-line *standard-input*) would do the same thing. Such behavior would not be seen if the stream being encapsulated were a socket or some other terminal stream.

The source code, along with commentary, is provided here.