| Allegro CL version 10.1 Unrevised from 10.0 to 10.1. 10.0 version | ||||||||
This document contains the following sections:
1.0 Introduction to Gray streams in Allegro CL
Since release 6.0, Allegro CL has a new streams model using
simple-streams, which have no element-type, but which act
as if they have element type (unsigned-byte
8). The new implementation is described in
streams.htm. This document describes the Gray
Streams implementation in Allegro CL, which was the stream
implementation in releases prior to 6.0. The
Gray Streams implementation is preserved for backward compatibility.
Gray stream code is not typically included in the Allegro CL
image. Load it with (require :streamc).
In general, a Gray stream is created whenever a file is opened (with
open) with an element-type
specified. If open is called
without an element-type specified, a simple-stream is created. If you
have code which does not specify an element-type, and you want a Gray
stream, specify :element-type 'character. Do note
that you cannot assume that system-created streams will be Gray
streams in version 6.1 or later.
String output streams are always simple-streams even though both make-string-output-stream and with-output-to-string accept an element-type keyword argument. Both operators create simple-streams regardless of whether a value is specified for the element-type keyword argument or not.
The transition from Gray streams to simple-streams should be easy and transparent unless you have done extensive stream customization.
The rest of this document is more or less unchanged from release 5.0.1. It describes the Gray stream implementation.
Symbols naming gray stream functionality are generally in
the :excl package.
Streams in Common Lisp have always been first-class objects and the stream type a first-class type. The inability of user code to customize or extend stream behavior has been an unfortunate limitation especially because the capabilities of the Common Lisp reader and printer can only be obtained through the interface of a stream. Suppose, for example, one wants to do normal Lisp printing while performing some simple output character translations. The only portable way to do this -- that is, without knowing about each implementation's internal functions -- would be to rewrite the whole printer from scratch for the single purpose of interposing a translation function around every call to write-char and write-string. This is clearly unacceptable. It should be possible to customize a stream to add such simple behavior without reimplementing large portions of Common Lisp.
When Common Lisp was first defined the Common Lisp Object System (CLOS) was not yet conceived. Some portable stream construction utilities were originally provided by the several "indirect" stream types - synonym-stream, concatenated-stream, etc. - but these provide only very limited kinds of customization. The subsequent ANSI standard for Common Lisp includes CLOS, so that is now the obvious mechanism to support user-customizable stream types.
Fairly late in the ANSI standardization process there was some consideration of closifying streams. The most complete proposal was by David N. Gray, then of Texas Instruments. Gray's proposal was only a draft and it acknowledged a number of problems and omissions. It was intended only as a starting point towards a complete specification. However, Gray and the standardization committee soon decided it best not to act because the change to the language would be large (causing a lot of work for implementors) and also because there was little actual experience with implementations of CLOS streams. Despite the feeling that it was not yet the time to adopt closified streams into the language standard, most members of the committee feel that redefining streams in terms of CLOS would be intrinsically worthwhile, and experimentation with extensions should be encouraged.
The implementation of streams in Allegro CL from release 4.0 (on Unix) through release 5.0.1 (on Unix and Windows) is based largely on the X3J13 issue writeup entitled "STREAM-DEFINITION-BY-USER, Version 1, 22-Mar-89 by David N. Gray." Some of the text in this chapter is taken from that proposal, but there are numerous additions, modifications, clarifications, and comments specific to the Allegro implementation.
An important feature of the Gray proposal is that it is upward compatible with both the current language standard and earlier implementations of Allegro CL. Programmers need not know anything about closified streams unless they actually use its features.
The symbols naming classes and generic functions for the CLOS Gray
stream interface are exported from the excl package.
The existing Common Lisp I/O functions cannot be made generic because
in nearly every case the stream argument is
optional and therefore cannot be specialized. It is therefore
necessary to define new generic functions which are called internally
by the standard Common Lisp functions. In order to make the meaning as
obvious as possible, the names of the generic functions have been
formed by prefixing "stream-" to the corresponding
non-generic function (e.g. terpri and stream-terpri). Note that for all of these
generic functions, the stream argument must be a stream object,
not t or nil.
Having the generic input functions consistently return
:eof at end-of-file, with the higher-level
functions handling the eof-error-p and eof-value
arguments, simplifies the generic function interface and makes it more
efficient by not needing to pass through those arguments. Note that
the functions that use this convention can only return a character or
integer as a stream element, so there is no possibility of
ambiguity.
There is much uncertainty in the industry about how to document an object-oriented protocol. There are many ways to do it wrong, and it is far from clear how to do it right. Of course, problems with documentation can often be ascribed to poor design of the underlying system itself or else design based on assumptions that are not made explicit.
For example, the default method for stream-write-string is
defined to do repeated calls to stream-write-char. The
character-translation problem mentioned at the beginning of
Section 1.0 Introduction to Gray streams in Allegro CL above could be implemented
minimally by defining an :around method for
stream-write-char, and everything ought to work. But what if
some stream specialization -- say, the
excl::file-gray-stream classes provided by the
implementation -- optimizes out the repeated calls to
stream-write-char in the interest of efficiency? The Gray
proposal is silent whether this would conform, but it is clear that
independently-written mixin classes will need to know each other's
assumptions about which publicly-specializable generic functions do or
do not call each other's publicly-specializable generic
function. There is also controversy whether there should be a default
method for a particular generic function or whether a method
definition should be required on specialized stream classes. The
difference is important if mixin methods normally do a
call-next-method, because the question is left open whether the
default method should be, should not be, or may optionally be
shadowed.
In attempting to answer these questions over the years, we have decided that this is not the best approach, and have opted to redesign streams per documentation in streams.htm.
This said, there are several kinds of information this document needs to specify:
It is a legitimate criticism that this chapter does not exhaustively cover all these details. The legitimate excuse is that cogent, defensible design decisions have not yet been made for all of them. In particular, there is something of a trade-off between (5) and (6) depending on whether one believes subclassing should be defined in terms of class inheritance or in terms of generic function behavior protocol. The new implementation described in streams.htm addresses these issues in a different way.
Two kinds of classes are mentioned here. Some are "mixin" classes intended to be used as super classes of user-defined stream classes. They are not intended to be directly instantiated; they primarily provide places to hang default methods. Others are classes actually instantiated by Allegro CL, for example, to service a call to open.
Those classes that are sufficiently complete to be meaningfully instantiated are labeled as Instantiable Class in their description pages while mixin classes are labeled simply as Class. You can, of course, further subclass both kinds of classes. Although most stream classes have their own description page, the pages do not contain more information than is present in this document and so we have not provided links.
| Table of mixin (non-instantiable) stream classes | |
| Class | Notes |
fundamental-stream |
This class is a subclass of
stream and of
standard-object. streamp will
return true for an instance of any class
that includes this. |
fundamental-input-stream |
A subclass of fundamental-stream. Its
inclusion causes input-stream-p to return true. Note: any user-defined stream class that
will do input must include this class. Bidirectional streams may be formed by including
subclasses of both fundamental-output-stream and fundamental-input-stream. |
fundamental-output-stream |
A subclass of fundamental-stream. Its
inclusion causes output-stream-p to return true. Note: any user-defined
stream class that will do output must include this class. Bidirectional streams may be
formed by including subclasses of both fundamental-output-stream and fundamental-input-stream. |
fundamental-character-stream |
A subclass of fundamental-stream. It
provides a method for stream-element-type which returns character. |
fundamental-binary-stream |
A subclass of fundamental-stream. The
Allegro CL implementation requires the :element-type keyword be provided to make-instance
for streams of this class. |
fundamental-character-input-stream |
Includes fundamental-input-stream and fundamental-character-stream.
Any user-defined stream class that will be used as an argument to read and friends
must include this class. |
fundamental-character-output-stream |
Includes fundamental-input-stream and fundamental-character-stream.
Any user-defined stream class that will be used as an argument to print, format,
etc. must include this class. |
fundamental-binary-input-stream |
Includes fundamental-input-stream and fundamental-binary-stream. |
fundamental-binary-output-stream |
Includes fundamental-output-stream and fundamental-binary-stream. |
file-gray-stream |
See just below. |
excl::string-gray-stream |
See just below. |
ANSI Common Lisp mandates these seven subtypes of type stream: file-stream, string-stream, echo-stream, concatenated-stream, two-way-stream, broadcast-stream, and synonym-stream. The first two are mixin classes
and the rest are instantiable classes. file-stream has two subclasses used in Allegro
CL: excl::file-gray-stream and
excl::file-simple-stream. string-stream also has two subclasses:
excl::string-gray-stream and
excl::string-simple-stream.
Since in Allegro CL's implementation stream is a CLOS class (more
precisely, a subclass of clos:standard-object) then
the subtypes of stream must
also be CLOS classes.
The stream types other than file-stream and string-stream are sometimes called indirect
streams. The creator functions for these stream types
(make-synonym-stream, make-echo-stream,
make-broadcast-stream, make-concatenated-stream, and
make-two-way-stream) are unmodified from the standard Common
Lisp definitions. These stream types represent an early (and awkward)
attempt in Common Lisp to obtain part of an extensible stream
facility. These stream types may not be defined in an image but the
module defining them, streama, will be loaded if it is
require'd or if (find-class 'x)
is evaluated or make-x is called (where
x is one of the classes). streama is also loaded
automatically if any of the Common Lisp slot-readers for one of these
streams is called. (streama stands for
stream-ansi. The other module is
streamc, which originally stood for
stream-clos, but that interpretation is no longer appropriate
since yet another stream module, streamd, is also
CLOS-based (so `c' and `d' are single letter file indentifiers
only). streamd implements simple-streams.
streamc is only loaded into a Lisp when gray
streams are used. Gray streams include the indirect streams --
synonym-stream, concatenated-stream, etc.)
The generic function approach does not integrate very well with
indirect streams because the indirect stream classes cannot anticipate
the full set of generic functions that user code may want to define
over all streams in order to indirect them to the contained
streams. It is arguable that no purely automatic mechanism can handle
this, not even by defining a general method for
no-applicable-method. For example, consider the problem a
two-way-stream has choosing whether to pass to its input stream,
output stream, or both a call to a generic function about which it
knows nothing. For this reason there has been no attempt to extend or
make customizable the five indirect streams.
The five indirect stream classes are instantiable, but at present cannot successfully be instantiated other than with their standard constructor functions (e.g. make-synonym-stream).
The excl::file-gray-stream class is customizable
since it embodies the interface to the file
system. excl::file-gray-stream is a mixin not intended to be
instantiated directly. Its subclasses are normally instantiated by the
open function (see below) although it is also possible to
create them directly with make-instance.
| Table of instantiable stream classes | |
| Class | Notes |
cl:echo-stream |
See discussion above. Created with cl:make-echo-stream. |
cl:concatenated-stream |
See discussion above. Created with cl:make-concatenated-stream. |
cl:two-way-stream |
See discussion above. Created with cl:make-two-way-stream. |
cl:broadcast-stream |
See discussion above. Created with cl:make-broadcast-stream. |
cl:synonym-stream. |
See discussion above. Created with cl:make-synonym-stream. |
input-terminal-stream |
All six classes implement streams intended
to support connection to sockets and input-output
devices that connect to a
"stream" of data rather than a fixed file
stored in a file system. A normal call
to open without the These classes all require an |
output-terminal-stream |
|
bidirectional-terminal-stream |
|
input-binary-socket-stream |
|
output-binary-socket-stream |
|
bidirectional-binary-socket-stream |
|
A character input stream class can be defined by including
fundamental-character-input-stream and defining methods
for the generic functions below. The builtin instantiable classes in
Allegro CL already have appropriate methods, of course.
| Generic function | Arguments | Notes |
| stream-read-char | stream | This reads one character from stream. |
| stream-unread-char | stream character | Undoes the last call to
stream-read-char, as
in unread-char.
Returns nil. |
| stream-read-char-no-hang | stream | This is used to implement
read-char-no-hang.
It returns either a character, or nil
if no input is currently available, or :eof
if end-of-file is reached. |
| stream-peek-char | stream | Used to implement
peek-char; this
corresponds to the case where the peek-type
argument is nil. It
returns either a character or :eof. |
| stream-listen | stream | Used by listen. Returns true or false as there is or is not a character available to be read. |
| stream-read-line | stream | Used by read-line. A string is returned as the first value. The second value is true if the string was terminated by end-of-file instead of the end of a line. |
| stream-clear-input | stream | Implements clear-input
for stream, returning
nil. |
A character output stream can be created by defining a class that
includes fundamental-character-output-stream and defining
methods for the generic functions below.
| Generic function | Arguments | Notes |
| stream-write-char | stream character | Writes character to stream and returns character. |
| stream-line-column | stream | This function returns the column
number where the next
character will be written, or nil
if that is not meaningful for stream.
The first column on a line is numbered 0. |
| stream-start-line-p | stream | This is a predicate which
returns t if stream is
positioned at the beginning of a line, else
returns nil. It is permissible to
always return nil. This is called by
stream-fresh-line. |
| stream-write-string | stream string &optional start end | Implements write-string.
It writes string to
stream, optionally delimited by start and
end, which default to 0 and nil.
The string argument is returned. |
| stream-terpri | stream | Writes an end of line, as for
terpri.
Returns nil. |
| stream-fresh-line | stream | Implements fresh-line. |
| stream-finish-output | stream | Implements finish-output |
| stream-force-output | stream | Implements force-output. |
| stream-advance-to-column | stream column | Writes enough blank space so that
the next character will be
written at the specified column. Returns true if the operation
is successful, or nil if it is not
supported for stream. |
| stream-clear-input | stream | Implements clear-input. |
A binary stream class can be defined by including either or both of
fundamental-binary-input-stream
and
fundamental-binary-output-stream,
and methods for one or
both of the following generic functions.
If you create a binary stream
other than by using open you must also include a keyword
initialization argument for :element-type.
| Generic function | Arguments | Notes |
| stream-read-byte | stream | Used by read-byte; returns either an integer, or the symbol :eof if stream is at end-of-file. |
| stream-write-byte | stream integer | Used by write-byte; writes the integer to stream and returns the integer as the result. |
In the public comments on the first Draft Proposed ANSI Standard several reviewers noted that except for the write-string function, the language provided no means to request efficient input-output on large blocks of data. The ANSI standard consequently added two new functions to the language read-sequence and write-sequence.
Allegro CL also defines generic function versions of these functions. The nongeneric versions are implemented by calling these generic functions. Methods are defined for these functions that handle all legal calls, but not all legal calls will have highly efficient execution. The existing methods will handle vector sequence arguments efficiently, and will also handle start and end arguments efficiently. The Allegro CL implementation will also accept and efficiently transfer data to and from higher-dimension arrays in the usual row-major order, although this is an extension to the language and not defined by ANSI Common Lisp.
| Generic function | Arguments | Notes |
| stream-read-sequence | stream sequence &optional start end | Destructively modifies sequence, storing in it elements read from stream. |
| stream-write-sequence | stream sequence &optional start end | Writes elements of sequence to stream. |
The definition of these functions does not permit the useful ability to operate on sequence elements different from the stream element type. Such capability would clearly be desirable. (It would, for example, permit floats to be transferred efficiently, and allow an application needing to store several different successive sequences of different types to write them to a single stream and later reread the data.) However, the behavior of any such operation would depend on bit representations of objects in an implementation- and machine-dependent way, and would be incompatible with the strict definition of read-sequence and write-sequence, which require automatic conversion of data to the correct element type of the stream or sequence. Any such capability would therefore need to be implemented by a different pair of functions.
Streams of arbitrary class may be constructed by an explicit call to
make-instance, but the Gray proposal did not address
how to customize the stream created by open. We
define a simple interface here. The Gray proposal also omits mention
of constructor functions such as
make-string-input-stream,
make-string-output-stream, and their associated
macros such as with-output-to-string. However, there
is nothing these various operators do that can't be performed
explicitly by user code including a call to
make-instance. Unfortunately, Allegro CL's current
string-gray-stream subclasses do not (reliably) support
specialization or even independent instantiation by
make-instance. This is a bug in that some required
initialization is performed by the
make-string-...-stream function.
The open function has been
extended to take a class keyword argument. open passes this
argument to make-instance when it creates the stream,
and as with make-instance, the argument may be a
stream class object or a symbol naming such a class. If the
class argument is not supplied or is nil, open selects one of the
following built-in classes according to the direction and
element-type arguments:
excl::character-input-file-stream excl::character-output-file-stream excl::character-bidirectional-file-stream excl::binary-input-file-stream excl::binary-output-file-stream excl::binary-bidirectional-file-stream
These classes all contain excl::file-gray-stream
and are variously mixed with
fundamental-character-input-stream fundamental-character-output-stream fundamental-binary-input-stream fundamental-binary-output-stream
Although the excl::file-gray-stream subclasses
returned by open are all instantiable, at present
they require hidden initialization (for element-type upgrading, buffer
allocation, etc.) and therefore they should only be created using
open. It is fine to further specialize them, but you
are required to create instances of your specializations of these
stream classes using the class keyword argument to
open rather than by calling
make-instance yourself.
On Unix platforms, open will select a terminal-stream
subclass rather than a excl::file-gray-stream
subclass if it detects the argument file is a character special device
or a named pipe. See socket.htm for more
information about creating sockets in Allegro CL.
The following functions are also defined (or in the case of standard Common Lisp functions, extended).
| Generic function | Arguments | Notes |
| close | stream &key abort | The existing function close is redefined to be a generic function but otherwise behaves as required by standard Common Lisp. |
| open-stream-p | stream | This function is made generic. A default method is provided by class fundamental-stream which returns true if close has not been called on stream. |
| streamp | object | The original proposal allowed but did not require these three existing predicates to be implemented as generic functions. In Allegro CL, streamp is not a generic function in order not to impact speed of opencoded type dispatching, but the other two functions are made generic. Normally, the default methods provided by classes fundamental-input-stream and fundamental-output-stream are sufficient. |
| input-stream-p | stream | |
| output-stream-p | stream | |
| stream-element-type | stream | This existing function is made generic, but otherwise behaves the same. Class fundamental-character-stream provides a default method which returns character. |
| stream-yes-or-no-p | stream &optional format-string &rest args | The generic function analogue of cl:yes-or-no-p. |
| stream-y-or-n-p | stream &optional format-string &rest args | The generic function analogue of cl:y-or-n-p. |
| stream-input-fn | stream | These accessors return
the input or output file
descriptor for streams that have them, or
nil. For some streams the input and
output file descriptors may be the same. |
| stream-output-fn | stream |
The following function is used by the pretty printer to determine the output width to be used while pretty printing. Actually, the pretty printer uses the first of the following three values that return true.
*print-right-margin*excl::*default-right-margin*,
an internal variable set when
Lisp starts up to the apparent width of
cl:*terminal-io*, if it can be
determined, and to 72 otherwise.| Generic function | Arguments | Notes |
| stream-output-width | stream | Returns an integer width of stream
or nil. The
default method for this function returns
nil for all streams. |
Copyright (c) 1998-2022, Franz Inc. Lafayette, CA., USA. All rights reserved.
This page was not revised from the 10.0 page.
Created 2019.8.20.
| Allegro CL version 10.1 Unrevised from 10.0 to 10.1. 10.0 version | ||||||||