|
Allegro CL version 11.0 | ||||||||
Since release 6.0, Allegro CL has a new streams model using simple-streams, which have no element-type, but which act as if they have element type (unsigned-byte 8). The new implementation is described in streams.html. This document describes the Gray Streams implementation in Allegro CL, which was the stream implementation in releases prior to 6.0. The Gray Streams implementation is preserved for backward compatibility.
Gray stream code is not typically included in the Allegro CL image. Load it with (require :streamc).
In general, a Gray stream is created whenever a file is opened (with open) with an element-type specified. If open is called without an element-type specified, a simple-stream is created. If you have code which does not specify an element-type, and you want a Gray stream, specify :element-type 'character. Do note that you cannot assume that system-created streams will be Gray streams in version 6.1 or later.
String output streams are always simple-streams even though both make-string-output-stream and with-output-to-string accept an element-type keyword argument. Both operators create simple-streams regardless of whether a value is specified for the element-type keyword argument or not.
The transition from Gray streams to simple-streams should be easy and transparent unless you have done extensive stream customization.
The rest of this document is more or less unchanged from release 5.0.1. It describes the Gray stream implementation.
Symbols naming gray stream functionality are generally in the :excl package.
Streams in Common Lisp have always been first-class objects and the stream type a first-class type. The inability of user code to customize or extend stream behavior has been an unfortunate limitation especially because the capabilities of the Common Lisp reader and printer can only be obtained through the interface of a stream. Suppose, for example, one wants to do normal Lisp printing while performing some simple output character translations. The only portable way to do this -- that is, without knowing about each implementation's internal functions -- would be to rewrite the whole printer from scratch for the single purpose of interposing a translation function around every call to write-char and write-string. This is clearly unacceptable. It should be possible to customize a stream to add such simple behavior without reimplementing large portions of Common Lisp.
When Common Lisp was first defined the Common Lisp Object System (CLOS) was not yet conceived. Some portable stream construction utilities were originally provided by the several "indirect" stream types - synonym-stream, concatenated-stream, etc. - but these provide only very limited kinds of customization. The subsequent ANSI standard for Common Lisp includes CLOS, so that is now the obvious mechanism to support user-customizable stream types.
Fairly late in the ANSI standardization process there was some consideration of closifying streams. The most complete proposal was by David N. Gray, then of Texas Instruments. Gray's proposal was only a draft and it acknowledged a number of problems and omissions. It was intended only as a starting point towards a complete specification. However, Gray and the standardization committee soon decided it best not to act because the change to the language would be large (causing a lot of work for implementors) and also because there was little actual experience with implementations of CLOS streams. Despite the feeling that it was not yet the time to adopt closified streams into the language standard, most members of the committee feel that redefining streams in terms of CLOS would be intrinsically worthwhile, and experimentation with extensions should be encouraged.
The implementation of streams in Allegro CL from release 4.0 (on Unix) through release 5.0.1 (on Unix and Windows) is based largely on the X3J13 issue writeup entitled "STREAM-DEFINITION-BY-USER, Version 1, 22-Mar-89 by David N. Gray." Some of the text in this chapter is taken from that proposal, but there are numerous additions, modifications, clarifications, and comments specific to the Allegro implementation.
An important feature of the Gray proposal is that it is upward compatible with both the current language standard and earlier implementations of Allegro CL. Programmers need not know anything about closified streams unless they actually use its features.
The symbols naming classes and generic functions for the CLOS Gray stream interface are exported from the excl package.
The existing Common Lisp I/O functions cannot be made generic because in nearly every case the stream argument is optional and therefore cannot be specialized. It is therefore necessary to define new generic functions which are called internally by the standard Common Lisp functions. In order to make the meaning as obvious as possible, the names of the generic functions have been formed by prefixing "stream-" to the corresponding non-generic function (e.g. terpri and stream-terpri). Note that for all of these generic functions, the stream argument must be a stream object, not t or nil.
Having the generic input functions consistently return :eof at end-of-file, with the higher-level functions handling the eof-error-p and eof-value arguments, simplifies the generic function interface and makes it more efficient by not needing to pass through those arguments. Note that the functions that use this convention can only return a character or integer as a stream element, so there is no possibility of ambiguity.
There is much uncertainty in the industry about how to document an object-oriented protocol. There are many ways to do it wrong, and it is far from clear how to do it right. Of course, problems with documentation can often be ascribed to poor design of the underlying system itself or else design based on assumptions that are not made explicit.
For example, the default method for stream-write-string is defined to do repeated calls to stream-write-char. The character-translation problem mentioned at the beginning of Introduction to Gray streams in Allegro CL above could be implemented minimally by defining an :around method for stream-write-char, and everything ought to work. But what if some stream specialization -- say, the excl::file-gray-stream classes provided by the implementation -- optimizes out the repeated calls to stream-write-char in the interest of efficiency? The Gray proposal is silent whether this would conform, but it is clear that independently-written mixin classes will need to know each other's assumptions about which publicly-specializable generic functions do or do not call each other's publicly-specializable generic function. There is also controversy whether there should be a default method for a particular generic function or whether a method definition should be required on specialized stream classes. The difference is important if mixin methods normally do a call-next-method, because the question is left open whether the default method should be, should not be, or may optionally be shadowed.
In attempting to answer these questions over the years, we have decided that this is not the best approach, and have opted to redesign streams per documentation in streams.html.
This said, there are several kinds of information this document needs to specify:
It is a legitimate criticism that this chapter does not exhaustively cover all these details. The legitimate excuse is that cogent, defensible design decisions have not yet been made for all of them. In particular, there is something of a trade-off between (5) and (6) depending on whether one believes subclassing should be defined in terms of class inheritance or in terms of generic function behavior protocol. The new implementation described in streams.html addresses these issues in a different way.
Two kinds of classes are mentioned here. Some are "mixin" classes intended to be used as super classes of user-defined stream classes. They are not intended to be directly instantiated; they primarily provide places to hang default methods. Others are classes actually instantiated by Allegro CL, for example, to service a call to open.
Those classes that are sufficiently complete to be meaningfully instantiated are labeled as Instantiable Class in their description pages while mixin classes are labeled simply as Class. You can, of course, further subclass both kinds of classes. Although most stream classes have their own description page, the pages do not contain more information than is present in this document and so we have not provided links.
| Table of mixin (non-instantiable) stream classes | |
| Class | Notes |
| fundamental-stream | This class is a subclass of
stream and of
standard-object. streamp will
return true for an instance of any class
that includes this. |
| fundamental-input-stream | A subclass of fundamental-stream. Its
inclusion causes input-stream-p to return true. Note: any user-defined stream class that
will do input must include this class. Bidirectional streams may be formed by including
subclasses of both fundamental-output-stream and fundamental-input-stream. |
fundamental-output-stream |
A subclass of fundamental-stream. Its
inclusion causes output-stream-p to return true. Note: any user-defined
stream class that will do output must include this class. Bidirectional streams may be
formed by including subclasses of both fundamental-output-stream and fundamental-input-stream. |
fundamental-character-stream |
A subclass of fundamental-stream. It
provides a method for stream-element-type which returns character. |
fundamental-binary-stream |
A subclass of fundamental-stream. The
Allegro CL implementation requires the :element-type keyword be provided to make-instance
for streams of this class. |
fundamental-character-input-stream |
Includes fundamental-input-stream and fundamental-character-stream.
Any user-defined stream class that will be used as an argument to read and friends
must include this class. |
fundamental-character-output-stream |
Includes fundamental-input-stream and fundamental-character-stream.
Any user-defined stream class that will be used as an argument to print, format,
etc. must include this class. |
fundamental-binary-input-stream |
Includes fundamental-input-stream and fundamental-binary-stream. |
fundamental-binary-output-stream |
Includes fundamental-output-stream and fundamental-binary-stream. |
|
|
See just below. |
|
|
See just below. |
ANSI Common Lisp mandates these seven subtypes of type stream: file-stream, string-stream, echo-stream, concatenated-stream, two-way-stream, broadcast-stream, and synonym-stream. The first two are mixin classes and the rest are instantiable classes. file-stream has two subclasses used in Allegro CL: excl::file-gray-stream and excl::file-simple-stream. string-stream also has two subclasses: excl::string-gray-stream and excl::string-simple-stream.
Since in Allegro CL's implementation stream is a CLOS class (more precisely, a subclass of clos:standard-object) then the subtypes of stream must also be CLOS classes.
The stream types other than file-stream and string-stream are sometimes called indirect streams. The creator functions for these stream types (make-synonym-stream, make-echo-stream, make-broadcast-stream, make-concatenated-stream, and make-two-way-stream) are unmodified from the standard Common Lisp definitions. These stream types represent an early (and awkward) attempt in Common Lisp to obtain part of an extensible stream facility. These stream types may not be defined in an image but the module defining them, streama, will be loaded if it is require'd or if (find-class 'x) is evaluated or make-x is called (where x is one of the classes). streama is also loaded automatically if any of the Common Lisp slot-readers for one of these streams is called. (streama stands for stream-ansi. The other module is streamc, which originally stood for stream-clos, but that interpretation is no longer appropriate since yet another stream module, streamd, is also CLOS-based (so c and d are single letter file indentifiers only). streamd implements simple-streams. streamc is only loaded into a Lisp when gray streams are used. Gray streams include the indirect streams -- synonym-stream, concatenated-stream, etc.)
The generic function approach does not integrate very well with indirect streams because the indirect stream classes cannot anticipate the full set of generic functions that user code may want to define over all streams in order to indirect them to the contained streams. It is arguable that no purely automatic mechanism can handle this, not even by defining a general method for no-applicable-method. For example, consider the problem a two-way-stream has choosing whether to pass to its input stream, output stream, or both a call to a generic function about which it knows nothing. For this reason there has been no attempt to extend or make customizable the five indirect streams.
The five indirect stream classes are instantiable, but at present cannot successfully be instantiated other than with their standard constructor functions (e.g. make-synonym-stream).
The excl::file-gray-stream class is customizable since it embodies the interface to the file system. excl::file-gray-stream is a mixin not intended to be instantiated directly. Its subclasses are normally instantiated by the open function (see below) although it is also possible to create them directly with make-instance.
| Table of instantiable stream classes | |
| Class | Notes |
cl:echo-stream |
See discussion above. Created with cl:make-echo-stream. |
cl:concatenated-stream |
See discussion above. Created with cl:make-concatenated-stream. |
cl:two-way-stream |
See discussion above. Created with cl:make-two-way-stream. |
cl:broadcast-stream |
See discussion above. Created with cl:make-broadcast-stream. |
cl:synonym-stream. |
See discussion above. Created with cl:make-synonym-stream. |
input-terminal-stream |
All six classes implement streams intended
to support connection to sockets and input-output
devices that connect to a
"stream" of data rather than a fixed file
stored in a file system. A normal call
to open without the These classes all require an |
output-terminal-stream |
|
bidirectional-terminal-stream |
|
input-binary-socket-stream |
|
output-binary-socket-stream |
|
bidirectional-binary-socket-stream |
|
A character input stream class can be defined by including fundamental-character-input-stream and defining methods for the generic functions below. The builtin instantiable classes in Allegro CL already have appropriate methods, of course.
| Generic function | Arguments | Notes |
| stream-read-char | stream | This reads one character from stream. |
| stream-unread-char | stream character | Undoes the last call to
stream-read-char, as
in unread-char.
Returns nil. |
| stream-read-char-no-hang | stream | This is used to implement
read-char-no-hang.
It returns either a character, or nil
if no input is currently available, or :eof
if end-of-file is reached. |
| stream-peek-char | stream | Used to implement
peek-char; this
corresponds to the case where the peek-type
argument is nil. It
returns either a character or :eof. |
| stream-listen | stream | Used by listen. Returns true or false as there is or is not a character available to be read. |
| stream-read-line | stream | Used by read-line. A string is returned as the first value. The second value is true if the string was terminated by end-of-file instead of the end of a line. |
| stream-clear-input | stream | Implements clear-input
for stream, returning
nil. |
A character output stream can be created by defining a class that includes fundamental-character-output-stream and defining methods for the generic functions below.
| Generic function | Arguments | Notes |
| stream-write-char | stream character | Writes character to stream and returns character. |
| stream-line-column | stream | This function returns the column
number where the next
character will be written, or nil
if that is not meaningful for stream.
The first column on a line is numbered 0. |
| stream-start-line-p | stream | This is a predicate which
returns t if stream is
positioned at the beginning of a line, else
returns nil. It is permissible to
always return nil. This is called by
stream-fresh-line. |
| stream-write-string | stream string &optional start end | Implements write-string.
It writes string to
stream, optionally delimited by start and
end, which default to 0 and nil.
The string argument is returned. |
| stream-terpri | stream | Writes an end of line, as for
terpri.
Returns nil. |
| stream-fresh-line | stream | Implements fresh-line. |
| stream-finish-output | stream | Implements finish-output |
| stream-force-output | stream | Implements force-output. |
| stream-advance-to-column | stream column | Writes enough blank space so that
the next character will be
written at the specified column. Returns true if the operation
is successful, or nil if it is not
supported for stream. |
| stream-clear-input | stream | Implements clear-input. |
A binary stream class can be defined by including either or both of fundamental-binary-input-stream and fundamental-binary-output-stream, and methods for one or both of the following generic functions. If you create a binary stream other than by using open you must also include a keyword initialization argument for :element-type.
| Generic function | Arguments | Notes |
| stream-read-byte | stream | Used by read-byte; returns either an integer, or the symbol :eof if stream is at end-of-file. |
| stream-write-byte | stream integer | Used by write-byte; writes the integer to stream and returns the integer as the result. |
In the public comments on the first Draft Proposed ANSI Standard several reviewers noted that except for the write-string function, the language provided no means to request efficient input-output on large blocks of data. The ANSI standard consequently added two new functions to the language read-sequence and write-sequence.
Allegro CL also defines generic function versions of these functions. The nongeneric versions are implemented by calling these generic functions. Methods are defined for these functions that handle all legal calls, but not all legal calls will have highly efficient execution. The existing methods will handle vector sequence arguments efficiently, and will also handle start and end arguments efficiently. The Allegro CL implementation will also accept and efficiently transfer data to and from higher-dimension arrays in the usual row-major order, although this is an extension to the language and not defined by ANSI Common Lisp.
| Generic function | Arguments | Notes |
| stream-read-sequence | stream sequence &optional start end | Destructively modifies sequence, storing in it elements read from stream. |
| stream-write-sequence | stream sequence &optional start end | Writes elements of sequence to stream. |
The definition of these functions does not permit the useful ability to operate on sequence elements different from the stream element type. Such capability would clearly be desirable. (It would, for example, permit floats to be transferred efficiently, and allow an application needing to store several different successive sequences of different types to write them to a single stream and later reread the data.) However, the behavior of any such operation would depend on bit representations of objects in an implementation- and machine-dependent way, and would be incompatible with the strict definition of read-sequence and write-sequence, which require automatic conversion of data to the correct element type of the stream or sequence. Any such capability would therefore need to be implemented by a different pair of functions.
Streams of arbitrary class may be constructed by an explicit call to make-instance, but the Gray proposal did not address how to customize the stream created by open. We define a simple interface here. The Gray proposal also omits mention of constructor functions such as make-string-input-stream, make-string-output-stream, and their associated macros such as with-output-to-string. However, there is nothing these various operators do that can't be performed explicitly by user code including a call to make-instance. Unfortunately, Allegro CL's current string-gray-stream subclasses do not (reliably) support specialization or even independent instantiation by make-instance. This is a bug in that some required initialization is performed by the make-string-...-stream function.
The open function has been extended to take a class keyword argument. open passes this argument to make-instance when it creates the stream, and as with make-instance, the argument may be a stream class object or a symbol naming such a class. If the class argument is not supplied or is nil, open selects one of the following built-in classes according to the direction and element-type arguments:
excl::character-input-file-stream
excl::character-output-file-stream
excl::character-bidirectional-file-stream
excl::binary-input-file-stream
excl::binary-output-file-stream
excl::binary-bidirectional-file-streamThese classes all contain excl::file-gray-stream and are variously mixed with
fundamental-character-input-stream
fundamental-character-output-stream
fundamental-binary-input-stream
fundamental-binary-output-streamAlthough the excl::file-gray-stream subclasses returned by open are all instantiable, at present they require hidden initialization (for element-type upgrading, buffer allocation, etc.) and therefore they should only be created using open. It is fine to further specialize them, but you are required to create instances of your specializations of these stream classes using the class keyword argument to open rather than by calling make-instance yourself.
On Unix platforms, open will select a terminal-stream subclass rather than a excl::file-gray-stream subclass if it detects the argument file is a character special device or a named pipe. See socket.html for more information about creating sockets in Allegro CL.
The following functions are also defined (or in the case of standard Common Lisp functions, extended).
| Generic function | Arguments | Notes |
| close | stream &key abort | The existing function close is redefined to be a generic function but otherwise behaves as required by standard Common Lisp. |
| open-stream-p | stream | This function is made generic. A default method is provided by class fundamental-stream which returns true if close has not been called on stream. |
| streamp | object | The original proposal allowed but did not require these three existing predicates to be implemented as generic functions. In Allegro CL, streamp is not a generic function in order not to impact speed of opencoded type dispatching, but the other two functions are made generic. Normally, the default methods provided by classes fundamental-input-stream and fundamental-output-stream are sufficient. |
| stream | ||
| stream | ||
| stream-element-type | stream | This existing function is made generic, but otherwise behaves the same. Class fundamental-character-stream provides a default method which returns character. |
| stream-yes-or-no-p | stream &optional format-string &rest args | The generic function analogue of cl:yes-or-no-p. |
| stream-y-or-n-p | stream &optional format-string &rest args | |
| stream-input-fn | stream | These accessors return
the input or output file
descriptor for streams that have them, or
nil. For some streams the input and
output file descriptors may be the same. |
| stream-output-fn | stream |
The following function is used by the pretty printer to determine the output width to be used while pretty printing. Actually, the pretty printer uses the first of the following three values that return true.
the value of *print-right-margin*
the value returned by stream-output-width
the value of excl::*default-right-margin*, an internal variable set when Lisp starts up to the apparent width of cl:terminal-io, if it can be determined, and to 72 otherwise.
| Generic function | Arguments | Notes |
| stream-output-width | stream | Returns an integer width of stream
or nil. The
default method for this function returns
nil for all streams. |
Copyright (c) Franz Inc. Lafayette, CA., USA. All rights reserved.
|
Allegro CL version 11.0 | ||||||||