ToCDocOverviewCGDocRelNotesFAQIndexPermutedIndex
Allegro CL version 8.1
Unrevised from 8.0 to 8.1. Moderate update since 8.1 release.
8.0 version

Details of the Allegro CL Implementation

This document contains the following sections:

1.0 Implementation introduction
2.0 Data types and array types
3.0 Arrays and short arrays
   3.1 Array short-ness
   3.2 Relationship of arrays to array-like structures
   3.3 Short-arrays in the type hierarchy
      3.3.1 String comparisons with short strings
4.0 Characters
5.0 Autoloading
   5.1 Where the autoloaded files are located
   5.2 Common Lisp symbols
   5.3 Major extensions
   5.4 How to load modules
6.0 Miscellaneous implementation details
   6.1 Extensions to cl:make-package, cl:intern, cl:disassemble, cl:truename, cl:probe-file, cl:open, cl:apropos, etc.
      6.1.1 Extensions to cl:make-package
      6.1.2 Extensions to cl:intern
      6.1.3 Extensions to cl:disassemble
      6.1.4 Extensions to cl:truename
      6.1.5 Extensions to cl:probe-file
      6.1.6 Extensions to cl:open
      6.1.7 Extensions to cl:apropos
      6.1.8 Extensions to cl:interactive-stream-p
   6.2 A comment about with-open-file and timing hazards
   6.3 cl:directory
   6.4 Reader macros and cl:*features*
      6.4.1 Features present or missing from *features* in Allegro CL
      6.4.2 The issue of nested conditionals in Allegro CL
   6.5 cl:random and cl:make-random-state
   6.6 cl:make-hash-table
   6.7 cl:make-array
   6.8 cl:namestring
   6.9 cl:defpackage
   6.10 cl:file-length
   6.11 cl:file-write-date
   6.12 cl:lisp-implementation-version
   6.13 Functionality for quickly writing and reading floats
   6.14 cl:provide and cl:require
   6.15 cl:macroexpand and cl:macroexpand-1
   6.16 cl:simple-condition-format-arguments and cl:simple-condition-format-control
   6.17 What user-homedir-pathname does on Windows
   6.18 Speed and pretty printing
   6.19 Floating-point infinities and NaNs, and floating-point underflow and overflow
   6.20 64 bit Allegro CL Implementations
7.0 Allegro CL and the ANSI CL standard
   7.1 Compatibility with pre-ANSI CLtL-1 in Allegro CL
   7.2 Other package changes and compile-time-too behavior
   7.3 The function data type
   7.4 CLOS and MOP
   7.5 CLOS and MOP conformance
   7.6 CLOS optimization
8.0 Function specs (fspecs)
9.0 Some low-level functionality
   9.1 Windows: GetWinMainArgs2
10.0 Conformance with the ANSI specification


1.0 Implementation introduction

The Common Lisp standard is deliberately vague on many of the specifics of an implementation. The authors of that book were aware that implementation details are dependent on the nature of the hardware and the operating system, as well as the differing priorities of the implementors and the different user communities. This document details some of the specifics of the implementation of and extensions in Allegro CL.



2.0 Data types and array types

Allegro CL contains all of the required Common Lisp data types. Fixnums are signed 30-bit quantities (29 bits of value, one sign bit) on 32-bit machines and signed 61-bit quantities (60 bits of value, one sign bit) on 64 bit machines. There are two distinct floating-point types on all platforms (32 bit and 64 bit floats). Short-float and single-float are equivalent and are 32 bit floats. Double-float and long-float are equivalent and are 64 bit floats.

The distinct array data types are shown in the following list (in the case of the simple arrays, we use suspension points, `...', to indicate that there may be any number of dimensions). When you specify an element-type to make-array, you will get an array whose element type is upgraded-array-element-type applied to the value specified. Arrays of type t are general arrays and arrays of any other type are called specialized arrays. After the list, we give some examples of the types of arrays created with particular values for element-type.

The new short-array type is not mentioned in this list. See Section 3.0 Arrays and short arrays for information on short-arrays and also on maximum array sizes.

Allegro CL allows most types of arrays to be allocated in static space (where they are never moved or even looked at the the garbage collector). See Section 6.7 cl:make-array for information on creating such arrays. Only (as noted) arrays of type t cannot be allocated in static space (because such arrays usually contain pointers to other Lisp objects whicxh must be looked at and updated by the garbage collector when the objects pointed to are relocated).

  (array t)  ;; cannot have :allocation :static, 
             ;; :malloc, or :static-reclaimable as noted above
  (array bit)
  (array (unsigned-byte 4))
  (array (unsigned-byte 8))
  (array (unsigned-byte 16))
  (array (unsigned-byte 32))
  (array (unsigned-byte 64)) [64-bit Lisps only]
  (array character)
  (array single-float)
  (array double-float)
  (array fixnum)
  (array (complex single-float))
  (array (complex double-float))
  (array (signed-byte 8))
  (array (signed-byte 16))
  (array (signed-byte 32))
  (array (signed-byte 64)) [64-bit Lisps only]
  (array nil) 
  (simple-array t (* ...))
  (simple-array bit (* ...))
  (simple-array (unsigned-byte 4) (* ...))
  (simple-array (unsigned-byte 8) (* ...))
  (simple-array (unsigned-byte 16) (* ...))
  (simple-array (unsigned-byte 32) (* ...))
  (simple-array (unsigned-byte 64) (* ...)) [64-bit Lisps only]
  (simple-array character (* ...))
  (simple-array single-float (* ...))
  (simple-array double-float (* ...))
  (simple-array fixnum (* ...))
  (simple-array (signed-byte 8) (* ...))
  (simple-array (signed-byte 16) (* ...))
  (simple-array (signed-byte 32) (* ...))
  (simple-array (signed-byte 64) (* ...)) [64-bit Lisps only]
  (simple-array nil (* ...))

Now let us look at some examples. When we specify (unsigned-byte 3) as the value of element-type, we get an array of type (unsigned-byte 4):

cl-user(2): (setq fn-arr 
                  (make-array 5 :element-type '(unsigned-byte 3) 
                                :initial-element 0))
#(0 0 0 0 0)
cl-user(3): (array-element-type fn-arr)
(unsigned-byte 4)
cl-user(4): (upgraded-array-element-type '(unsigned-byte 3))
(unsigned-byte 4)
cl-user(5): 

Note that upgraded-array-element-type applied to (unsigned-byte 3) returns (unsigned-byte 4). Note too that we have specified 0 as the value of the initial-value. If we had not, the initial value would be nil, which is not of type (unsigned-byte 4).

Here is what is returned by upgraded-array-element-type for some other common types:

cl-user(7): (upgraded-array-element-type 'single-float)
single-float
cl-user(8): (upgraded-array-element-type 'double-float)
double-float
cl-user(9): (upgraded-array-element-type 'float)
t
cl-user(10): (upgraded-array-element-type 'integer)
t
cl-user(11): (upgraded-array-element-type 'character)
character
cl-user(12): (upgraded-array-element-type '(signed-byte 6))
(signed-byte 8)
cl-user(13): (upgraded-array-element-type '(unsigned-byte 100))
t

Note that specifying float and integer both result in arrays of type t, not in specialized arrays. Specifying signed or unsigned bytes of particular sizes results in that size or bigger, or possibly t.

It is good programming practice to use upgraded-array-element-type to determine exactly what sort of array you will get.

Stack allocating vectors

Certain types of vectors can be stack allocated, thus saving space in applications. See Stack consing, avoiding consing using apply, and stack allocation in compiling.htm for details.



3.0 Arrays and short arrays

Arrays sre stored internally as vectors. The underlying vector associated with an array is accessible with the macro excl:with-underlying-simple-vector.

Release 7.0 contains a new array implementation with larger array size limits: now most-positive-fixnum. In earlier releases, the limit in 32-bit Lisps was (expt 2 24), a value 32 times smaller. In 64-bit images, the limit was (expt 2 56), 16 times smaller than the new most-positive-fixnum limit.

Because the structure of arrays had to change in order to implement this change, and because there exists the possibility that users have done some coding which assumes a particular arrangement for arrays (such as is the case for the lisp.h file for compiling C code to recognize lisp structure), we have retained the older array types with their smaller limits, and have renamed them to be short arrays.

The make-array function now accepts the additional :short keyword argument. :short defaults to nil and when nil, a (long) array is produced, and when specified true, a short array (that used in earlier releases) is produced, with these exceptions:

The next section discusses these and other anomalies.

The new functions short-vector and short-string create short vectors and short strings, analogously to the standard functions vector and string.


3.1 Array short-ness

Most of the array set is symmetrical with respect to short-ness; i.e. a call to make-array for most element types will either produce a simple-array or short-simple-array of the specified element-type, based on the :short argument, (and, for specifications which normally create non-simple arrays, these results will be either arrays or short-arrays of the specified element-type) with the following exceptions:

  1. The fixnum element-type has been added to (long) arrays only, if specified as short array, the element-type is upgraded to (unsigned-byte 32). For example:
    CL-USER(2): (type-of (make-array 10 :element-type 'fixnum))
    (SIMPLE-ARRAY FIXNUM (10))
    CL-USER(3): (type-of (make-array 10 :element-type 'fixnum :short t))
    (SHORT-SIMPLE-ARRAY (SIGNED-BYTE 32) (10))
    CL-USER(4): 
    
  2. An element-type specification of nil always results in a (long) array:
    CL-USER(4): (type-of (make-array 0 :element-type nil))
    (SIMPLE-ARRAY NIL (0))
    CL-USER(5): (type-of (make-array 0 :element-type nil :short t))
    (SIMPLE-ARRAY NIL (0))
    CL-USER(6): 
    
  3. An element-type of excl::foreign always results in a short array:
    CL-USER(6): (type-of (make-array 5 :element-type 'excl::foreign))
    (SHORT-SIMPLE-ARRAY FOREIGN (5))
    CL-USER(7): (type-of (make-array 5 :element-type 'excl::foreign :short t))
    (SHORT-SIMPLE-ARRAY FOREIGN (5))
    CL-USER(8): 
    

In all other cases, the "short-ness" of arrays depends only on the :short argument to make-array:

CL-USER(8): (type-of (make-array 5 :element-type 'double-float))
(SIMPLE-ARRAY DOUBLE-FLOAT (5))
CL-USER(9): (type-of (make-array 5 :element-type 'double-float :short nil))
(SIMPLE-ARRAY DOUBLE-FLOAT (5))
CL-USER(10): (type-of (make-array 5 :element-type 'double-float :short t))
(SHORT-SIMPLE-ARRAY DOUBLE-FLOAT (5))
CL-USER(11): 

3.2 Relationship of arrays to array-like structures

At a low level, and below the level most programmers will ever need to know, some other CL objects retain the same basic structure (and thus the allocation limitations) as short arrays, though these can certainly be reviewed and addressed as necessary in the future.

They are:

These objects should never be arguments to svref, even if they had been punned on simple-vectors in unsafe code (`punned' means declared to be simple-vectors even when they are not). If such punning is still needed for these objects, use ssvref.


3.3 Short-arrays in the type hierarchy

Short-arrays are not Common Lisp standard types. Some of the relationships between short-arrays and normal (long) arrays are intuitive, but some are not. For example, a short-vector of element-type character is arrayp, and is short-array-p, but is not stringp, though it is short-string-p (this is because only (array character (*)) is stringp. And a short-simple-vector (i.e. of (short-simple-array t (*)) type) is short-simple-vector-p, but is not simple-vector-p, because only (simple-array t (*)) is a simple-vector.

Most other relationships between short array types are consistent and type-of, typep, and subtypep know about them.

The list of short array types, classes, and utility functions follows. The symbols naming them are the standard Common Lisp symbol names with short- prepended. All are in the excl package.

All short array types are subtypes of array, but not subtypes of any other Common Lisp array type. Their type hierarchy is the same as the corresponding Common Lisp array type hierarchy.

The various predicates also correspond to their standard Common Lisp counterparts. arrayp and (where appropriate) vectorp return true when applied to short arrays, but no other Common Lisp array predicate returns true when applied to a short array.

There are the following two types. Each is defined by a deftype form, the source is shown.

aref and other accessors

aref and its setf works on short arrays and normal arrays. But the specialized accessors sbit, schar, and svref and their setf's only work on normal arrays (it is an error to pass a short array to them). The following three specialized short array accessors work, in the same way as their Common Lisp counterparts, on short arrays.

In optimized code, care must be taken to match the kind of array with its accessor; svref will open-code to a single instruction access that assumes a normal (long) vector of type t. If the vector is instead a short vector, the access might be to a nonexistent slot beyond the allocation of the short-simple-vector. In the other direction, ssvref will open-code to a single instruction access that assumes a short array. An inverse-ssvref of the zeroth "slot" of a normal (long) array will overwrite the length word, and will result in eventual GC corruption.

An aref in optimized code will generate the correct code if the declaration is missing or matches the kind of array that will actually be accessed. If it is unknown whether the array being accessed will be short or normal (long), then a declaration of dual-simple-array or dual-simple-vector will generate the right code (but code which is still much faster than an out-of-line call to aref or its inverse).

It is strongly recommended, in the face of all of these dangers, that use of short-arrays is kept at a minimum. The space-savings of short-arrays over normal arrays is on average one word per array (depending on the parity of the size; odd-sized short-arrays will save 2 words, and even-sized short-arrays will not save any memory) so the desirability of short arrays is very small when compared to the risks.


3.3.1 String comparisons with short strings

Because short strings are not true strings (i.e. are not stringp), short strings are not permitted as arguments to string comparison functions such as string= and string-lessp. An error will be signaled if you pass a short string as an argument to a string comparison function.

If you need to compare short strings with each other or with regular strings, you can use equalp for equality tests. You must write your own functions for greater than or less than tests, such as the following:

(defun my-string-lessp (s1 s2)
  (let ((l1 (length s1)) (l2 (length s2)) minl)
    (setq minl (min l1 l2))
    (dotimes (i minl)
      (if (char-lessp (aref s1 i) (aref s2 i)) 
        (return-from my-string-lessp i)))
    (cond ((>= l1 l2) (return-from my-string-lessp nil))
          (t (return-from my-string-lessp l1)))))


4.0 Characters

X3J13, the ANSI subcommittee chartered to propose a specification for the forthcoming ANSI Common Lisp, has voted to make several changes to Common Lisp's treatment of characters. The intent of these changes is to clean up ideas that are felt not to have worked out in pre-ANSI Common Lisp as well as to allow for Common Lisp to be extensible to international languages. Unfortunately, some of these changes affect backward compatibility and storage efficiency. The result is that Franz Inc. has had to make some user-visible changes that may affect code which explicitly makes arrays or vectors of type character.

X3J13 has removed discussion of bit and font attributes of characters from the Common Lisp language. The string-char type specifier has also been removed from the language by X3J13. Finally, strings are now equivalent to (vector character) for creation purposes. X3J13 allows characters to be attributed with bit/font features as described in CLtL, but in an implementation-dependent way.

ANSI compatible Allegro CL continues to support font/bit attributes of characters. For example, the reader and printer acts on such characters in the pre-ANSI CL way (e.g., #\control-a is #\a with the control bit set, #3\meta-b is #\b with font 3 and the meta bit set). What's more, functions operating on bits and fonts from pre-ANSI CL (e.g., string-char-p, char-bits, char-font, make-char) are available in the cltl1 package.

Because Franz Inc. wants to achieve as much backward compatibility as possible with code using pre-ANSI font/bit attributed characters, and because Franz Inc. also wants to represent strings at least as efficiently as they have been in pre-ANSI versions of Allegro CL, difficulties arise in representing attributed characters in strings (which are now vectors of characters instead of vectors of string-chars). What ANSI-compatible Allegro CL does is to specify that it is an error to store attributed characters in a string. What in fact happens if one tries to do so is that the attributes are stripped. Thus an attributed character that has been stored in an array and extracted is no longer attributed and no longer EQL to its previous value.

Although this behavior violates the spirit of how elements are stored in arrays, this behavior was chosen by Franz Inc. because (a) pre-ANSI CL code using fonts/bits will not have been storing attributed characters into strings since it has always been an error to do so, and (b) representing strings as arrays that can hold attributed characters would have made strings less efficient and incompatible with existing foreign function code that uses strings.

In other words, portable ANSI CL code should not notice this compromise and pre-ANSI CL code should mostly be able to run as before with very little source change. The one area where portable pre-ANSI CL may run into problems is in places where the character type specifier is explicitly specified in calls to make-array, or to sequence functions that create a vector. (Such sequence functions include coerce, map, concatenate, etc.) These places in pre-ANSI CL where the character type specifier is used should most likely be changed to specify the t type specifier. In pre-ANSI versions of Allegro CL (array character) was equivalent to (array t).



5.0 Autoloading

Allegro CL has the ability to autoload certain files and modules. In order to keep the size of the system down by excluding parts not always needed, some of Allegro CL is not included in the system when it is built. These parts must be loaded in when they are required. This section describes how that code is loaded in.

Autoloads are triggered by referencing certain objects associated with an unloaded module. Typically, calling a function triggers an autoload, but autoloads can also be triggered by referencing a package or a class associated with an unloaded module. Note that only certain objects associated with a module trigger autoloads. If you reference unloaded functionality that does not trigger an autoload, the functionality may seem to be undefined.

An autoload is an automated form of load. When an autoload occurs, a message is printed unless *load-verbose* is nil, in which case the autoload is done silently. The autoload message is sent to the stream specified by the variable *system-messages*.


5.1 Where the autoloaded files are located

All the fasl files which have the potential to be autoloaded are part of the Allegro CL library. All the files are collected into a single file called the bundle file. Its filename is files and its type depends on the version of Allegro CL, but is always some variant of [letter]bu, for example files.bu and files.ebu. The bundle file is located in the Allegro directory. It contains a set of fasl files which can be loaded individually (the whole file is not loaded when a part is). The function bundle-pathname returns the pathname of the bundle file.


5.2 Common Lisp symbols

Code for some Common Lisp functions and macros (notably trace, inspect, and step) are contained in modules separate from the default binary. (The modules are called :trace, :inspect, and :step.) Whenever any Common Lisp function or macro is called, the necessary module will be loaded automatically. Note that using auxiliary features provided as extensions (such as referring to the variable *trace-print-length*) will not cause the module to be loaded. Even though the modules can be automatically loaded, we recommend explicitly loading those that you need with a call to require, as described below.


5.3 Major extensions

The code for major extensions, such as the foreign function interface or multiprocessing, also is loaded when needed instead of being in the default Lisp binary. Again, calls to some functions will cause the correct module to be loaded, but we recommend loading the module before using the facility, using require, as described next.


5.4 How to load modules

While most modules will be loaded automatically when an important function or macro defined in the module is called, you have to load modules explicitly to use some of the less central functionality. Some users also prefer to explicitly load modules in order to save waiting when the module is actually needed.

To load a module with require, simply enter the form:

(require :module-name)

It is useful to put this form at the beginning of any source file containing code which uses symbols in the module. It is not an error to call require when the module is already loaded.



6.0 Miscellaneous implementation details

This section describes implementation details and extensions to Common Lisp operators.

An extension is additional functionality beyond what is specified in the ANSI spec. The section Section 6.1 Extensions to cl:make-package, cl:intern, cl:disassemble, cl:truename, cl:probe-file, cl:open, cl:apropos, etc. describes extensions to a number of SL functions. Usually, these extensions use an additional (non-standard) argument. Portable programs should conditionalize any use of that argument so that it is only used when run in Allegro CL.

An implementation detail either clarifies some part of the spec that is intentionally or unintentionally under specified. The spec usually says that details are left to the implementation when it intentionally under specifies. Unintentional under specification is more subtle: the spec simply says nothing about what should be done in a particular situation. (So for example, should a defpackage call which defines an existing package completely redefine the package according to the new description or should it add features to the package without removing existing features -- see Section 6.9 cl:defpackage for details on this issue.) A number of subsections discuss such details of various Common Lisp operators (and some variables).


6.1 Extensions to cl:make-package, cl:intern, cl:disassemble, cl:truename, cl:probe-file, cl:open, cl:apropos, etc.

Certain standard Common Lisp functions have been extended in minor ways in Allegro CL. Elsewhere we describe changes to load: Using the load function in loading.htm for the general implementation, Load foreign code with cl:load in foreign-functions.htm (for loading foreign code) and sleep (in Process functions and variables (both models) in multiprocessing.htm, making it work on a per-process basis). Those functions were extended to do something essentially new (load to load foreign functions and fasl files in libfasl mode, sleep to work on single processes). The extensions mentioned in this section refer to changes in the semantics of some Common Lisp functions which affect the way they are ordinarily used. The sort of changes done include allowing strings denoting objects as input as well as the object itself. In some cases we have added boolean variables which control the extended behavior, allowing you to decide exactly how you want Lisp to work.

The following Common Lisp operators are dicussed in subsections of this section:


6.1.1 Extensions to cl:make-package


make-package

Function

Package: common-lisp

Arguments: package-name &key use implementation-packages

The implementation-packages keyword argument is an Allegro CL extension described fully in packages.htm. Its value should be a list. Otherwise, this function works as specified in the ANSI specification. The default for the use argument is implementation-dependent. The default in Allegro CL is a list containing one element, the common-lisp package.



6.1.2 Extensions to cl:intern


intern

Function

Package: common-lisp

Arguments: string &optional packages

Allegro CL may allow a symbol as the first (string) argument to intern. Standard Common Lisp requires that the first argument be a string, but specifies no consequences if it is not. Allegro CL controls the behavior with the variable *intern-allows-symbol*, which, if true, causes intern to also accept a symbol as its first argument. If *intern-allows-symbol* is nil, passing a symbol as the first argument signals an error.



6.1.3 Extensions to cl:disassemble


disassemble

Function

Package: common-lisp

Arguments: name-or-compiled-function &key absolute references-only recurse

The standard disassemble does not have any keyword arguments. The keyword arguments are extensions which are likely not supported in implementations of Common Lisp other than Allegro CL.

In standard CL, name-or-compiled-function should be a function-object, a lambda expression, or a symbol with a function definition. Allegro CL also accepts function names which are lists as well (see Section 8.0 Function specs (fspecs) for a discussion of function names which are lists).

name-or-compiled-function can also be a string. A string is interpreted as naming a foreign (C or Fortran) function. The string must match the name identified by applying nm (or similar system function) to the current symbol table. This is often the result of applying convert-to-lang to the routine name, but there are exceptions -- e.g. Lisp internal routines typically do not have a prepended underscore. name-or-compiled-function can also be a codevector. These are extensions to Common Lisp.

The :absolute keyword argument

If the value of the absolute keyword argument is nil (the default), then relative pc addresses are given, starting at 0. If the value of absolute is true, addresses are given as absolute addresses. Note that these addresses are consistent within a single disassembly, but any gc activity may have moved the code vector by the time the disassembly is done.

The :recurse keyword argument

The recurse keyword argument, if true, causes internal functions to be disassembled after the specified function. It defaults to t if the name-or-compiled-function represents a function and if references-only is nil. Otherwise it defaults to nil.

The :references-only keyword argument

If the references-only keyword argument is specified true (its default value is nil) then no disassembly is printed. Instead, a list is returned of all references the function identified by the required argument makes (from either the function object or the global table) to any Lisp object. When references-only is non-nil, recurse defaults to nil.

There are other keyword arguments to disassemble but they are not for programmer use.



6.1.4 Extensions to cl:truename


truename

Function

Package: common-lisp

Arguments: pathname &key (follow-symlinks t)

Until a 6.2 patch, truename did not reliably follow symbolic links to find the canonical name of the file, as required by section 20.1.3.1 of the ANS. Now, truename does correctly follow symbolic links if the follow-symlinks keyword arguments is true (the default). It returns the symbolic link pathname if follow-symlinks is specified nil.

Note that when p evaluates to a pathname that references a symbolic link, (delete-file (truename p)) will delete the actual file while (delete-file (truename p :follow-symlinks nil)) will delete the symbolic link.



6.1.5 Extensions to cl:probe-file


probe-file

Function

Package: common-lisp

Arguments: filespec &key (follow-symlinks t)

probe-file checks to see whether the file named by filespec exists and returns its truename if it does. The value of the follow-symlinks keyword argument is passed as the value of that argument to truename in order to get the pathname to return. If filespec evaluates to a pathname that references a symbolic link, the symbolic link is returned if follow-symlinks is nil, the canonical name of the file if follow-symlinks is true, the default. See the description of the Allegro CL implementation of truename just above.



6.1.6 Extensions to cl:open


open

Function

Package: common-lisp

Arguments: file &key direction element-type if-exists if-does-not-exist class follow-symlinks external-format &allow-other-keys

The specification of this Common Lisp function allows a great deal of latitude to the implementation since interfacing with file systems is hard to specify generally. Here we discuss the if-exists, class, and (briefly) the if-does-not-exist keyword arguments. For a discussion of the external-format keyword argument, see Streams in iacl.htm.

The if-exists argument is looked at only if the direction argument is specified as :io or :output. In that case the following values are allowed for if-exists and have the effect described.

The if-does-not-exist keyword argument also accepts the value :always-append when a file is opened for output. This value causes the file to be created and opened using O_APPEND. See the description of the :always-append value for if-exists described just above for details of the effect of specifying :always-append.

The class keyword argument

The open function has been further extended to take a class keyword argument. open passes this argument to make-instance when it creates the stream, and as with make-instance, the argument may be a stream class object or a symbol naming such a class. If the class argument is not supplied or is nil, open selects one of the following built-in classes according to the direction and element-type arguments:

  excl::character-input-file-stream 
  excl::character-output-file-stream
  excl::character-bidirectional-file-stream
  excl::binary-input-file-stream 
  excl::binary-output-file-stream
  excl::binary-bidirectional-file-stream

These classes all contain file-stream and are variously mixed with

  fundamental-character-input-stream
  fundamental-character-output-stream
  fundamental-binary-input-stream
  fundamental-binary-output-stream

Although the file-stream subclasses returned by open are all instantiable, at present they require hidden initialization (for element-type upgrading, buffer allocation, etc.) and therefore they should only be created using open. It is fine to further specialize them, but you are required to create instances of your specializations of these stream classes using the :class keyword argument to open rather than by calling make-instance yourself.

Missspelled keyword arguments

open is also modified with &allow-other-keys and &rest to pass all keyword arguments as initialization arguments to make-instance. This has the unfortunate side effect of removing error checking for misspelled keyword arguments.

See streams.htm, particularly the discussion of using open to create streams in Implementation of Common Lisp Functions for simple-streams.

The follow-symlinks keyword argument

When called with :direction :probe, open essentially works like probe-file and checks to see whether the file named by file exists and returns its truename if it does. The value of the follow-symlinks keyword argument, which is ignored unless direction is :probe, is passed as the value of that argument to truename in order to get the pathname to return. If file evaluates to a pathname that references a symbolic link, the symbolic link is returned if follow-symlinks is nil, the canonical name of the file if follow-symlinks is true, the default. See the description of the Allegro CL implementation of truename just above.



6.1.7 Extensions to cl:apropos


apropos

Function

Package: common-lisp

Arguments: string &optional package external-only (case-insensitive t)

apropos in Allegro CL accepts two additional optional arguments. The second is external-only. If a package designator is specified as the value of the first (standard) optional argument, only symbol external in that package will be considered as candidates for output. If package is specified nil (some value must be given if external-only is to be specified), the external-only is ignored. external-only defaults to nil.

CL-USER(1): (apropos :defun nil t)
DEFUN               [macro] (name varlist &rest body)
COMP::PA-DEFUN-PROTO-1 [function] (xform)
COMP::QC-DEFUN-IN-RUNTIME [function] (node target cc)
COMP::COMPILE-P-DEFUN [function] (form)
EXCL::DEFUN-PROTO-1
EXCL::DEFUN-LIKE    [function] (xp list &rest args)
EXCL::RECORD-SOURCE-FILE-DEFUN [function] (fspec &optional icsp)
DEFUN-PROTO         [macro] (name varlist &rest body)
FF::DEFUN-FOREIGN-CALLABLE-1 [function] (name arglist body)
FF:DEFUN-FOREIGN-CALLABLE [macro] (name arglist &rest body)
FF:DEFUN-C-CALLABLE [macro] (&whole form &rest args)
:DEFUN              value: :defun
CL-USER(2): (apropos :defun (find-package :excl) t)
DEFUN-PROTO         [macro] (name varlist &rest body)
CL-USER(3): (apropos :defun (find-package :excl) nil)
EXCL::DEFUN-PROTO-1
EXCL::DEFUN-LIKE    [function] (xp list &rest args)
EXCL::RECORD-SOURCE-FILE-DEFUN [function] (fspec &optional icsp)
DEFUN-PROTO         [macro] (name varlist &rest body)

The third optional argument is case-insensitive. If true (which is the default starting in release 7.0), comparisons between string and symbol names are done in a case-insensitive fashion. Thus, in an ANSI (case-insensitive, symbols are named with uppercase strings) image,

(apropos "car" (find-package :common-lisp) nil nil)
  PRINTS nothing (as no symbols in the CL package have "car" in
         their names)
(apropos "car" (find-package :common-lisp))
  PRINTS:

   MAPCAR
   CAR

And in a modern image (case-senstive, symbols are named with lowercase strings),

(apropos "CaR" (find-package :common-lisp) nil nil)
  PRINTS nothing (as no symbols in the CL package have "CaR" in
         their names)
(apropos "CaR" (find-package :common-lisp))
  PRINTS:

   mapcar
   car


apropos-list

Function

Package: common-lisp

Arguments: string &optional package external-only case-insensitive

Like apropos, as described just above, apropos-list accepts two additional optional arguments, external-only and case-insensitive. If external-only is true and a package designator is specified for the standard optional argument package, only external symbols in that package are included in the result. If case-insensitive is true (the default is nil), comparisons between string and symbol names are done in a case-insensitive fashion.



6.1.8 Extensions to cl:interactive-stream-p


interactive-stream-p

Function

Package: common-lisp

Arguments: stream

The Common Lisp function interactive-stream-p returns true if its argument is an interactive stream, which is a stream "on which it makes sense to perform interactive querying". Allegro CL extends this function so that it is setf'able.

When (setf (interactive-stream-p stream) t) is evaluated, not only does (interactive-stream-p stream) return true, but also any writing that is done is encapsulated into blocks of output that are forced out by a call to force-output at the end of the call. This makes the stream seem like it is unbuffered, yet without sacrificing as much performance as a raw unbuffered stream would require, since the actual output takes place only at the end of each group of write operations.



6.2 A comment about with-open-file and timing hazards

with-open-file tries to guarantee that the file stream opened for the evaluation of its body is closed, thus avoiding open but unused files. (Such open files can cause an error if the number, set by the operating system, of allowable open files is reached.)

But note that there is a hazard between the time Lisp calls out to the operating system to open a file and the time Lisp sets the stream variable to the newly opened file stream. Between those events, an interruption that causes a non-local exit may leave the file open, but Lisp, lacking any handle on the newly opened stream object, cannot in fact close it.

The risk is small, but can be exacerbated by the following:


6.3 cl:directory

The directory function has had a keyword argument added to it to assist in recursive walks down a directory tree. (Note that even though the new argument is not specified, Common Lisp: the Language says the following about directory: `It is anticipated that an implementation may need to provide additional parameters to control the directory search. Therefore directory is specified to take additional keyword arguments so that implementations may experiment with extensions, even though no particular keywords are specified here.')


directory

Function

Package: common-lisp

Arguments: path &key directories-are-files

Returns a list of pathnames matching path, which may be a pathname, string, symbol or stream. Returns nil if there is no match.

If the keyword argument directories-are-files is specified true (the default), this function will return directories as files (that is pathnames with name and/or type components true). If the argument is nil, directories are returned as directories (pathnames with name and type components nil). In the latter case it is possible to walk down a directory tree recursively using directory.

The elements of the list returned by directory is in the same order as returned by the associated system function (e.g. readir() on UNIX).

If directory is given wildcards, for example "*/*.cl", it will ignore files which are symbolic links that point to other directories. This prevents directory recursing into these symbolically named directories. For example, (directory "*/*.cl") will no longer, in the face of a `foo' symlink to a directory, would descend into `foo'. (This issue affects UNIX and UNIX like platforms only as Windows does not support symbolic links.)

Wildcard handling

directory uses pathname-match-p, which, when presented with wildcards in path (when path is a string), converts the pathname into Allegro CL regular expressions, according to the rules given next. (See regexp.htm for information on regular expression handling.)

Handling of non-directory components:

    . turned into \.
    * turned into .*
    ? turned into .
    ^ prepended onto beginning
    $ appended onto end

Handling of directory components:

    . turned into \.
    * turned into .[^/]* (or .[^\\]* on windows)
    ** matches any number of directory levels
    ? turned into .
    ^ prepended onto beginning
    $ appended onto end


6.4 Reader macros and cl:*features*


#+(version>= ...)/#-(version>= ... )

Reader Macro

We have extended the #+ and #- reader macros to accept (version>= N [ M]) as an argument. It is interpreted to mean that the form following will only be read if the version (also called release) of Allegro CL is greater than or equal to N.M. The N must be supplied. The M is optional. Both must be integers. With #+, version>= signifies read the next form only if the version is greater than or equal to N.M. With #-, it means read the next form only is the version is less than N.M. For example, because of an X3J13 change, the element type for an array of characters is character starting in release 4.1 and string-char in earlier releases. To have code work in all Allegro CL releases, do the following:

(make-array 3 
            :element-type #+(version>= 4 1) 'character
                          #-(version>= 4 1) 'string-char)

Warning: while most Common Lisp implementations (including Allegro CL prior to version 4.1) ignore `(version>=...)', it is possible that an implementation would signal an error upon encountering it. As a workaround for truly portable code, use:

#+(and allegro-version>= (version>=...))

Because :allegro-version>= is (presumably) only on the *features* list of Allegro CL 4.1 and later, this will fail in all versions without version>= having to have a definition.



*features*

Variable

Package: common-lisp

This standard Common Lisp variable can be used with the #+ and #- reader macros to conditionalize code for different Lisp implementations and releases. The exact value is different in every version of Allegro CL. Here are some useful values which may or may not be in your version. Please check the value of *features* in your version to see exactly what is there. The function featurep can be used to test whether a feature is present or not.



6.4.1 Features present or missing from *features* in Allegro CL

This is a partial list.

Feature

Meaning and use

:allegro Unique to Allegro CL. Present in all versions on all platforms. Use this to distinguish Allegro CL from other Lisp implementations.
:ignore Absent in all versions on all platforms. Thus a form marked #+ignore is never evaluated. Used, for example, in custom.cl.
:x3j13 Purports to conform to some version of Common Lisp specified by the ANSI X3J13 committee. Present in Allegro CL since version 4.2.
:cltl2 Purports to conform to Common Lisp: the Language, 2nd ed. Since ANSI Lisp has diverged, :x3j13 and :cltl2 should not both be present. Not present in Allegro CL 4.2 or later. Present in some earlier versions.
:draft-ansi-cl-2 Purports to conform to the second draft ANSI standard. Allegro CL does so, so :draft-ansi-cl-2 is present in Allegro CL 7.0
:ansi-cl Purports to conform to ANSI Common Lisp standard. The standard is now (since early 1996) final. Present in Allegro CL starting with version 4.3.
:dynload Foreign loading is done by dynamic linking of shared libraries/objects. The next several features are types of dynamic loading. See foreign-functions.htm.
:dlfcn Uses dlopen() to link foreign code. OS examples: Solaris, IRIX, Dec Unix, AIX. See foreign-functions.htm.
:dlhp Uses shl_load to link foreign code. HP only. See foreign-functions.htm.
:dlwin Uses LoadLibrary to link foreign code. Windows machines only. See foreign-functions.htm.
:dlmac Uses the Mac OS X system loader NSLoadModule to link foreign code. Mac OS X machines only. See foreign-functions.htm.
:dlld Loads .o files into image with ld. No Allegro CL version uses this.
:ics Supports International Character sets. Characters are 16-bits (rather than 8 bits). Allegro CL comes in both International and non-International versions (the International version is standard). Use this feature to distinguish the versions. See iacl.htm.
:os-threads Multiprocessing model uses native threads. Other model is the non :os-threads model (no special feature) where processes are managed within Lisp. See multiprocessing.htm.
:mswindows Appears in versions running on Windows machines. Use #-mswindows for Unix.
:sparc This feature appears on versions that run on machines with a Sparc processor (e.g. Sun 4's and Sparcstations). A similar platform-naming feature appears in all implementations and allows differentiating between machines. Look for the feature in your version.
:big-endian Platform uses the big-endian method of representing numbers.
:little-endian The platform uses the little-endian method of representing numbers.
:verify-stack Checking how close the stack is to overflowing is expensive. See verify-stack-switch.
:allegro-vN.M Present in Allegro CL version N.M. (Examples :allegro-7.0, :allegro-8.0, :allegro-8.1, etc.) See also #+(version>=...) reader macro defined Section 6.4 Reader macros and cl:*features* above. Both it and this feature are useful for conditionalizing code to run on different releases of Allegro CL.

6.4.2 The issue of nested conditionals in Allegro CL

Assume :allegro is on the *features* list and that :foo is not. Consider the following two forms and their evaluations:

;; CASE 1
(list #+allegro :allegro #-allegro #+foo :foo #-foo :default)

  Versions of Allegro CL prior to 8.0 return (:allegro :default)
  Allegro CL 8.0 and many other implementations return (:allegro)

;; CASE 2
(list #+allegro :allegro #-allegro #+foo :foo)

  Versions of Allegro CL prior to 8.0 return (:allegro)
  Allegro CL 8.0 and many other implementation signal an error

We will explain these disparate behaviors below, but first we recommend that conditions be nested using not, or, and and within the #+ or #- test expressions (the expression which follows the #+ or #-) as that is always unambiguous in any Lisp. Thus the first conditional below implements the old Allegro CL behavior and the second implement the current behavior:

(list #+allegro :allegro #+(and (not allegro) foo) :foo #-foo :default)
(list #+allegro :allegro #+(and (not allegro) foo) :foo
                         #-(or allegro foo) :default)

Using nesting within the test expression for Case 2 should make clear what is desired when :allegro holds and :foo does not -- presumably:

(list #+allegro :allegro #+(and (not allegro) foo) :foo)

In older Allegro CL implementations, when a conditional fails (like #-allegro fails), a conditional in the associated form (the one that will be ignored) is not further considered. Thus that conditional and its associated form are taken to be the form to be ignored. So in the first example, #-allegro #+foo :foo is considered to be a (failing) conditional and its associated form. It is ignored and the reader then encounters #-foo :default. The #-foo conditional succeeds so the subsequent form -- :default -- is evaluated.

Other Lisp implementations resolve the conditionals following a conditional as part of determining what the form associated with a conditional is. Allegro CL has been changed to match that behavior. As a result, conditionals following a conditional (i.e. nested conditionals) are considered and resolved as part of determining the form that follows a conditional, the form that should be ignored (when the original conditional fails) or evaluated (when it succeeds). So in #-allegro #+foo :foo #-foo :default the inner conditionals #+foo :foo #-foo :default are resolved to :default. Thus #-allegro #+foo :foo #-foo :default resolves to #-allegro :default which is then ignored.

In the second example, #-allegro #+foo :foo resolves to #-allegro which signals an error because no form follows the #-allegro conditional, and that is erroneous code. (The conditional doing the nesting within the test expression, show above, does not error.)

We believe (although we do not present our analysis here) that the ANSI standard is ambiguous on the handling of these cases and so both the older Allegro CL behavior and the newer behavior are within standard. However, since the #+/#- conditonals are designed to allow for using the same code in various implementations of Common Lisp, we believe it is most important that all implementation do the same thing. Since other implementations of Common Lisp resolve inner conditionals to produce the form that outer conditionals apply to, Allegro CL has been changed (starting in release 8.0) to do that as well. Again, we recommend doing the nesting in the test expressions rather than nesting #+/#-'s.

The change in the handling of nested #+/#-'s is a non-backward-compatible change in Allegro CL 8.0, and a rather obscure one which may cause difficult to diagnose errors in user code which has heretofore worked correctly. To mitigate this, in Allegro CL 8.0 (and later), a warning is signaled when nested conditionals are detected. This warning remarks on the behavior change. The warning is suppressed when the variable *warn-on-nested-reader-conditionals* is set to nil (its initial value is t). Users who want to revert to the old behavior (not resolving inner conditionals before applying outer) can do so by setting the variable *sharp-plus-de-facto-standard-compatible* to nil (its initial value is also t). We do recommend that user change their code to conform to the new behavior where that is possible rather than reverting to the old behavior.


6.5 cl:random and cl:make-random-state

There are two random number generators used in Allegro CL, depending on the argument to random. One is fast, efficient, and does no consing. It is used in compiled code when the argument is a (positive) single or double floating point constant or any (positive) fixnum. Other reasonable calls also invoke the fast algorithm. The other, which is slower and less efficient and conses a great deal, is used when the fast algorithm cannot be.

We recommend that users interested in random floats of magnitude X do

(* x (random 1.0f0))

rather than

(random x)

Initial values returned by random

Because random may be called by any Lisp function at any time, there can be no guarantee that the sequence of numbers seen by your calls to random will be the same each time you invoke Lisp even if your actions are seemingly identical. However, absent specific action on your part, often the values returned by random are the same from invocation to invocation. If either repeatability or ensuring different runs are different are important to you, you should manage random by specifying the optional random-state argument with random-state objects you have created and stored (see make-random-state). Printed versions of random-state objects are readable so values can be stored in text files.

random and multiple processes

When a new process is created, the value of *random-state* may be bound as part of the initial bindings for the process (see mp:make-process and mp:process-run-function). *random-state* is one of the variables included in the suggested list of bindings which is the value of *cl-default-special-bindings*, but that list is used for processes you create only if you specify that it be used. The binding is to a copy of an existing random-state object. This means that if that list is used, different processes may start with copies of the same random-state object or with random-states that produce similar (i.e. slighly displaced) random number sequences. This may or may not be what is required for your application. As we suggest with managing random numbers in general, we suggest that if the nature of random sequences is important to your application, you manage the random sequences for processes that you create by creating your own random-state objects (with make-random-state) and using them in the processes you create.


random

Function

Package: common-lisp

Arguments: number &optional state

Returns a pseudo-random number uniformly distributed between 0 and (- number 1) if number is an integer and between 0 (inclusive) and number (exclusive) if number is real but not an integer. number must be real and positive. state should be a random-state object. If supplied, it will be made the state while the returned value is calculated.

Pseudorandom numbers are generated using The Mersenne-Twister algorithm, MT179937. MT179937 is described in detail in the paper "Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator" by Makoto Matsumoto (Keio University/Max-Planck-Institut fuer Mathematik) and Takuji Nishimura (Keio University), which appeared in the issue 1/1998 of the ACM Transactions on Modeling and Computer Simulation.



make-random-state

Function

Package: common-lisp

Arguments: &optional state seed

This standard Common Lisp function returns a random-state object. If state is already a random-state, it is returned. If state is t (and seed is nil), make-random-state uses get-universal-time for its starting value. We have ensured that even in a tight loop, different states will be produced by each call in the loop.

The seed argument is provided as an extension to standard Common Lisp. If state is specified as t, and seed is given and is an integer, then instead of using its own internal method for generating a seed, that specified seed is accepted and used to create the new random-state. Two such calls to make-random-state with the same seed will produce equivalent random-states. Only the least significant 32 bits of the integer are used to seed the random state.

This capability allows the user to generate starting seeds from any random-number generation source. The question of how long that source requires to generate truly random data is the user's responsibility. (Thus /dev/random, if available, may block indefinitely waiting for the entropy pool to be replenished. /dev/urandom can also be used to generate random seeds, if it is available).



6.6 cl:make-hash-table


make-hash-table

Function

Package: common-lisp

Arguments: &key test size rehash-size rehash-threshold hash-function values weak-keys

Hash tables with standard tests (eq, eql, equal, and equalp) have been optimized in Allegro CL to make putting values into and getting values from a hash table fast. eq hashtables are the fastest, followed closely by eql, and then equal and equalp.

The size argument to make-hash-table

The maximum size of a hash table is one less than the value of array-dimension-limit. In safe code, if the value specified by the size is greater than or equal to array-dimension-limit, then array-dimension-limit minus 1 will be used instead and a warning will be signaled.

Extensions to make-hash-table

Allegro CL has also extended make-hash-table in several ways:

  1. to accept the (non-standard) hash-function keyword argument,
  2. to allow test functions other than the standard four,
  3. to allow for weak hashtables, and
  4. to allow for valueless hashtables.

The :hash-function keyword argument

The hash-function keyword argument allows further specialization when standard functionality is inefficient (usually because of excessive collisions caused by bunching of the hash codes of the data). Code that uses the hash-function argument is not portable Common Lisp, of course.

If specified, the value from hash-function should be a a symbol naming a function of one argument which reproducibly returns an integer in the correct range when applied to any Lisp object intended to be used as a hash key. (The value must be a symbol, not a function object.)

The correct range is between 0 and (1- (expt 2 24)) (inclusive) in 32-bit Lisps and between 0 and (1- (expt 2 32)) (inclusive) in 64-bit Lisps. Reproducibly here means the function will return the same value on equivalent objects whenever it is called. hash-function defaults to sxhash except when test is one of the four standard tests (eq, eql, equal, equalp) when hash-function defaults to an internal function optimized for that test. (For equal and equalp, the hash-function is an internal version of sxhash.)

The :test keyword argument

test should be a symbol naming a function of two arguments that returns t or nil as two keys are or are not equivalent. The standard values for test are eq, eql, equal, and equalp (or the associated function objects #'eq etc.) but any test function can be specified. (But note (1) that symbol is reserved for internal use; test should not be specified 'symbol in application or user code; and (2) the value must be a symbol naming a function, not a function object; the four standard function objects listed just above are accepted as values but no other function objects.) If hash-function is specified, it is the programmer's responsibility to ensure the test function and the hash function work together correctly and consistently.

The :weak-keys keyword argument

weak-keys defaults to nil, which specifies the default behavior. When weak-keys is specified as t, the keys of the resulting hash table are treated specially by the garbage-collector: when a key in such a hash table has no more references to it, the entire entry is removed from the hash table, and the hash-table-count is decremented. This entry removal will occur regardless of whether :values :weak is specified (which by itself will never affect the hash-table-count, but only the value of an entry). See gc.htm for information on weak objects.

If weak-keys is given the value :tenurable, then the key vector (the part of the weak-key hash-table that is normally kept in newspace) is allowed to be tenured. Any other true value for weak-keys causes the key vector to be forced to stay in newspace (but it is best to use t as this allows other non-nil values which have special meaning to be added later). The :tenurable option allows the amount of data copied between newspace halves to remain smaller than if the key vector were forced to remain in newspace. This difference can be large if the hash-table is large. Allegro CL now uses this option internally. If a tenurable weak-keys hash-table must be rehashed due to growth, the new key vector is allocated in newspace, but is still allowed to be tenured. (This means the vector is not created with :allocation :old described below.)

The downside of tenuring the weak-key vector is that references to the values will remain until a global garbage collection examines the weak-key vector. An untenured weak-key vector is examined whenever there is a scavenge. Global gc's are typically rare, but scavenges occur regularly. A decision to use the :tenurable option should take this into consideration.

The :values keyword argument

values can be t (the default), :weak, or nil.

:values t or :values unspecified

When values is t, the hash table will contain both a key and a value for each entry (that is, it will be a normal hash table). As said above, t is the default value for values.

:values :weak

When :values :weak is specified, then the hash table will hold a value only as long as it is referenced non-weakly by some other object. If no other objects reference the value, it becomes nil and a gethash on the key will return nil for the value (the value is collected by the gc).

:values :weak example

;;  We create a :values :weak hashtable:
cl-user(26): (setq ht (make-hash-table :values :weak))
#<eql hash-table with weak values, 0 entries @ #x48aef52>
;;  We create an object to store aa a value:
cl-user(27): (setq a (list 1 2 3))
(1 2 3)
;;  We store the list as the value of the key 100:
cl-user(28): (setf (gethash 100 ht) a)
(1 2 3)
;;  And the list is returned when we ask for it:
cl-user(29): (gethash 100 ht)
(1 2 3)
t
;;  We break the link from the symbol A to the list:
cl-user(30): (setq a nil)
nil
;;  We break the links from variables like *, **, and *** to the list
;;  (this works here but be aware that links may exist that you are
;;  unaware of, and it make take longer for those links to disappear).
cl-user(31): t
t
cl-user(32): t
t
cl-user(33): t
t
cl-user(34): (gc)
cl-user(35): (gc)
;;  Now when we get the value associated with 100, it is NIL
cl-user(36): (gethash 100 ht)
nil
t
;;  Note, second value is T as 100 still has a value.  But the
;;  value is now NIL, not the list which was the original value.
cl-user(37): 

:values nil

When :values nil is specified, a sans values hash table is created, and only keys are stored. gethash returns the key as its first return if the key is in the table, and t as the second value in that case. As usual, gethash returns nil and nil if the key is not in the table. You can use setf and gethash to store a key. You must specify a value but that value is ignored. You can also use the function excl:puthash-key to store a key in the table.

On sans-value hash tables, maphash will call its argument function with the key as both arguments (as the key argument and as the value argument), as there is no value to pass.

One use of :values nil (sans-value) hash tables is to identify a set of objects, such as those objects which have a particular property, in a space efficient way. Suppose, for example, you have many instances (millions of them) of a particular class, and only 20 are XYZ-positive. You could have an xyz-positive instance slot in the class, but that could use megabytes of space. A sans-value hash table with the 20 objects as keys uses just a few hundred bytes. That table could be the value of a class slot of the class and a method that looked to the user like an ordinary reader could test whether an instance was in the hash table or not, while a writer could add an instance to the hash table.

Sans-value hash tables are also a good way to store conses. If you have a bunch of conses you will need many times, place each as you first create it as a key into a sans-value hash table with the appropriate test function (say equal). Then, if you need that cons, create one and test it using excl:puthash-key or gethash, and always using the return value (unless nil in the case of gethash) and discarding the test value. Only one permanent copy of the cons will then be stored no matter how may you create. (See the second example below.)

:values nil example

;; We create a sans-value hashtable:
cl-user(49): (setq svht (make-hash-table :values nil))
#<eql hash-table (sans values) with 0 entries @ #x4a94bb2>
;;  We store as keys all CL symboles with more than 3 e's in
;;  the symbol name. Note we use puthash-key to store the key.
;;  We do not need a value because being in the hashtable indicates
;;  the key has the desired property (more that 3 e's).
;;
cl-user(50): (do-external-symbols (s (find-package :cl))
	       (if (> (count #\e (symbol-name s) :test 'char-equal) 3)
		   (puthash-key s svht)))
nil
;;  There are 39 such symbols:  #x4a94bb2
cl-user(51): svht
#<eql hash-table (sans values) with 39 entries @ #x4a94bb2>
;;  We use MAPHASH to print out the 39 symbols. Note the value
;;  passed to the MAPHASH argument function is the key (that
;;  is, the key is passed as both the K and the V arguments).
;;  We have added line breaks for clarity in some cases
cl-user(52): (maphash #'(lambda (k v)
			  (format t "~S, value is ~S~%" k v))
		      svht)
integer-decode-float, value is integer-decode-float
least-negative-normalized-single-float, 
  value is least-negative-normalized-single-float
set-difference, value is set-difference
update-instance-for-different-class, 
  value is update-instance-for-different-class
double-float-negative-epsilon, 
  value is double-float-negative-epsilon
make-sequence, value is make-sequence
make-instances-obsolete, value is make-instances-obsolete
stream-element-type, value is stream-element-type
least-negative-normalized-double-float, 
  value is least-negative-normalized-double-float
delete-package, value is delete-package
least-negative-normalized-long-float, 
  value is least-negative-normalized-long-float
delete-file, value is delete-file
array-element-type, value is array-element-type
upgraded-array-element-type, value is upgraded-array-element-type
encode-universal-time, value is encode-universal-time
least-negative-normalized-short-float, 
  value is least-negative-normalized-short-float
internal-time-units-per-second, 
  value is internal-time-units-per-second
type-error-expected-type, value is type-error-expected-type
ensure-generic-function, value is ensure-generic-function
delete-duplicates, value is delete-duplicates
define-setf-expander, value is define-setf-expander
least-negative-single-float, 
  value is least-negative-single-float
read-sequence, value is read-sequence
get-decoded-time, value is get-decoded-time
concatenated-stream-streams, 
  value is concatenated-stream-streams
invoke-restart-interactively, value is invoke-restart-interactively
read-preserving-whitespace, value is read-preserving-whitespace
get-internal-real-time, value is get-internal-real-time
least-positive-normalized-double-float, 
  value is least-positive-normalized-double-float
decode-universal-time, value is decode-universal-time
*compile-file-truename*, value is *compile-file-truename*
least-negative-double-float, value is least-negative-double-float
nset-difference, value is nset-difference
ensure-directories-exist, value is ensure-directories-exist
make-concatenated-stream, value is make-concatenated-stream
update-instance-for-redefined-class, 
  value is update-instance-for-redefined-class
write-sequence, value is write-sequence
least-positive-normalized-single-float, 
  value is least-positive-normalized-single-float
single-float-negative-epsilon, 
  value is single-float-negative-epsilon
nil
;;  GETHASH works as usual but returns the KEY as if it 
;;  were the value:
cl-user(53): (gethash 'write-sequence svht)
write-sequence
t
;;  You can use SETF of GETHASH instead of PUTHASH-KEY. Note
;;  the value specified (10 in this case) is discarded:
cl-user(54): (setf (gethash nil svht) 10)
nil
;;  GETHASH returns the key as the value. The value specified
;;  just above (10) is not stored so is not available:
cl-user(55): (gethash nil svht)
nil
t
;;  NIL is returned because the KEY is NIL. When the key is NIL,
;;  you must look at the second return value to see if NIL is 
;;  in the hash table.
cl-user(56): 

;;  In the second example, we create a EQUAL sans-value hash table
;;  and store some conses in it. We create a new cons and use
;;  PUTHASH-KEY to store it if necessary.  PUTHASH-KEY returns
;;  the stored cons if there, or the cons if just stored.  
cl-user(59): (setq cons-storer-ht (make-hash-table :test 'equal :values nil))
#<equal hash-table (sans values) with 0 entries @ #x4b78e3a%gt;
;;  We put some conses in the hash table:
cl-user(60): (puthash-key (list 'baltimore 'md) cons-storer-ht)
(baltimore md)
cl-user(61): (puthash-key (list 'boston 'ma) cons-storer-ht)
(boston ma)
cl-user(62): (puthash-key (list 'berkeley 'ca) cons-storer-ht)
(berkeley ca)
cl-user(63): (puthash-key (list 'reno 'nv) cons-storer-ht)
(reno nv)
;;  Here is a cons. We put it in the hashtable if necessary.
;;  PUTHASH-KEY returns the one there if present:
cl-user(64): (setq a (list 'boston 'ma))
(boston ma)
cl-user(65): (puthash-key a cons-storer-ht)
(boston ma)
;;  Note the new one is not the one returned:
cl-user(66): (eql a *)
nil
;;  So we break the link to the new one, and use the stored
;;  one so only one copy is live in the image:
cl-user(67): (setq a **)
(boston ma)


6.7 cl:make-array


make-array

Function

Package: common-lisp

Arguments: dims &key allocation element-type weak short [and other standard CL keyword args not listed here]

allocation is discussed first and then weak. short is discussed briefly after the discussion of weak and in detail in Section 3.0 Arrays and short arrays.

allocation: make-array, a standard Common Lisp function, has been extended to accept the allocation keyword argument. The value of this argument must be one of the following keywords (the default is :new, which produces the behavior of earlier releases).

Value of allocation argument Meaning
:new Allocate the new array data in new space (the usual behavior). Any array element type accepted. This is the default.
:old Try to allocate the new array data in old space immediately (without waiting for it to survive for the required number of scavenges). Any array element type accepted.

If there is not enough contiguous oldspace available to allocate the array, it will be allocated in newspace. resize-areas can be used before the allocation in order to ensure that there is enough oldspace available.

:static Allocate the new array in aclmalloc (foreign) space. The array will never be touched by the garbage collector and must be deallocated explicitly. The arrays must have a specialized element type since arrays of type t may contain pointers that the garbage collector may need to update. See the list of array types in Section 2.0 Data types and array types for a list of specialized array types. Note that if the upgraded-array-element-type of an element type is t, that array may not be allocated :static or :malloc.

You must explicitly free the space if it is no longer needed, as described below. :malloc and :static are synonyms. (Despite the :malloc argument name, aclmalloc is used to allocate space, not malloc. It is preferable to use :static rather than :malloc to avoid confusion about how the space is allocated.)

:malloc
:static-reclaimable Allocate the new array data in aclmalloc (foreign) space and the header in Lisp space. The data will never be touched by the garbage collector but it will be deallocated when there are no pointers from Lisp (using a finalization). Only specialized arrays (not arrays of type t) can be allocated in this way, as with :static/:malloc allocations. See the description of those allocation types for more details.
:lispstatic-reclaimable Allocate the new array in malloc (foreign) space. The array will never be touched by the garbage collector (except to update pointers back into Lisp space) until there are no pointers from Lisp, at which point the whole array will be deallocated explicitly. Any Lisp type can be contained in the array.

allocation is not a standard Common Lisp argument to make-array so programmers may wish to conditionalize it with #+allegro to preserve code portability.

Having created a static array, you may wish to free it. To do this, first pass the array to the function lispval-other-to-address, which will return an address (an integer). That address can be passed to aclfree. Note: if you reference the array after it has been freed, you will get garbage values. If you set a value in the array after it has been freed, you may cause Lisp to fail.

weak: make-array, a standard Common Lisp function, has been extended to accept the weak keyword argument. weak is not a standard Common Lisp argument to make-array so programmers may wish to conditionalize it with #+allegro to preserve code portability. weak may be true (meaning create a weak array) or nil (meaning create a standard array). The default is nil.

A Lisp object becomes garbage when nothing points to or references it. The way the garbage collector works is it finds and identifies live objects (often then moving them somewhere). Whatever is left is garbage. Weak arrays allow pointers to objects which will not, however, keep them alive. If one of these pointers exists, the garbage collector will see the item and (depending on the circumstances), either keep it alive or abandon it.

If you specify weak true, you cannot specify the non-standard allocation argument or the standard displaced-to argument. The only values accepted for the standard element-type argument are those for which no specialized array type for that element-type is defined (i.e. upgraded-array-element-type applied to element-type should return t, which in essence means you should not specify element-type).

short: Allegro CL supports two fundamental kinds of arrays: standard and short. Short arrays (equivalent to the array type in releases prior to 7.0) have a smaller maximum size than standard arrays. See Section 3.0 Arrays and short arrays for details. When :short t is specified, a short array is produced. Otherwise a standard array is produced.

See Weak arrays and hashtables in gc.htm for more information on weak arrays.



6.8 cl:namestring


namestring

Function

Package: common-lisp

Arguments: pathname &key syntax

cl:namestring takes a pathname designator and returns the full namestring of the pathname. Allegro CL adds an additional keyword argument: syntax. The value of syntax can be nil or :unix. The behavior of the syntax argument is different on Unix and Unix-like platforms and on Windows.

On Unix and Unix-like platforms

The syntax argument is ignored.

On Windows

If syntax is :unix, any backward slashes in the pathname are converted to forward slashes. If syntax is nil (the default), no slashes are converted and cl:namestring behaves normally.

Thus, on Windows only,

(namestring "\\ftp\\pub\\patches\\8.0\\ftp.001" :syntax :unix)
  returns "/ftp/pub/patches/8.0/ftp.001"

while

(namestring "\\ftp\\pub\\patches\\8.0\\ftp.001")
  returns "\\ftp\\pub\\patches\\8.0\\ftp.001"

The argument was added to assist ftp functions called from Windows. Functions like map-over-ftp-directory called on Windows generates pathnames of the files in an ftp directory, but these generated pathnames use Windows syntax (with backward slashes delimiting directories). In order for these pathnames to be used in calls to other ftp functions, such as ftp-stream-file-mod-time, they must be first converted to Unix syntax. Users writing their own mapping functions for ftp directories may find this added feature of cl:namestring useful. The ftp client module is described in ftp.htm.



6.9 cl:defpackage


defpackage

Function

Package: common-lisp

Arguments: defined-package-name &rest options

The specification of defpackage is silent on whether, when there are two defpackage forms for the same package, the second should augment the first or the second should replace the first.

Consider, for example, the following two defpackage forms:

(defpackage :newpack (:use :excl :cl))
(defpackage :newpack (:use :net.uri))

What is the package-use-list after the second defpackage form returns: a list of three packages (excl, common-lisp, and net.uri) or a list of a single package (net.uri)? Allegro CL augments the package specification rather than replacing it, as illustrated by the following transcript:

cl-user(1): (defpackage :newpack (:use :excl :cl))
#<The newpack package>
cl-user(2): (package-use-list (find-package :newpack))
(#<The excl package> #<The common-lisp package>)
cl-user(3): (defpackage :newpack (:use :net.uri))
#<The newpack package>
cl-user(4):  (package-use-list (find-package :newpack))
(#<The net.uri package> #<The excl package> #<The common-lisp package>)


6.10 cl:file-length


file-length

Function

Package: common-lisp

Arguments: stream

Allegro CL allows stream to be a pathname or a namestring as well as a stream open to a file (ANSI CL specifies only a stream open to a file). For a pathname or a namestring argument, the file-length function returns the size (number of octets, that is 8-bit bytes) of the associated file.

We also do not signal an error when the argument to file-length is a string stream or a buffer stream (instead of just a stream open to a file). See the discussion of file-length in Section 10.0 Conformance with the ANSI specification for further details.



6.11 cl:file-write-date

There are two implementation details for cl:file-write-date:

  1. A setf method has been provided for file-write-date. It sets the mtime (the modification time on UNIX) of the file. On Windows, the comparable value is set.
  2. If the file specified by the pathspec argument does not exist, nil is returned (rather than an error being signaled).

The fact that nil is returned when the argument file does not exist is arguably an ANSI non-compliance. The Spec says: "An error of type file-error is signaled if the file system cannot perform the requested operation". But it also says: "returns nil if such a time cannot be determined". Returning nil in this situation is longstanding behavior in Allegro CL and is being maintained.


6.12 cl:lisp-implementation-version

In Allegro CL, cl:lisp-implementation-version returns two value. The first is a string which is of the form

"[Version number] [[Platform]] ([Date and time of build])"

For example

"8.1 [64-bit Linux (AMD64)] (Feb 7, 2007 14:55)"

The second return value is a list of strings (there may be only one) that identify the Allegro shared library build version. For example:

("lisp_build_NNN")

NNN is an integer which is increased with each new build. (The library is often updated between releases because of patches.) So, a call looks like:

cl-user(2): (lisp-implementation-version)
"8.1 [64-bit Linux (AMD64)] (Feb 7, 2007 14:55)"
("lisp_build_1")
cl-user(3): 

These examples are for illustration only. The values you see will be different. This information may be helpful when trying to identify a potential problem, allowing users at different sites to be sure they are running the same versions of all parts of Allegro CL.


6.13 Functionality for quickly writing and reading floats

Often you wish to write floating point numbers to a file which is read later, perhaps in the same Lisp invocation but more likely in a different one. The simple way to do this, writing the decimal representation of the floats, suffers from being very inefficient and somewhat inexact. (It is expensive to convert the internal binary representation to decimal for writing, and then the decimal representation back to binary when the values are read.)

Allegro CL provides functions that write and read the binary representation of floats (rather than the decimal representation), thus saving the time and loss of accuracy associated with conversion to and from decimal format. The functions are single-float-to-shorts, double-float-to-shorts, shorts-to-single-float, and shorts-to-double-float. Note that machine binary representations are used by most languages, and so, just as Allegro CL can read the files produced by writing the integers returned by single-float-to-shorts and convert them back to floating point numbers, programs easily written in other languages can do so as well.


6.14 cl:provide and cl:require

An additional version optional argument has been added to cl:provide and cl:require. It allows specifying a minimal version acceptable for loading a module.


provide

Function

Package: common-lisp

Arguments: module-name &optional version

The non-standard version argument, if specified, should be a positive real number (a float or an integer). This value is checked when the module specified by module-name is loaded. If the cl:require form also specified a version, it is compared (numerically) with the version in the provide form. If the require version is less than the provide version, a continuable error is signaled. If either form does not have a version specified, there will be no error.



require

Function

Package: common-lisp

Arguments: module-name &optional pathname min-version

The non-standard min-version argument, if specified, should be a positive real number (a float or an integer). This value is compared with the version specified in the cl:provide form in the module. An error will be signaled if the cl:provide form has a version less than the value of min-version. If the cl:provide form has no version specified (or there is no cl:provide), no error will be signaled.



6.15 cl:macroexpand and cl:macroexpand-1

Both macroexpand and macroexpand-1 are enhanced to receive a new argument, special-operator-stop, and to add behavior for specific kinds of environments.


macroexpand

Function

Package: common-lisp

Arguments: form &optional environment special-operator-stop

A second (non-standard) optional argument, special-operator-stop, has been added to allow controlling behavior for specific kinds of environments.

If the environment passed in is a :compiler environment (as opposed to a :compilation, :interpreter, :evaluation, or :macros-only) then compiler-macros will be expanded by both macroexpand and macroexpand-1. This is to simulate what happens when the compiler does its macro expansions. Note that this behavior is an extension to the ANSI Spec, which states that compiler-macros are not expanded by macroexpand/macroexpand-1. Note also that any portable code-walker which expects to receive an ansi-compliant environment must condition the environment by using sys:ensure-portable-walking-environment on the argument.

If a :compiler environment is passed in to macroexpand/macroexpand-1 and special-operator-stop is true, then a special-form (a form whose car is a special-operator) will not be macroexpanded, as is otherwise usually the situation. This feature is provided to allow a specialized code-walker (not necessarily a portable one) to see what special forms the compiler sees. If the walker then knows how to interpret the syntax of the special-operator, it can do so in an implementation-dependent way; otherwise, it can always do the macroexpansion again with the special-operator-stop set to nil, in order to get a full macroexpansion through the special form.

Example

cl-user(1): (setq env (sys:make-compilation-unit-environment))
#<Augmentable compiler environment @ #x40c916aa>
cl-user(2): (macroexpand '(case num
                            (1 (foo "one"))
                            (2 (foo "two"))
                            (3 (foo "three"))
                            (otherwise (foo "unknown")))
                            )
(let ()
  (cond ((eql '1 num) (foo "one"))
        ((eql '2 num) (foo "two"))
        ((eql '3 num) (foo "three"))
        (t (foo "unknown"))))
t
cl-user(3): (macroexpand '(case num
                            (1 (foo "one"))
                            (2 (foo "two"))
                            (3 (foo "three"))
                            (otherwise (foo "unknown")))
                            env)
(let ()
  (cond ((eql '1 num) (foo "one"))
        ((eql '2 num) (foo "two"))
        ((eql '3 num) (foo "three"))
        (t (foo "unknown"))))
t
cl-user(4): (macroexpand '(case num
                            (1 (foo "one"))
                            (2 (foo "two"))
                            (3 (foo "three"))
                            (otherwise (foo "unknown")))
                            env t)
(excl::simple-case num (1 (foo "one")) (2 (foo "two")) (3 (foo "three"))
                   (otherwise (foo "unknown")))
t
cl-user(5): (macroexpand-1 * env t)
(excl::simple-case num (1 (foo "one")) (2 (foo "two")) (3 (foo "three"))
                   (otherwise (foo "unknown")))
nil
cl-user(6): (macroexpand-1 * env)
(let ()
  (cond ((eql '1 num) (foo "one"))
        ((eql '2 num) (foo "two"))
        ((eql '3 num) (foo "three"))
        (t (foo "unknown"))))
t
cl-user(7): 


macroexpand-1

Function

Package: common-lisp

Arguments: form &optional environment special-operator-stop

The effect of the second (non-standard) optional argument to macroexpand-1 is the same as described just above in the description of macroexpand in Allegro CL. See that description and the associated examples for further details.



6.16 cl:simple-condition-format-arguments and cl:simple-condition-format-control

The generic functions cl:simple-condition-format-control and cl:simple-condition-format-arguments take condition arguments and return the values of the respective slots. Allegro CL extends the condition system to define and sometimes bind the format-control and format-arguments slots in all conditions. That is, the slots always exist but are only sometimes bound. The slot names are internal in the excl package, and so are excl::format-control and excl::format-arguments.

Because the slots are not always bound (except for actual simple-conditions), code should check that there are bound before trying to access them, with tests like:

(slot-boundp instance 'excl::format-control)
(slot-boundp instance 'excl::format-arguments)

Because these slots need not exist in Lisps other than Allegro CL (again except for simple-conditions) code which tries to access them should be conditionalized for Allegro CL.

Many conditions in Allegro CL which are not simple conditions bind these slots (including, for example, undefined-function and and unbound-variable). We do not give a list, however, because it will likely go out of date. Users who wish to make use of the slot value should, again, test whether they are bound before accessing them.


6.17 What user-homedir-pathname does on Windows

user-homedir-pathname is a Common Lisp function that "determines the pathname that corresponds to the user's home directory on host." host is an optional argument.

In Allegro CL, the host argument is ignored in all cases. Allegro CL simply polls the Operating system in which it is running asking for the current user's home directory. On UNIX, this concept is well defined. On Windows, the notion of a home directory is more murky. Here is what Allegro CL does on Windows.

  1. If there is a value for the HOME environment variable, it is obtained. It will be a string.
  2. If HOME has no value, then the HOMEDRIVE and HOMEPATH environment variables are examined and if both exist, a string is made from concatenating them together.

If a value has been determined from HOME or the combination of HOMEDRIVE and HOMEPATH, that value is processed and returned as a pathname if it exists. HOME or HOMEDRIVE and HOMEPATH are not set, or if the home directory they specify does not exist, #P"C\\" is returned. (A warning is signaled if the environment variables name a directory which does not exist.)

Thus, on Windows, you can change what user-homedir-pathname returns in a running Lisp by setting the value of the HOME environment variable to be a string naming the desired existing directory, for example:

(setf (sys:getenv "HOME") "C:\\mydir\\")

Only the Lisp process see this value of HOME. You are not setting it for all processes or permanently.


6.18 Speed and pretty printing

While investigating ways to speed up Allegro CL, developers at Franz determined that pretty printing was a significant user of compute cycles, and that turning pretty printing off produced significant speedup of code that did output. This conclusion is not particularly suprising, of course. It takes work to produce pretty output. The question is, what to do about it. Turning off pretty printing sounds easier than it is.

Allegro CL starts with *print-pretty* set to t and further, the value in *cl-default-special-bindings* is (essentially) t as well. So simply setting *print-pretty* to nil will not work because the true value will tend to return unexpectedly (in new processes, for example).

Further, user code may depend on the initial value of *print-pretty* being t, so the initial value could not be changed.

However, we can make suggestions to users so that they can achieve the speedups when desired.

In a development image

There are three steps.

(1) change the value of *print-pretty* in *cl-default-special-bindings* to nil by evaluating

(setq *print-pretty* nil)
(tpl:setq-default *print-pretty* nil)

(See setq-default.)

(2) avoid using format strings that are pretty-printing by nature (such as ~< ... ~:$gt;).

(3) Set the value of *pprint-gravity* to nil. Code in Allegro CL that used to bind *print-pretty* to t now bind it to *pprint-gravity*. That variable is not set on the *cl-default-special-bindings* list.

In custom images

There is a module pprint.fasl. When loaded into an image, it sets *print-pretty* and *pprint-gravity* to t. This module is loaded automatically when an image is built with a standard top-level. However, when an image is built with a minimal top-level (as described in Minimal top levels in building-images.htm, the pprint module is not loaded.

So when building an image, you can include them for a development image (above), putting them in, say, custom.cl, or you can build the image with a minimal top-level.

Note that the guts of pretty-printing are in the pprint module. Whenever a format statement, or a print statement with *print-pretty* set to t, is executed, the pprint module is required, so that the machinery is present to do the pretty-printing. So you have to be careful to avoid such cases. If you need pretty printing, you should use the strategy presented above rather than a minimal top-level strategy.

Starting in release 6.2, the new variable *print-circle-gravity* acts with respect to *print-circle* as *pprint-gravity* does with *print-pretty*: no Allegro CL code sets the value of *print-circle*. Instead, Allegro CL code binds it where necessary to the value of *print-circle-gravity*, and *print-circle-gravity* is only set in two places: initially to nil and in the :pprint module to t.

Further notes

Please check the Allegro CL FAQ from time to time to see if there is new information on this issue.


6.19 Floating-point infinities and NaNs, and floating-point underflow and overflow

The IEEE floating-point standard calls for infinities and NaNs (Not-a-Number) to be represented and used. So division of a non-zero finite float by zero produces an infinity, while a division of zero by zero produces a NaN.

the Common Lisp standard does not call for these special floats (as they are often called), but does allow for implementation of IEEE. Further, compiled code which dispatches directly to an IEEE floating-point processor may get back a special float result which it may just return, particularly if it is compiled at high speed and low safety. So consider the following from Allegro CL:

cl-user(115): (defun foo-err (sf)
                (declare (single-float sf))
                (declare (optimize (speed 1) (safety 1)))
                (/ 1.0 sf))
foo-err
cl-user(116): (compile *)
foo-err
nil
nil
cl-user(117): (foo-err 0.0)
Error: Attempt to divide 1.0 by zero.
  [condition type: division-by-zero]
[1] cl-us