Loading

1.0 Using the load function

In Allegro CL, load has additional keyword arguments, searchlist, system-library, script, and foreign, described in the bullets below. Note that if the argument is a foreign library file (such as a so/sl/dll file) without any directory information, Allegro CL passes the filename off to the Operating System without merging the name with *default-pathname-defaults* or using the search list.

In earlier versions of Allegro CL, there was an additional libfasl keyword argument which is no longer supported.

• search-list, if specified, should be a search list which will be used instead of *load-search-list*. (See Search lists.)

• system-library, if true (default is nil) tells the system that the file being loaded is a system library file. The file should be a foreign library file, that is a shared object (.so) file on most Unix platforms, a shared-library file (.sl) on HP's, or a dynamic link library (.dll) file on Windows. Specifying this argument true has no practical effect on the running Lisp image but it tells generate-application not to copy the file to the application directory (or any subdirectory) when building an application. (It is a bad idea to copy system libraries to an application directory because (1) it may be illegal to distribute them; (2) having an application use a personal copy of a system library which may be different from the version installed on the system can lead to subtle errors. For example, when CLIM is loaded on Windows, it loads the system library user32.dll with system-library t, and so user32.dll does not get copied to CLIM application directories created with generate-application.) See delivery.html, particularly Installation of your application on Windows using the Install Wizard, for more information.

• script, if true (the default), causes load to ignore the first line of a source file if its first two characters are ## followed by #!. This is to allow script files to serve as both script files and ordinary source files. Using script files to start Lisp is described in Starting using a shell script in startup.html.

• The foreign keyword argument, if true (default is nil), tells the system that the file being loaded is a foreign library file. Typically, the system checks the file extension and if it is one listed in the value of *load-foreign-types*, it treats the file as a foreign library file and acts appropriately. However, on some systems, extensions of foreign library files is not consistent (some may add version numbers to the extension, for example). Users should not modify *load-foreign-types*, but if a file is a foreign library file whose extension is not listed in *load-foreign-types*, it will be loaded as a foreign library if the foreign keyword argument is specified true.

Certain global variables are bound when load is loading a file. Therefore, setting those variables in a file such as an initialization file (e.g. .clinit.cl) may not have the desired effect. (Setting global values in an initialization file is discussed in Setting global variables in initialization files and in The package on startup (which discusses *package* specifically), both in startup.html.) The following table shows what variables are bound by load:

The top-level command :ld calls the load function. :ld will accept multiple filenames but it does not accept keyword arguments. :ld reads its arguments as strings and does not accept pathnames or streams as filename arguments. Therefore, to load the file test.cl, the following :ld command will work:

:ld test.cl 

The file is found via the load search list, described below. With the default value of the load search-list (it is the value of the variable *load-search-list*), the path provided is merged with *default-pathname-defaults*.

There are other keyword arguments for load in Allegro CL other than those specified by the ANSI standard, such as unreferenced-lib-names, used when load loads foreign (C or Fortran) object files. See foreign-functions.html for details of these additional keyword arguments.

1.1 EOF encountered error during a load

If you try to load a file that contains an incomplete form (because of, e.g., a missing closing parenthesis), load signals an error with condition end-of-file. Consider the following file missing-paren.cl:

(defun foo nil nil)
(defun bar (a b) (+ a b)

The closing parenthesis is missing from the definition of bar. When Lisp tries to load this file, it signals an error:

USER(1): :ld missing-paren.cl
; Loading /net/rubix/usr/tech/dm/missing-paren.cl.
Error: eof encountered on stream #<EXCL::CHARACTER-INPUT-FILE-STREAM
#p"/net/rubix/usr/tech/dm/missing-paren.cl" pos 45 @ #xa91f02>
starting at position 20.
 [condition type: END-OF-FILE]

Restart actions (select using :continue):
 0: retry the load of missing-paren.cl
[1] USER(2):

Note the line:

starting at position 20.

That indicates that the incomplete form starts at position 20 in the file. Opening the file in Emacs and entering the command C-u 20 C-f (assuming standard keybindings) should bring you to the beginning of the incomplete form.

1.2 Special utilities for loading applications

Allegro CL does provide certain utilities for loading collections of files, in particular collections of files which comprise an application. The primary utility for this purpose is the defsystem utility, described in defsystem.html. A couple of simpler utilities are also available: the macros load-application and tenuring.

tenuring allows evaluation of forms with all live objects tenured at the end of the evaluation. You might want to call load within that macro. See the discussion in gc.html for more information.

load-application allows you to specify the environment in which the loading takes place.

2.0 Search lists

Now that logical pathnames are available in Lisp, search lists may be obsolete. However, they are preserved for backward compatibility.

This section describes how load and require find the files they are called upon to load. Novice users, who keep the files to load in the same directory as they run Lisp need not concern themselves with the subtleties of this section. However, the facilities defined in this section are very powerful, allowing complicated mechanisms to be defined for finding files specified without directory paths or with only relative directory paths. If such capability will be useful to you, you should read on.

The functions require and load (and the top-level command :ld) read files. The Common Lisp standard specifies how the filename argument to these functions is processed to yield the true name of the file to be read: the argument is merged (using the Common Lisp function merge-pathnames) with the value of *default-pathname-defaults*. This file-finding mechanism is inadequate for all but the simplest of applications although logical pathnames may be adequate. Note that using the initial value of the load search-list, pathnames are merged with *default-pathname-defaults*. Note further that the directory specified by *default-pathname-defaults* may be different from the directory returned by current-directory (see the description of chdir).

Addressing the need for a more flexible file-finding mechanism, Allegro CL incorporates search lists, data structures embodying information about how the functions require and load will find a file to load into Lisp. Both functions have their own search list.

Autoloading (where an action like calling a function triggers the loading of the module containing the associated functionality) also uses a search list. We do not discuss it much in this document since autoloading is usually invisible to the user.

Search lists permit the user to specify a sequence of pathnames to be merged with the filename argument to require or load. We henceforth refer to this filename argument as the supplied name. From among those pathnames resulting from the sequence of merging operations, the first pathname that satisfies the given criteria will be loaded. Further, it is possible to specify side effects, e.g. to compile a file if the source file has been modified more recently than the corresponding compiled (fasl) file. This very general mechanism provides considerable control over file-loading operations.

In general, most of the default Lisp functionality stored in a file called the bundle file, whose name is files and whose extension is bu or ebu. When you load a module (for example, inspect, foreign-functions, trace, etc.) the system must know to look in the bundle file instead of looking for the fasl file in the Lisp library. The :lib-bundle search list keyword argument provides this functionality.

2.1 Search List Structure

A search list is a recursive data structure. It may be a symbol, string, pathname, or a list of search lists. (The car of a list is interpreted specially.) The elements of a search list are processed sequentially. If the search list is recursive, it is processed in depth-first fashion. The goal of processing a search list is to locate a file to be loaded -- in effect a search list returns the pathname of a file. It is often convenient to refer to search lists as returning a pathname, especially when discussing recursive search lists. It is an error if a file cannot be located within the given constraints. The different kinds of search lists are handled as follows:

• Symbols. If the symbol is bound, then its value is interpreted as a search list. If the symbol is unbound, it is ignored. If the symbol is nil, the denoted file is the filename supplied as an argument to load or require.

• Strings and pathnames. The file is determined by merging the string or pathname with the supplied name. If search-element is a string or pathname search list, the denoted file will be (merge-pathnames supplied-name search-element).

• Lists. The car of the list determines how the elements of the list are processed. The car may be a keyword or a search list. If it is a keyword, it must be one of :lib-bundle, :first, :newest, :newest-ask-compile , :newest-do-compile, or :call. If it is not a keyword, the list is treated as if it began with the keyword :first.

We now define the effect of the keywords that can be the car of a search list which is a list. In the definitions below, we borrow the Common Lisp notation &rest search-list to represent the rest of the list.

(enough-namestring (merge-pathname supplied-name sub-path)
default-pathname)

For an example using :call, a user supplied function could look like:

(defun my-call 
 (target search-list order check-bundle)
 ;; User supplied code, eventually returning a pathname,
 ;; a string or nil.
 )
;; A sample searchlist that uses :call.
(:newest
 (:call my-call . . .)
 (:call my-call . . .))
;; The order argument to my-call will be :newest instead
;; of :first. 

2.2 Search lists variables

The search list for the load function is bound to the symbol *load-search-list*. The default value on many platforms for this symbol is

(#p"" #p(:type "fasl") #p(:type
      "cl") #p(:type "lisp"))

Thus load function looks for its file first by its supplied name in the directory which is the value of *default-pathname-defaults*. If it cannot find the file there, it looks in the same place for a file with a "fasl" file type, then with a "cl" file type, and then with a "lisp" file type. If no file is found, then the search list returns nil indicating no satisfactory file corresponding to the supplied name was found.

The search list for the require function is bound to the symbol *require-search-list*. The initial value for this symbol changed in release 6.0 compared to earlier releases. Its value is now:

((:first #p"sys:;code;.fasl"
     #p"sys:;private;.fasl"
     #p"sys:;clos;.fasl"
     #p"sys:;sys;.fasl"
         (:lib-bundle #p"sys:;code;.fasl")
         (:lib-bundle #p"sys:;private;.fasl")
         (:lib-bundle #p"sys:;clos;.fasl")
         (:lib-bundle #p"sys:;sys;.fasl"))
 #p"src:;code;.cl"
 #p"src:;private;.cl"
 #p"src:;clos;.cl"
 #p"src:;sys;.cl")

require now only looks for fasl files in subdirectories of sys: and for source files in subdirectories of src:. In sys: it also looks in the bundle file. A problem with require in earlier releases, where it looked in the current directory as well as library directories, was that files with names the same as module name ("build" for example) were loaded in preference to the library file, with undesired results.

Note that looking in the bundle file is preceded by looking in code/. If a file exists in both the bundle and in the code/ subdirectory, then the first file encountered will be in code/ and will be used. (This ensures that a fixed fasl file which was originally part of the bundle, supplied as a patch and placed in code/, will be used instead of the version in the bundle file.)

There is also an autoload search list, the value of *autoload-search-list*, which is used by the system when an autoload is triggered. Autoloads are triggered by actions such as calling a function whose functionality is not presently in the system. (Not all such functions trigger autoloads. Typically only the important functions of a facility.) Autoloading is usually nearly invisible to users. Most modules are autoloaded from the bundle. The structure of the search list, however, ensures that updated fasl files which replace fasl files in the bundle are found and used correctly.

2.3 Search list example

Suppose that you maintain a personal Lisp library in a subdirectory of your home-directory, and that you wish to have the load function look for files in this subdirectory. Let us assume this subdirectory is named lisplib. It would be reasonable to search your library directory after searching the directory which is the value of *default-pathname-defaults*. (Note that the directory specified by *default-pathname-defaults* may be different from the directory returned by current-directory. See the description of chdir.)

Further, say you would like to compile automatically all files that are in your working or library directories. The following definition of system:load-search-list in an initialization file (such as .clinit.cl) will accomplish this. Your home directory pathname is denoted with a tilde (~).

(setq system:*load-search-list*
  '(:first
    (:newest-do-compile 
      #.(make-pathname :type "fasl")
      #.(make-pathname :type "cl"))
     (:newest-do-compile
     #.(make-pathname :directory "~/lisplib"
:type "fasl")
      #.(make-pathname :directory "~/lisplib"
:type "cl"))
 ))

If you start up Lisp in the directory ~/work, the expression

(load "foo") 

will result in the following pathnames being generated.

>~/work/foo.fasl ;; these two point to
~/work/foo.cl ;; the current directory
~/lisplib/foo.fasl ;; these two point to the 
~/lisplib/foo.cl ;; lisplib directory

If ~/work/foo.cl exists, it will be compiled, if necessary, and ~/work/foo.fasl will be loaded. If ~/work/foo.fasl exists but ~/work/foo.cl does not, ~/work/foo.fasl will be loaded. If neither of these files exists, the next two files are examined. If ~/lisplib/foo.cl exists, it will be compiled, if necessary, and ~/lisplib/foo.fasl will be loaded. If ~/lisplib/foo.fasl exists but ~/lisplib/foo.cl does not, ~/lisplib/foo.fasl will be loaded. If neither of these files exist, an error is signaled.

Note: if you change the value of *require-search-list*, you must include

(:newest (:lib-bundle #p"sys:;code;.fasl") 
         #p"sys:;code;.fasl"
         #p"sys:;code;.cl" 

so that Lisp will be able to find library files such as foreign.fasl.

3.0 The Allegro Presto facility has been removed

The Allegro Presto facility was a space saving option in releases of Allegro CL prior to 7.0. It is no longer supported. It worked by only partially loading function definitions when a fasl file was loaded in libfasl mode, loading the full definition only when the function was actually called. It has been removed bacause the space savings can, for the most part, be obtained in other ways, for example by using pll files, and the residual space savings no longer justify the costs of Allegro Presto. (The main cost was extra complexity, both within Allegro CL and in user code and applications. This complexity could result in bugs which were often subtle and difficult to fix.)

For the most part, few or no changes to user code are necessitated by the removal of the Allegro Presto facility. The operators, variables, and arguments associated with Allegro Presto are retained for backward compatibility (although use in some cases signals a warning), but are effectively no-ops. Here is a list of specific changes:

• :libfasl argument to load and to load-patches: the functions load and load-patches took a libfasl keyword argument which is now ignored.

• *libfasl*: this variable provided the default for the libfasl keyword argument to load. It still exists but is ignored. References to it should be removed. It may be removed in a later release.

• :presto, :presto-flush-to-code-file, and :presto-lib arguments to build-lisp-image: the function build-lisp-image took these arguments but they are now ignored and signal a warning when used.

• sys:prestop: The operator, which is obsolete, still exists and always returns nil.

• resident-function: this obsolete function now effectively acts like identity applied to the same argument.

• sys:presto-build-lib: this obsolete function is now a no-op (i.e. does nothing) and a warning is signaled when it is called.

• sys:presto-fasl-set: this function returned a list of files containing partially loaded definitions. The operator still exists and always returns nil.

• Integrated Development Environment tools for building projects: options allowed building Allegro Presto images. These options have been removed.

• compiler-no-in-package-warning: this condition class was signaled when a file without an in-package form was compiled or loaded. (Such fasl files could not be libfasl loaded.) This condition is no longer signaled under any circumstances, but the class is still defined for backward compatibility (since handlers will error if they are defined with non-existent condition classes).

4.0 Loading and case modes

Allegro CL supports both modes regarding case of letters in symbols naming Common Lisp objects: a case-insensitive mode where foo, FOO, foO, Foo, etc. all name the same symbol; and a case-sensitive mode where they all name different symbols. See case.html for information on deciding upon, setting, and changing the case mode in Allegro CL. (Besides case-sensitivity, the mode also controls case preference, specifying the case of standard Common Lisp symbols, among other things, as described in case.html.)

A problem with allowing different case modes is that a fasl file compiled in an image using one mode will not behave as expected when loaded into an image. Consider for example the file containing two function definitions:

(in-package :cl-user)

(defun FOO (x) (+ 1 x))

(defun foo (y) (* 2 y))

In an ANSI-mode (case-insensitive) image, loading this file (as a source file) would result in one function definition, on the symbol FOO, which multiplied the argument by 2 and returned the result. In a modern-mode (case-sensitive) image, loading this file would result in two function definitions, one on FOO, and one on foo.

That problem (different behavior when the same source file is loaded into images using different modes) is, however, not surprising given the nature of case-sensitivity. Users working in different modes must manage the issue of symbols differing only in the case of the alphabetic letters in the name of the symbol.

A more serious problem occurs when dealing with compiled (fasl) files. If the file above is compiled in an ANSI-mode image, it will contain two (compiled) definitions of the function named by the symbol FOO. If the file is compiled in a modern-mode image, it will contain, again, two definitions: one on FOO, and one on foo. Now consider what happens when these fasl files are loaded. Let us consider all the cases (we are assuming the setting of convert-mixed-case-symbols is its initial value t, see below in this section for the behavior when the value is nil) :

• Compiled in modern-mode, loaded into modern-mode: two definitions, one on FOO, and one on foo.

• Compiled in modern-mode, loaded into ANSI-mode: one definition, on FOO.

• Compiled in ANSI-mode, loaded into ANSI-mode: one definition, on FOO.

• Compiled in ANSI-mode, loaded into modern-mode: one definition, on FOO.

  Note the fourth case: when the file is compiled in ANSI-mode and loaded into modern-mode, the behavior is different from the source file being loaded into a modern-mode image. In all other cases, loading the source file and loading the compiled file produces the same behavior. Therefore, Allegro CL signals a continuable error with condition type excl:fasl-casemode-mismatch when an attempt is made to load a fasl file compiled in an ANSI-mode image into a modern-mode image:

  ; Fast loading x.fasl
  Error: #p"x.fasl" was compiled in case-insensitive-upper mode.  
    Continuing to load this file may result in symbol-name mismatches.
   [condition type: fasl-casemode-mismatch]
  
  Restart actions (select using :continue):
  0: Continue loading x.fasl
  1: retry the load of x.fasl
  2: skip loading x.fasl
  3: Return to Top Level (an "abort" restart)
  4: Abort #<process Initial Lisp Listener>
  [1] cl-user(2): 

  The correct thing to do at this point is to recompile the source file in the modern-mode image (assuming that is possible). The results of choosing to continue (by, e.g. calling :continue) are undefined (but will not be untoward if in fact there are no name conflicts).

  Note that the setting of convert-mixed-case-symbols does not affect compilation: all (unescaped) symbol names are converted to uppercase in ANSI-mode images during compilation. The setting does affect loading of fasl files, however. If the setting is nil (the initial setting is t), then the second case listed above:

• Compiled in modern-mode, loaded into ANSI-mode: one definition, on FOO. produces the results: two definitions, one on FOOand one on |foo| (we are using escapes to indicate that the print name of the symbol is all lower case).

Copyright (c) Franz Inc. Lafayette, CA., USA. All rights reserved.