|
Allegro CL version 11.0 | ||||||||
Note that there are many classes defined as part of the defsystem facility and named by exported symbols in the defsystem package. The facility is in the defsys module (and associated modules). Defsystem code is normally included automatically in development images, but if you need to load it, evaluate (require :defsys) (it is not an error to evaluate that form when the module is already loaded).
Defsystem is a tool designed to help developers deal with large programs contained in multiple files. The defsystem facility improves the development environment by providing quick and easy ways to perform complicated tasks on a set of files.
For example, let's say a developer has a program with 20 different files and has just edited 3 of the files. Those 3 files will need to be recompiled. There may also be other files that will need to be recompiled because the code in them depends in some way on the code in the files that were just edited. Without defsystem, the developer would have to separately issue commands to compile and load each of the files in the correct order. With defsystem, the developer can set up a system by indicating which files should be included in the system and in what order they need to be compiled. Then after editing some of the files, the developer can just issue one function call and the defsystem will figure out which files need to be recompiled and will do the compilation in the correct order.
The defsystem facility is strongly based on CLOS, which means that the defsystem can be easily extended and customized to meet individual needs. The second part of this chapter describes the ways in which the defsystem facility can be extended and customized.
Many symbols associated with the defsystem facility are in the defsystem package. It has nicknames defsys and ds. However, most operators (functions, macros, etc.) are named by symbols in the excl package.
This chapter uses a hierarchy of terms. We define those terms below:
Systems are defined with the defsystem macro. There are two parts to a system definition: the system options and the module-specification. System options specify attributes of the system or define default values for attributes of modules in the system. Module-specifications describe the module-groups of the system and dependencies between modules and module-groups.
An item in a module-specification can be specified with a file name, the name of a named module group defined earlier in the same system, or the name of another system. Supplying the name of another system in a module-specification causes the named system to be a component system of the system being defined. This means that system operations will be performed recursively on the component system (if the :include-components keyword to the system operation method has the value t).
The defsystem macro is defined on its own page. It takes a module name (a symbol) and a possible empty list of system-options as required arguments followed by a list of module-specifications.
The allowable system-option keywords are shown in the table below. After the table, we describe module specifications.
Keyword |
value type |
default value |
Notes |
:pretty-name |
string | print-name of system-name symbol | This name will be used for printing. |
:default-module-class |
symbol identifying a class | ds:lisp-module |
Predefined classes are:
Users may define additional classes, see the paragraphs under the heading Defining new classes below. |
:default-package |
package object or string or symbol naming a package | The value of *package* | Can be overridden for a module or module-group with the
:package option in the long module form. Also can be overridden during an operation with
the :package argument. |
:default-pathname |
pathname object or string or symbol naming pathname | *default-pathname-defaults* | This pathname will be merged with the filenames of the modules of the system in order to find files. |
:default-file-type |
string | depends on value of :default-module-class
- see notes at right. |
Specifies file type (extension) for source files for modules
in system. If :default-module-class is
ds:lisp-module, default is value of
*source-file-types*; if ds:text-module, default is nil; if ds:c-module, default is
"c". |
:property-list |
property list | nil | The value is added to the property list of the system. |
Table 1: system-option keywords and values |
|||
More than one module-specification may be specified when defining a system. Module-groups can be specified using a short form module-specification or a long form module-specification. We describe short and long form module-specifications below. Note that we use the terms short form specification and short form module-specification interchangeably. They mean the same thing. We do the same thing with long form specification and long form module-specification.
(:module ...) and (:module-group...) create module groups, as do all other defsystem module-specifications. Module-specifications which begin with the keywords :module and :module-group provide ways of associating a name with a module-group.
Note that a module-specification which begins with :module really creates what we call a module-group.
The (:module ...) syntax for long-form module-specifications is used to maintain syntax compatibility with the defsystem facility available on Symbolics machines. Note that our term module-group is similar to what is called a module in the Symbolics' documentation of their defsystem.
Short form module-specifications provide a simple syntax for describing systems that:
More complex needs can be met by using long form module-specifications.
A short form module-specification is a list beginning with one of :serial, :parallel, :definitions, or :module-group. The table below gives a description of each of the short forms. In the table, module-spec-element may:
Short form name |
Arguments |
Discussion |
:parallel |
{module-spec-element}+ | This module-specification keyword specifies that the elements
listed have no dependencies between them. A :parallel
module-specification list causes each element to be processed as an individual. This
module-specification list does not cause each element to be loaded before the next element
is compiled. |
:serial |
{module-spec-element}+ | This module-specification keyword specifies that each element depends on the element listed before it in the list. Therefore, during compilation, each module-spec-element specified is compiled and loaded before the next. |
:definitions |
primary-module-spec-element {module-spec-element}+ |
This module-specification keyword option specifies that element has a serial dependency on primary and also has a compile-dependency on primary such that if primary is compiled, element must be compiled. In other words element uses definitions in primary. If primary is touched, then element should be recompiled or reloaded. This keyword is often used to describe dependencies between files containing macros and files that use macros. |
:module-group |
name short-form {long-form-option}* |
This module-specification keyword is used to name a
short-form. :module-group gives the name name
to the short-form system described by short-form. name can then be used to
refer to short-form in other specifications. name is a symbol (it is not
evaluated). short-form is a list that describes a module-group. long-form-option
can be any option which can be used for long form specifications as described below. Zero
or more long-form specifications can be specified. |
Table 2: Short Form module specifications |
||
Most simple needs can be met by use of the :parallel and :definitions short forms. The following are examples of defining short form systems.
(defsystem :foo ()
(:parallel "file1"
(:serial "filea" "fileb")
"file2"))In this example, filea would be compiled before fileb, but files filea and fileb are not dependent on file1 and file2. Files file1 and file2 are not dependent on each other.
(defsystem :my-sys ()
(:serial "file1" "file2"
(:parallel "filea"
my-other-system)))In this example, file2 is dependent on file1. Also, filea and the system my-other-system are dependent on file2.
The :definitions module-specification keyword is different from the :serial module-specifications keyword in that :serial does not force recompilation of modules farther down the list. As an example, suppose we have:
(defsystem :bar ()
(:definitions "macro-def-file" "file-that-uses-macros"))In this example, file-that-uses-macros is dependent on macro-def-file. If macro-def-file is compiled, then file-that-uses-macros will be compiled too.
(defsystem :named-system ()
(:module-group my-group (:serial "file1" "file2")))This example associates the name my-group with file2 being dependent on file1.
Long form module-specifications should be used when more control over some module-group or module-groups is desired. With the long form it is possible to override the system default values for various parameters and to specify more complex dependencies.
There are two types of long form module-specifications, one allows a group of modules to be given a name which can be referenced by other module-specifications in the system and the other type provides a simple way to specify non-default options for just one module.
A long form specification is a list of one of the following forms:
(:module name system-or-files {long-form-option}*)or
("filename" {long-form-option}*)Here name is a symbol that names the module-group. name can be used by other module-groups in the system to refer to the module-group it names. system-or-files can be a string or (non-empty) list of strings representing the one or more source files of the module-group, or a symbol representing the name of another system.
Note that :module-group is used to name short form module-specifications, while :module is used to name long form module-specifications. In both cases, the name is associated with the resulting module-group.
("filename" {long-form-option}+) is functionally equivalent to
(:module foo "filename" {long-form-option}+) except
("filename" {long-form-option}+) does not have the added name foo associated with it.
Zero or more keyword options can be specified in the long-form-option position. The allowable keyword options are given in the following table. Note that some options have more than one or a variable number of arguments. All have at least one argument. The `+' after an argument indicates at least one value must appear but as many as you like can appear.
Keyword option name |
Arguments |
Discussion |
:package |
package-name | Must be a package object or a symbol or string identifying a package. If specified, this will override the default package for the system |
:module-class |
module-class-name | Argument must be a symbol naming a module class. If
specified, this overrides the default module class of the system. Pre-defined valid values
are ds:lisp-module, ds:text-module, and ds:c-module.
Users may define additional classes. See the information under
the heading Defining new classes
below. |
:in-order-to |
operation requirement |
operation can be :compile
or :load or the list (:compile :load). requirement must be a list whose first element is either
:compile or :load
and whose other elements (there must be at least one) are symbols each denoting a
module-group. This option says that before operation can be performed on the module,
requirement must first be satisfied. |
:uses-definitions-from |
definition-module-group+ | Argument(s) must be symbols naming module-groups. This option says that (1) the module-group must be recompiled if any definition module is recompiled; (2) if the module-group is to be compiled, then all definition-modules must be recompiled first; (3) if the module-group is to be loaded, all definition-modules must be loaded first. |
:recompile-on |
needed-module-group+ | Argument(s) must be symbols naming module-groups. Same as :uses-definitions-from except needed-module-groups
do not have to be loaded if module-group is loaded. |
:compile-satisfies-load |
boolean | If true, tells system that compiling module-group makes the module up-to-date in the lisp image as if it had been both compiled and loaded. |
:load-before-compile |
module-groups-or-systems | Argument may be a symbol naming a module-group or system or a list of module-group and/or system names. The module-groups and systems specified must be loaded before module is compiled. |
:force-dependent-recompile |
boolean | Causes all modules examined, after a module with this option, to be recompiled even if they are up-to-date. The main use of this is to cause recompilation of the rest of a system because critical definitions exist in a particular module that is out-of-date. |
:concatenate-system-ignore |
boolean | A true value causes concatenate-system to ignore this module. |
:force-load |
boolean | Always load this file during compile-system. |
:force-compile |
boolean | Always compile this file during compile-system. |
Table 3: Long form module specification options |
||
When a system is redefined the following events occur:
The full description of each function described in this table is given in the page description of the function. :simulate means to print the actions that would be taken but do not actually perform them.
| Operator | Arguments | Notes |
| load-system | system &key
:interpreted :compile :no-warn :reload :silent :simulate :include-components |
Loads system into the current environment. See the description for descriptions of the keyword arguments. |
| compile-system | system &key
:recompile :no-warn :reload :silent :simulate :include-components |
Compiles (if necessary) each module in system. Returns t if any action was taken to bring system up-to-date and returns nil if no action was necessary. See the description for descriptions of the keyword arguments. |
| map-system | system function
&key :silent :simulate :include-components |
Maps function (which should accept one argument) over each module-group in system. See the description for descriptions of the keyword arguments. See the example below the table. |
| clean-system | system &key
:silent :simulate |
Removes the product files of the modules in system. See the description for descriptions of the keyword arguments. |
| concatenate-system | system destination-file
&key :silent :simulate |
Concatenates the fasl files of the modules in system into destination-file. See the description for descriptions of the keyword arguments. |
| touch-system | system &key
:silent :simulate |
Makes the fasl files of the modules in system up-to-date (with the touch command on Unix). See the description for descriptions of the keyword arguments. |
| find-system | system-name &optional
errorp |
Returns the system named by system-name (returns nil or errors as errorp is nil or true and no
system can be found). See the example below. |
| show-system | system |
Prints a description of system. See the example below. |
| undefsystem | system-name |
Removes system-name from the list of defined systems. |
| *default-file-type* | [variable] | The default file type (extension) used by objects of class
default-module. Initially nil. |
In this example using map-system, we pretty print each module in a system.
USER(9): (defsystem :my-serial-sys ()
(:serial "my1" "my2"))
:MY-SERIAL-SYS
USER(10): (map-system :my-serial-sys
#'(lambda (module) (pprint module)))
#<DEFSYSTEM:LISP-MODULE "my1" @ #x68c9a6>
#<DEFSYSTEM:LISP-MODULE "my2" @ #x69197e>
NIL
USER(11): Example using find-system:
USER(18): (defsystem :my-serial-sys ()
(:serial "my1" "my2"))
MY-SERIAL-SYS
USER(19): (find-system :my-serial-sys)
#<DEFSYSTEM:DEFAULT-SYSTEM "MY-SERIAL-SYS" @ #x6b5b4e>
USER(20): Example using show-system:
USER(20): (defsystem :my-serial-sys ()
(:serial "my1" "my2"))
:MY-SERIAL-SYS
USER(21): (show-system :my-serial-sys)
; System: "MY-SERIAL-SYS"
; default package: #<The COMMON-LISP-USER package>
; default pathname: #p"./"
; default module class: DEFSYSTEM:LISP-MODULE
; the system contains the following modules:
; Module-group: "<unnamed>"
; default module class: DEFSYSTEM:LISP-MODULE
; the module-group contains the following modules:
; Module: "my1"
; source file: "my1"
; Module: "my2"
; source file: "my2"
; Dependencies:
; before COMPILE dependencies:
; LOAD #<DEFSYSTEM:LISP-MODULE "my1" @ #x6b638e>
; before LOAD dependencies:
; LOAD #<DEFSYSTEM:LISP-MODULE "my1" @ #x6b638e>
NIL
USER(22):Example using undefsystem:
USER(42): (defsystem :my-serial-sys ()
(:serial "my1" "my2"))
:MY-SERIAL-SYS
USER(43): (undefsystem :my-serial-sys)
NILBecause defsystem is CLOS-based, it can be easily extended in many ways in order to add functionality or meet special needs. It is possible to extend existing system operations or to define totally new operations. The module actions that implement the operations on the modules of a system can also be customized. New classes of system, module group and module can be created to have specialized behavior and new slots. It is also possible to extend the syntax of the defsystem macro that defines systems. Each of these extension methods is described below.
The following is a description of the defsystem class hierarchy. Users can build new classes based on these classes. The following can be used as a guide for choosing which classes to build on.
defsys::defsystem-base-class: all defsystem classes inherit from this class.defsys:module-container: this is a class of objects that contain modules. Objects of this class have a list of modules and a default-module-class which indicates the default class to be used when modules are created for this module-container.defsys:default-module-group and defsys:default-system both inherit from defsys:module-container. Exported accessors for defsys:module-container:
defsys:default-system has the following exported accessors:
nil if there is no such objectdefsys::default-module-group has the following exported accessors:
defsys:default-module has the following exported accessors:
defsys:lisp-module, defsys:text-module, and defsys:foreign-module inherit from defsys:default-module.
defsys:lisp-module has the following specialized methods:
defsys:text-module has the following specialized methods:
defsys:foreign-module has the following specialized methods:
defsys:c-module inherits from defsys:foreign-module and should be used for C code modules. The following methods are specialized for defsys:c-module:
By defining new subclasses of existing defsystem classes, it is possible to add new slots to defsystem objects and to write specialized methods on defsystem generic functions which will affect only objects of the new class.
If new classes are defined, defsystem needs to be told when to use them for creating defsystem objects. The following variables determine which classes the defsystem macro will use when creating defsystem objects.
Another way to alter the module class used by the defsystem macro is with the :default-module-class option, as described in table 1 above.
The syntax of the defsystem macro can be extended through the use of shared-initialize methods. In order to understand how to extend the defsystem syntax, first it is necessary to explain how a defsystem form is processed to create the systems, module-groups and modules.
When defsystem is called, the system name is looked up in a table of systems to determine if this is a new system definition or the system already exists. In the former case, a system object of class *default-system-class* is created for the system and the options list is passed to make-instance as a list of initialization arguments. In the case of reinstalling an existing system, the existing system object is reinitialized with the new options list. In either case, shared-initialize is called to perform initialization of the object. The options are handled either by the shared-initialize method for the system class which handles all slots that have initargs or by keyword arguments to an after-method of shared-initialize defined for the system class. So new system options can be added in either of two ways:
by defining a new system class with a new slot where the initarg of the slot will be a valid option.
by defining an after-method of shared-initialize for the new system class which has a keyword argument for the new option.
The list of module-specifications in the defsystem form is processed in a similar manner by shared-initialize methods and after-methods handling the options for module-groups and modules.
The following defsystem methods can be specialized to obtain customized behavior.
In this section, we provide a number of examples of extending defsystem. The examples are:
As an example of defining and using a new system class, let's say we want a form to be evaluated after a system is loaded. We will need to create a subclass of defsys:default-system which will contain a new slot, eval-after-load. Then we will need a way to initialize this slot at system creation time, and a way to evaluate the form at system load time. For initialization we will want to add a new option to the defsystem macro, which can be done through the use of a shared-initialize method for our new system class. Evaluation of the form can be implemented with an around-method on the load-system generic function.
First we will need to create the new system class. The following code defines a subclass of defsys:default-system with the new slot.
(defclass mysystem (defsys:default-system)
((eval-after-load :initform nil
:initarg :eval-after-load
:accessor eval-after-load)))Now to enable initialization of the eval-after-load slot, we will need to extend the defsystem macro syntax for this class by creating a shared-initialize after-method that takes an :eval-after-load keyword.
(defmethod shared-initialize :after
((new-system mysystem)
slot-names
&key (eval-after-load nil e-a-l-supplied))
(declare (ignore slot-names))
(when e-a-l-supplied
(setf (eval-after-load new-system) eval-after-load)))Next, we need to define an around-method on load-system so that the contents of the eval-after-load slot can be evaluated at system load time. In this method, we will first call the other methods to load the system, then check the return value to see if the system was loaded or not. If it was, then we evaluate the eval-after-load method.
(defmacro mydefsystem (system-name options &body modules)
`(let ((defsys:*default-system-class* 'mysystem))
(excl:defsystem ,system-name ,options ,@modules)))Finally, we can define our system, which has an eval-after-load form to set a variable when the system is successfully loaded.
(defvar *foo-system-loaded* nil)
(mydefsystem :foo-system
(:eval-after-load (setf *foo-system-loaded* t))
("foo"))The following is a description of how system operations and module actions work together.
System operations recursively invoke methods on each of the components of the system. The methods that are invoked are named foo-module-action, where foo-system is the name of the system operation. The foo-module-action method for a component system invokes foo-module-action on its components if ds::.include-components. is true. The foo-module-action method for a module-group invokes foo-module-action on each of its components. The foo-module-action for a module decides if the foo-module method needs to be called and calls it if necessary. The foo-module-action methods also print messages about the actions that are being performed. In simulation mode, the foo-module method is not called by the foo-module-action method.
For example, load-module-action checks if the module has been loaded and if not, calls the load-module method to perform the actual load. compile-module-action and load-module-action are the only actions that do checking on the modules. The other module-actions unconditionally invoke the method to perform the action unless ds::.simulate-mode. is non-nil.
The following example demonstrates how to define a new system operation and the module-action methods that go along with the operation. This example implements a grep-system operation which searches each source file in the system for a given string. Note that this could be implemented more easily using map-system, but this is just an example of defining new system operations and module actions.
(defgeneric grep-system (system string &rest keys))
(defmethod grep-system ((system-name symbol) string
&rest keys)
(apply #'grep-system (find-system system-name t) string keys))
(defmethod grep-system ((system ds:default-system) string
&rest keys)
(dolist (module (ds:modules system))
(apply #'grep-module-action module string keys)))
(defmethod grep-module-action ((system ds:default-system) string
&rest keys
&key &allow-other-keys)
(declare (ignore string keys))
;; we want to ignore component systems, so do nothing
nil)
(defmethod grep-module-action ((module-group ds:default-module-group)
string
&rest keys
&key &allow-other-keys)
(dolist (module (ds:modules module-group))
(apply #'grep-module-action module string keys)))
(defmethod grep-module-action ((module ds:default-module)
string &rest keys
&key (silent nil))
(declare (ignore keys))
(let ((source-namestring (namestring (ds:source-pathname module))))
(unless silent
(format t "Looking at ~a: ~%" source-namestring))
(excl:shell
(concatenate 'string "grep " string ""
source-namestring))))This example involves creating new system, module-group and module classes and specializing defsystem methods for the new classes.
If a developer wants to edit a few files of a large file tree, but doesn't want to make a complete copy of the tree, he/she can define a system that looks for files first in the developer's copy of the tree, then in the master tree. If some modification is done that requires the recompilation of a source file not yet copied to the developer's tree, then defsystem will compile using the source in the master directory and put the fasl (product) file in the developer directory.
The default system and module classes will be subclassed with the new slots master-directory and developer-directory, that will contain the pathnames of the master and developer directories.
The source-pathname method will first look for the source file in the developer directory and then in the master directory.
The product-pathname method will always look in the developer directory.
(defclass md-system (defsys:default-system)
((master-directory :initform nil :initarg :master-directory
:accessor master-directory)
(developer-directory :initform nil :initarg :developer-directory
:accessor developer-directory)))
(defclass md-module-group (defsys:default-module-group)
((master-directory :initform nil :initarg :master-directory
:accessor master-directory)
(developer-directory :initform nil :initarg :developer-directory
:accessor developer-directory)))
(defclass md-module (defsys:lisp-module)
((master-directory :initform nil :initarg :master-directory
:accessor master-directory)
(developer-directory :initform nil :initarg :developer-directory
:accessor developer-directory)))
(defmethod get-master-directory ((system md-system))
(master-directory system))
(defmethod get-master-directory ((module-group md-module-group))
(or (master-directory module-group)
(get-master-directory (ds:parent-object module-group))))
(defmethod get-master-directory ((module md-module))
(or (master-directory module)
(get-master-directory (ds:parent-object module))))
(defmethod get-developer-directory ((system md-system))
(developer-directory system))
(defmethod get-developer-directory ((module-group md-module-group))
(or (developer-directory module-group)
(get-developer-directory (ds:parent-object module-group))))
(defmethod get-developer-directory ((module md-module))
(or (developer-directory module)
(get-developer-directory (ds:parent-object module))))
(defmethod defsys:source-pathname ((module md-module))
;; if file exists in developer directory then return that,
;; else return the master directory pathname
(let* ((developer-directory (get-developer-directory module))
(file-name (defsys:module-file module))
(file-type (defsys:default-file-type module))
(pathname nil))
(setf pathname (merge-pathnames developer-directory
(make-pathname :name file-name
:type file-type)))
(if (probe-file pathname)
pathname
(merge-pathnames (get-master-directory module)
(make-pathname :name file-name :type file-type)))))
(defmethod defsys:product-pathname ((module md-module))
;; always return pathname in developer directory
(merge-pathnames (get-developer-directory module)
(make-pathname :name (defsys:module-file module)
:type excl:*fasl-default-type*)))
;; This macro binds the class variables to our new classes
(defmacro md-defsystem (system-name options &body modules)
`(let ((defsys:*default-system-class* 'md-system)
(defsys:*default-module-group-class* 'md-module-group)
(defsys:*default-module-class* 'md-module))
(excl:defsystem ,system-name ,options ,@modules)))
;; Here is an example of defining a system with our
;; new system options.
(md-defsystem :sys1 (:master-directory #p"master/"
:developer-directory #p"devel/"
:default-file-type "cl")
(:parallel "filea" "fileb"))Given a system A and a second system B, how can one get system B to reference system A?
;; file cross.cl:
(defsystem :a ()
(:parallel "file1"
(:serial "filea" "fileb")))
(defsystem :b ()
(:parallel :a
"file2"))
user(3): :ld cross.cl
t
user(4): (compile-system :b)
; Compiling system: "b".
; Compiling system: "a".
; Compiling module "file1" because the product file does not exist.
; --- Compiling file /tmp/file1.cl ---
; Writing fasl file "/tmp/file1.fasl"
; Fasl write complete
; Compiling module "filea" because the product file does not exist.
; --- Compiling file /tmp/filea.cl ---
; Writing fasl file "/tmp/filea.fasl"
; Fasl write complete
; Loading product for module: "filea".
; Fast loading /tmp/filea.fasl.
; Compiling module "fileb" because the product file does not exist.
; --- Compiling file /tmp/fileb.cl ---
; Writing fasl file "/tmp/fileb.fasl"
; Fasl write complete
; Loading product for module: "fileb".
; Fast loading /tmp/fileb.fasl.
; Compiling module "file2" because the product file does not exist.
; --- Compiling file /tmp/file2.cl ---
; Writing fasl file "/tmp/file2.fasl"
; Fasl write complete
t
user(5): (load-system :b)
; Loading system: "b".
; Loading product for module: "file1".
; Fast loading /tmp/file1.fasl.
; Loading product for module: "file2".
; Fast loading /tmp/file2.fasl.
t
user(6):
Copyright (c) Franz Inc. Lafayette, CA., USA. All rights reserved.
|
Allegro CL version 11.0 | ||||||||