| Allegro CL version 10.1 Unrevised from 10.0 to 10.1. 10.0 version | ||||||||
This document contains the following sections:
1.0 Introduction to excl:dumplisp, a tool for saving an imageThe function dumplisp writes an image file from the current running Lisp image. It has many options and many uses. Unlike the two other functions that create new images, build-lisp-image and generate-application, dumplisp preserves much of the environment of the running image. Both build-lisp-image and generate-application create new images from external constituent parts that inherit nothing from the running image.
If you are building images for general use (for example, an image incorporating an application), we recommend that you use build-lisp-image or generate-application rather than dumplisp as those functions give more control over the resulting images and unnecessary functionality which may be in your running image will not be included.
A new dumplisp feature on Windows only (added by a post-9.0 patch) allows for what is essentially a pseudo-fork operation on Windows and is useful for very large images. See Section 22.0 Virtual dumplisp: for very large images and pseudo-forks on Windows for more details.
One advantage of the small executable/large image model is that all image files use the same executable, so there needs only be one executable file on a machine (or on a network). This means that the location of that executable can be used as a starting location which can be a reference point for finding other necessary files.
The location of the executable is taken as the reference point for finding files so no specific location must be supplied when Lisp starts up. (You can specify a different location with the -H argument if you wish. However, that should not be necessary.)
You might use dumplisp for the following purposes (as well as others not listed):
In order for dumplisp to work, it must examine the currently running image. For this reason, dumplisp will fail if:
dumplisp will fail if there
are too many oldspaces. The limit depends on a complex formula, but is
around 2000. You can see how many oldspaces there are by
calling (room t). You can consolidate oldspaces
with sys:resize-areas. The
error message when there are too many is Error: Too many
regions. Consolidate oldspace with sys:resize-areas and retry if you get this
error.
With the dxl-header keyword argument to dumplisp, you can have the text from an appropriately formated file prepended as the header to a dumped image (dxl) file. The file must contain octets and must be an exact multiple of 8192 octets in length, up to a maximum of (* 9 8192) octets. The first character in the file must be one of the characters in the range from #\1 to #\9.
The header can be view by (Unix) utilities like more and head.
The function dribble-for-dxl-header captures input like dribble does and produces a file suitable for being a dxl header.
Allegro CL does a number of things when it starts up before it prints the first prompt and is ready to accept input. Programmers may want to know what it does and in what order, so that programmer modifications and customizations can be done at the right time. Allegro CL calls the function excl::start-lisp-execution on startup. The source for that function and for several associated functions (that process command-line arguments and read initialization files, e.g.) can be found in the file sys:;src;aclstart.cl, equivalently [Allegro directory]/src/aclstart.cl. See also startup.htm where startup is described in English rather than Lisp code.
Please see delivery.htm for a detailed discussion of creating images for delivery.
A typical customization is loading a complicated program and thus saving the time to load whenever you start Lisp (but quicker startup should be balanced against additional disk space usage).
When a Lisp Listener (which presents the prompt to users and
accepts and processes typed input) starts up, bindings are set up for
many Common Lisp and Allegro CL-specific global variables. The
bindings come from alists, one of which is the value of *cl-default-special-bindings*. Here is
part of *cl-default-special-bindings*:
;; From excl:*default-cl-special-bindings* (a few values only) ;; (*PRINT-LENGTH*) is equivalent to (*PRINT-LENGTH* . NIL). (*PRINT-LENGTH*) (*PRINT-LEVEL*) (*PRINT-RADIX*) (*PRINT-BASE* . 10) (*PRINT-PRETTY* . T) (*PRINT-ESCAPE* . T)
Note that each variable in the portion of the alist shown is
associated with a specific value (nil for
*print-length*, *print-level* and
*print-radix*, 10 for *print-base*, and
t for *print-pretty* and
*print-escape*). When a Lisp listener is started in the
dumped image, *print-length* in that listener will be
nil regardless of its value when dumplisp was called, as this
example shows:
;; We bring up Allegro CL and note that *print-length* has initial value NIL. USER(1): *print-length* NIL ;; We set the value of *print-length* to 20: USER(2): (setq *print-length* 20) 20 USER(3): *print-length* 20 ;; and we dump the image: USER(4): (excl:dumplisp :name "my-image") USER(5): (exit) ;; Now we start my-image: ============================================================== Starting image `my-image' [...] ;; Note that *print-length* is NIL, not 20: USER(1): *print-length* NIL USER(2):
The listener in the restarted image gets the value of
*print-length* from the *cl-default-special-bindings* alist,
where *print-length* is associated explicitly with
nil. The value in the parent image is not
relevant. In order to ensure that *print-length* (or any
variable whose name appears on either alist) has the desired value in
the listener when the dumped image is started, you must modify the
appropriate alist. The easiest way to do this is with tpl:setq-default. This macro
is like setq: its first argument should be a symbol and is not
evaluated; its second argument should be a value and is evaluated. It
modifies the alist appropriately (see the definition for a complete
description) but does not affect the current value. Thus:
;; We bring up Allegro CL and note that *print-length* has initial value NIL. USER(1): *print-length* NIL ;; We use TPL:SETQ-DEFAULT to modify the EXCL:*CL-DEFAULT-SPECIAL-BINDINGS* ;; entry referring to *PRINT-LENGTH* so new bindings will be to 20, not NIL: USER(2): (tpl:setq-default *print-length* 20) 20 USER(3): *print-length* ;; its value in the current listener is unchanged NIL ;; and we dump the image: USER(4): (excl:dumplisp :name "my-image") USER(5): (exit) ;; Now we start my-image: ============================================================== Starting image `my-image' [...] ;; Note that *print-length* is 20, not NIL: USER(1): *print-length* 20 USER(2):
Recall that changes to the reader (such as setting a
macro-character) are stored in *readtable* and that
variable appears on *cl-default-special-bindings*.
This same issue affects setting values from an initialization file. A discussion very similar to this one appears in startup.htm.
Command-line arguments are processed as usual or ignored as the
ignore-command-line-arguments keyword argument to dumplisp is nil (the default) or t. The emacs-lisp
interface uses command-line arguments to start up. See below under the
heading
Section 14.0 The emacs-lisp interface and dumped images
for information on starting the
interface when command-line arguments are ignored.
Command-line arguments are available, of course, with sys:command-line-arguments and related functions. They are just ignored by Allegro CL's startup routine when ignore-command-line-arguments is true.
There are various initialization files that might be read:
~/.clinit.cl, [working-directory]/.clinit.cl (if
[working-directory] is different than ~), ~/ and
[working-directory]/clinit.cl, and
sys:siteinit.cl. These files will be looked for if *read-init-files* is
t when the image is dumped (just
[working-directory]/.clinit.cl and clinit.cl and
sys:siteinit.cl, if *read-init-files* is
:no-home). This variable should be set (not bound) prior
to the call to dumplisp. Setting this variable after
Lisp has started up only affects its value in dumped images. It is not
used by Lisp after startup has completed.
Command-line arguments can suppress the reading of some or all initialization files (see startup.htm). Since command-line arguments are processed before initialization files are read, the value of this variable can be changed using them -- e.g. specify `-e (setq excl:*read-init-files* t)' on the command-line -- assuming command-line arguments are not ignored.
The variable *restart-actions*, if
true, should be a list of functions of no
arguments (function names or function objects are acceptable). These
functions are called in order, after the command-line arguments are
processed and the init files are read. The purpose of this variable is
to do system initializations. Do not set this variable or you may
eliminate values put on the list by Allegro CL and its associated
products (CLIM, for example, uses this variable). Instead, use
push or pushnew to add items if desired.
Programmers are in fact discouraged from using this variable. *restart-init-function*
and *restart-app-function*
(both described under the next heading) are better suited to application code while *restart-actions* is best
left to Allegro CL and its associated products. Both *restart-init-function*
and *restart-app-function*
are processed at the end of the startup procedure. *restart-actions* is
processed early in the procedure, too early for certain actions (like starting CLIM) to
succeed.
At the end of the startup procedure, Allegro CL examines the
variable *restart-init-function*. If it is
true, its value is assumed to be a function of no
arguments (either a function name or a function object) and that
function is called. The function typically will return. It is designed
for late initializations (perhaps based on information supplied by
initialization files), possibly cleanup of state from the pre-dumplisp
image, application tasks like printing an information message or an
application banner, reading an application-specific initialization
file, processing application command-line arguments, etc.
When the *restart-init-function* returns, Allegro
CL examines *restart-app-function*. If it is nil, a standard Lisp listener is started. If it is
true, its value is assumed to be a function of no
arguments (either a function name or a function object) and that
function is called. The function should not return (the behavior if it
does return is undefined.) *restart-app-function* is designed to
provide an application top-level when the application does not want to
use the standard Lisp listener as a top-level.
There are two restart functions specifically to divide the two
purposes of a restart function in earlier versions of Allegro CL:
application-specific initializations and providing an application
top-level. Note that, a listener is started if *restart-app-function* is nil but not when *restart-app-function* returns (indeed,
the behavior if *restart-app-function* returns is
undefined). Note that *restart-app-function* can start its own
Lisp listener by evaluating the following form:
(tpl:start-interactive-top-level *terminal-io* #'tpl:top-level-read-eval-print-loop nil)
See the definitions of tpl:start-interactive-top-level and tpl:top-level-read-eval-print-loop.
This issue is also discussed in eli.htm. The Emacs-Lisp interface provides a close coupling of Allegro CL and Emacs. What is important with respect to dumped images is how the interface is started. It can only be started by invoking Allegro CL within Emacs with the function fi:common-lisp. Images invoked in that way cause the Emacs-Lisp interface to be started with the command line arguments "-e (excl::new-start-emacs-lisp-interface t)" (plus any additional command line arguments you specify). This will be ineffective if processing of those command line arguments is suppressed because the image was dumped with the ignore-command-line-arguments argument to dumplisp specified true.
If you create an image which ignores command-line arguments and you
want the emacs-lisp interface started, you must take responsibility
for evaluating (excl::new-start-emacs-lisp-interface
t). This could be done as part of the *restart-init-function*.
There is one minor complication: you may want to handle the case where you are starting the image in some fashion other than calling fi:common-lisp within Emacs (for example, starting Lisp in a shell). In that case, calling excl::new-start-emacs-lisp-interface will not, of course, do anything effective. Calling the function is not an error. However, it does print a cryptic string to the listener, which you may want to avoid. The solution to that problem is, instead of unconditionally calling excl::new-start-emacs-lisp-interface, to look yourself for "-e (excl::new-start-emacs-lisp-interface t)" among the command line arguments and only call excl::new-start-emacs-lisp-interface if it appears. The following code (which actually simply looks for the first -e argument and evaluates its companion argument) will do that:
(let ((e-arg (member "-e"
(system:command-line-arguments :application nil)
:test #'string=)))
(when e-arg
(eval (with-standard-io-syntax
(read-from-string (cadr e-arg))))))
In earlier releases, there were issues with dumping an image using the Allegro Presto feature and with some functions partially loaded. In 7.0, the Allegro Presto facility has been removed so interaction between dumplisp and Allegro Presto is no longer an issue. See The Allegro Presto facility has been removed in loading.htm for further information.
Because Allegro CL now requires an executable file and an image file, standalone images (images that depend on no other files) are no longer possible.
Foreign loading is done on Unix by `loading' .so or .sl files with dlopen() or an equivalent and on Windows by loading .dll files. Dumped images must find these files when it starts up.
The pathnames of loaded .so files are stored. So long as those pathnames are valid when the image is restarted, things should work correctly. However, to guarantee that the dumped image finds the .so files when it restarts, even if run on a different machine or network, you can use one of the following methods.
*restart-init-function*).
We describe both methods in some detail.Using logical pathnames and making them valid in the dumped image. Lisp stores the pathnames of all loaded .so files as they are specified to load (or :ld). It uses these stored pathnames when a dumped image is restarted to find the files and reload them. We recommend these steps for loading .so files. Because logical pathnames are used and because a translations file is specified (~/myhosts), all that is needed is to ensure that file contains the correct translation for the logical host.
For example, suppose you want to load /usr/mydir/mysos/foo.so into an image that will be dumped. The following transcript shows the actions in the pre-dumped image and modifications to the external logical hosts file.
USER(1): (setf (logical-pathname-translations "myso")
'((";**;*.*"
#p"/usr/mydir/mysos/")))
((#p";**;*.*" #p"/usr/mydir/mysos/"))
USER(2): :ld myso:b.so
; Foreign loading myso:b.so.
USER(3): (push "~/myhosts" (logical-pathname-translations-database-pathnames))
("~/myhosts" "sys:hosts.cl")
USER(4): (dumplisp :name "mycl")
Warning: shared object files have been loaded into this image, and so the
resulting image will depend on myso:b.so for successful operation.
USER(5):
;; ~/myhosts contains:
"myso" '(";**;*.*" #p"/usr/mydir/mysos/")
Unloading all .so files and reloading them upon
startup. In this example, we unload the (in our case single)
.so file with ff:unload-foreign-library and
show how to modify *restart-init-function* so the .so file
is reloaded. We end with a couple of notes and warnings.
USER(1): :ld /usr/mydir/mysos/b.so ; Foreign loading /usr/mydir/mysos/b.so. USER(2): (ff:unload-foreign-library "/usr/mydir/mysos/b.so") NIL USER(3): (defun reload-my-so () (load "/usr/mydir/mysos/b.so")) RELOAD-MY-SO USER(4): (setq excl:*restart-init-function* 'reload-my-so) USER(5): (dumplisp :name "mycl") ;; NOTE: no warning about loaded .so files. USER(6): ;; The new image comes up as follows (note the message about foreign loading): Allegro CL Copyright (C) Franz Inc., Berkeley, CA, USA. All Rights Reserved. ;; Starting socket daemon and emacs-lisp interface... ; Foreign loading /usr/mydir/mysos/b.so. USER(1):
Notes and warnings:
*restart-init-function* in our example is
very simple and not very useful. In real situations, programmers
would likely get information from a command-line argument or an
initialization file about the location of the needed files and act
on that information.All logical pathname translations are flushed as the first action when a dumped image restarts. This is a feature. There is no way for Lisp to tell whether the translations are valid upon restart (and they almost certainly will not be valid if the image is restarted on a different machine on a different network). Rather than devise a way to communicate to Lisp whether to flush translations on startup (and thus overload the few ways available to communicate with restarting images), Allegro CL instead provides tools for properly re-establishing translations using specific files.
The function logical-pathname-translations-database-pathnames returns a list of pathname namestrings. Initially, it returns the list ("sys:hosts.cl"). You may use push or pushnew to add other strings naming files. For example,
USER(5): (logical-pathname-translations-database-pathnames)
("sys:hosts.cl")
USER(6): (pushnew "~/myhosts"
(logical-pathname-translations-database-pathnames))
("~/myhosts" "sys:hosts.cl")
USER(6): (logical-pathname-translations-database-pathnames)
("~/myhosts" "sys:hosts.cl")
When looking for a translation for a logical pathname, Lisp will look first in ~/myhosts and then in sys:hosts.cl. It is not an error if a named file does not exist. Therefore, your translations should be added to hosts.cl in the Allegro directory or to a file of your own which you push onto logical-pathname-translations-database-pathnames as described above. (Many users of Lisp may not have permission to modify hosts.cl. Indeed, having several users modifying that file is impractical.)
The Allegro directory is the location of the executable file (such as mlisp or mlisp.exe). In the separate executable/image model, finding the directory using the executable is simple and should work without problems.
When lisp starts from a dumped image, it has exactly one process that is runnable, the Initial Lisp Listener process, and that is a new process, completely unrelated to the Initial Lisp Listener process that was running when the image was dumped.
Any other processes that were in the image when it was dumped must be process-reset before being run again. It would be legal to process-kill any or all of them instead of resetting and restarting them.
The image is specified to the executable with the -I argument, Thus, if the dumped image is named myimage.dxl, and mlisp.exe or mlisp is the executable, then this starts a Lisp using myimage.dxl:
[Windows] % mlisp.exe [+ args if any] -I <path>/myimage.dxl [other args] [Unix] % mlisp -I <path>/myimage.dxl [other args]
If no -I argument is specified, the executable looks for an image in the same directory with the same name as itself (thus mlisp.exe will look for mlisp.dxl). If you have created an image for general use, you may wish to copy it to the Allegro directory (assuming you have permission to do so) and copy the executable to have the same name.
dumplisp on Windows only
recognizes a new keyword
argument virtual. If specified
and non-nil, it requests dumplisp to create a two-piece lisp image. The
file named by the name keyword argument will be one piece and will be
relatively small. Most of the dumped image will be in the second
piece, a memory-mapped virtual file. The real file contains a
reference to the virtual file.
(dumplisp :name "vapp.dxl" :virtual t)
will write a relatively small file named vapp.dxl and create a large virtual file holding most of the image data. vapp.dxl will have a symbolic pointer to the virtual file. As long as the virtual file exists, executing the shell command
lisp -I vapp
will get its image data from the virtual file.
One important issue is "How long does that virtual file exist"? The answer is that it exists as long as there is a Lisp program using it. The Lisp program that did the virtual dumplisp is a initially the only user. When it exits, it will no longer be a user, obviously, and the virtual data will vanish unless there is another user. A Lisp that is loading data from a virtual image by copying it is a user as long as it is loading the data. Once the data has been copied, that Lisp program is no longer a user. A Lisp program that maps the data rather than copying it will be a user until it exits.
A Lisp program that creates a virtual image file can release the virtual file before exiting, and may wish to do so to release the OS resources dedicated to supporting the virtual file. To do so it must capture the result returned from dumplisp and then call excl:release-dumplisp-virtual-image when it wishes to discard the file.
dumplisp returns no values
when it is called with :virtual nil or
unspecified. When called with :virtual
non-nil, dumplisp returns a value that can later be
used to release the virtual file. One usage pattern would be something
like this:
(let ((vmem (dumplisp :name "vapp.dxl" :virtual t))) (run-shell-command "lisp -I vapp.dxl -- initial") (run-shell-command "lisp -I vapp.dcl -- cleanup") (excl:release-dumplisp-virtual-image vmem))
This runs one Lisp from the image with initial as a command line argument, then when that one finishes it runs another with command line argument cleanup. Then it releases the virtual image file.
Virtual image files are only available on Windows; requesting a virtual image file on a non-Windows lisp results in a warning and an ordinary file-based dumplisp.
Once a virtual image file has been released, the file part of the dumplisp that references it is no longer a valid image file for starting lisps. An attempt to use it will result in an error.
Calling excl:release-dumplisp-virtual-image a second time on the same otherwise legitimate argument is legal and does nothing.
Calling excl:release-dumplisp-virtual-image with an argument that was not previously returned by dumplisp is an error that may or may not be recognized and signaled.
Copyright (c) 1998-2022, Franz Inc. Lafayette, CA., USA. All rights reserved.
This page was not revised from the 10.0 page.
Created 2019.8.20.
| Allegro CL version 10.1 Unrevised from 10.0 to 10.1. 10.0 version | ||||||||