| Allegro CL version 10.0 Unrevised from 9.0 to 10.0. 9.0 version | ||||||||
This document contains the following sections:
1.0 IntroductionThis document describes functionality that does not naturally fit in any of the other overview documents.
The functions fasl-write and fasl-read provide a mechanism for writing Lisp data and subsequently reading it back into a Lisp image. It can handle many of the common Lisp data types. It can optionally detect circularity and structure sharing in the data and recreate the same topology up to eql-ness of components. The data is written in a binary file format similar to that used for compiled Lisp files, not in ASCII.
Among the advantages of fasl-read and fasl-write over standard Common Lisp read and print functions is that data does not have to be converted to its printed representation prior to being written (that conversion takes a significant amount of time for complex objects). The main disadvantage is that the files (unlike ASCII files with printed representations of Lisp objects) are not at all portable between versions of Lisp (or even between versions of Allegro CL on different platforms or between major releases of Allegro CL).
Not all Lisp objects may be written with fasl-write (and thus are not available to be read by fasl-read). The objects that cannot be written are CLOS objects, streams, stacks, or c-allocated data (cstructs). You can use fasl-open to open a file suitable for numerous fasl-write's. (fasl-read reads all of whatever file is specified to it.)
fasl-read reads the
entire contents of its argument file. However, fasl-write, if its argument is a stream rather
than a filename, writes data to the stream but does not close it. Thus
you can open a stream and do multiple fasl-write's to it. The stream must have
:element-type (unsigned-byte 8). fasl-open opens an appropriate
stream, as shown in the example.
Here is a simple-minded example to show the correct syntax and give an idea of the effect:
(excl:fasl-write '(a b #1=#:c (#1#) #*1011 #(1 2 3)
1 1.1 1.1d0 1/2
#c(1 1) #c(1.5 1.5) "abc")
"test.fw")
(excl:fasl-read "test.fw")
(setq f (fasl-open "test2.fw"))
(excl:fasl-write '(a b c) f)
(excl:fasl-write '#1=(a b . #1#) f t)
(excl:fasl-write '#(1 2 3 4.5) f)
(close f)
(excl:fasl-read "test2.fw")
fasl-read and fasl-write require the compiler so they are not available in application images without the compiler.
The tables describes those extensions to Common Lisp that do not naturally fit elsewhere in the documentation. We only provide brief information in the table. Please follow the link to the documentation page for a full description.
| Name | Arguments | Notes |
| excl:dribble-bug | &optional file | This function is an extension of the Common Lisp function dribble.
dribble-bug called with the optional file
calls several information functions whose output is then placed at the beginning of the
dribble file. See also excl:*dribble-bug-hooks*. |
| excl:file-older-p | file-1 file-2 | If file-1 and file-2 both exist, and if file-1 is older than file-2, this function returns t. Otherwise, it returns nil. |
| excl:gen-sym | x | Similar to cl:gensym except that a symbol is accepted as the optional argument (as well as the standard string or integer), in which case the symbol-name is used. This is useful for programs that are intended for use in either ANSI or modern Lisps, where it is desired to print any gensyms without requiring escapes (see case.htm). |
| excl:if* | test-form {then then-form+ | thenret} {elseif else-test-form {then else-then-form+ | thenret}}* [else else-form+] | This extension to cl:if allows symbols like then, else, elseif, and thenret in the body allowing a complex number of cases and outcomes to be specified. |
| excl:named-readtable | name &optional errorp | This function looks up a readtable by name. name must be a symbol, but it is coerced to a keyword (so readtables are named by keywords). setf may be used to associate a name to a readtable. The association can be broken by setting to nil. See also with-named-readtable. |
| excl:prefixp | prefix sequence |
This function returns non-nil if
sequence starts with prefix,
returning the index in sequence where prefix ends (so
(prefixp "foo" "foobar") returns 3). While this can
be done using regular-expression matching (see
regexp.htm), this function is easier to use for very
simple cases.
|
| excl:uncompile | function-name | If the function function-name was compiled with the compile function (as opposed to having been in a file that was compiled with compile-file and subsequently loaded), then the function is `uncompiled,' i.e. its function definition is replaced by the original interpreted definition. This function will only work when
definitions are saved. See the description page for excl:uncompile and the page for |
| Name | Arguments | Notes |
| excl:bignump | object | These functions, like similar ones in standard Common Lisp return t if object is of the type specified, and nil otherwise. |
| excl:fixnump | ||
| excl:ratiop | ||
| excl:single-float-p |
A pll file can be used in association with a Lisp image. It contains constant code vectors and strings that can be shared among many Lisp objects. When an image uses a pll file and a function is compiled, the new codevector is compared to codevectors in the pll file. If a match is found, the match is used and no new codevector is allocated. Similarly, if a constant string is specified (with excl:pure-string) and a matching string appears in the pll file, no new string is allocated and the match is used. Strings in a pll file cannot be modified. Attempting to do so causes an error. (Neither can codevectors be modified but there is no user-visible way to modify codevectors as there is with strings.)
Strings and codevectors in a pll file are not also (after
being garbage collected) in the Lisp heap. Thus if a string has been
successfully purified, it will not be in the heap after a global gc. A
total count of strings and codevectors is shown in the output of
(room t).
Constant strings in pll files cannot be overwritten (constant strings stored in the heap can be overwitten although doing so is actually illegal). Strings naming Common Lisp symbols are usually stored in pll files, for example. This example shows in that case, the strings cannot be modified:
cl-user(1): (pll-file) #P"/usr/fi/dcl.pll" cl-user(2): (nstring-upcase (symbol-name 'car)) Error: Attempt to store into purespace address #x2d7865e8. [condition type: purespace-write-error]
Here we tried to upcase the string naming the
symbol car (this is a modern Lisp where symbol
names are lowercase -- see case.htm). The change
failed and signaled purespace-write-error, which is the condition
specific to this issue. The next example shows that modifying a
constant strings not in the pll file does not signal an error:
cl-user(88): (setq teststring "this-string-is-likely-not-in-the-pll-file") "this-string-is-likely-not-in-the-pll-file" cl-user(89): (dotimes (i 4) (gc)) nil cl-user(90): (nstring-upcase teststring) "THIS-STRING-IS-LIKELY-NOT-IN-THE-PLL-FILE" cl-user(91): teststring "THIS-STRING-IS-LIKELY-NOT-IN-THE-PLL-FILE" cl-user(92):
The call to nstring-upcase is in fact illegal: you may not modify a constant, but it is an error that does not signal an error unless the string is located in the pll file.
pll files are created with the cvdcvt program described next.
cvdcvt
Arguments: [
-o outfilename] [-u] [file1 file2]A .pll file,
outfilename, is created holding all the unique code vectors and strings. Ifoutfilenameis omitted it defaults to code.blob.If
-uis specified, then no duplications of strings are done, otherwise for every string that has no lowercase characters in it and at least one uppercase character, a lowercase copy is added to the output file. This is the default and is useful forset-case-mode. If no files are specified, stdin is used..str (string) files and .cvs (code vector) files are combined without redundancies; if two files of the same extension have identical objects, the object from the file specified first to cvdcvt is retained, and the latter object is removed from the output. This allows for files (.cvs files especially) to be arranged by code vectors in order of execution, to provide for locality of reference. Those .cvs files that were produced by training techniques should be placed first in order to have the desired effect.
As said in the description, pll files are built out of cvs files and str files. cvs files are created with sys:write-codevectors and can be created by sys:flush-codevectors. str files are created with record-strings. See also record-code-vectors.
The following functions can be used to associate a pll file with an image, to find out which pll file is used with an image, and to use strings in the pll file.
| Name | Arguments | Notes |
| excl:pll-file | [none] | Return the location of the current .pll file, or nil if there is no .pll file associated with this Lisp. |
| excl:use-pll-file | name &key (global-gc t) | Associates the current Lisp with the pll file specified by name. It is an error to associate an image already using a pll file with another pll file. |
| excl:pure-string | x | When not in the body of sys:record-strings, returns a pure-string if there is one identical to the argument and a heap-allocated string if there isn't. When in the body of sys:record-strings, also write the string to the str file being created. |
Allegro CL provides various message digest algorithms. Various algorithms are supported, including MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, and RMD160. Rather than have functions for each of these types, there are general functions for them, named digest-* and listed first. HMAC support is available for :md5, :sha1, :sha224, :sha256, :sha384, and :sha512. The functions hmac-init, hmac-update, hmac-string, and hmac-final implement these algorithms.
The MD*, SHA* and RMD160 functions are all are cryptographic hash functions. A hash function takes a long string (or message) of any length as input and produces a fixed-size output. This output is sometimes termed a message digest.
The keyed-hash algorithm HMAC is designed to work with algorithms MD5 and SHA* (for *=1, 224, 256, 384, and 512, to list what Allegro CL currently supports). Keyed hash algorithms work by utilizing an existing hash function (such as MD5 or SHA1) and using a secret "key" as part of the information to hash. If one party sends a message to another party and also includes an HMAC with the message, the receiver (if he/she shares the same secret key) can verify that the message hasn't been altered by running the message through the HMAC functions using the same key. If the two HMACs match, then the message is considered, with high probability, to be unaltered
Many of these function require SSL support (see Secuere Socket Layer (SSL) in socket.htm).
The functions available are:
These functions are in the :digest module loaded
with (require :digest).
The following functions for specific encodings are also loaded when
the :digest module is loaded.
These functions are in the :hmac module loaded
with (require :hmac).
Allegro CL provides support for Base64 encoding within Lisp. Base64 encoding is a 64-bit representation scheme that uses the ASCII characters A-Z, a-z, 0-9, + and /. Since padding could be needed in converting multiples of 8-bits into base64, = characters are used, when necessary, as padding at the end of a converted string. Base64 encoding is described in the RFC2045 document (www.ietf.org/rfc/rfc2045.txt).
These functions provide the Base64 support in Allegro CL:
Here are some examples, first using integer-to-base64-string and base64-string-to-integer:
cl-user(2): (integer-to-base64-string #xfeedfacefeedface) "/u36zv7t+s4=" cl-user(3): (base64-string-to-integer "/u36zv7t+s4=") 18369614222061337294 cl-user(4): (format t "~x" *) feedfacefeedface nil cl-user(5):
And now using usb8-array-to-base64-string and base64-string-to-usb8-array:
;; The encoding results may differ between Windows and
;; UNIX/Linux/Mac OS X (the transcript is from a
;; UNIX machine).
cl-user(5): (setq a (string-to-octets
(setq s
"
(defun deep-thought ()
(sleep (years2secs 7500000))
42)
")
:external-format (crlf-base-ef :latin1)))
#(10 40 100 101 102 117 110 32 100 101 ...)
cl-user(6): (usb8-array-to-base64-string a)
"CihkZWZ1biBkZWVwLXRob3VnaHQgKCkKICAoc2xlZXAgKHllYXJz
MnNlY3MgNzUwMDAwMCkpCiAgNDIpCgA="
cl-user(7): (base64-string-to-usb8-array *)
#(10 40 100 101 102 117 110 32 100 101 ...)
cl-user(8): (setq a2 *)
#(10 40 100 101 102 117 110 32 100 101 ...)
cl-user(9): (equalp a a2)
t
cl-user(10): (octets-to-string a2 :external-format (crlf-base-ef :latin1))
"
(defun deep-thought ()
(sleep (years2secs 7500000))
42)
"
61
61
cl-user(11):
Allegro CL provides implementations of some publicly available encryption algorithms: RSA (see Section 7.2 Support for rsa encryption) and RC4 (see Section 7.3 Support for the RC4 cipher). Please note that we make no claims about the actual security provided by these encryption schemes. For backward compatibility reasons, we also support blowfish (see Section 7.1 Support for Blowfish encryption), whose use is not recommended because it is no longer considered secure.
Blowfish is an old cipher, designed in 1993. We recommend it not be used, because it is susceptible to various attacks and is no longer considered secure. We maintain support to allow for encrypted material to be decrypted.
The Blowfish algorithm, described on this Wikipedia page (and links from it), is a high speed symmetric cryptographic algorithm (or cipher). The same key is used to encrypt and decrypt the data. Blowfish encrypts blocks of 64 bits (8 octets) at a time. The functions below can automatically pad out the data to encrypt to be a multiple of 8 octets. Blowfish was designed by Bruce Schneier, a leading authority on cryptography and author of the book Applied Cryptography. Schneier writes in his book published in 1996: "I know of no successful cryptanalysis against Blowfish."
Here are some examples of Blowfish encryption and decryption:
;; Example 1. string encrypting
cl-user(12): (blowfish-encrypt "my secret message"
:key "my key")
#(57 27 110 242 191 19 182 150 1 5 ...)
24
cl-user(13): (blowfish-decrypt * :key "my key" :string t)
"my secret message"
cl-user(14):
;; Example 2. (unsigned-byte 8) encrypting:
;; Here we allocate an (unsigned-byte 8) array with a size
;; that is a multiple of 8 and fill it with data.
;; We can do in-place encryption and decryption.
;; We specify no padding (since otherwise 8 bytes of padding would
;; have to be added and there's no room in this array for that):
;; Create our array:
cl-user(12): (setq aa (make-array 8 :element-type '(unsigned-byte 8)
:initial-contents '(2 4 6 8 10 12 14 16)))
#(2 4 6 8 10 12 14 16)
;; Encrypt it in place
cl-user(13): (blowfish-encrypt aa :key "my key"
:in-place t :pad nil)
#(129 144 108 210 20 227 10 58)
8
;; Verify that it has been modified:
cl-user(14): aa
#(129 144 108 210 20 227 10 58)
;; Now decrypt in place. Notice how the arguments to
;; blowfish-decrypt match those to blowfish-encrypt:
cl-user(15): (blowfish-decrypt aa :key "my key"
:in-place t :pad nil)
#(2 4 6 8 10 12 14 16)
cl-user(16):
;; And verify that the array is now back to normal:
cl-user(16): aa
#(2 4 6 8 10 12 14 16)
cl-user(17):
;; Example 3. use of contexts
;; Create context which holds the key processed by
;; blowfish to prepare it for encryption/decryption:
cl-user(21): (setq cc (blowfish-init "my key"))
#(141 90 172 196 250 88 140 57 179 211 ...)
;; Encrypt something using the context:
cl-user(22): (blowfish-encrypt "my message" :context cc)
#(75 202 37 143 4 243 181 205 211 126 ...)
16
;; And now decrypt it using the same context
;; to show the original string
cl-user(23): (blowfish-decrypt * :context cc :string t)
"my message"
cl-user(24):
It is a common practice to send Blowfish keys to intended recipients using their RSA public keys. See Section 7.2 Support for rsa encryption for information on RSA encryption.
RSA is a public key cipher named after its inventors: Rivest, Shamir and Adleman. A public key cipher differs from a symmetric cipher like Blowfish (see Section 7.1 Support for Blowfish encryption) in two important ways:
With RSA you can encrypt with the Public key and decrypt with the Private key or encrypt with the Private key and decrypt with the Public key. Typically one encrypts with the Public key to send a message to the person with the Private key.
RSA has never been proven to be secure. However the obvious way to crack the encryption involves factoring a very large number. There is no published way of factoring a large number that's better than a brute force attempt of trying all possible factors. Thus by making the key big enough you can be sure that it won't be possible to compute the factors by brute force search in a very long time. There may be other ways to crack RSA encryption that are simply not published yet.
One major downside to RSA is that it is roughly 1000 times slower to encrypt and decrypt than a symmetric cipher like Blowfish. As a result people usually use RSA as means of transmitting a key for a symmetric cipher. For example if Alice wants to send Bob a large document securely she'll first go to Bob's web site and copy down his Public RSA key. Then she'll use a random number generator to create a 64 bit blowfish key. She'll encrypt the blowfish key with Bob's Public key and send the result to Bob. Then she'll encrypt her document using Blowfish and the key she generated. Bob will decrypt the first message from Alice using his Private RSA key. That will give him the Blowfish key he'll need to decrypt the second message from Alice.
Because the public key is known to all you have to be careful to not encrypt small values with an RSA public key since that gives you very little security. For example, suppose you decide to encrypt a 4 digit security code using an RSA public key. A person willing to steal your code need only encrypt the values 0000 through 9999 and compare them to your encrypted value to determine what the value encrypted was. If you want to encrypt a 4 digit security code XXXX then it's best to encrypt instead YYYYYYYYXXXX where the Y's are digits chosen randomly.
An RSA key pair consists of three integers: a modulus, a private exponent and a public exponent. The only number that must be kept secret is the private exponent. The public exponent is usually one of a set of common small numbers. The Allegro RSA key generator always chooses 17 as the public exponent.
An RSA key is represented in Allegro as a vector of three values:
t if this is the public key, nil if this is the private key. This value is to help
you distinguish one key from the other and is not used in the
encryption/decryption code.
RSA is a block cipher: a sequence of octets is encrypted at once. The block size isn't fixed but is usually determined by the size of the modulus. In order to encrypt data whose length is not a multiple of the block size padding is done at the end of the value and information about the padding is added to the value. The format of this padding information is not standard among rsa encryption functions, thus you can't expect any function except rsa-decrypt to be able to decrypt a value encrypted with rsa-encrypt.
The functions associated with RSA encryption and decryption are:
Because RSA encryption is resource intensive compared to symetric encoders like Blowfish, it is a common practice to encode using Blowfish and send Blowfish keys to intended recipients using their RSA public keys. See Section 7.1 Support for Blowfish encryption for information on Blowfish encryption.
Here are some examples of Blowfish encryption and decryption:
;; A call to generate-rsa-keys, such as the following, can take ;; on the order of 10 minutes to complete. The example call ;; could have been made with ':verbose t' to get progress ;; information as it runs. ;; ;; The return value is a list of the public and ;; private keys, both of which are vectors. This list ;; is made the value of the variable 'keys'. cl-user(12): (setq keys (generate-rsa-keys :verbose nil)) (#(t 4696616306992156162791359909817969438301590857320651704912\ 5099728659553054438846018512904176959283177314807123575693727\ 9515543419344057970899365859403176313951068268266882944649562\ 1008090347981854919956845970556254842289211552574616675107428\ 9213609596618613446079618857135830766959762009927055865884710\ 796501 17) #(nil 46966163069921561627913599098179694383015908573206517049125\ 09972865955305443884601851290417695928317731480712357569372795\ 15543419344057970899365859403176313951068268266882944649562100\ 80903479818549199568459705562548422892115525746166751074289213\ 609596618613446079618857135830766959762009927055865884710796501 38678016645817756634752375727912689491895454119111249334573\ 61154124904369189081436818709755749588026367101763117998307007\ 77506345342165387799477766567321557421166007590816670669406236\ 43773618649101682703783799537062586749082756929026484684965113\ 373035085734095558864183533679809881862774879514342408924287321)) ;; Here we encrypt with the public key and decrypt with the private key ;; we could have encrypted with the private key and decrypted with the ;; public key as well. cl-user(13): (rsa-encrypt "my secret message" (car keys)) #(102 136 69 180 180 27 185 63 132 137 ...) cl-user(14): (rsa-decrypt * (cadr keys) :string t) "my secret message" cl-user(15):
RC4 is a stream cipher algorithm. It is used to encrypt streams of data.
The functions available are:
These functions are in the :rc4 module loaded
with (require :rc4).
The inflate and deflate modules allows you to compress
data as it is written to files, and to open streams to files containing gzip
compressed data and to uncompress the data while reading the file. To
load the modules, evaluate (require
:inflate) and (require
:deflate). Symbols in the modules are in the
util.zip package. (There are two modules because they were
added at different times.)
The :deflate module requires that the libz
library be available on your computer (and in the correct version). If
you do not have that library or do not have the correct version,
deflation will not work. Allegro CL functionality that uses deflation
(such as AllegroServe, see aserve/aserve.html)
will warn that they cannot compress data but will continue to work
without data compression.
The inflation function util.zip:inflate can be applied
to instances of the class util.zip:inflate-stream.
The various utility functions util.zip:skip-gzip-header, util.zip:skip-gzip-trailer, util.zip:skip-zlib-header, and util.zip:skip-zlib-trailer can be applied to an input stream to position the file position to the correct location for uncompressing and to avoid any trailers.
The deflation functions util.zip:deflate-target-stream, util.zip:deflate-stream-vector, and util.zip:deflate-stream-vector-combined can
be applied to instances of the class util.zip:deflate-stream.
The value of the variable sys:*zlib-system-library* is the name of the
gzip library used by the deflate module.
Arguments: input-stream output-stream
The compressed information from the input-stream is read and the uncompressed information is written to the output-stream.
Both streams must support (unsigned-byte 8) element reading and writing.
Arguments: input-stream
If the input stream is positioned on the header of a gzip'ed file then
skip that header. input-stream is not an instance
of the class util.zip:inflate-stream. It is
simply an input stream (opened, for example, with open). The file position must be moved to the
beginning of the compressed data before unziping, and only at that
point should an instance of util.zip:inflate-stream be
created (see util.zip:inflate-stream).
If the input stream is not positioned on a gzip header then nothing is done.
Arguments: input-stream
Skips past the next 8 bytes in input-stream. Note that gzip trailers have no byte-markers to identify them so care should be taken to only call this function after the final data block is read from the stream.
Arguments: input-stream
If input-stream is positioned at the header of a zlib'ed stream, then skip past it, returning the number of bytes read.
If input-stream is not positioned at a zlib
header, return nil
If input-stream appears to be positioned at a zlib header but turns out to not be, signal an error.
Arguments: input-stream
Skips past the next 4 bytes in input-stream. Note that zlib trailers have no byte-markers to identify them so care should be taken to only call this function after the final data block is read from the stream.
The stream class for instances of files containing comressed gzip'ed data. Instances of this class are suitable as arguments to util.zip:inflate.
To create an instance of this class, do the following:
(make-instance 'inflate-stream :input-handle <stream created in 1>
[:compression compress-spec])
compress-spec can be one of the following three values:
:gzip (the default): automatically read past gzip
header and trailer.
:zlib: read past zlib header/trailer.
:deflate: no headers in this stream.
(header-fn trailer-fn): users may specify their own
functions for skipping past headers and trailers wrapping the
deflate-stream. This can be used to support less common encoding
methods, such as zip or pkzip, etc. The inflate module
does not capture any information from header and trailers, so the
custom functions can also be used to capture this information if
needed. The function accepts a single argument, which is the
input-handle of the inflate-stream. It should
return nil or the number of bytes read.
nil: equivalent to
the :deflate option. No callbacks to utility
functions needed.
The documentation until April, 2016 incorrectly left out the :compress-spec keyword argument. The documentation is now correct.
The call to make-instance will return a stream which can be read to recover the uncompressed data.
Closing the inflate-stream will also close the stream created in step 1.
To write compressed data, you create a deflate-stream and at that time
specify an ultimate taget, which is either a regular stream (perhaps
open to a file) or a octet vector (of element
type (unsigned-byte 8)). You then write data to the
deflate-stream and that data is compressed and eventually written to
the target. We say eventually because there is a lot of
buffering so you do not see data in the target immediately. When you
are done, you close the deflate-stream and that causes any remaining
data to be written to the target.
Closing the deflate-stream does not close the target if it is a stream. You must close the target stream yourself.
You create deflate-streams with make-instance, as described next.
The stream class for instances of deflation streams which accepts characters and bytes and causes them to be compressed and sent to a target.
You created a deflate-stream with make-instance. You must specify a
target when you create a deflate-stream.
Here is a sample make-instance call:
(make-instance 'deflate-stream :target target-spec
[:compression compress-spec])
target-spec can either another stream, or it can be a vector (that is an actual stream or an actual vector). The stream must be writable. The vector must have element-type (unsigned-byte 8).
compress-spec can be one of the following three values:
:gzip (the default): create a deflate stream with
gzip headers.
:zlib: create a deflate stream with zlib headers.
:deflate: create a deflate stream with no
headers.
The documentation until April, 2016 incorrectly said there were two
choices: :gzip and :deflate,
with :deflate creating a stream with zlib
headers. The documentation is now correct.
:gzip is the preferred
format as the result can be uncompressed with the util.zip:inflate-stream (be sure
to specify :skip-gzip-header t to
the make-instance creating the inflate-stream or to
call util.zip:skip-gzip-header after
the input stream is opened). The :gzip format
output can also be uncompressed with the gunzip program found
on Unix.
If you pass a stream as the target-spec then as you write characters and bytes to the deflate-stream, the bytes resulting from deflation will be written to the given stream. There is a lot of buffering going on in this stream and the compression library. Therefore you may not see the results in your target-spec stream immediately.
When you close the deflate-stream the last bytes in all the buffers will be sent through deflation and the end of deflation record will be written to the target-spec stream.
Again, the target-spec stream will not be closed when the deflate-stream is closed. It is the callers responsibility to close the stream passed in as the target-spec.
The function deflate-target-stream will return that target-spec stream used by the deflate-stream.
Passing a simple vector of type (unsigned-byte 8) as the target-spec is telling the deflate-stream that you wish to collect the deflation result in vectors in the lisp heap. The size of the vector passed is not important. Additional vectors will be created as necessary to hold data as it is written.
Once you have closed the deflate-stream after all data has been written to it, you can retrieve the result with util.zip:deflate-stream-vector or util.zip:deflate-stream-vector-combined, as described in the descriptions of those functions. The deflate-stream-vector-combined combines all results into a new vector if the target vector is not large enough. deflate-stream-vector returns the vectors created by writing to the deflate-stream, along with additional information.
Arguments: deflate-stream
Returns the stream which is the target
of deflate-stream which must be
a deflate-stream. Returns
nil if deflate-stream
has a vector as its target.
Arguments: deflate-stream
deflate-stream must be
a deflate-stream whose target is a
vector. In that case, this function returns three values:
For example, if the three returns values are:
v
100
(c b a)
then the deflated result is found by combining in this order:
all of a
all of b
all of c
the first 100 bytes of v
This function signals an error if deflate-stream has a stream as its target.
Arguments: deflate-stream
deflate-stream must be
a deflate-stream whose target is a
vector. In that case, this function returns two values:
The octet vector is newly created if necessary (if the target vector
specified when the deflate-stream was created is
not large enough to hold the compressed data).
This function signals an error if deflate-stream has a stream as its target.
Suppose we wish to create a compressed file foo.cl.gz from the following text:
;; file foo.cl begin (in-package :user) (defpackage :foo (:use :cl :excl)) (defun foo (y) (bar y)) ;; file foo.cl end
In the following transcript, we use a deflate stream to create foo.cl.gz and then we inflate foo.cl.gz by opening the file, stripping the gzip header, creating an inflate-stream instance, reading the file line by line, closing the inflate-stream, and closing the file. We could also inflate the while file by calling inflate.
;; We load the modules and then write a compressed file.
;; Once it is written, we read it back uncompressing it.
cl-user(61) (require :deflate)
; [loading messages]
tcl-user(62) (require :inflate)
; [loading messages]
t
cl-user(63): (setq myfile (open "foo.cl.gz" :direction :output :if-exists :supersede))
#<file-simple-stream #P"foo.cl.gz" for output pos 0 @ #x100439f9f2>
cl-user(64): (setq *df* (make-instance 'deflate-stream :target myfile))
#<deflate-stream in 0 / out 0 @ #x10043ad922>
cl-user(65): (format *df* ";; file foo.cl begin~%~%")
nil
cl-user(66): (format *df* "(in-package :user)~%~%")
nil
cl-user(67): (format *df* "(defpackage :foo (:use :cl :excl))~%")
nil
cl-user(68): (format *df* "(defun foo (y) (bar y))~%~%")
nil
cl-user(69): (format *df* ";; file foo.cl end")
nil
cl-user(70): (close *df*)
#<deflate-stream in 120 / out 106 @ #x10043ad922>
cl-user(71): (close myfile)
t
cl-user(72): (setq s (open "foo.cl.gz" :direction :input))
#<file-simple-stream #P"foo.cl.gz" for input pos 0 @ #x1004400732>
cl-user(73): (util.zip:skip-gzip-header s)
10
cl-user(74): (setq is
(make-instance 'util.zip:inflate-stream :input-handle s))
#<inflate-stream
inflating #<file-simple-stream #P"foo.cl.gz" for input pos 10 @
#x1004400732>ef :latin1-base, in: 0, inflated 0, used: 0 of
@ #x1004417412>
cl-user(75): (read-line is nil s)
";; file foo.cl begin"
nil
cl-user(76): (read-line is nil s)
""
nil
cl-user(77): (read-line is nil s)
"(in-package :user)"
nil
cl-user(78): (read-line is nil s)
""
nil
cl-user(79): (read-line is nil s)
"(defpackage :foo (:use :cl :excl))"
nil
cl-user(80): (read-line is nil s)
"(defun foo (y) (bar y))"
nil
cl-user(81): (read-line is nil s)
""
nil
cl-user(82): (read-line is nil s)
";; file foo.cl end"
t
cl-user(83): (read-line is nil s)
#<file-simple-stream #P"foo.cl.gz" for input pos 98 @ #x1004400732>
t
cl-user(84): (close is) ;; this also closes s.
t
cl-user(85):
The source code to the gzip utility is included with the Allegro CL distribution, in [Allegro directory]/src/inflate.cl.
PAM stands for Pluggable Authentication Modules. It is a flexible mechanism for authenticating users. An Allegro CL module provides a Lisp wrapper around the PAM API on Linux, Solaris, and some other unixlike operating systems. We do not discuss PAM in detail here. See www.kernel.org/pub/linux/libs/pam/FAQ.
PAM is supported on the following platforms:
PAM is not supported in Allegro CL on the following platforms:
The PAM API is is loaded by evaluating (require
:pam). Symbols naming functionality are in the
util.pam package. Depending on your system configuration, your
program may be required to run with 'root' privileges to successfully
make use of PAM.
There is one class and several operators defined in the pam module. They are:
pam (class)
The class of pam objects. A pam object is
created by pam-start.
Arguments: service-name user &key conversation data
This function provides a wrapper around the pam_start(3) PAM
library function. service-name should be
a string naming the desired PAM service.
user may be nil
or a string.
If conversation is specified, it should be
a function (or a symbol naming a function) which will perform the PAM
"conversation", when necessary. data is
optional user-defined data that will be passed to the conversation
function. If conversation is nil (the default), a default conversation function
will be used. For more information on conversation functions, see
Section 9.1 PAM conversation functions.
This function returns a pam object on success. You will
pass this object to other functions and methods. If pam-start
fails, an error is signalled.
Arguments: pam &optional status
The default method is analogous to the pam_end(3) PAM library
function. pam must be a pam object returned by pam-start. If
status is specified, it should be an
integer. See the pam_end(3) description (in PAM documentation
not supplied here) for details on the use of
status. If status is nil, excl.osi:*pam-success* will
be used.
This method returns t on success, otherwise
it signals an error.
Arguments: (var &rest rest) &body body
with-pam is a convenience macro which evaluates body with var bound to the result of calling pam-start with the arguments specified in rest. pam-end will be called when body terminates, either normally or abnormally.
(util.pam:with-pam (pam "login" "jimmy") (format t "This is the body~%"))
Arguments: pam &key flags password
This default method is analogous to the pam_authenticate(3) PAM library function. pam must be a pam object returned by pam-start. If password is specified, it should be a string. It will be used when needed if the default conversation (see pam-start) is used. If flags is specified, it should be an integer.
On success, this function returns t.
If the call is not successful, the function returns two values,
nil and a status value. The status value
will be a keyword or an integer. Possible status value keywords are:
:auth-err
:cred-insufficient
:authinfo-unavail
:user-unknown
:max-tries
If pam_authenticate(3) returns an unrecognized status code, it will be returned without being converted to a keyword.
Arguments: pam microseconds
This method is used to request a delay of at least the specified number of microseconds (which must be an integer) before returning from an unsuccessful pam-authenticate call. Setting a delay slows down attempts to rapidly try different passwords for an account.
If multiple calls to set-pam-fail-delay are made, the largest requested delay will be used. On some systems, it is possible that the PAM modules themselves may request delays, so you might notice a delay longer than one you requested (and in particular, you might notice a delay even though you hadn't called set-pam-fail-delay).
The actual delay will is computed pseudorandomly and may differ by as much as 25% above or below the maximum requested value.
Regardless of the success or failure of a pam-authenticate call, the delay is set back to 0 before returning from pam-authenticate. This means that you should generally call set-pam-fail-delay before each call to pam-authenticate.
The default conversation function will display prompts and request
input from *terminal-io*.
If the password argument is supplied to pam-authenticate, it is probable
that no interaction with *terminal-io* will occur
at all. However, if your system configuration or application has
different requirements, you can provide your own conversation
function.
The conversation function will be called by the PAM API when it needs to collect information to move the authentication process along.
If you supply your own conversation function, it should accept two required arguments, and one keyword argument:
Arguments: messages data &key password
messages will be a list of pam-message
structures. data will be the same value
that was supplied to pam-start.
password will be the password that was
passed to pam-authenticate (and may
possibly be nil).
The conversation function should return a list of pam-response structures. The list must have the same length as the messages list. The first entry in the list should be the response that corresponds to the first message. The second entry should correspond to the second message, and so forth. See pam_conv(3) for details.
The pam-message structure has two slots, style and
message. The style slot will be one of the
following keywords (:prompt-echo-off, :prompt-echo-on,
:error, :text) or an integer (indicating an unrecognized
style). message will be a string which may be used
to prompt the user.
The pam-response structure has two slots, response and
code. response should be a string with the
data requested by the corresponding message. code
should be an integer (the default is 0). Again, see
pam_conv(3) for details.
The popular system definition facility, asdf, is included with
Allegro CL. Evaluate (require :asdf) to load it
into a running Lisp. See [Allegro
directory]/code/asdf.readme for more information,
[Allegro directory]/code/asdf.license for the
license, and [Allegro directory]/src/asdf.lisp for
the source code. asdf documentation can be found on the web
at
http://constantly.at/lisp/asdf/.
Allegro CL provides support for extracting contents from tar files. To use this facility, evaluate
(require :tar)
The tar functions are named by symbols exported from the util.tar package.
The tar functions take streams (rather than pathnames) as arguments. util.tar:list-tar lists the contents of the tar file. util.tar:extract-tar extracts the contents into a specified directory.
Here are example forms using the tar functions:
(with-open-file (s "foo.tgz") (util.tar:list-tar s :gzip t)) (with-open-file (s "foo.tgz") (util.tar:extract-tar s :gzip t :directory "tmp/"))
Arguments: stream &key gzip
List, to *terminal-io*,
the contents of stream, which should be a
stream opened to a tar file. If stream is
compressed with gzip compression, specify a
non-nil value
for gzip. bzip2 compression is not
supported.
Arguments: stream &key gzip directory verbose
Extract, to directory, the contents
of stream, which should be a stream opened
to a tar file. If stream is compressed with gzip
compression, specify a non-nil value
for gzip. bzip2 compression is not
supported.
directory defaults
to nil, which means extract to the current
directory (as returned by current-directory).
If verbose is specified
non-nil, information about what is being done
will be printed.
When testing a program, the coverage is a measure of how much of the source code has actually been tested: have all branches of a conditional been taken, have all defined functions been called, have all error handlers been triggered, and so on (coverage is described in Wikipedia here).
Allegro CL has a macro, with-coverage, which executes code and when done, prints information about how well a specified list of functions are excercised. The type of coverage, following the description in the Wikipedia article linked to above, is statement coverage, where "lines of code" is understood to mean Lisp S-expressions.
See the with-coverage page for further details and an example. Note that source file recording must be on for the coverage tool to work.
Allegro CL has a facility for checking format forms during compilation. What is checked is whether there seem to be sufficient arguments in the form for the format control string to be processed without error when the form is evaluated at run time. If enabled, the facility will warn when it detects that there are two few arguments, note when there are too many arguments, and note when the control string is too complex for analysis. (Having too many argument is not an error and is indeed explicitly permitted by the ANS. However, if it is not what is intended, it is useful to know that it has happened.)
The analysis will occur when the
comp:verify-format-argument-count-switch
is non-nil and not 0. This switch differs
from most compiler switches in that its value can be 0, 1, 2, 3, or 4
as well as nil (equivalent to 0)
and t (equivalent to 2).
Here is the behavior for the various values of the switch:
nil: Collect no statistics, signal no
warning or compiler notes.
t, 1, 2, 3, or 4: Collect statistics about
format argument counts compared to what is called for in format
strings; about format strings too complex to analyze; and about
syntactic errors in format forms.
t or 2: Gather statistics, signal
style-warning if not enough arguments.
Statistics are only kept if the value of the
variable *format-arg-count-stats* is a list of six
fixnums (representing all, missing args, args equal in number to
needed, args more than needed, too complex, and syntax error). These
numbers are incremented appropriately as format forms are
analysed. Warnings and compiler-notes are printed if the switch value
calls for them. The function format-arg-count-stats prints the
collected statistics.
A heap walker is a tool which examines the Lisp heap and gathers information about it. The Lisp heap is where most Lisp objects are stored. It comprises the old space and new spaces described in the garbage collection document gc.htm.
The two built-in tools which walk the heap are get-objects and get-references. Each returns a heapwalk
vector. This is a simple vector with element
type t, whose size depends on the number of
objects found. The first element of a heapwalk vector is the integer
number of objects of interest found). These objects are stored in the
vector elements after the first. The actual size of the vector is a
little bit bigger than strictly necessary to allow for including
objects created while the heap is being examined. Note that the
heapwalk vector itself is an object in the heap. It is persumably not
an object of interest (it did not even exist when get-objects or get-references was called) so its existence
can skew the results slightly but unavoidably.
The function get-objects takes a code as its
required argument and has keyword agguments old
and new. The keyword arguments default
to t. If
specified nil, that particular space will not
be searched. The code is the type code of a lisp
object.
Type codes are shown in room output and also by the
function print-type-counts. (Neither lists all possible
type counts, just those for which at least one object of the
particular type exists.) Here is a (slightly truncated) output
of (room t):
cl-user(2): (room t)
area area address(bytes) cons other bytes
# type 8 bytes each
(free:used) (free:used)
Top #x20d7a000
New #x20994000(4087808) 916:15225 843936:3035024
New #x205ae000(4087808) ----- -----
0 Old #x20000aa0(5952864) 711:78771 2098888:3209704
Root pages: 126
Lisp heap: #x20000000 pos: #x20d7a000 resrve: #x20fa0000
Aclmalloc heap: #xa0000000 pos: #xa0048000 resrve: #xa00fa000
Pure space: #x1f8ec000 end: #x1ffff888
code type items bytes
126: (simple-array (unsigned-byte 16)) 10767 2153400 31.4%
112: (simple-array t) 8891 1445976 21.1%
1: cons 93996 751968 11.0%
7: symbol 20360 651520 9.5%
8: function 9681 602864 8.8%
133: sv-vector 20549 340840 5.0%
120: (simple-array fixnum) 259 270272 3.9%
119: (simple-array code) 367 192064 2.8%
117: (simple-array character) 2396 148960 2.2%
125: (simple-array (unsigned-byte 8)) 19 98720 1.4%
12: standard-instance 3900 62400 0.9%
9: closure 2897 50432 0.7%
15: structure 1159 47856 0.7%
127: (simple-array (unsigned-byte 32)) 11 12744 0.2%
[...]
total bytes = 6856032
aclmalloc arena:
[...]
Now we call get-objects on
type code 127, which is (simple-array (unsigned-byte
32)). There are eleven such objects, according to the list
printed by room (code 127 is
the last item before the list is truncated).
cl-user(3): (get-objects 127)
#(11 #(255 65535 16777215 4294967295) #(32 8224 2105376 538976288)
#(3960924350 2165561044 562617442 2270225120 1264129478 758582028 172407450
2782512936 595962478 1609675396 ...)
#(0 2567483615)
#(546914304 4087808 916 15225 529448960 536869000 553254912 536870912 843936
3035024 ...)
#(3960924350 2165561044 562617442 2270225120 1264129478 758582028 172407450
2782512936 595962478 1609675396 ...)
#(3960924350 2165561044 562617442 2270225120 1264129478 758582028 172407450
2782512936 595962478 1609675396 ...)
#(200235464 1375865383 2472949741 3729159793 443451277 421802134 4188904507
2175392005 408067652 1254986169 ...)
#(3960924350 2165561044 562617442 2270225120 1264129478 758582028 172407450
2782512936 595962478 1609675396 ...)
...)
cl-user(4): (length *)
31
cl-user(5):
The result is a heapwalk vector. The first element is 11, the number of objects of the type of interest. Then those eleven objects. Then some extra elements (19 extra in this case, for a total vector size of 31).
Once you have objects of a particular type, you can find references to that object with get-references, which returns a heapwalk vector filled with objects in the Lisp heap or in lispstatic space which point directly to the object of interest. Note that compound structures might not point to the object directly; for example, if a list has an element within it, the first call to get-references will return only the cons cell whose car is that object; you would have to repeat the get-references on that cons cell to find the cons whose cdr points to it, and so on backward through the list until you find a recognizable structure, symbol, or function that points to the head of the list. The stack can be included in the search if the include-stacks keyword argument to get-references is specified true.
Heapwalker vectors themselves point to lots of objects but are not what you are looking for when you search for references. Therefore, heapwalk vectors are marked and are not included in get-references output.
When using get-references, note the following:
(gc t) (global gc) before you
start using this tool. Heapwalkers don't care if an object is dead or
not, so it may be that you are grabbing objects from the heap which
would have otherwise been collected because it was dead.
*, **, and *** are assigned results from previous
invocations of get-references, so it is likely that one of
them will be in the next result-vector.
This functionality in the JSON-RPC module was added as a patch in December, 2015. You must have that patch in order to use this module. See sys:update-allegro for information on getting patches.
The JSON-RPC module depends on
the ST-JSON open
source module. A compiled version of the module, the
file st-json.fasl is included with the Allegro CL
distribution and can be loaded with (require
:st-json). You can download the sources and documentation
from github
at https://github.com/marijnh/ST-JSON. ST-JSON
is a 'Common Lisp library for encoding and decoding JSON values (as
specified on json.org)' (quoting from
the ST-JSON home page). In what follows, we assume that module has
been downloaded and loaded in Allegro CL. The symbols in the ST-JSON
module (useful for manipulation JSON objects) are in
the st-json package.
JSON-RPC is lightweight remote procedure call protocol similar to XML-RPC. (See xml-rpc.htm for information on the XMP RPC interface in Allegro CL.) The JSON-RPC module in Allegro CL provides an API within Lisp to JSON-RPC.
Symbols naming functionality in the module are in
the net.json.rpc package. You load the module with
the following require
form:
(require :json-rpc)
The JSON-RPC module is initialized with the following function.
Arguments: &key client server converter methods global
This function initializes various JSON-RPC features. The various keyword arguments are described in later sections as they apply to client or server operations.
The converter keyword must be omitted,
nil, :default, or a
function of one argument. If omitted or nil,
there is no effect.
When a function is specified, it must be a function that transforms a string (denoting a JSON method or member name) to a corresponding Lisp symbol, or Lisp symbol to a corresponding JSON name string. This function is used in all cases where derived-symbol or derived-string is specified.
The value :default specifies a built-in function with the following behavior. This is the initial setting as well.
Convert a string to a symbol or a symbol to a string using simple case and hyphenation ruled:
"fooBarBaz" <--> foo-bar-baz
This function will try to signal a warning when troublesome cases are found.
A
non-nil server
keyword argument specifies a default transport protocol for JSON-RPC
servers. If the global argument
is nil, the only effect is to perform any
initialization required for the transport protocol (such as loading a
required fasl or library). If the global
argument is non-nil (the default) then any
running JSON-RPC servers are stopped.
When the methods keyword argument is
non-nil, all current JSON-RPC method
definitions are deleted, and any desired methods must be defined again
by evaluating or loading the defining forms.
A
non-nil client
argument specifies a default destination for client calls. If the
global argument
is nil, the only effect is to perform any
initialization required for the transport protocol (such as loading a
required fasl or library). If the global
argument is non-nil, the value of
*json-rpc-destination* is
updated.
Arguments: name bv &body body
Defines a Lisp function that can be dispatched as the destination of a JSON-RPC message to the server. The arguments are:
name: the value can be
(def-json-rpc-method (s1 "method1") ...)
defines a Lisp function s1 that is called when the server received a JSON-RPC request where the method name is "method1".
(def-json-rpc-method foo ...)
defines a Lisp function foo called when method "foo" is
invoked. Equivalent to
(def-json-rpc-method (foo "foo") ...).
(def-json-rpc-method "fooBar" ...)
defines a Lisp function foo-bar called when method "fooBar" is
invoked. This is equivalent to (def-json-rpc-method (foo-bar
"fooBar") ...)
options: options may only be specified using the list value for name. The options can name a set or sets in which to include the method. If no set is named, the method is part of all sets.
The bv argument: the value should be a list of one of the following forms:
([var]... [&rest tailvar] [&whole listvar])
The "params" member in the JSON message must be a JSON array. Each var specifies a required element in the array, and is bound in body to the corresponding array element.
If &rest is present, additional arguments are allowed in the array, and the list is bound to this tailvar. By-position methods are dispatched by matching all methods without &rest first, then all methods with &rest. All methods are matched in order of decreasing number of required args.
(&key [label]... [&allow-other-keys] [&whole wholevar])
The "params" member in the JSON message must be a JSON object. Here:
label --> (var member-name)
--> symbol == (symbol derived-string)
--> string == (derived-symbol string)
Each var is a keyword argument bound in body to the named member of the object. If &allow-other-keys is present, additional members are ignored; they are accessed by the member name from the pointer bound to wholevar.
By-name methods are dispatched by matching methods without &allow-other-keys first.
In each group, the methods are ordered by decreasing number of required members. The member names are sorted lexicographically and the signatures are sorted lexicographically within each set of equal length.
The dispatch strategy may cause some methods to be unreachable. Some unreachable methods may be detected at compile time, others may not.
body returns one value, a JSON object or a Lisp object that is converted to JSON by default rules, or the body signals an error that is returned as a JSON error object.
The same JSON method name may apply to several actual methods with distinct signatures; the Lisp names of these methods must be distinct (therefore only one can be a derived name).
This variable specifies the behavior when def-json-rpc-method redefines a JSON-RPC method or a one of the Lisp functions associated with the method. The value can be:
:warn -- signal a Lisp warning.
:error -- signal a Lisp error.
:allow-same -- signal an error if the JSON name or
Lisp name differ.
:ignore -- allow redefinitions silently.
The initial value is :warn.
Arguments: &key transport export receive send &
Start a JSON-RPC server, or enhance an existing server with JSON-RPC features.
The transport argument specifies how JSON-RPC messages will arrive. The format of the argument depends on the nature of the transport.
The inital implementation supports a simple stream or a web server running AllegroServe. The default for this argument can be specified with init-json-rpc. Additional arguments are interpreted by specific transport implementations.
The receive argument must be a list that
specifies the JSON-RPC versions accepted by the server. The first
entry in the list is the default reply version. The initial value is
(:2.0 :1.0 :batch). The
entry :batch specifies that batch requests will be
honored.
The send argument specifies the version of
reply messages. The value :same specifies that the
reply message will be the same version as the request message.
The export argument is a string or symbol or a list of strings and/or
symbols. A symbol denotes a Lisp function defined with
def-json-rpc-method, or a set of
methods specified with the :rpc-set option; if a
symbol denotes both a set and a function, the set definition prevails.
A string denotes a JSON-RPC method (in that case all Lisp names are
included).
This function returns an object that can be used as the argument to stop-json-rpc-server.
The transport argument must
be :stream or a list (:stream
stream-instance). If the argument
is :stream, then a stream-instance must be
specified with a :stream keyword argument. The
stream-instance must be subclass of dual-channel-simple-stream.
The transport argument must
be :aserve or a list (:aserve
. aserve-start-options).
Additional transport protocols may be defined in the future.
Arguments: &key server
Stop a server. The server argument should be the return value of the call to start-json-rpc-server that started the server.
See the description of init-json-rpc for information on
initializing JSON-RPC for servers. A
non-nil server keyword argument to that
function specifies a default transport protocol for JSON-RPC servers.
If the global argument
is nil, the only effect is to perform any
initialization required for the transport protocol (such as loading a
required fasl or library). If the global
argument is non-nil (the default) then any
running JSON-RPC servers are stopped.
When the methods keyword argument to that
function is non-nil, all current JSON-RPC
method definitions are deleted, and any desired methods must be
defined again by evaluating or loading the defining forms.
A destination where JSON-RPC call messages can be sent, such as a URL.
The default version for method calls from a client. The initial value is :2.0.
Arguments: method-name json-parameters &key notification id destination version &
Call the server at destination with a JSON-RPC message where method-name is the method member and json-parameters is the parameters member.
Default destination is *json-rpc-destination*. Default
version is *json-rpc-call-version*.
The id argument is used only in the scope of a with-json-rpc-batch expression. It specifies an id value that can be used in the call to json-rpc-batch-result.
Additional keywords are interpreted or ignored by methods specific to a particular destination.
Within the call body of a with-json-rpc-batch, the
returned value is the unique JSON-RPC call id. In other contexts, the
returned value is the JSON-RPC
result. When notification is
non-nil, no value is returned in either case.
Arguments: name bv &key destination version notification id
This macro defines a Lisp function that will make a JSON-RPC call to a JSON-RPC method.
name: the value can be
(def-json-rpc-call (s1 "method1") ...)
defines a Lisp function s1 that makes the call to a JSON-RPC method named "method1".
(def-json-rpc-method foo ...)
defines a Lisp function foo that makes the call to a JSON-RPC
method named "foo". Equivalent to
(def-json-rpc-call (foo "foo") ...).
(def-json-rpc-call "fooBar" ...)
defines a Lisp function foo-bar that makes the call to a
JSON-RPC method named "fooBar". This is equivalent
to (def-json-rpc-method (foo-bar "fooBar") ...)
The bv argument: the value should be a list of one of the following forms:
([var]... [&rest rvar])
The Lisp function expects arguments that get sent as a single JSON array. The argument names are ignored.
(&key label... [&optional label...])
The Lisp function expects keyword arguments that are sent as members of a JSON object. The labels before &optional specify required keyword args. The trailing labels specify optional keyword args. (We ask the reader to excuse this deviant usage of the &optional marker)
label --> (keyword-name member-name)
--> symbol == (symbol derived-string)
--> string == (derived-symbol string)
Arguments: bv options callbody &rest body
Call several JSON-RPC methods as a single batch call. The options are passed to methods specific to the destination.
The variables in bv are bound and visible in callbody and in body.
The callbody expression is evaluated first for its side-effects, and then any values are discarded. Any JSON-RPC calls during this evaluation add to the content of a batch; the calls themselves do not return a JSON result, instead, they return the unique JSON-RPC id of the call in the batch. The calls in the batch are sent when the progn exits.
The expressions in body are evaluated as a progn after the result of the batch call have arrived. Within body, the function json-rpc-batch-result can be called to extract the result of a specific call.
Arguments: id
When called in the body of a with-json-rpc-batch expression, this function returns the result of the request with the specified id. The id argument can be the unique id returned in the callbody of the with-json-rpc-batch expression, or the local id specified in the JSON-RPC call.
When init-json-rpc is called with a
non-nil client argument, it specifies a
default destination for client calls. If
the global argument
is nil, the only effect is to perform any
initialization required for the transport protocol (such as loading a
required fasl or library). If the global
argument is non-nil, the value of
*json-rpc-destination* is
updated.
Application program must supply a stream to the server or the client.
The stream must be a dual-channel-simple-stream instance such as a
socket or a pipe stream.
In a call to start-json-rpc-server, the value
of the transport keyword argument must be
:stream or a list (:stream
stream). An additional stream
keyword argument is accepted (so the stream to be used may be part of
a list value of transport or separately specified
as the value of stream argument). The stream must
be a dual-channel-simple-stream instance.
In a call to call-json-rpc-method, the value of the destination argument must be a stream.
To use the JSON-RPC module with AllegroServe, you must load the :json-rpc-aserve module:
(require :json-rpc-aserve)
In calls to start-json-rpc-server,
the transport argument must be either
:aserve or the dotted list (:aserve
. start-arguments) or
a net.aserve:wserver instance.
Additional accepted keyword arguments include:
:url "/json-rpc" :content-type "text/html" :publish additional-publish-args
In a call to call-json-rpc-method, the value
of the destination argument must be a uri
string or a net.uri:uri instance.
The steps for adding a new transport method are:
json-rpc-server.
(keyword class-name) onto
*json-rpc-transports*.
*json-rpc-transport* to the
keyword (or a list starting witht he keyword). This is used as the
default transport argument to
start-json-rpc-server.
nil, or a fully formed JSON-RPC reply
message. If the result is nil, the message
was a notification or an array consisting entirely of notifications;
in that case the server does nothing. If the result is
not nil, the server must send the reply
message to the client; the ST-JSON package includes several methods to
serialize a JSON object for transmission if the transport method uses
characters.
As an example, this very trivial implementation assumes the client and server are in the same address space:
(defclass local-server (json-rpc-server) ())
(push (list :local 'local-server) *json-rpc-transports*)
(defmethod start-json-rpc-server-implementation
((server local-server) &key &allow-other-keys)
())
(defmethod json-rpc-send-message ((server local-server) message notification)
(declare (ignore notification))
(json-rpc-dispatch server message)
Here are the descriptions of some of the needed functionality.
This class used in defining a new transport method. This class must be a superclass of any new transport server.
This variable is used in defining a new transport method. The value of this variable is a list of the form
((transport-keyword name-of-server-class) ... )
Additional entry formats may be defined in the future.
This variable is used in defining a new transport method. The value of this variable is the default transoprt method for JSON-RPC calls. This must be a transport name keyword or a list beginning with such a keyword. The tail of the list contains additional transport-specific arguments.
Arguments: server &rest options &key &
This method must be implemented when defining a new transport method.
Keyword arguments provide transport-specific inital values. This
method is called after the server instance is created and initialized
at the json-rpc-server level.
Arguments: server
This method must be implemented when defining a new transport method.
This method is called before any generic json-rpc-server shutdown
operations are performed.
Arguments: destination message notification &rest options &key &
This method, specialized on a new destination type, must be
implemented when defining a new transport method. The message
argument is a jso instance formatted for the specified JSON-RPC
version. The notification argument
is t for a notification or for a batch
consiting entirely of notifications.
The options argument will contain all the options specified in the call to call-json-rpc-method.
Arguments: server message
This function is called from the body of a start-json-rpc-server-implementation method for a new transport method. The server argument is the server argument passed to this method, and the message argument is a parsed JSON-RPC message received by the server.
This function returns nil if the message was
a notification or an array consisting entierly of notifications.
Otherwise, this function returns a JSON-RPC reply object that contains
the result(s) of the message.
To call a method with positional args to add two numbers:
;; Server def: (def-json-rpc-method "adder" (x y) (+ x y)) ;; Client call: (call-json-rpc-method "adder" (list 2 5)) ;; ==> 7
The client can also define a Lisp function that makes this call
(def-json-rpc-call (json-add "adder") (x y)) (json-add 2 5) ;; ==> 7
To call a json-rpc method with named args:
;; Server def:
(def-json-rpc-method "combineMembers" (&key (p1 "part1") (p2 "part2"))
(jso "combined" (append p1 p2)))
;; Client call:
(setq r (call-json-rpc-method "combineMembers"
(jso "part1" (list 2 4) "part2" (list 6 8))))
;; ==> jso instance
(getjso "combined" r)
;; ==> (2 4 6 8)
;; or
(def-json-rpc-call (jappend "combine-members")
(&key (m1 "part1") (m2 "part2")))
(setq r (jappend :m1 (list 2 4) :m2 (list 6 8)))
(getjso "combined" r)
;; ==> (2 4 6 8)
*json-rpc-call-version*
(variable)
*json-rpc-destination*
(variable)
*json-rpc-redefinition-action*
(variable)
json-rpc-server (class)
*json-rpc-transport*
(variable)
*json-rpc-transports*
(variable)
The websocket module implements the websocket protocol specified in RFC 6455. This document describes the Lisp functionality that support this specification; it is not a websocket tutorial by any measure. Many websocket tutorials are available on the web; the Mozilla Firefox tutorials at https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API is one starting point.
This module implements both a server and a client API. Extensions are not implemented at this time, but protocol and extension negotiation is performed in the initial handshake, and extension data are parsed and transmitted correctly in websocket message payloads.
The module is included in a running Lisp with the expression
(require :websocket)
The module requires the AllegroServe component and includes the
necessary calls to require. Most of the documented symbols
are in the net.aserve package. Symbols specific to
the client API are in the net.aserve.client
package.
The main server operator is publish-websocket.
Arguments: &key path host port content-type format server authorizer timeout hook headers remove on-open on-message on-close on-error on-ping on-pong on-idle on-fragment (class (quote net.aserve:websocket-message-entity)) contract-class plist protocols extensions origins debug ply close-timeout fragment-size ping-interval
This function enables a websocket service in an AllegroServe web server. It has many keyword arguments, which are divided into three groups:
See the description of the AllegroServe publish function in aserve/aserve.html for details on these arguments.
The value of these arguments must be nil or
an appropriate function which accepts the specified number of
arguments. See Section 16.3 Websocket event handlers for
further information. The names of these arguments
are: on-open,
on-message,
on-close, on-error,
on-ping, on-pong,
on-idle,
and on-fragment.
websocket-entity. The
default (and only implemented class) is websocket-message-entity.
websocket-message-server-contract.
In the current implementation, extension names are compared in the initial handshake as specified in the protocol, but there is currently no other effect on the behavior.
:only,
then only requests with a matching Origin header are accepted.
nil, 0, or 1: all server code in response to
a websocket connection runs in the AllegroServe worker thread that
handled the initial connection. The default value is 1.
nil, there is no automatic fragmentation
of messages. If specified, the value must be a positive integer and
messages are fragmented to this maximum fragment size. Text fragment
size may vary to accomodate complete UTF-8 octet sequences in each
fragment (this behavior is not required by the websocket
specification, but should make life much easier between Allegro CL
clients and Allegro CL servers. Binary fragments will all be the
specified size, except possibly the last one.
nil the value must be a positive real
number. Ping messages will be sent repeatedly with an interval
approximately that number of seconds. The application payload in the
message will be the string
"Heartbeat ping at xxxxxxx"
where xxxxxx is the ISO 8601 time string for the time when the ping was sent.
Arguments: contract
This generic function can be specialized on
the contract argument. A method is
predefined on the class websocket-message-server-contract.
That method returns the value of the plist slot
initialized by the
plist argument to publish-websocket. This value is
typically used with getf and (setf getf) to fetch and
modify values specific to an application. Any changes are seen by all
connections to this endpoint.
Symbols naming websocket client functionality are in
the net.aserve.client package.
Arguments: url &key cookies proxy proxy-basic-authorization basic-authorization digest-authorization query user-agent ssl on-open on-message on-close on-error on-ping on-pong on-idle on-fragment headers class ply protocols extensions wait plist close-timeout fragment-size ping-interval
This function opens a websocket connection to a server to enable sending and receiving websocket messages. The function operates in two modes depending on the specified threading behavior, as specified by the ply keyword argument. In one mode the function returns a websocket instance that is used subsequently to perform operations on this connection. In the other mode, the function returns only after the connection is terminated; all application actions take place in event handlers.
The required argument to this function is url. This argument is a string containing the URL of the desired websocket server endpoint. The URL must specify the websocket scheme "ws" or "wss".
The keyword arguments to this function are divided into four groups:
This first argument group, which includes the following arguments, are passed unchanged to the AllegroServe function net.aserve-client:make-http-client-request:
The second group of arguments are event handlers. The value of these
arguments must be nil or an appropriate
function which accepts the specified number of
arguments. See Section 16.3 Websocket event handlers for
further information. The names of these arguments
are: on-open,
on-message,
on-close, on-error,
on-ping, on-pong,
on-idle,
and on-fragment.
websocket-message-client-contract.
nil. The arguments are interpreted
as follows:
nil: the function call
returns only after the connection is terminated. All application
actions take place in event handlers. Receiving and sending of
messages takes place in the same thread.
websocket-contract
instance that is used as an argument in subsequent websocket calls by
the application. One new thread is created to handle both receiving
and sending of messages.
nil (the default),
a string, a symbol, or a list of strings or symbols. The
value nil means the client is requesting the
basic unmodified websocket protocol. When given a
non-nil value, the server will choose the
first supported protocol in the list. If the server was unable to
negotiate a protocol, the connection is closed and an error is
signaled. The negotiated protocol is queried with
websocket-protocol.
:open state is reached. The argument
may be a positive number to specify a wait of that many
seconds; nil or 0 to specify an immediate
return, in which case the state will
be :connecting; or any other value to specify an
indefinite wait. The default value is t
(meaning wait indefinitely until the :open state is
reached).
websocket-contract instance. It
allows a program to fetch and modify values specific to an
application. Any changes are seen only by one connection instance.
These final arguments are the same as the identically named arguments to publish-websocket. See that description for details. For each argument, there is a link to the description in the publish-websocket documentation.
Arguments: contract &key code message wait
This generic function can be specialized on the contract argument. A
method is predefined on the class websocket-message-contract. This
method begins and possibly waits for the completion of a closing
handshake on the websocket connection specified by contract.
The keyword arguments are:
nil or a string. If a string, the text is
included in the closing message sent to the server. The text is
transmitted as UTF-8 octets. The text is truncated if the UTF-8
representation is longer than 125 octets.
nil, the
function initiates the closing handshake and
returns nil. If an integer greater than zero,
wait that many seconds for completion; if not completed in that
interval, return 2 values, nil
and :timeout. If another
non-nil value, then wait indefinitely (such
values are not recommended). If omitted, the default is the
close-timeout value in the contract.
On successful completion, the method returns two values: the status code sent by the server and a string containing the message text if any.
Event handlers are application functions (written by the application programmer) called by a websocket implementation when specified messages arrive at the endpoint. The functions are specified by the corresponding keyword arguments to publish-websocket and open-websocket.
The first argument is in all cases the instance of
websocket-message-contract
specific to the current websocket connection.
Arguments: contract data ext
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-message) or the associated function object should be passed as the value of the on-message keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when an endpoint receives a complete message.
When the message is a text message,
the data
and ext arguments
are nil or strings.
When the message is a binary message,
the data
and ext arguments
are nil or arrays of octets.
Arguments: contract message-contract in-message ftype appdata ext
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-fragment) or the associated function object should be passed as the value of the on-fragment keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
When this function is specified, it is called for every message fragment.
If the function returns nil, it must
return nil for every fragment in a single
message, and the fragments are collected and handled as if no function
was specified. The function is used simply to note the passing of
fragments. The on-message function is called
after the final fragment has been collected.
If the function returns a non-nil value,
it must return a non-nil value for every
fragment in a single message. The function is responsible for
collecting and handling the content of the message. The
on-message function is not
called for that message.
If the on-fragment function returns an inconsistent sequence of values, the results are unspecified.
The in-message argument is one of the
keywords :text or :binary.
The ftype argument is one of the
keywords :only, :first,
:continuation, or :final.
The appdata
and ext arguments are
always nil or arrays
of (unsigned-byte 8) elements. Note that when
the ftype is :text, these
arrays may begin or end with incomplete UTF-8 multi-octet
characters.
Arguments: contract code data-string
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-close) or the associated function object should be passed as the value of the on-close keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when a close message is received.
Arguments: contract data-usb8
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-ping) or the associated function object should be passed as the value of the on-ping keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when a ping message is received and is called before the required pong response is sent, and its behavior does not affect the standard response.
Arguments: contract data-usb8
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-pong) or the associated function object should be passed as the value of the on-pong keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when a pong message is received.
Arguments: contract
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-error) or the associated function object should be passed as the value of the on-error keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when an endpoint detects an error that causes an attempt to send an abnormal termination message. The message may or may not actually be sent.
Arguments: contract
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-open) or the associated function object should be passed as the value of the on-open keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when a connection is first opened after a successful handshake.
In a client application where open-websocket is called
with ply
value nil or 0, this is the where the
application would normally begin operation on the client side (unless
it expected an unsolicited server message instead).
Arguments: contract
This function is not already defined. The application writer must define a function which accepts the arguments listed and returns values as specified in the function description below. The symbol naming the function (which need not be on-idle) or the associated function object should be passed as the value of the on-idle keyword argument to publish-websocket and/or open-websocket (different functions can be specified for each).
This function is called when the websocket handler is setup to handle receiving and sending in the same thread. The function is called when there is no incoming message and the send queue is empty.
If the function returns nil, the handler
enters an operating system wait state waiting for input from the other
endpoint. If the function return a non-nil
value, the handler polls for input or queued messages to send.
Symbols naming general websocket functionality are in
the net.aserve package.
Arguments: contract message &key ext fragment-size
There is a predefined method on this generic function specialized on
(websocket-message-contract t):
That method sends a websocket message to the other endpoint and
returns the keyword :text
or :binary depending on the message type.
The contract argument must be an open websocket connection instance.
If the message argument is a string, the
message type is :text. If
the message argument is an array of
octets, the message type is :binary. Any other
type causes an error to be signaled.
The ext argument when specified must be the same type as the message argument. The data is included in the message payload and will be presented as extension data to the receiver. The current implementation in Allegro CL does not support any named extensions.
The fragment-size argument if specified overrides the fragment-size default for the connection. Messages above this size will be sent in fragments of approximately the specified size. If the ext argument is specified, any fragment-size specification is ignored; the message is always sent in a single payload.
Arguments: contract &key message
The predefined method specialized on websocket-message-contract
does the following:
The method sends a ping message to the other
endpoint. The message argument must be a
string or nil.
Arguments: contract &key message
The predefined method specialized on websocket-message-contract
does the following:
The method sends a pong message to the other
endpoint. The message argument must be a
string or nil.
Arguments: contract &key code message
The predefined method specialized on (websocket-message-contract t)
does the following:
This method sends a close message to the other endpoint. If
a code is specified it should be a valid
websocket close code. The message argument
must be a string or nil.
Once the close message is sent, subsequent send calls are ignored.
Arguments: contract &optional state wait
The predefined method specialized on websocket-message-contract does
the following:
The method queries the state of the connection, returning one of the
keywords :connecting, :open,
:closing,
or :closed.
If the state argument is specified, it
should be one of the keywords that can be returned by this
function. The function returns nil if the
contract is not in the specified state, or the
keyword if that state is in effect.
If the wait argument is specified, then
the function will wait until the state specified by
the state argument is
reached. The wait argument may be a positive
number to specify a wait of that many
seconds; nil or 0 to specify an immediate
return; or any other value specifies an indefinite wait.
Arguments: contract
The predefined method specialized on websocket-message-contract does
the following:
Returns a symbol or a string naming the protocol negotiated for the
connection idenified by contract. Returns
nil if the negotiated result is the
unmodified websocket protocol.
Arguments: contract
The predefined method specialized on websocket-message-contract does
the following:
this method returns a list of strings or symbols naming the extensions negotiated for this connection.
Arguments: contract fq &key begin final data ext
The predefined method specialized on (websocket-message-contract t)
does the following:
The method allows an application to exert more control over the fragmentation of a message. Each call prepares a separate message fragment. Any number of fragments can be prepared before any sending takes place. As soon as the first fragment is sent, no other messages will be sent (except for ping or pong) until the final fragment is sent.
The fq argument must
be nil for the first call of a fragmented
message. The value returned is an opaque handle used as
the fq argument in subsequent calls.
The keyword arguments have the following effect:
t,
begin sending this or previously prepared fragments. Once the first
fragment is sent, new messages will be queued and will wait until the
final fragment in this message is sent.
t,
the fragment is the last, and no more will be accepted.
When fq is nil,
return new fq, required in subsequent calls.
The data and ext arguments are as in websocket-send. Note that this function allows an application to scatter extension data among fragments.
This method may be called with a nil value
for data and begin specified
as t.
Arguments: contract
This generic function can be specialized on
the contract argument. A method is
predefined on the class websocket-contract. That method
returns the value of the plist slot in the contract
instance.
In server code this slot is initialized
to nil. In client code it is initialized
with the plist argument to open-websocket.
This value is typically used with getf and (setf getf) to fetch and modify values specific to an application. Any changes are seen only by one connection instance.
Websocket classes are named by symbols in
the net.aserve package.
The superclass of any websocket implementation for
AllegroServe. Currently the only subclass is websocket-message-entity.
This class implements the server part of the websocket protocol specified in RFC 6455.
The superclass on any any websocket contract classes. Currently the
only subclass is websocket-message-contract.
An instance of this superclass is the opaque handle for all websocket operations in both server and client applications. This class implements the websocket protocol specified in RFC 6455.
The subclass of websocket-message-contract used
by client code.
The subclass of websocket-message-contract used
by server code.
All operators and classes associated with the websocket API are listed below.
websocket-contract: class
websocket-entity: class
websocket-message-client-contract: class
websocket-message-contract: class
websocket-message-entity: class
websocket-message-server-contract: class
In a multi-threaded application it can be difficult to trace the sequence of events leading to an unexpected outcome. This is especially true in an SMP lisp. Allegro CL provides a very low-level log of multiprocessing events, including arrival of interrupts, starting and exiting of threads, opening and closing gates, garbage collections and socket events, that can sometimes be helpful in these situations.
Logging is not enabled by default. You start (and stop) logging with the mplog:recording function. The log is maintained in an internal memory buffer of fixed and limited size, which can hold approximately one thousand entries. When it fills up, new entries push the oldest ones out.
You often want to examine a log while debugging a misbehaving program, but because the running Lisp is stopped in the debugger, you cannot execute the calls which return log information. A feature of the facility allows you to store the log in a shared memory buffer so a separate Lisp image can be used to examine it. See mplog:share.
On Windows only, it is possible to construct a log from information available in debuggers. See Section 17.1 Constructing mplog files from the debugger on Windows for more information.
The mplog functionality is in the :mplog module. To
load it, evaluate
(require :mplog)
Operators are named by exported symbols in
the :mplog package except the
function sys:mplog is named by a symbol in the system package.
Arguments: on-or-off &optional include-process-events
Enables (on-or-off
non-nil) or disables
(on-or-off
nil)
mplogging.
If include-process-events is specified
true, high-level process events like creation and termination, are
recorded. If nil (the default) such events
are not recorded.
As said in the introduction above, the mplog is stored in an internal buffer of fixed size, holding about 1000 entries. Older entries are dropped when the buffer fills up and new entries are added. Entries can be viewed with mplog:show. On Linux, it is possible for one running Lisp to view the entries for another, a useful feature when an image is stopped for debugging and so cannot itself call mplog:show: see mplog:share.
Arguments: &key read write format
This function formats the data in an mplog. The keyword arguments are
nil, the source is the current
Lisp's mplog buffer. If a string, the source is a file whose format is
described below. If :shared, the source is found in
the shared-memory area set up by the mplog:share function described
below. No other values are valid.
nil: display on *standard-output*; if a string or
pathname, create or supersede the named file (meaning an existing
file, if any, will be destroyed).
:msdev
or :gdb. See Section 17.1 Constructing mplog files from the debugger on Windows
below for information on capturing mplog data from a debugger.
Arguments: mode &optional filename
This function is only available on Linux systems. It allows a running Lisp to put it's mplog buffer in shared memory so a second Lisp can read it for formatting. This is useful when debugging a lisp application that hangs and does not respond to the console.
mode is either :write,
:read, or :close. The lisp
that is being debugged executes (mplog:share :write
filename) before enabling recording. The lisp that will be
used to format the result executes (mplog:share :read
filename) before calling (mplog:show
:shared). The lisp displaying the shared buffer does not
itself put data in the buffer, and should call (mplog:share
:close) when it is done.
Arguments: code &optional data
The function (named by a symbol in the system package, not the mplog
package, note) places an entry in the mplog. This can be useful for
syncing the data in the formatted output to program actions. The
function returns nil.
code must be an integer in the range #xf000 (61440) to #xffff (65535) inclusive.
data can be anything, even a Lisp value, but what is actually written is the tagged address of the object (its LispVal) at the time of the call and that value will not be maintained by the garbage collector. For this reason it's best to limit this to fixnum values (which will display as 4 (32-bit) or 8 (64-bit) times the values passed in) or addresses of non-moving items.
;; Suppose mplog recording is on. Then we execute cl-user(17): (sys::mplog #xf1f1 20) nil ;; Then in the log we have cl-user(18): (mplog:show) index tcb id name [...] 2138786: Initial Lisp Listener(7fec6954d700) (#xf1f1 #xa0) ??? [...] ;; #xf1f1 is the code we specified and #xa0 is 160, 8 times the data ;; value, which was the fixnum 20. (#xa0 is the value with the ;; positive fixnum tag. This example was run in a 64-bit Lisp.)
;; In a running Lisp image, load the mplog module: (require :mplog) ;; Start mp logging: (mplog:recording t) ;; Show the current log: (mplog:show) ;; Stop logging (mplog:recording nil) ;; To share date with another running Lisp, load ;; the module and start logging: (require :mplog) (mplog:recording t) ;; Share the data: (mplog:share :write "myfile") ;; In a second running Lisp, load the module and link to ;; the shared data: (require :mplog) (mplog:share :read "myfile") ;; Still in the second Lisp, show the data (mplog:show :shared) ;; When done, in the second Lisp stop sharing: (mplog:share :close)
mplog:share is only available on Linux machines. On Windows, it is possible to construct mplog results when debugging a running Lisp from information available in either the msdev or gdb debuggers.
To do that you create a text file holding data pulled from the
debugger display using cut-and-paste or any other technique that
captures the data as text. Optionally edit that file with a text
editor to supply identifying names for the different threads. Then
evaluate (mplog:show :read fname)
or (mplog:show :read fname :format
:gdb). See mplog:show.
To get the data that should be copied in the MSDEV debugger, get a
memory-display window showing the address and 5 32-bit entries per
line, all in hex. Display the region at acl_mplog. Use copy-paste to
copy the whole buffer, as is, into an editor. This will be the main
body of a file for readlog to process. The rest of the file consists
of (mplog:name-thread ...) entries and a marker
symbol. Here's how a typical file might look:
(mplog:name-thread 1 #x08c #x52008002 "Initial Lisp Listener") (mplog:name-thread 2 #x3e4 #x52008802 "portmapper") (mplog:name-thread 3 #x738 #x54008002 "GWHandler") (mplog:name-thread 4 #x4c0 #x54008802 "mountd") (mplog:name-thread 5 #x44c #x54009002 "nfsd") (mplog:name-thread 6 #x538 #x54009802 "open file reaper") (mplog:name-thread 7 #x5e0 #x5400a002 "attr cache reaper") (mplog:name-thread 8 #x708 #x5400a802 "dir cache reaper") (mplog:name-thread - #x744 - "lisp console") (mplog:name-thread - #x1b8 - "lisp timer") (mplog:name-thread - #x3d4 - "Socket Control") mplog 638A9720 00003E80 0000D6BB 000004C0 00000008 00000004 638A9734 00003E81 0000D6BB 0000044C 00000200 00000000 638A9748 00003E82 0000D6BB 000004C0 00000080 00000000 638A975C 00003E83 0000D6BB 0000044C 00000204 00000000 638A9770 00003E84 0000D6BB 0000044C 00000202 00000000
with the hex-dump lines continuing to the end of the file.
To get the info for the name-thread entries, display acl_thread_control in the MSDEV watch window, then expand it and scroll down to the registry array. Expand that and you'll see the hex addresses of all the allocated threads. Use the index displayed there with each address as the first number in the name-thread form. The second number is the thread id. The third number is the address of the thread control block itself; the string is the thread name. These last three can be extracted from the memory display since you have the address of the control block in the watch window. The name-thread entries with "-" in the index and control block slots represent non-lisp threads. Get the thread id's from the MSDEV debug/threads panel, and put in whatever you want for the thread names.
The symbol mplog following the name-thread
entries is the marker that tells readlog the rest of the file is the
log dump.
Now you can (mplog:show :read "xxx"),
where xxx is the name of the file with all this
stuff in it, and see a human-readable display of the log info pour out
onto standard-output.
Note: mplog:show expects the buffer data to be ordered per the usual wrap-around regime. It finds the split point and cuts the deck there to put it in order. It then checks that the sequence numbers of the entries are kosher and complains if they aren't. This would most likely be due to not grabbing all (and only) the mplog buffer data in the cut-and-paste operation.
Copyright (c) 1998-2019, Franz Inc. Oakland, CA., USA. All rights reserved.
This page was not revised from the 9.0 page.
Created 2015.5.21.
| Allegro CL version 10.0 Unrevised from 9.0 to 10.0. 9.0 version | ||||||||