Allegro CL Modern Mode

Introduction

Allegro CL 6.0 users will notice new options for starting Lisp. One of the most prominent of these new choices is that between "Modern" and "ANSI" mode Lisps. This article explains these different modes of Common Lisp. [Note: written at the time Allegro CL 6.0 was released, the above statement is not true about the Allegro CL 6.1 Trial Edition. Allegro CL 6.1 was released with ANSI mode images.]

The Modern/ANSI mode choice refers to the Lisp image's case mode. Briefly, a case mode is a combination of two things: (a) the Lisp symbol reader's default behavior, and (b) which case (upper case or lower case) is used to store pre-defined Lisp symbols which name functions, variables, classes, etc. The case mode being used has a big effect on how source files and data files are read and how their data may be represented internally or output by Lisp. Thus, it is important to understand how the case mode choice affects one's Lisp programming. Before we go into details, however, we wish to reassure users concerned about compatibility.

Support

Franz Inc. is and always has been committed to providing fully ANSI-compliant Common Lisp development and runtime environments. With each release, Franz Inc. includes a fully supported ANSI-compliant Common Lisp executable. As Franz Inc. introduces new features that extend the ANSI Common Lisp specification, the highest priority is given to ensure not only that ANSI-compliant Common Lisp code will run, but also that such new features are as backwards compatible as possible with prior Franz Inc. releases.

Modern mode is provided separately from the backwards compatible ANSI executable(s) because Modern mode directly contradicts the ANSI specification in two very specific user-visible ways which can break ANSI-compliant Common Lisp programs. If you have Allegro CL 6.0 installed, you may find, for example, files named "alisp.dxl" and "mlisp.dxl". These correspond to the base lisp images for ANSI mode and Modern mode respectively. (Please see Allegro CL 6.0 documentation for information on starting Allegro CL images.) Windows users may find "ANSI" and "Modern" as Allegro CL 6.0 start menu items. Other than the case mode, however, the images are equivalent. Users unable to switch to Modern mode can simply stick to using the ANSI mode versions.

In order to increase Common Lisp's effectiveness with modern day, externally accessible, case-sensitive programming environments, Franz Inc. is bringing its Modern mode to the fore as a way to encourage Lisp users and developers to consider new ways of thinking about how Lisp symbols can be used. In Modern mode, Lisp symbols can easily be used not only for Lisp function/variable/object names, but also for external names such as classes/functions in Java/C++, or data elements in XML or CORBA. By following a few simple guidelines (described further below), it is generally easy to write and maintain Lisp code which runs in either ANSI or Modern mode.

Background

The ANSI Common Lisp specification states that all pre-defined Common Lisp functions, variables, classes, etc. are represented as symbols whose names are stored in all upper case. Thus, for example, suppose you wish to evaluate the following Lisp source expression: 



 (member 'cat '(dog cat mouse))

When Lisp reads this expression, either by loading a source file, or from a user typing it to a Lisp Listener, the symbol reader internally converts all symbol names to their upper case equivalents. [Aside: The case of some non-English letters can actually depend on a user's locale. Thus, it is possible to have the Lisp reader's behavior depend on the environment in which it is being run. Other than mentioning it here, we disregard this problem in the rest of this article.]

Thus, after being read by Lisp, the form above is equivalent to the following: 



 (MEMBER 'CAT '(DOG CAT MOUSE))

When Lisp evaluates this expression, it finds that the first element, MEMBER, matches the pre-defined Lisp symbol, MEMBER, whose function when applied to the arguments as described, returns the following: 



 (CAT MOUSE)

As can be seen by the examples below, the ANSI Lisp symbol reader is case-insensitive. In other words, each of the following expressions, when read in ANSI mode, returns the same result: 



 (Member 'cat '(dog cAT mOUSE))   -> (CAT MOUSE)

 (MEMBER 'CAT '(Dog Cat Mouse))   -> (CAT MOUSE)

 (mEmBEr 'Cat '(dog CAT mouse))   -> (CAT MOUSE)

 (member 'CAT '(dog cat mouse))   -> (CAT MOUSE)

 (MEMBER 'CAT '(DOG CAT MOUSE))   -> (CAT MOUSE)

 and so on...
One of the features of such a case-insensitive reader is that it allows programmers to write source code using mixed case knowing that Lisp will canonicalize (and thus ignore the case variants of) the symbol names. Some programmers may choose to adopt a coding style where they indicate predefined Lisp symbols in upper case: 


 (DEFUN my-function (arg1 &OPTIONAL arg2)

  (WHEN (my-predicate arg1 arg2)

   (PRINT "Success")))

An alternate style might be simply to capitalize all symbols: 



 (Defun My-Function (Arg1 &Optional Arg2)

  (When (My-Predicate Arg1 Arg2)

   (Print "Success")))

One can have many different such case-variant styles even within a single file. Whether it is good or not to have such styles is another matter. However, as far as Lisp is concerned, all (unescaped) symbols are converted to upper case as they are read.

ANSI Mode Problems

As long as Lisp programs only require that symbol names be in upper case, there would be no problem using the default ANSI specified Lisp reader. However, difficulties occur when interfacing to non-Lisp programming systems which are increasingly using lower case case-sensitive identifiers. The case-insensitive nature of the Common Lisp symbol reader creates hurdles for programmers wishing to interface to such systems.

For example, if a user wants to establish a foreign function associated with the Microsoft Windows API function CreateDIBitmap() (which is used to Create a Device Independent Bitmap), the default Lisp reader action is to turn the name into upper case. Thus, one may have source code which reads as follows: 



 (defun call-windows ()

  (CreateDIBitmap ...))

If, however, an error occurs during the execution of this call, the Lisp stack backtrace displayed by the debugger is hard to read due to missing word boundary information in the function's name: 



 (CREATEDIBITMAP ...)

Processing XML data is another example which demonstrates Lisp's power as well as the ANSI-mode hurdle with case-sensitive data. Suppose we have employee information in XML format as shown in the following fragment: 




 <Employee OfficePhone="510-548-3600"

           SSNumber="111-22-3333"

           HomePhone="510-111-2222">

 Franz Liszt

 <IdPhoto href="liszt.jpg"/>

 </Employee>
A natural way to represent such elements in Lisp is with keyword-style lists, and this is what Franz' open-source xml-parser returns. XML is intrinsically case sensitive, and the DTD (Document Type Definition) mandates the case of all tag and attribute names. Thus, the above expression can be represented with case-preserving tags, but escapes are needed when the list is printed in ANSI-mode: 



 ((|Employee| |OfficePhone| "510-548-3600"

              |SSNumber| "111-22-3333"

              |HomePhone| "510-111-2222")

  "Franz Liszt"

  ((|IdPhoto| |href| "liszt.jpg")))
A reason to choose symbols for tags is that we can then use powerful Lisp tools such as destructuring-bind to further process the parsed structure. Element attributes in XML consist of key/value pairs, but the attributes attached to any node are unordered, and it can be tedious to extract some set of attributes from a node. This pattern is so very similar to Lisp keywords that exploiting Lisp lambda-list binding is an obvious mechanism for extracting and collecting attributes and their values. In ANSI mode, however, Lisp code which deals with case-sensitive symbol tags needs a lot of escaping, making the code both hard to read and hard to write. 



 ;; An XML phone directory processor

 (case (car element)

   ((|CompanyOfficer| |Manager| |Employee| |Consultant|)

    (destructuring-bind

        ((tag &key ((|OfficePhone| office-phone))

                   ((|HomePhone| home-phone))

                   ((|SSNumber| ss-number))

              &allow-other-keys)

         &rest content)

        element

      (my-formatted-output content tag office-phone

                           home-phone ss-number)

      ...))

   (|Customer| ...)

   ...)
The above code works in either ANSI mode or Modern mode Lisp, but we'll see below how this code can be rewritten more felicitously in Modern mode.

Many Lisp programmers have chosen to avoid the symbol case-insensitivity problem by using strings instead of symbols to represent externally named objects. Strings are always read with their case preserved. Unfortunately, by using strings, a Lisp programmer gives up using Lisp variables for case-sensitive external identifiers. Furthermore, one gives up other features unique to symbols such as fast eq comparisons or being able to store the identifiers in separate package name spaces.

Readtable-Case

The ANSI specification does, in fact, address limited case-sensitivity by providing readtable-case functionality. Franz Inc. fully supports this readtable-case functionality in both ANSI and Modern modes. An Allegro CL 6.0 patch, 'update/p4a005.001' from ftp.franz.com/pub/patches/, is needed for Modern mode readtable-case support. The easiest way to get patches is via the Allegro CL function sys:update-allegro.

While the ANSI readtable-case solution may be suitable for programs with case-sensitive needs, this functionality is somewhat counterintuitive due to ANSI limitations. Specifically, the only readtable-case setting which can effectively be used that doesn't require all source code to be converted to upper case and which still provides case-sensitivity is the :invert setting. The way this setting works is by having the Lisp symbol reader turn symbol names with all lower case characters into all upper case and vice versa.

For those not already aware of the interaction between *print-case* and the :invert readtable-case, ANSI mode Common Lisp can create some surprising results. For example, a symbol whose name is "ZEBRA" is printed in all lower case even when *print-case* is specified to be :upcase. Also, the symbol whose name is "Zebra" is printed as 'Zebra' (i.e., capitalized) even if *print-case* is either :upcase or :downcase. While these results are understandable, given how the reader and printer must operate with the ANSI specified case mode, they are not intuitive.

Modern Mode

Coercing names to upper case may have been useful in Lisp's infancy when computer text representation was mainly limited to upper case (e.g., for punched cards or now ancient programming languages such as FORTRAN). Modern programming environments, however, almost universally use case sensitive lower case identifiers. As we shall see, Modern mode in Lisp not only eases interfacing to such environments, it also simplifies the Lisp reader.

As opposed to ANSI Mode, symbols in Modern mode are simply treated as WYSIWYG (What You See Is What You Get). In other words, symbols are simply read with case preserved. Thus, following up on an example above, the call to (CreateDIBitmap ...) shows up as (CreateDIBitmap ...) when displayed in a stack backtrace, or trace output, etc. Further examples show how in a Modern mode Lisp session, one gets straightforward results: 



 (quote MyDogSpot)   -> MyDogSpot



 (symbol-name 'MyDogSpot)   -> "MyDogSpot"

Thus, in Modern mode, one is easily able to use symbols as names for externally defined case-sensitive identifiers (e.g., foreign functions).

As a further example, the Lisp / XML fragment from above can be greatly simplified by using unescaped symbols to represent XML tags, and by using the corresponding case-sensitive variable names directly in Lisp code: 




 ;; An XML phone directory processor

 (case (car element)

   ((CompanyOfficer Manager Employee Consultant)

    (with-xml-element (tag (OfficePhone HomePhone SSNumber)

                           &rest content)

        element

      (my-formatted-output content tag

                           OfficePhone HomePhone SSNumber)))

   (Customer ...)

   ...)
The helper with-xml-element macro is defined as follows: 



 (defmacro with-xml-element ((tag (&rest attributes)

                                  &rest contents)

                             element

                             &body body)

   `(destructuring-bind

        ((,tag

          &key ,@(mapcar #'(lambda (x)

                             (list (list x x)))

                         attributes)

          &allow-other-keys)

         ,@contents)

        ,element

      ,@body))

As for readtable-case, there is little need for it at all in Modern mode. In fact, the original implementation of Modern mode, which predates readtable-case and even the full ANSI Common Lisp specification itself by several years, ignored readtable-case settings. The reason for ignoring readtable-case was to allow greater portability for ANSI mode programs by allowing the same readtable-case settings in Modern mode as in ANSI mode. User feedback, however, suggested that Franz Inc. reconsider this particular decision. Thus, the above mentioned Allegro CL 6.0 patch, 'update/p4a005.001', is now available to provide full readtable-case functionality in Modern mode. The default readtable-case setting for Modern mode is :preserve.

ANSI / Modern Code Portability

In general, Common Lisp programs which don't depend on case-sensitivity, and which are written using lower case for pre-defined Lisp names will work either in Modern mode or in ANSI mode. There are a few areas, however, that programmers must look for when developing code that is to be portable between both Modern and ANSI modes. Specifically, places where literal upper case strings are used as string designators can portably be replaced by keyword symbols. For example, the following form works only in ANSI mode: 



 (in-package "COMMON-LISP-USER")

If the string designator is changed to be a keyword, then the form will work in both ANSI and Modern modes: 



 (in-package :common-lisp-user)

Some functions may deal with symbol-names directly in other ways. Again, these can also be made portable between ANSI and Modern modes by using keywords as string designators. For example, the following form works only in ANSI mode: 



 (defun my-new-symbol (arg)

   (intern (concatenate 'string

             "MY-NEW-SYMBOL-"

             (format nil "~a" arg))))

If written as follows, however, the function works exactly the same way in ANSI mode, but also can be used in Modern mode: 

 

 (defun my-new-symbol (arg)

   (intern (concatenate 'string

             (symbol-name :my-new-symbol-)

             (format nil "~a" arg))))

Conclusion

Modern mode simplifies the Lisp symbol reader by removing the requirement that symbols be canonicalized in upper case. What we gain by this is a notational convenience that gives easier access to the underlying expressiveness and power of Common Lisp. This is particularly useful and important as Lisp is increasingly used to interact with case-sensitive programming languages and environments in the modern programming world. Modern mode is yet another tool that helps Allegro CL users meet this challenge.

Copyright © 2023 Franz Inc., All Rights Reserved | Privacy Statement Twitter