Some notes on the Lisp heap size

If you load a lot of data into Lisp, you may get an error that looks like this:

Error: An allocation request for XXXX bytes caused a need for YYYY more
bytes of heap.
The operating system will not make the space available.
  [condition type: STORAGE-CONDITION]

XXXX and YYYY and numbers, the second usually being in the millions and the first usually smaller, sometimes much smaller. What this message is telling you is that the Lisp heap needs to expand but for some reason, it is unable to do so. The most common reason that the Lisp heap cannot expand is that it has grown to the point where it has run into something else. And the most common thing that it has hit is the C heap used by Lisp. That is the situation we will discuss in this note though we will mention other potential problems.

First, let us quickly deal with some issues:

  • The amount of physical memory is not relevant to this issue. The operating system uses a virtual memory model where processes, libraries, etc. are mapped from the physical memory and physical disk onto a virtual address space, and it is conflicts in that virtual space that cause the problem. The amount of physical memory may affect performance but it does not affect the amount of space allocated to a process or a portion of a process.
  • Allegro CL has several components and each must be mapped onto a contiguous portion of virtual memory. The components include the Lisp heap, the C heap, pure space, the Lisp library, and others. The two that concern us are the Lisp heap and the C heap. If the Lisp heap runs out of space, even if there are large blocks of space elsewhere in virtual memory, the Lisp heap cannot grow.
  • The desired location of the start of the Lisp heap and the start of the C heap are specified when the image is built. At run time, those values will be used if possible, but if those values cannot be used, other values will be used and Lisp will start up.
  • The two most important values are the Lisp heap start and the C heap start. Other parameters, such as Lisp heap size and C heap size, occasionally have significance, but in the majority of cases, it is the two heap start values that must be adjusted if there is a space problem and specifying appropriate values for those two parameters solves the problem. It is those two parameters we will focus on in this note.

Using (room) to see where the Lisp and C heaps start

Evaluating (room) prints out information about space used by Lisp. This is from a 64-bit Lisp:

cl-user(2): (room)
area area  address(bytes)        cons         other bytes
  #  type                   8 bytes each
                             (free:used)      (free:used)
     Top #x1001c06000
     New #x100182e000(4030464)  461:32494     711376:2731456
     New #x1001456000(4030464)   -----            -----
   3 Old #x10013d6000(524288)    0:0         351616:171808
   2 Old #x1001396000(262144)    0:0             16:261520
   1 Old #x1001316000(524288)  327:3222           0:466080
   0*Old #x1000001250(20008368)    0:127257         0:17923152
     OTot(Old Areas)           327:130479    351632:18822560
* = closed old area

Root pages: 137
 Lisp heap:  #x1000000000  pos: #x1001c06000 resrve: #x1004000000
 C heap:     #x2000000000  pos: #x2000039000 resrve: #x20000fa000


For our purposes, the two relevant lines are:

 Lisp heap:  #x1000000000  pos: #x1001c06000 resrve: #x1004000000
 C heap:     #x2000000000  pos: #x2000039000 resrve: #x20000fa000
These lines tell us where the Lisp heap starts (#x1000000000), how much space is used (the pos value, #x1001c06000), how much has been reserved (the resrve value, #x1004000000) and where the C heap starts (#x2000000000). These are all hex values. If you cannot think in hex, Lisp will translate them to decimal:

cl-user(3): #x1000000000
68719476736  ;; the Lisp heap start, 64 Gigabytes
cl-user(4): #x1001c06000
68748861440  ;; the current top of the Lisp heap
cl-user(5): (- #x1001c06000 #x1000000000)
29384704     ;; the current Lisp heap is using about 28 Mbytes
cl-user(6): #x1004000000
68786585600  ;; the top of the reserved area
cl-user(7): #x2000000000
137438953472 ;; the C heap start
cl-user(8): (- #x2000000000 #x1000000000)
68719476736  ;; the max potential for the Lisp heap (64 Gbytes)

I started with a 64-bit Lisp to indicate that only very large amounts of data are likely to cause problems in 64-bit Lisps. Here are some 32-bit Lisp numbers (the numbers you see may differ):

 Lisp heap:  #x10000000  pos: #x11312000 resrve: #x20000000
 C heap:     #x54000000  pos: #x54022000 resrve: #x540fa000

cl-user(3): #x10000000
268435456  ;; Lisp heap start
cl-user(4): #x11312000
288432128  ;; Current Lisp heap top
cl-user(5): #x20000000
536870912 ;; top of reserve
cl-user(6): #x54000000
1409286144 ;; C heap start
cl-user(7): (- #x54000000 #x10000000)
1140850688 ;; Maximum potential Lisp heap size

The reserve area is usually not important. If the Lisp heap grows bigger than that value, as long as there is nothing in the way (see below), it just keeps growing, until it hits the C heap start. So the important value is the difference between the C heap start and the Lisp heap start. In our cases, the maximum for our 64-bit image is about 64 Gbytes while the maximum for the 32-bit image is 1 Gbyte.

Now the 32-bit image is bound by the effective address size (32 bits, or 4 Gbytes) and OS restrictions (which further limit things to 2 or sometimes 3 Gbytes). Our image allows about 1 Gbyte and that may be increased. The way to increase it is to specify different values for the :lisp-heap-start and :c-heap-start arguments to build-lisp-image. This will create an image that uses these new values and will therefore allow for a Lisp heap as large as the difference between the values. A call looks like:

(build-lisp-image "biggerimage.dxl" :lisp-heap-start "100M"
                  :c-heap-start "1610M")

Note that we specify the values as strings and use M to stand for Megabyte. The value can be an integer or a string and strings can use the K or M abbreviations. We recommend that you use strings with the M abbreviation as that avoids any problem that might occur with integer values overflowing.

If the image is built successfully, we will have increased the maximum Lisp heap size to 1.5 Gbytes.

What if something else is in the way?

The (room) output only reports on where Lisp components are mapped in virtual memory. It is possible that some other non-Lisp component (such as a shared library) is mapped in the area between the top of the Lisp heap and the start of the C heap. In that case, the Lisp heap will stop growing when it hits whatever is mapped. (Nothing can be mapped in the reserve space, because the OS has allocated that space to Lisp. But if you get the cannot grow Storage Condition error mentioned at the start of this note when there seems to be sufficient space between the reserve location and the C heap start, then it is likely something is mapped in that unreserved area.)

If you suspect some object is mapped where it interferes with Lisp growing, you have to find out what it is and decide what to do. It is possible sometimes to move objects, and you can also move the Lisp heap start (and also the C heap start) up so they are above the object. But the first step is finding out what is there. That process is beyond the scope of this note, but there is a FAQ entry describing what to do (the process is different on UNIX and Windows -- Windows users scroll down in the entry)

Links to other documentation

The following links to the documentation and the FAQ deal with the issues discussed in this note:

Copyright © 2014 Franz Inc., All Rights Reserved | Privacy Statement
Delicious Google Buzz Twitter Google+