The shell module interface

This document contains the following sections:

1.0 Introduction

The shell module is intended to provide UNIX shell-like commands, such as you find on a modern UNIX system, and in addition shortcuts for some common Perl idioms.

This module does not provide functionality to spawn a shell (as older functions like run-shell-command and shell and newer ones like command-output do). Instead, many commands that are available in a shell are available in this module. These commands are available on UNIX and Windows and work in a uniform fashion whatever the actual operating system. No external programs are used to implement the functions in this module, and as a result security issues associated with executing external programs by spawning shells are finessed.

The shell module's package, :excl.shell, is autoloaded when referenced. The module name is shell. The module can be loaded by evaluating

(require :shell)

See also os-interface.htm, in particular The Operating System Interface (OSI) module in that document, for other operators releated to those described in this document.

2.0 Operators, variables, and classes in the Shell module

Operators

• concat
• cp
• die
• glob
• join
• ln
• mv
• rm
• split
• touch
• -d
• -e
• -f
• -l
• -r
• -s
• -w
• -x
• =~

Variables

• $1
• $2
• $3
• $4
• $5
• $6
• $7
• $8
• $9

Classes

• shell-not-supported-condition

=~

Function

Package: excl.shell

Arguments: regexp string &rest keys &key &allow-other-keys

Return t if the regular expression regexp matches string As a side-effect, set the $1...$9 variables to the corresponding group match. When there is no corresponding group match, the associated variable is set to nil. If regexp is a string, it is cached in case it is used repeatedly -- it will only be compiled once for consecutive calls to this function.

See `compile-regexp' for details on compilation of regular expressions.

The keywords accepted by match-regexp are accepted by this function.

For example:

(33): (excl.shell:=~ "\\([^.]+\\)\\.\\(.*\\)" "foo.bar")
t
cl-user(34): excl.shell:$1
"foo"
cl-user(35): excl.shell:$2
"bar"
cl-user(36): excl.shell:$3
nil
cl-user(37): 

$1

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $2, $3, $4, $5, $6, $7, $8, and $9.

$2

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $3, $4, $5, $6, $7, $8, and $9.

$3

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $4, $5, $6, $7, $8, and $9.

$4

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $5, $6, $7, $8, and $9.

$5

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $4, $6, $7, $8, and $9.

$6

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $4, $5, $7, $8, and $9.

$7

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $4, $5, $6, $8, and $9.

$8

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $4, $5, $6, $7, and $9.

$9

Variable

Package: excl.shell

This variable is used by =~ and is set to a group match by that function (or to nil if there is no corresponding group match).

There are nine variables associated with =~. This is one. The other eight are: $1, $2, $3, $4, $5, $6, $7, and $8.

join

Function

Package: excl.shell

Arguments: delimiter list

Join the elements of list together with delimiter. This function is shorthand for (list-to-delimited-string list delimiter) (see list-to-delimited-string).

For example:

cl-user(37): (excl.shell:join #\, '(1 2 3 4))
"1,2,3,4"
cl-user(38): 

split

Function

Package: excl.shell

Arguments: regexp string &rest args &key &allow-other-keys

split is a synonym for the function split-regexp.

For example:

cl-user(38): (excl.shell:split "," "1,2,3,4")
("1" "2" "3" "4")
cl-user(39): 

concat

Function

Package: excl.shell

Arguments: &rest things

Concatenate objects into a simple-string. This function is similar to concatenate, except that conversion of pathnames, numbers and symbols is done automatically into a string. The conversion of these three types is done with princ-to-string. For example: (concat 1 2 3) will yield "123", whereas (concatenate 'simple-string 1 2 3) will signal an error.

glob

Function

Package: excl.shell

Arguments: string-or-pathname &key &allow-other-keys

Expand string-or-pathname, a pathname or string, into a list of files matching the pattern in the argument. The call is equivalent to calling common-lisp:directory on string-or-pathname and signalling an error if it returns nil.

This function accepts the same keywords as common-lisp:directory and merely passes them onto directory.

See Pathname wildcard syntax in pathnames.htm for information on how wildcard characters (like * and ?) are handled.

For example:

cl-user(39): (excl.shell:glob "*.cl") 
(#p"/stuff1/acl/acl70/src/boot70.cl" #p"/stuff1/acl/acl70/src/foo.cl"
 #p"/stuff1/acl/acl70/src/eol.cl" #p"/stuff1/acl/acl70/src/npacl.cl"
 #p"/stuff1/acl/acl70/src/concat.cl" #p"/stuff1/acl/acl70/src/doit.cl"
 #p"/stuff1/acl/acl70/src/constants.cl" #p"/stuff1/acl/acl70/src/xx.cl"
 #p"/stuff1/acl/acl70/src/develenv.cl" #p"/stuff1/acl/acl70/src/gdbm.cl" ...)
cl-user(40): (mapcar #'enough-pathname (glob "*.cl"))
(#p"boot70.cl" #p"foo.cl" #p"eol.cl" #p"npacl.cl" #p"concat.cl" #p"doit.cl"
 #p"constants.cl" #p"xx.cl" #p"develenv.cl" #p"gdbm.cl" ...)
cl-user(41): 

-r

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and is readable. A filespec must be a string or pathname.

For example:

cl-user(42): (excl.shell:-r "makefile")
t
cl-user(43): (excl.shell:-r "this-file-does-not-exist")
nil
cl-user(44): 

-w

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and is writable. A filespec must be a string or pathname.

-x

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and is executable. filespec must be a string or pathname. On Windows this function always returns t if filespec exists.

-e

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists. A filespec must be a string or pathname.

-f

Function

Package: excl.shell

Arguments: filespe

Return a non-nil value if file filespec exists and is a regular file. filespec must be a string or pathname. Note, -f will return true if given an existing file that is not a directory, including, for example, a symbolic link. This may be unintuituve (you might expect -f to return true only when given an actual file), but it is how the test program works on UNIX.

-d

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and is a directory. filespec must be a string or pathname.

-l

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and is a symbolic link. filespec must be a string or pathname. Always returns nil on Windows.

-s

Function

Package: excl.shell

Arguments: filespec

Return a non-nil value if file filespec exists and has a size greater than zero. filespec must be a string or pathname.

touch

Function

Package: excl.shell

Arguments: filespec &key (if-does-not-exist :create) atime mtime time reference-file

Update the access and modification times of filespec to the current time. The various keyword arguments augment this behavior:

• if-does-not-exist: :create is the default value, and other values are :error or :ignore. :error causes touch to signal an error if the file does not already exist. :ignore causes touch to do nothing if the file does not already exist.
• atime: set the access time to the given universal time.
• mtime: set the modification time to the given universal time.
• time: set the access and modification times to the given universal time.
• reference-file: set the access and modification times to the corresponding times on the file given by the value of this keyword.

If a file does not exists and if-does-not-exist is :ignore, touch returns nil. Otherwise, touch returns t if successful, or signals an error.

rm

Function

Package: excl.shell

Arguments: filespec &key force recurse verbose no-execute

Remove filespec from the file system. By default it does not remove directories. The keyword arguments to rm are:

• force: ignore non-existent files
• recurse: remove the contents of the directory recursively
• verbose: explain what is being done as it happens
• no-execute: when true, do not execute any actions; just print information on what would be done.

If successful t is returned, otherwise an error is signaled.

cp

Function

Package: excl.shell

Arguments: from to &key make-links make-symbolic-links sync update to-execute force remove-destination ignore-files one-filesystem preserve-symbolic-links preserve-times preserve-ownership prune-directories recurse verbose

This function copies the file specified by from to a file specified by to. If from is a directory and the recurse keyword argument is is nil, or from and to are the same file, then an error is signaled. Other situations can also result in an error: they are described below. from and to should be objects accepted by pathname (such as pathnames and strings).

The large number of keyword arguments allows for support of the modes described next and support for fine tuning exactly how the copying is done. Simply copying one file to another can be done with specifying values for any keyword arguments.

This function has five modes:

1. link: this is the mode when the make-links keyword argument is given a non-nil value. In this mode, cp makes hard links instead of copying files. Windows does not support links, so this mode is not supported on Windows.
2. symlink: this is the mode when the make-symbolic-links keyword argument is given a non-nil value. In this mode, cp makes symbolic links instead of copying files. If the value is `:relative', then relative symlinks are created instead of absolute ones. Windows does not support links, so this mode is not supported on Windows.
3. update: this is the mode when the update keyword argument is given a non-nil value. In this mode, cp only copies a file when the source file (from) is newer than the destination file (to), or when the destination file is missing, or when the destination file is a different size.
4. sync: this is the mode when the sync keyword argument is given a non-nil value. This mode is like update mode, except when directories are involved delete files which are in the destination (to) but not in the source (from).
5. normal: this is the mode when the none of the above keywords (make-links, make-symbolic-links, update, and sync) have a non-nil value. Plain vanilla copying of files is done

It is an error if more than one of make-links, make-symbolic-links, update, and sync is non-nil.

The following options control how the copy is done, aside from the modes given above.

• force: when specified true, if an existing destination file (to) cannot be opened, remove it and try again.
• remove-destination: when specified true, remove each destination file before copying.
• ignore-files: if non-nil, the value should be a string which is a regular expression, or the keyword :backup, or a list of such strings with or without :backup. When non-nil, ignore files that match the provided regular expression, or any regular expression in the provided list of regular expressions. :backup is shorthand for a regular expression that matches GNU Emacs backup files (files that start with # or end with ~). ignore-files is only applied to files in subdirectories found when recursing.
• one-filesystem: when true, skip subdirectories that are on different filesystems from the one that the copy started on.
• preserve-symbolic-links: if non-nil, then symbolic links themselves are copied, instead of the files or directories they reference. If nil, symbolic links which point to directories will be followed, if the recurse keyword is given and non-nil. Windows does not support links, so this argument is not supported on Windows.
• preserve-times: when true, preserve file creation and modification times, if possible.
• preserve-ownership: when true, preserve file ownership (user and group, must be root).
• recurse: when true, descend into directories when copying.
• prune-directories: when given an appropriate non-nil, allow copy to abort descent into certain directories. The value of this keyword has the same meaning as the value of the keyword of the same name to the function map-over-directory.
• no-execute: when true, do not execute any actions, just print information on what would be done.
• verbose: when true, print information about what is being copied. The value can be a function which should expect arguments: what (which will be one of the following keywords: :skipping, :unlink, :creating-directories, :link, :symlink or :copy), file1, a filename, and an optional file2, a second filename. Specifying t is roughly equivalent to this functional value:

    (lambda (what file1 &optional file2)
      (if* file2
        then (format t ";; ~a ~a to ~a~%" what file1 file2)
        else (format t ";; ~a ~a~%" what file)))
  

  what will vary depending on the mode.

The type of the arguments (i.e. whether a file has a bearing on the operation in each mode. The meaningful combinations are given in the table below.

In the Comments column of the table the following notation is used:

• @ means a symbolic link is dereferenced (the file to which the symlink points to is what is used instead of the argument itself), and if the file is not a symbolic link then the @ can be ignored.
• a -> b means a is copied to b
• [n]: refer to note n after the table.

The first two sections of this table are the default behavior, without any keyword arguments (i.e., `normal' mode) specified non-nil. The first shows what happens when neither file argument is a link. The second (which applies to UNIX only since Windows does not support links) shows what happens when one or both of the file arguments is a link. The third section of the table represents non-default behavior using a non-nil value of preserve-symbolic-links.

In this second section, one or both of the file arguments is a link. Again, all keyword arguments are assumed to be nil. In particular preserve-symbolic-links is nil so links are dereferenced. Windows does not support links so this section of the table applies to UNIX only.

In this third section, one or both of the file arguments is a link. Again, all keyword arguments are assumed to be nil. In particular preserve-symbolic-links is t so links are not dereferenced. Windows does not support links so this section of the table applies to UNIX only.

Notes:

1. only an error in the default case, where remove-destination is nil.
2. destination file is overwritten if permissions and ownership allow, and if they do not an error is signalled -- if the force keyword value is non-nil, then the destination is forcibly removed before the copy.
3. similiar to 2 except the file to which the destination symbolic link points is the object of the operation.

cp return value

If successful the return value (an integer specifying a number of files) for each mode is given by the following table, otherwise an error is signaled:

mv

Function

Package: excl.shell

Arguments: from to &key force verbose no-execute dereference-source-directory

Move file from to to. If a directory is moved across filesystem boundaries, then it is copied and then the original is removed.

The keyword arguments are as follows:

• force: when non-nil, the destination is forcibly removed before the move.
• dereference-source-directory: when non-nil and the source directory is a symbolic link, then instead of moving the symbolic link, the file it points to is moved. A non-nil value is the same as a trailing slash on a source argument to GNU mv.
• no-execute: when non-nil, do not execute any actions, just print out what would be done.
• verbose: when true, print information about what is being copied. The value can be a function which should expect arguments: what (which will be one of the following keywords: :remove, :move, or :copy), file1, a filename, and an optional file2, a second filename. Specifying t is roughly equivalent to this functional value:

    (lambda (what file1 &optional file2)
      (if* file2
        then (format t ";; ~a ~a to ~a~%" what file1 file2)
        else (format t ";; ~a ~a~%" what file)))
  

  what will vary depending on whether a directory is being moved to another filesystem.

The type of the file arguments (from and to) has a bearing on the operation of mv. The meaningful combinations are given in the table below.

In the Comments column of the table the following notation is used:

• @ means a symbolic link is dereferenced (the file to which the symlink points to is what is used instead of the argument itself), and if the file is not a symbolic link then the @ can be ignored.
• a -> b means a is moved to b
• rm b means b is removed before the move.
• [n]: refer to note n after the table.

The first two sections of this table are the default behavior, without any keyword arguments specified non-nil. The first shows what happens when no directory specified as a value of the from is a link. The second shows what happens when a directory specified as a value of the from is a link and dereference-source-directory is nil. The third section of the table shows what happens when a directory specified as a value of the from is a link and dereference-source-directory is non-nil.

In this second section, the from is a linked directory and dereference-source-directory is nil.

In this third section, the from is a linked directory and dereference-source-directory is non-nil.

Notes:

[1] this operation leaves a dangling symbolic link. The GNU mv in fileutils version 4.1 has a bug, in our estimation. If you do this:

mkdir dir
ln -s dir link
mv link/ dir2

The trailing slash is the same as :dereference-source-directory t, according to the fileutils documentation. The above commands cause dir to be copied to dir2, and not the correct result, a symbolic link link that points to dir and dir renamed to dir2. The fileutils documentation says this:

    Warning: If you try to move a symlink that points to a directory,
    and you specify the symlink with a trailing slash, then `mv' doesn't
    move the symlink but instead moves the directory referenced by the
    symlink.

In other words, the fileutils documentation says that it will do the right thing, but it does not.

mv return value

If mv is successful, t is returned; otherwise an error is signalled.

ln

Function

Package: excl.shell

Arguments: from to &key symbolic no-dereference-destination force relative no-execute verbose

This function creates a link to the from with the name to. It works on UNIX only since Windows does not support symbolic links.

The keyword arguments are:

• symbolic: when true, create a symbolic instead of hard link.
• force: when true, remove existing destination files before linking.
• no-dereference-destination: if non-nil then treat to that is a symbolic link to a directory as if it were a normal file.
• relative: when true, do not do full merging on from and to so that a relative symbolic link can be created.
• no-execute: when true, do not execute any actions, just print out a description of what would be done.
• verbose: when true, print information about what is being linked. The value can be a function which should expect arguments: what (which will be one of the following keywords: :symlink or :link), file1, a filename, and an optional file2, a second filename. Specifying t is roughly equivalent to this functional value:

    (lambda (what file1 &optional file2)
      (if* file2
        then (format t ";; ~a ~a to ~a~%" what file1 file2)
        else (format t ";; ~a ~a~%" what file)))
  

  what will vary depending on whether symbolic or hard links are being created.

The type of the arguments has a bearing on the operation of ln. The meaningful combinations are given in the table below.

In the Comments column of the table the following notation is used: b -> a means a link is made from b to a.

The first two sections of this table are the default behavior, without any keyword arguments specified non-nil. The first shows what happens when no directory specified as a value of the to is a link. The second shows what happens when a directory specified as a value of the to is a link and no-dereference-destination is nil. The third section of the table shows what happens when a directory specified as a value of the to is a link and no-dereference-destination is non-nil.

The second section of the table shows what happens when a directory specified as a value of the to is a link and no-dereference-destination is nil.

The third section of the table shows what happens when a directory specified as a value of the to is a link and no-dereference-destination is non-nil (such as when --no-dereference is specified to ln).

Value returned by ln

If successful, ln returns t; otherwise an error is signaled.

die

Function

Package: excl.shell

Arguments: format-string &rest format-args

Print an error message and then exit Lisp. The error message is sent to a stream bound to stderr.

This function does not return, and the exit status of the process executing it will be 1.

shell-not-supported-condition

Class

Package: excl.shell

Any function which is not supported on a particular version of Lisp signals an error of this type. It is a subclass of `error'.

Copyright (c) 1998-2022, Franz Inc. Lafayette, CA., USA. All rights reserved.
This page was not revised from the 10.0 page.
Created 2019.8.20.