simple-stream-read-line

Arguments: stream eof-error-p eof-value &optional given-string

This function implements read-line for simple-streams. In addition, it allows an optional string as a buffer into which to store characters being read. Unless an EOF is encountered immediately, in which case an error is signaled or eof-value is returned, the first return value is a string. If the given-string is supplied but is too small to hold the whole line, a new string of the exact size needed is allocated and returned. (If a new string is allocated, the contents of a provided given-string when this function returns are undefined -- they will likely have been changed in anticipation of the string being used.)

Two values are always returned (following the behavior of read-line) and a third value is sometimes returned. The first two return values are:

1. The result string, or the eof-value. If an EOF is not encountered immediately, and if the given-string is supplied but is not large enough, this value will be a newly allocated string large enough to hold the entire line exactly.
2. A flag indicating why reading stopped: nil, if the line was terminated with a newline; and t if the line was not terminated with a newline (that is, the end of file was reached).

A third value is returned when given-string is large enough to hold the line read. (No third value is returned when given-string is not supplied or when given-string is too small). The third value is:

3. The number of characters read and stored in given-string, (which does not include a newline if read).

Since you do not typically know in advance whether the given-string or a new string is returned, the number of characters read (not counting the newline) is the value of

  (or end (length string))
  ;; END is the third return value, STRING the first

given-string may be simple or non-simple, but must be a string. The other arguments are as for read-line (note there is no recursive-p argument and stream, eof-error-p, and eof-value are required, not optional).

The motivation for this function is twofold: to provide a direct implementation for read-line for simple-streams, and to provide a tradeoff for consing between that of read-line and that of excl:read-line-into (excl:read-line-into does no consing but requires more bookkeeping). If the supplied string is large enough, then most of the time consing will not occur, but for the occasional larger-than-expected line, the allocation of a string is automatic.

See also read-line-into. That function is more space efficient than this one, but has a more complicated interface.

Examples

;; Assume the file myfile.txt contains (no trailing spaces):
12345

12345678901234567890
123456789
1234567890
123456789012345

;; That is 6 lines or lengths 5, 0, 20, 9, 10, 15, 0, not counting
;; the newlines.
;;
;; We read the file one line at a time:
(with-open-file (f "myfile.txt" :direction :input)
  (let (res stopped end
	(given-string (make-string 10 :initial-element #\a)))
    (loop
      (multiple-value-setq (res stopped end)
	(simple-stream-read-line f nil f given-string))
      (when (eq res f) (return))
      (format t " res is ~S, eq to given-string ~S~% ~
                     length ~D, given-string is ~S~%"
	      res (eq res given-string) (or end (length res)) given-string))))
 res is "12345aaaaa", eq to given-string t
 length 5, given-string is "12345aaaaa"
 res is "12345aaaaa", eq to given-string t
 length 0, given-string is "12345aaaaa"
 res is "12345678901234567890", eq to given-string nil
 length 20, given-string is "1234567890"  ;; actual value may be different
 res is "1234567890", eq to given-string t
 length 9, given-string is "1234567890"   ;; given-string is used
 res is "1234567890", eq to given-string nil
 length 10, given-string is "1234567890"  ;; given-string is not used
                                       ;; see below
 res is "123456789012345", eq to given-string nil
 length 15, given-string is "1234567890"
nil
;;  One trick is, when a new string is allocated, use that string 
;;  as the given-string from then on. (It already exists, so no 
;;  extra allocation and it is bigger so more likely not to overflow.)
;;  Here is the same example with using any newly allocated string:
(with-open-file (f "myfile.txt" :direction :input)
  (let (res stopped end
	(given-string (make-string 10 :initial-element #\a)))
    (loop
      (multiple-value-setq (res stopped end)
	(simple-stream-read-line f nil f given-string))
      (when (eq res f) (return))
      (format t " res is ~S, eq to given-string ~S~% ~
                     length ~D, given-string is ~S~%"
	      res (eq res given-string) (or end (length res)) given-string)
      (unless end (setq given-string res))
      )))
 res is "12345aaaaa", eq to given-string t
 length 5, given-string is "12345aaaaa"
 res is "12345aaaaa", eq to given-string t
 length 0, given-string is "12345aaaaa"
 res is "12345678901234567890", eq to given-string nil
 length 20, given-string is "1234567890"  ;; actual value may be different
 res is "12345678901234567890", eq to given-string t
 length 9, given-string is "12345678901234567890" ;; Note: given-string
                                              ;; is now last returned string
 res is "12345678901234567890", eq to given-string t
 length 10, given-string is "12345678901234567890"
 res is "12345678901234567890", eq to given-string t
 length 15, given-string is "12345678901234567890"
nil

How large must the given-string be?

The examples above include a case where the line has 10 characters, and the given-string is length 10 but it is not used. Instead, a new string of length 10 is allocated. Why is the given-string insufficient? The problem is that the most efficient time to allocate a new string when necessary is to do it when the given-string is full and there is known to be at least one more character to read, but before it is known what that character is. If that additional character turns out to be a newline, then in fact the given-string was big enough but by this time the new string is already allocated.

Therefore, given-string must be larger than the anticipated maximum line length in order that it always be used. (For common external formats, it must be one larger. There may be external formats where more that one extra character is needed.)

This need for a longer given-string than the maximal line length (not counting the newline) is suboptimal behavior, and may be replaced in a future release if an efficient implementation can be discovered. However, as currently implemented, too much extra time would have to be spent in order to determine whether a line is or is not exactly as big as the given-string.

Copyright (c) 1998-2022, Franz Inc. Lafayette, CA., USA. All rights reserved.
This page was not revised from the 10.0 page.
Created 2019.8.20.