Go to the tutorial main page.
These other tutorials also deal with AllegroServe:
AllegroServe is the opensource webserver written by Franz Inc. You can download and read documentation on AllegroServe at http://opensource.franz.com/aserve/.
One of the main advantages to using AllegroServe is that it is possible to process forms directly in lisp rather than dispatching out to a shell or cgi script.
There are two methods by which a form can be submitted to a web server, the get method and the post method. Processing of "get" method forms is discussed in Tutorial on GET requests in AllegroServe.
Form data submitted via a post request can be passed in one of two ways. Data passed in the body of the request as a urlencoded query string is discussed in Tutorial on POST requests in AllegroServe.
This tutorial shows how to process forms data submitted via the multipart/form-data encoding type. To begin, we need to get an AllegroServe server up and running, with the following three preliminary steps. (Yhe list of packages used by our demo package is different from that in the related tutorials. Otherwise the procedure is the same.)
Start Allegro CL in the usual way. Once you have a prompt, enter the following form to load AllegroServe (it is not an error to enter that form if AllegroServe happens to already be loaded):
(require :aserve)
We define a package in which to define the code for this tutorial. The AllegroServe functionality we will be using is exported from the :net.aserve package, so our package will :use it. Since we will also be generating html, we must also use the :net.html.generator package. This package is defined by the htmlgen module, which is loaded as part of AllegroServe. Lastly, we'll be making some use of the URI module, so we'll include the :net.uri package as well. Evaluate the following forms to define our package, which we are calling :demo, and make it the current package:
(defpackage :demo (:use :excl :cl :net.aserve :net.html.generator :net.uri)) (in-package :demo)
Now let's get a server running. You will need to choose a port that is available and that the operating system will permit you to use. Web servers normally listen on port 80. On UNIX-like systems (macosx included) port 80 can only be allocated by the the superuser (called root). On windows any user can open port 80 as long as it's not yet allocated. In order to make this tutorial work on both Unix and Windows (and not require that you run as root on Unix), we'll put our web server on the localhost at port 8000. Evaluate the following form to start the server. (The function start is in the :net.aserve package. We do not need a package qualifier because we are in our :demo package which uses :net.aserve.
(start :port 8000)The webserver is now up and running, but we've provided it with no content to deliver to curious clients. If we visit http://localhost:8000/ right now, we'll get AllegroServe's standard "404 Error: Not found" response:
Not Found The request for http://localhost:8000/ was not found on this server. AllegroServe 1.2.43We'll get to making content available as we explore reasons for using "multipart/form-data" post requests.
The alternative encoding method for submitting form data involves urlencoding form data and either appending it to the request URI or adding it to the request body. urlencoding has various requirements as specified in RFC1738. For instance, the space character is replaced by the plus ('+') character. Further, all characters must have a representation in the US-ASCII character set. What if you want to transmit data in other character sets? What if you want to transmit binary data? While urlencoding can be considered insufficient for such data types, it can also be considered inefficient for large form data submissions that otherwise can be submitted via urlencoding.
The multipart/form-data encoding method submits form data in the body of the request as a MIME data stream, which are outlined in rfc2045.
AllegroServe provides an API for processing MIME data streams in the body of the request. It is worth noting that this tutorial is not intended to provide a comprehensive understanding of how to process MIME data, but rather to introduce the user to said API via an example illustrating a particular usage.
For this tutorial we'll create a page that could be part of a larger bug-tracking and patch submission framework. We'll present a form that allows a user to type in a large text message (like a bboard post) describing a patch. The form will also include two input fields for uploading files, such as diffs or screenshots. The submitted form will be echoed back to the user. For each attached file, we will adopt the following behavior:
;; large form submission. textarea and two file uploads.
(defun generate-patch-form (req ent)
(with-http-response (req ent)
(with-http-body (req ent)
(html
(:html (:head (:title "Patch Submission Form"))
(:body (:h2 "Patch Submission")
((:form :action "postpatch"
:method "post"
:enctype "multipart/form-data")
"Please enter a description of the patch." :br
((:textarea :name "msg" :rows "15" :cols "50"))
:br "Attachments:"
:br "File 1 "
((:input :type "file" :name "attach1"))
:br "File 2 "
((:input :type "file" :name "attach2"))
:br
((:input :type "submit" :name "submit")))))))))
(defparameter *known-form-items* '("msg" "attach1" "attach2"))
(defparameter *text-output-limit* 1024)
(defparameter *attachments* (make-array 10 :fill-pointer 0 :adjustable t))
;;; create a size length buffer and retrieve multipart body into it from
;;; request. if length is nil, then retrieve the entire sequence.
;;; returns multiple values: buffer containing sequence and bytes read.
(defun fetch-multipart-sequence (req &key (length nil) (format :binary))
(if* length
then (let ((buffer (make-array length :element-type (if (equal format :text)
'character
'(unsigned-byte 8))))
(start 0)
(end length))
(do* ((bytes-read start index)
(index start (get-multipart-sequence req buffer :start index :end end)))
((or (null index) (= index end)) (values buffer (or index bytes-read)))))
else (let ((buffer (get-all-multipart-data req :type format)))
(values buffer (length buffer)))))
(defun process-patch-form (req ent)
(with-http-response (req ent)
(with-http-body (req ent)
(html
(:html
(:head (:title "Patch Submission Result"))
(:body
(do ((header (get-multipart-header req) (get-multipart-header req)))
(nil)
(multiple-value-bind (type item-name filename content-type)
(parse-multipart-header header)
(when (equal type :eof) (return t)) ;; no more headers.
(when (member item-name *known-form-items* :test #'equal)
;; it's a form item we know about, handle it.
(case type
((:data)
(html (:p
(:b "This is what you typed in:")
:br
(:princ-safe (fetch-multipart-sequence req :format :text)))))
((:file)
(if* (equal content-type "text/plain")
then ;; display up to *text-output-limit* of text.
(multiple-value-bind (buf len)
(fetch-multipart-sequence req :length *text-output-limit* :format :text)
(html
(:p (:b "Text of " (:princ-safe filename)
" (up to " (:princ *text-output-limit*) " bytes)"))
(:pre
(:princ-safe (subseq buf 0 len)))))
else ;; save attachment and generate URL for retrieving file.
(let* ((mp-data (fetch-multipart-sequence req))
(index (vector-push-extend (cons content-type mp-data)
*attachments*))
(temp-uri (format nil "http://localhost:8000/attach?id=~a"
index))
)
(if* (equal content-type "image/jpeg")
then (html
(:p (:b "You uploaded the following image: ("
(:princ-safe filename) ")"))
((:img :src temp-uri)))
else (html
(:p (:b "You uploaded the following file: ")
((:a :href temp-uri) (:princ-safe filename))))))))))))))))))
(defparameter *response-method-not-allowed*
(net.aserve::make-resp 405 "Method Not Allowed"))
(push *response-method-not-allowed* net.aserve::*responses*)
(publish :path "/postpatch"
:content-type "text/html"
:function
#'(lambda (req ent)
(let ((request-type (request-method req)))
(case request-type
(:post ;; process submission and display
(process-patch-form req ent))
((:head :get) ;; generate form
(generate-patch-form req ent))
(t ;; we don't support this request method.
(with-http-response (req ent :response *response-method-not-allowed*)
(setf (reply-header-slot-value req :allow) "get, head, post")
(with-http-body (req ent)
;; empty body
))) ))))
(publish :path "/attach"
:format :binary
:function
#'(lambda (req ent)
(let* ((id (ignore-errors
(parse-integer (request-query-value "id" req :test #'equal))))
(data (and id (ignore-errors (aref *attachments* id)))) )
(if* data
then (with-http-response (req ent :content-type (car data))
(with-http-body (req ent)
(write-sequence (cdr data) *html-stream*)))
else ;; 404.
(with-http-response (req ent :response *response-not-found*)
(with-http-body (req ent)
(html
(:html (:head (:title "404 - NotFound"))
(:body
(:h1 "Not Found")
"The request for "
(:b (:princ-safe (render-uri (request-uri req) nil)))
" was not found on this server."
:br :br :hr
(:i "AllegroServe "
(:princ-safe net.aserve::*aserve-version-string*))))))) ))))
Evaluate the above code in your running image. Visit http://localhost:8000/postpatch to try out the page. Try attaching .txt files, .jpgs, and other file types. Browsers may use different methods for determining the content-type of files. A file may contain ASCII text but still be passed with content-type application/octet-stream.
Processing of "multipart/form-data" post requests is handled by four routines:From the AllegroServe documentation:
If you create a form with <form method="post" enctype="multipart/form-data"> then your url handler must do the following to retrieve the value of each field in the form:
- Call (get-multipart-header req) to return the MIME headers of the next field. If this returns nil then there are no more fields to retrieve. You'll likely want to call parse-multipart-header on the result of get-multipart-header in order to extract the important information from the header.
- Create a buffer and call (get-multipart-sequence req buffer) repeatedly to return the next chunk of data. When there is no more data to read for this field, get-multipart-sequence will return nil. If you're willing to store the whole multipart data item in a lisp object in memory you can call get-all-multipart-data instead to return the entire item in one Lisp object.
- Go back to step 1
It's important to retrieve all of the data sent with the form, even if that data is just ignored. This is because there may be another http request following this one and it's important to advance to the beginning of that request so that it is properly recognized.
process-patch-form contains the code implementing the above algorithm. It loops across all multipart-headers (step 1.) until there are none left. Each header is then parsed, and we decide which MIME part is worth handling. In this example, we only handle parts for form components that we know about, as specified in *known-form-items* (excluding the "submit" button). fetch-multipart-sequence handles the details of step 2 of the algorithm, calling get-all-multipart-data or get-multipart-sequence depending on if we want to limit the amount of the file read.
Inlining of jpeg images and providing links to other file types is handled by saving the file contents in a buffer and storing it in the *attachments* array. We publish an entity at
http://localhost:8000/attach
which accepts an id query parameter. This value is used to look up the content-type and data of the attachment and send it as the reply. If an invalid id argument is passed, a 404 response is generated.
This method of handling attachments has the advantage of not requiring this tutorial to write to the local filesystem. The drawback is that machines tend to have more disk space than address space and one could easily crash their server by uploading too many large files. Also, storing file contents in arrays means file size is limited to the maximum size of arrays, which tends to be less than the maximum file size allowed by the servers local filesystem.
The attachments stored in *attachments* will grow, but you can replace the array at any time to remove old stuff. Or, you could publish a function to allow you to clear all attachments by visiting a URL.
(publish :path "/clear" :content-type "text/html" :function #'(lambda (req ent) (setq *attachments* (make-array 10 :fill-pointer 0 :adjustable t)) (with-http-response (req ent) (with-http-body (req ent) (html (:html (:head (:title "Clear Attachments")) (:body "Done.")))))))
This completes the tutorial. Additional AllegroServe tutorials are listed at the top of this page.
The AllegroServe documentation is available in doc/aserve/aserve.html. The HTML generator is described in doc/aserve/htmlgen.html.
Allegro Webactions is a framework on top of AllegroServe for developing entire web sites using the Model-View-Controller paradigm. Among other things, this paradigm separates web page design tasks from dynamic content programming tasks. It is described in doc/using-webactions.html and doc/webactions.html.
Go to the tutorial main page.