Allegroserve, Webactions and Ajax

The interface to the web is rapidly evolving. Initially the user clicked on a link or button and waited while the browser sent a request to the server which then returned a whole new page to display. Next, frames were introduced so that clicks could affect only part of the screen. Dynamic html was then introduced. It uses client-side scripting to react to user input (keyboard and mouse) and modify what the user was seeing without even a message being sent to the server. Now we have Ajax which adds to dynamic html the ability for the client to send background messages to the server and have responses trigger changes to the user's screen. Ajax adds what was missing in dynamic html: the ability to harness the considerable processing power of the server as compared to running interpreted javascript on the client.

Ajax is an acronym for Asynchronous Javascript And Xml. Our example will make use of XML but mainly for illustrative purposes. In many situations you can avoid the overhead of XML.

The main idea behind Ajax is that user actions in the browser cause javascript functions to be run. These functions can send requests to web servers asynchronously, meaning that the requests are done in the background. The user does not have to sit and wait for a response and indeed often has no idea the requests are even being made. When and if the web server responds a javascript function is run to handle the results. That javascript function will more than likely modify the page the user is viewing. To the user the experience is more similar to using a standard windowized application running on his machine than to using a web browser.

This note is not a tutorial on Javascript programming nor on Ajax. We just want to demonstrate how easy it is to add an Ajax component to an AllegroServe Webactions site. Hopefully this will inspire you to try this on your web site. Please feel free to use the Javascript code we have provided to get you started.

This note describes a simple web application that uses Ajax. This application is a tool to improve your memory. A good way to learn facts is to quiz yourself repeatedly until you pass the quiz. It is the result of repeatedly thinking about the associations of objects that moves that association from your short term to your long term memory. Once the association has made it into your long term memory you have learned it.

The sample quizzes we provide with this example will help you learn the capital cities of the US states. You can also run the quiz in reverse and learn the states associated with the capital cities. We have divided the 50 states randomly into five groups of 10 states each. By learning 10 states at a time you get a chance to reinforce each association much more often than if you tried to learn all 50 states at once. If you wait too long between reinforcments the association will not make it into long term memory.

You may wonder if using Ajax is critical to this application and the answer is no. It would have been simpler to write this this using the old fashioned request/response paradigm but this is why we wrote it using Ajax:

  1. We want to demonstrate the principles of Ajax programming
  2. This example is small enough to be understood by the reader without long hours of study.

We are using the Webactions framework in the quiz web application. This isn't necessary for a single user version but we want the quiz server to serve any number of users at the same time.

How the quiz application works

The quiz application works as follows:

  1. When the user connects to the quiz server he's offered a list of the quizzes currently loaded into the server.
  2. He selects any quiz (or its reverse where he must quess the question given the answer).
  3. The page then displays the first question.
  4. The user should then think or say out loud his answer and then click on any key (but usually the space key) to reveal the actual answer.
  5. Now the user must grade himself. If he got the answer right he presses the space key. If he got the answer wrong he presses some letter key.
  6. The running tally of correct and incorrect results is updated and a new question is posed.
  7. Eventually the test is over and the user must select a new quiz (or the same quiz) to continue.

The files in this project

In the bulleted list, the filename is linked to the discussion below (where the whole file is reproduced interspersed with commentary). The link in the next paragraph will download a zip file containing all three files.

  • quiz.cl - the Lisp code which drives the quiz.
  • quiz.clp - the html sent to the browser after processing by the webactions clp processor.
  • states.cl - the definitions of the state capital quizzes.

Click here to download a zip file containing the three application files.

The file quiz.cl part 1

We discuss part of quiz.cl here, then discuss quiz.clp, and return to quiz.cl after. quiz.cl begins with a definition of the webaction project.

The project's map has only three entries:

  1. "quiz" - show the quiz page
  2. "start-quiz" - starts a new quiz and then shows the quiz page
  3. "keydown" - this is the Ajax callback into the web server. This called asynchronously by the client after each key is pressed when the mouse is over the window showing the quiz. It is key presses that move the quiz forward.
(eval-when (compile load eval)
  (require :aserve)
  (require :webactions))


(defpackage :user (:use :net.aserve :net.html.generator))
(in-package :user)

(webaction-project "quiz"
		   :project-prefix "/"
		   :index "quiz"
		   :destination ""
		   :map '(("quiz" "quiz.clp")
			  ("start-quiz" action-start-quiz "quiz.clp")
			  ("keydown" action-key-down)))


The map references two action functions and we'll explain what they do later after we establish some context.

While the application is running there is just html file being shown in the browser. That html file is generated from the following quiz.clp file (we return to our discussion of quiz.cl below):

The file quiz.clp

<html>
<head>
<title>Quiz using Ajax</title>

<script language='JavaScript' type='text/javascript'>

function createRequest() {
  var request

  try {
    request = new XMLHttpRequest();
  } catch (trymicrosoft) {
    try {
      request = new ActiveXObject('Msxml2.XMLHTTP');
    } catch (othermicrosoft) {
      try {
        request = new ActiveXObject('Microsoft.XMLHTTP');
      } catch (failed) {
        request = false;
      }
    }
  }

  if (!request)
    alert('failed to create XMLHttpRequest');

  return request
}

var myreq = false;

function keyisdown(code) {
  // key with code pressed.. send request to server
  if (code >= 32) {
    var  myreq = createRequest()  
    myreq.open('GET', "/keydown?key=" + code, true)
    myreq.onreadystatechange = function () {updateform(myreq)}
    myreq.send(null);
 }
}

function updateform(myreq) {
   if ((myreq.readyState == 4) && (myreq.status == 200)) {

      
      var root = myreq.responseXML.documentElement;
      // fill in the form based on return value
    
      document.getElementById('question').innerHTML =
        root.getElementsByTagName('question')[0].firstChild.data ;

      document.getElementById('answer').innerHTML =
        root.getElementsByTagName('answer')[0].firstChild.data ;

      document.getElementById('correct').innerHTML =
        root.getElementsByTagName('correct')[0].firstChild.data ;

      document.getElementById('incorrect').innerHTML =
        root.getElementsByTagName('incorrect')[0].firstChild.data ;

    }
}
            

</script>

</head>
<body onKeyDown="keyisdown(event.keyCode)"  onLoad="keyisdown(32)">


<clp_ifdef name="quizready" session>
<table>
<tr>
<td><b>Question:</b>:  </td><td><span id="question"> </span></td>
</tr>
<tr>
<td><b>Answer:</b></td><td><span id="answer"> </span></td>
</tr>
</table>
</clp_ifdef>

<clp_ifndef name="quizready" session>
<h2>Pick A Quiz</h2>
</clp_ifndef>

<clp_ifdef name="quizready" session>
<br><br>
Correct: <span id="correct"></span>,  Incorrect: <span id="incorrect"></span>
</clp_ifdef>


<hr>

<quiz_quizes/>

</body>
</html>

Let's look at quiz.clp in detail. Most of this file is sent directly to the browser as html.

In the <head> of the html page are the javascript function definitions. The createRequest function definition is one that you'll want to include in all your Ajax enabled pages. There are three different ways to create the javascript object you'll need to send a message to the server and this function tries each way until it find a way that works.

The keyisdown function is called whenever a key is pressed while the mouse is over the application's window. This is because of the way we specified the <body> element

    <body onKeyDown="keyisdown(event.keyCode)"  onLoad="keyisdown(32)">

The keyisdown function looks for a significant key being pressed and when seen it notifies the server by sending asynchronously a GET http request to the /keywdown url. The keyisdown function returns immediately after the request is queued up to be sent to the server, keyisdown doesn't wait around for the response. However the request object is set so that the function updateform will be called as the request is processed.

This line

    myreq.onreadystatechange = function () {updateform(myreq)}

creates javascript's equivalent of a a closure function in Lisp. The function created is anonymous and when invoked will call the updateform function passing it in the request object allocated in this call to keyisdown.

This closure is called whenever the 'readyState' of of the request objects changes, which it does as the request it represents is processed by the browser.

For our application all we care about is when the request has finished processing which is indicated by the readyState having the value 4. We also only care about successful responses from the web server which is indicated by the status return value of 200. When we have a finished successful response we are ready to process the response from the web server.

The server can send either a string or an xml form back to the client. The server indicates what it is sending back by the content-type of the response. If the content type is "text/xml" or "application/xml" the the response must be a well formed xml expression.

If the server wants to send back a single string then the server sets to content type to "text/plain" or even "text/html" and the client retrieves that response from the request object using myreq.responseText.

In our case we want to send back four pieces of information so we decided to use xml to show how this can be done. We could have sent back a string containing the four pieces of information separated by special field separator characters and then on the client side broken the string up using javascript programming or even regular expression parsing. We chose to use xml just to show how this done as there will be situations where you really do want to return complex structured data.

When the content-type indicates an xml expression is being returned the client side will parse the xml and put the single topmost xml element in the slot of the request object (myreq in this case):

  myreq.responseXML.documentElement

Now you can use dom functions to search the xml structure to find elements with a given name and for such elements find the data between the tags.

After setting root the top of the xml structure returned

        root = myreq.responseXML.documentElement;

we can find the data associated with the first (or only) <question> element using

        root.getElementsByTagName('question')[0].firstChild.data 

Once we have the data from the server response we have to find a place to put it. The span and div html elements are very useful here. You'll see in our table we've defined a span with an id of "question".

    <td><b>Question:</b>:  </td><td><span id="question"> </span></td>

the value between the <span> and </span> and be read or written using this form.

        document.getElementById('question').innerHTML

Thus to extract the value from between the <question> and </question> in the xml response from the server and to place it into the table on the page visible to the user we write:

      document.getElementById('question').innerHTML =
        root.getElementsByTagName('question')[0].firstChild.data ;

It is not required that the span be named 'question' just because the xml element is named 'question'.

Looking past the head of the html we find this definition of the body:

<body onKeyDown="keyisdown(event.keyCode)"  onLoad="keyisdown(32)">

This is where we link keyboard actions to the keyisdown function we wrote. Also when this page is first loaded we pretend that the user pressed the space key. What this will do is cause the first question to be shown when the user selects a quiz to take.

Near the bottom of quiz.clp we find this form

     <quiz_quizes/>

This invokes a user defined clp function which will show all the quizzes currently defined.

The file quiz.cl part 2

Now let's return to quiz.cl.

The following code defines the data structures we'll use to hold a quiz while we ask the questions. Also we create a hash table holding all the quizzes loaded into the server

; We can define any number of quizzes, they are all put 
; into the *quizes* hash table.  This table maps quiz number
; to quiz name and quiz name to quiz questions
;
(defvar *quizes* (make-hash-table :test #'equal))
(defvar *quiz-counter* 0)


(defstruct quiz 
  ;; when a quiz is being run this is stored in the session
  ;; object associated with the user this tracks the users
  ;; progress through the quiz
  ;;
  qarray        ; vector of qelts for each question
  (lasti 0)     ; index of last question asked
  (pass 0)	; number of questions answered correctly
  (fail 0)	; number of questions answered incorrectly
  current       ; most recent (question answer)
  (state :get-show-question)
  )

(defstruct qelt
  ;; one quiz question
  count  ; number of times to ask this question
  q  ; the (question answer)
  )

(defun def-quiz (name questions)
  (if* (null (gethash name *quizes*))
     then (setf (gethash (incf *quiz-counter*) *quizes*) name))
  
  (setf (gethash name *quizes*) questions))

The is the action function called when the user selects a quiz. We create the quiz object and store it in the session object so this quiz becomes private to this user.

(defun action-start-quiz (req ent)
  (declare (ignore ent))
  ;;
  ;; user click on link specifying which quiz to run
  ;;
  (let ((ws (websession-from-req req))
	(qnumber (request-query-value "q" req))
	(reverse (request-query-value "r" req)))
    
    (let ((qname (gethash (parse-integer qnumber) *quizes*)))
      (if* qname
	 then (setf (websession-variable ws "quiz") 
		(create-quiz (gethash qname *quizes*)  3 reverse))))
    
    (setf (websession-variable ws "quizready") t)
    
    :continue))

This is the function that's invoked asychronously by the web server. A key pressed advances us through the quiz and at one stage reports whether the user got the question correct or not.

The response from this function is an xml expression. Note that an xml expression has exactly one top level element. It doesn't matter what the name of that element is (as long as it doesn't match the names of any of the elements whose content you do care about).

(defun action-key-down (req ent)
  ;; this is the ajax called function.. we return xml
  
  (let* ((key 
	  (code-char (parse-integer (request-query-value "key" req))))
	 (ws (websession-from-req req))
	 (quiz (and ws (websession-variable ws "quiz")))
	
	 (question " ")
	 (answer "  ")
	 
	 )
    
    (if* quiz
       then ;
	    (case (quiz-state quiz)
	      ((:get-show-question
		:get-pass-fail)
	       (if* (eq (quiz-state quiz) :get-pass-fail)
		  then ; user's told us if he got the question
		       ; right (space key) or wrong (other key)
		       (if* (eq key #\space)
			  then (incf (quiz-pass quiz))
			  else
			       (incf (quiz-fail quiz))
			       ; ask  question again
			       (incf (qelt-count
				      (svref
				       (quiz-qarray
					quiz)
				       (quiz-lasti quiz))))))
	      

	       ; find next question to ask
	       
	       (let ((q (next-q quiz)))
		 (setf (quiz-current quiz) q)
		 (if* (null q)
		    then (setq question " -- Test Over --")
			 (setf (quiz-state quiz) :test-over)
		    else (setq question (car q))
			 (setf (quiz-state quiz)
			   :show-answer))))
	      
	      (:test-over
	       (setq question " -- Test Over --"))
	      
	      (:show-answer
	       (setq question (car (quiz-current quiz)))
	       (setq answer   (cadr (quiz-current quiz)))
	       
	       (setf (quiz-state quiz) :get-pass-fail))))
    
    
    (with-http-response (req ent :content-type "text/xml")
      (with-http-body (req ent)
	;; our response is a simple xml expression
	(html "<document><question>" (:princ-safe question)
	      "</question><answer>"  (:princ-safe answer)
	      "</answer><correct>"   (:princ-safe (and quiz (quiz-pass quiz)))
	      "</correct><incorrect>" (:princ-safe (and quiz (quiz-fail quiz)))
	      "</incorrect></document>")))
    
    nil  ; indicate to webaction that a response was sent
    ))

These functions support the application. One creates a quiz and the other selects the next question to ask in the quiz

(defun create-quiz (qs count reverse)
  ;; create a quiz asking the given questions 'qs' count numbers
  ;; times.  Possibly reverse the questions and answers
  (let ((arr (make-array (length qs))))
    (do ((qq qs (cdr qq))
	 (i 0 (1+ i)))
	((null qq))
      (setf (svref arr i) 
	(make-qelt :count count 
		   :q (if* reverse 
			 then (reverse (car qq))
			 else (car qq)))))
    
    (make-quiz :qarray arr :state :get-show-question)))


		   
(defun next-q (quiz)
  ;; compute the next question to ask.
  ;; find a question that has a positive question count
  ;; and try not to ask the same question twice in a row
  ;;
  (let ((next-i (mod (+ (quiz-lasti quiz)
			(random (length (quiz-qarray quiz))))
		     (length (quiz-qarray quiz)))))
    (let (looped)
      (do ((i (mod (1+ next-i) (length (quiz-qarray quiz)))
	      (mod (1+ i) (length (quiz-qarray quiz)))))
	  ((if* (eq i next-i)
	      then (if* looped
		      then t
		      else (setq looped t)
			   nil))
	   nil)
      
	(let ((ent (svref (quiz-qarray quiz) i)))
	  (if* (> (qelt-count ent) 0)
	     then (if* (and (eq i (quiz-lasti quiz))
			    (not looped))
		     then ; don't duplicate unless necessary
			  (setq looped t)
		     else 
			  (decf (qelt-count ent))
			  (setf (quiz-lasti quiz) i)
			  (return (values (qelt-q ent) 
					  i)))))))))		   

Finally we have the clp function which creates the html which contain the links to all the quizzes loaded into the server:

(def-clp-function quiz_quizes (req ent args body)
  ;;
  ;; display a link for every quiz loaded, and one for the reverse quiz
  ;;
  (declare (ignore body args))
  (let ((wa (webaction-from-ent ent))
	(ws (websession-from-req req))
	(tests))
    
    ; compute list of tests and sort by test number 
    (maphash #'(lambda (k value)
		 (if* (integerp k) then (push (cons k value) tests)))
	     *quizes*)
    
    (setq tests (sort tests #'(lambda (a b) (< (car a) (car b)))))
    
    (dolist (test tests)
    
      (html ((:a :href (locate-action-path
			wa
			(format nil "start-quiz?q=~a" 
				(car test))
			ws))
	     (:princ-safe (cdr test)))
			     
	    " -- "
	    ((:a :href (locate-action-path
			wa
			(format nil "start-quiz?q=~a&r=1" 
				(car test))
			ws))
	     "Reverse")
	    :br
	    :newline))))

The file states.cl

Our sample quizzes are defined this way. We've randomized the list of states and we create five quizzes each with a set of 10 states. Finally we define a quiz of all the states which you can use after you've passed the first five quizzes to verify that you know know all the information.

(defvar *state-capitals* 
  '(("Connecticut" "Hartford") 
    ("Nevada" "Carson City") 
    ("Kansas" "Topeka") 
    ("Idaho" "Boise") 
    ("Pennsylvania" "Harrisburg") 
    ("Kentucky" "Frankfort") 
    ("New York" "Albany") 
    ("South Carolina" "Columbia") 
    ("Missouri" "Jefferson City") 
    ("Iowa" "Des Moines")

    ("Massachusetts" "Boston") 
    ("New Mexico" "Santa Fe") 
    ("Minnesota" "St. Paul") 
    ("Utah" "Salt Lake City") 
    ("Michigan" "Lansing") 
    ("Virginia" "Richmond") 
    ("Oregon" "Salem") 
    ("Montana" "Helena") 
    ("Mississippi" "Jackson") 
    ("North Carolina" "Raleigh") 

    ("Hawaii" "Honolulu") 
    ("Washington" "Olympia") 
    ("Nebraska" "Lincoln") 
    ("Alabama" "Montgomery") 
    ("Alaska" "Juneau") 
    ("Wyoming" "Cheyenne") 
    ("Wisconsin" "Madison") 
    ("Indiana" "Indianapolis") 
    ("North Dakota" "Bismarck") 
    ("Florida" "Tallahassee") 

    ("New Hampshire" "Concord") 
    ("South Dakota" "Pierre") 
    ("Vermont" "Montpelier") 
    ("Maryland" "Annapolis") 
    ("Arizona" "Phoenix") 
    ("New Jersey" "Trenton") 
    ("Colorado" "Denver") 
    ("Louisiana" "Baton Rouge") 
    ("California" "Sacramento") 
    ("Oklahoma" "Oklahoma City") 

    ("Illinois" "Springfield") 
    ("Maine" "Augusta") 
    ("Rhode Island" "Providence") 
    ("Arkansas" "Little Rock") 
    ("West Virginia" "Charleston") 
    ("Delaware" "Dover") 
    ("Ohio" "Columbus") 
    ("Tennessee" "Nashville") 
    ("Georgia" "Atlanta") 
    ("Texas" "Austin")))


(def-quiz "State Capitals 1" (subseq *state-capitals* 0 10))
(def-quiz "State Capitals 2" (subseq *state-capitals* 10 20))
(def-quiz "State Capitals 3" (subseq *state-capitals* 20 30))
(def-quiz "State Capitals 4" (subseq *state-capitals* 30 50))
(def-quiz "State Capitals 5" (subseq *state-capitals* 40 50))

(def-quiz "All State Capitals" *state-capitals*)

Running the application

To run this application yourself start Lisp, and also make the directory containing quiz.cl and quiz.clp the current directory. Then:

  1. Compile and load quiz.cl -- :cl quiz.cl
  2. Load states.cl -- :ld states.cl
  3. Finally start the web server, by, for example:
    (setq quizserver (net.aserve:start :port 8000))
    

Now you can go to a browser and visit the url (replacing my_machine_name with your actual machine name, of course):

http://my_machine_name:8000/
You should see the quiz window come up. Click on one of the quizzes and the quiz will begin.

The Question will be a state (or a city, if playing the Reverse quiz). Think of the answer and press the space key to see the answer. Then press the space key if you were right or any letter key if you were wrong, and the Correct/Incorrect totals will be incremented appropriated. The Quiz cycles (asking its questions from the beginning after all questions have been asked). When you have finished one quiz, click on another.

When done, you can close the browser window and shutdown the server with net.aserve:server:

(net.aserve:shutdown)

No argument is needed if no other server has been starter; if one has do

(net.aserve:shutdown :server quizserver)
Copyright © 2014 Franz Inc., All Rights Reserved | Privacy Statement
Delicious Google Buzz Twitter Google+