Elastic Compute Cloud (EC2) Update and a useful script when using EC2

Late in 2014, the Allegro CL API to the Amazon Elastic Compute Cloud (EC2) was updated to use the Version 4 signing process for authenticating requests to AWS/EC2 ('AWS' is Amazon Web Services). This change is important because

  • Version 4 uses a more secure hashing algorithm (SHA256 vs SHA1).
  • New regions brought online after January 1, 2014 only support the Version 4 signing.

For those not familiar with the Allegro CL API for EC2, it is a handy way to manipulate or extract information from instances running on AWS. It is described in the document ec2.htm. That document has been updated to describe the behavior introduced with the 2014 patch. Note that the patch (and other changes) are for Allegro CL 9.0 only. Earlier releases, particularly Allegro CL 8.2 have a different API. This note does not apply to releases before Allegro CL 9.0.

In the remainder of this document, we present a Lisp program that can be used to inform about resources for which you will be charged if you use them. On UNIX platforms, you can run the program as a script. On Windows, you run it within Lisp. You have to edit the code so it refers to your situation.

Here is some sample output of the program (running it as a script on Linux):

$ checkec2.cl -v
;; Doing identity: amazon-ws...
;;   region: ap-southeast-1
;;   region: ap-southeast-2
;;   region: us-west-2
;;   region: us-west-1
;;   region: us-east-1
;;   region: eu-west-1
;;   region: ap-northeast-1
;;   region: sa-east-1
;;   region: eu-central-1
m1.large instance is running
  IDENTITY: amazon-ws
  instance: i-24eXXXXX
  hostname: ec2-11-222-33-10.compute-1.amazonaws.com
  state: running
  launched: Tuesday, November 04, 2014 08:33:14 AM 
  key-name: amazon-ws
  load: 1.17
  approximate cost: $0.16
     (does not include S3, EBS, etc)

;; Doing identity: amazon-devel-layer...
;;   region: ap-southeast-1
;;   region: ap-southeast-2
;;   region: us-west-2
;;   region: us-west-1
;;   region: us-east-1
;;   region: eu-west-1
;;   region: ap-northeast-1
;;   region: sa-east-1
;;   region: eu-central-1
;; Doing identity: amazon-sa...
;;   region: ap-southeast-1
;;   region: ap-southeast-2
;;   region: us-west-2
;;   region: us-west-1
;;   region: us-east-1
;;   region: eu-west-1
;;   region: ap-northeast-1
;;   region: sa-east-1
;;   region: eu-central-1
Excluded instance: #
                      reservation-id="r-9acXXXXX"
                      owner-id="403860000000"
                      identity=#
                      @ #x10001072492>
;; Doing identity: amazon-test...
;;   region: ap-southeast-1
;;   region: ap-southeast-2
;;   region: us-west-2
;;   region: us-west-1
;;   region: us-east-1
;;   region: eu-west-1
;;   region: ap-northeast-1
;;   region: sa-east-1
;;   region: eu-central-1
m3.xlarge instance is running
  IDENTITY: amazon-test
  instance: i-0ec00000
  hostname: ec2-11-22-144-12.compute-1.amazonaws.com
  state: running
  launched: Monday, November 03, 2014 09:34:08 AM 
  key-name: medical-dw-key
  approximate cost: $6.57
     (does not include S3, EBS, etc)

m3.2xlarge instance is running
  IDENTITY: amazon-test
  instance: i-02c11111
  hostname: ec2-54-172-0-13.compute-1.amazonaws.com
  state: running
  launched: Monday, November 03, 2014 09:37:04 AM 
  key-name: medical-dw-key
  approximate cost: $13.12
     (does not include S3, EBS, etc)
$ 

The -v argument prints the information. The -e argument sends an email (as directed by the function notify-by-email, which you must edit) containing the same information:

$ checkec2.cl -e
$ 

Here is the program. It is set up as a script for running on a UNIX platform (with argument -v for verbose output, -e for sending information in email, and no argument for non-verbose output. If you wish to run it on Windows, comment out the call to sys:with-command-line-arguments and uncomment the definition of the function run-aws-info-in-lisp, and call that function with optional argument "v" (for verbose printed output -- this is the default) or "e" for email output, or nil for non-verbose output. Be sure to modify the function notify-by-email appropriately and also make other changes indicated in comments. Also, create the file aws.cl with information about your EC2 usage.

#! /fi/cl/9.0/bin/linuxamd64.64/mlisp -#!

;; above line should be included only if you will run this
;; as a script in UNIX. Cannot be used this was on Windows.

(in-package :user)

(eval-when (compile eval load)
  (require :ssl)
  (require :ec2)
  (require :smtp)
  
  (use-package :net.ec2)

  (load "~/src/aws.cl")) ;; modify to show location of aws.cl file 
                         ;; (see below).

;; Modify the following function to indicate where email should be sent
(defun notify-by-email (body)
  (net.post-office:send-letter
   "localhost" "someone@somewhere.com" "someone@somewhere.com"
   body
   :subject "IMPORTANT: EC2 instances consuming your $$$"))

(defun string-to-keyword (string)
  (intern string (load-time-value (find-package :keyword))))

(defun calculate-cost (instance)
  (let ((hours (float (/ (- (get-universal-time)
			    (ec2-instance-launch-time instance))
			 3600))))
    ;; Return dollars
    (* hours
       (ecase (string-to-keyword (ec2-instance-instance-type instance))

         ;; from http://aws.amazon.com/ec2/pricing/
 
	 ;; Current generation
	 (:t2.micro    0.013)
	 (:t2.small    0.026)
	 (:t2.medium   0.052)
	 (:m3.medium   0.070)
	 (:m3.large    0.140)
	 (:m3.xlarge   0.280)
	 (:m3.2xlarge  0.560)
	 (:c3.large    0.105)
	 (:c3.xlarge   0.210)
	 (:c3.2xlarge  0.420)
	 (:c3.4xlarge  0.840)
	 (:c3.8xlarge  1.680)
	 (:g2.2xlarge  0.650)
	 (:r3.large    0.175)
	 (:r3.xlarge   0.350)
	 (:r3.2xlarge  0.750)
	 (:r3.4xlarge  1.400)
	 (:r3.8xlarge  2.800)
	 (:i2.xlarge   0.853)
	 (:i2.2xlarge  1.705)
	 (:i2.4xlarge  3.410)
	 (:i2.8xlarge  6.820)
	 (:hs1.8xlarge 4.600)
	 
	 ;; Older generations
	 (:t1.micro    0.02)
	 (:m1.small    0.08)
	 (:m1.large    0.32)
	 (:m1.xlarge   0.64)
	 (:m2.xlarge   0.24)
	 (:m2.2xlarge  0.90)
	 (:m2.4xlarge  1.80)
	 (:c1.medium   0.165)
	 (:c1.xlarge   0.66)
	 (:cc1.4xlarge 1.30)
	 (:cc1.8xlarge 2.40) ;; not listed in 2011-12-15 API 
	 (:cg1.4xlarge 2.10)
	 (:hi1.4xlarge 3.10)))))

(defparameter *exceptions*
    ;; these instances are known to be running and are exceptions.
    ;; This is an example. You instances will be different, if you
    ;; have any.
    '(("amazon-sa" ;; account
       "i-dc111111")))

(defun my-describe-instance (instance)
  (with-output-to-string (s)
    (format s "~a instance is ~a~%"
	    (ec2-instance-instance-type instance)
	    (ec2-instance-state-name instance))
    (format s "  IDENTITY: ~a~%"
	    (ec2-identity-keypair-name (ec2-instance-identity instance)))
    (format s "  instance: ~a~%" (ec2-instance-id instance))
    (format s "  hostname: ~a~%" (ec2-instance-dns-name instance))
    (format s "  state: ~a~%" (ec2-instance-state-name instance))
    (format s "  launched: ~:@/locale-format-time/~%"
	    (ec2-instance-launch-time instance))
    (format s "  key-name: ~a~%"
	    (ec2-instance-key-name instance))
    (let ((load (ignore-errors ;; depends on the state of the instance
		 (query-load instance))))
      (when load (format s "  load: ~a~%" load)))
    (format s "  approximate cost: $~,2f~%" (calculate-cost instance))
    (format s "     (does not include S3, EBS, etc)~%")
    (format s "~%")))

;; The following form is executed when running as a script on UNIX.
;; If you are running as a Lisp program, comment this form out and
;; uncomment the definition of RUN-AWS-INFO-IN-LISP below.
;;
(sys:with-command-line-arguments ("ve" verbose email) (rest)
  (declare (ignore rest))
  (dolist (identity *identities*)
    (when verbose
      (format t ";; Doing identity: ~a...~%"
	      (ec2-identity-keypair-name identity)))
    (let ((exceptions (cdr (assoc (ec2-identity-keypair-name identity)
				  *exceptions*
				  :test #'string=)))
	  instances temp)
      (dolist (region (describe-regions :identity identity))
	(when verbose
	  (format t ";;   region: ~a~%" (ec2-region-name region)))
	(when (setq temp
		(describe-instances
		 :identity (copy-ec2-identity identity :region region)))
	  (push temp instances)))
      (when instances
	(setq instances (apply #'append instances))
	(dolist (instance instances)
	  (if* (member (ec2-instance-id instance) exceptions
		       :test #'string=)
	     then (when verbose
		    (format t "Excluded instance: ~a~%" instance))
	     else (let ((body (my-describe-instance instance)))
		    (if* email
		       then (notify-by-email body)
		       else (format t "~a" body)))))))))

#!
;; Comment out the form above and uncomment this definition
;; if you wish to run within Lisp. The call in Lisp is
;; (run-aws-info-in-lisp "v") for printed verbose output
;; (run-aws-info-in-lisp) same as (run-aws-info-in-lisp "v")
;; (run-aws-info-in-lisp "e") for emailed output
;; (run-aws-info-in-lisp nil) for non-verbose output
;;
(defun run-aws-info-in-lisp (&optional (mode "v"))
  (dolist (identity *identities*)
    (when (string-equal mode "v")
      (format t ";; Doing identity: ~a...~%"
	      (ec2-identity-keypair-name identity)))
    (let ((exceptions (cdr (assoc (ec2-identity-keypair-name identity)
				  *exceptions*
				  :test #'string=)))
	  instances temp)
      (dolist (region (describe-regions :identity identity))
	(when (string-equal mode "v")
	  (format t ";;   region: ~a~%" (ec2-region-name region)))
	(when (setq temp
		(describe-instances
		 :identity (copy-ec2-identity identity :region region)))
	  (push temp instances)))
      (when instances
	(setq instances (apply #'append instances))
	(dolist (instance instances)
	  (if* (member (ec2-instance-id instance) exceptions
		       :test #'string=)
	     then (when (string-equal mode "v")
		    (format t "Excluded instance: ~a~%" instance))
	     else (let ((body (my-describe-instance instance)))
		    (if* (string-equal mode "e")
		       then (notify-by-email body)
		       else (format t "~a" body)))))))))

The file ~/src/aws.cl (loaded near the top of the script/file) defines *identities*, which contain the access key and secret access key for each identity that you want the script to check for running instances. You could create the value for this variable like so:

(setq *identities*
 (list
  (make-instance 'ec2-identity
    :ssh-user "ec2-user"
    :keypair-name "amazon-test"
    :ssh-identity-file "~/.ssh/amazon-test.pem"
    :region *default-region*
    :access-key "...your access key..."
    :secret-access-key "... your secret access key..."
    :account-number "... your account number..."
    :certificate "...the path to your public key..."
    :private-key "... the path to your private key...")
  ...))
Copyright © 2017 Franz Inc., All Rights Reserved | Privacy Statement Twitter Google+