Introduction

The Lisp client's run-sparql function can accept a pre-built query plan, a string in the SPARQL query syntax, or a Lisp s-expression that encodes the query. The s-expression can be created via a call to parse-sparql or by hand. This document describes the s-expression format.

Note that the s-expression syntax is stable but is still subject to change as the query engine it supports improves and the SPARQL language continues to mature. Franz intends to provide backward compatibility for all existing features.

Example

As an example, the s-expression for query 8 from from the SP2 benchmark: 

SELECT DISTINCT ?name  
WHERE {  
  ?erdoes rdf:type foaf:Person .  
  ?erdoes foaf:name 'Paul Erdoes'^^xsd:string .  
  {  
    ?document dc:creator ?erdoes .  
    ?document dc:creator ?author .  
    ?document2 dc:creator ?author .  
    ?document2 dc:creator ?author2 .  
    ?author2 foaf:name ?name  
    FILTER (?author!=?erdoes &&  
            ?document2!=?document &&  
            ?author2!=?erdoes &&  
            ?author2!=?author)  
  } UNION {  
    ?document dc:creator ?erdoes.  
    ?document dc:creator ?author.  
    ?author foaf:name ?name  
    FILTER (?author!=?erdoes)  
  }  
}

is 

(sparql.parser:sparql :select :vars (?name) :where  
   (:join  
    (:join nil  
	   (:bgp #(?erdoes !rdf:type !foaf:Person)  
		 #(?erdoes !foaf:name !"Paul Erdoes"^^<http://www.w3.org/2001/XMLSchema#string>)))  
    (:union  
     (:filter  
      (:join nil  
	     (:bgp  
	      #(?document !dc:creator ?erdoes)  
	      #(?document !dc:creator ?author)  
	      #(?document2 !dc:creator ?author)  
	      #(?document2 !dc:creator ?author2)  
	      #(?author2 !foaf:name ?name)))  
      (and (sparql.sop:!= ?author ?erdoes)  
	   (and (sparql.sop:!= ?document2 ?document)  
		(and (sparql.sop:!= ?author2 ?erdoes)  
		     (sparql.sop:!= ?author2 ?author)))))  
     (:filter  
      (:join nil  
	     (:bgp  
	      #(?document !dc:creator ?erdoes)  
	      #(?document !dc:creator ?author)  
	      #(?author !foaf:name ?name)))  
      (sparql.sop:!= ?author ?erdoes))))  
   :distinct :distinct)

This is the raw output of AllegroGraph's parse-sparql function 1 . There are many possible s-expressions for a given query. Before execution, the query engine does additional analysis and simplification. This means that you can be flexible in the way you generate queries and let the query engine convert them into its preferred form.

Every SPARQL s-expression starts with a description of the query type and verb: 

(query-type query-verb &key)

where query-type can be either sparql.parser:sparql-update or sparql.parser:sparql and query-verb must be one of:

The rest of the s-expression is a keyword list whose permitted values vary depending on the verb. In the example above, the :vars argument specifies which variables to project and the :where argument describes the query plan to be executed. It shows that the first step of the query is a :join between another :join and a :union.

Note that query-verb and query-type are redundant but both are kept for historical reasons.

Filters and Expressions

Many query clauses involve expressions that calculate values or that are used to filter the result set. These are also s-expressions that use a restricted set of functions and constants. For example, the SPARQL string filter "?a + ?b < 34" becomes the filter expression: 

(< (+ ?a ?b) !"34"^^<http://www.w3.org/2001/XMLSchema#integer>)

The complete list of expressions is outlined below.

Packages

Note that the SPARQL parser places all query variables into the query.variables package for consistency during parsing. Your s-expressions do not need to do this and you are free to use any package that you would like.

ASK, CONSTRUCT, DESCRIBE and SELECT

These four SPARQL verbs share the same basic query algebra (though some verbs have their own unique parameters).

The following keyword parameters and arguments are allowed in these verbs' s-expressions (these will be described in more detail below): 2

distinct, from, limit, offset, order

These forms control the number and order of the solutions as well as the SPARQL dataset used for the query and whether or not the solutions should be distinct (or reduced). For example a query like: 

select distinct * { ?A ?b ?c } order by ?c desc(?a) limit 10 offset 20

would be represented by the s-expression: 

(sparql.parser:sparql :select :vars (?A ?b ?c) :where (:bgp #(?A ?b ?c))  
 :distinct :distinct  
 :order ((:asc ?c) (:desc ?a))  
 :limit 10  
 :offset 20)
:distinct
This can be nil or one of the keywords :reduced or :distinct and controls whether the query returns all results, is allowed to omit duplicate results or must omit duplicates.
:from dataset-list
from describes the SPARQL dataset to use in the query. It is in the form of a dataset-list which is a list of pairs where each pair has the car :from or :from-named and the cdr is the name of the graph to include in the dataset (either a UPI or a future-part).
:limit number
limit indicates the maximum number of solutions to emit.
:offset number
offset indicates how many solutions to skip before beginning to return them.
:order expression-list
order controls the order of the results returned from the query. It is list of lists. Each sub-list must have the first element of either :asc or :desc (for ascending or descending). The second element of the sublist is a SPARQL expression.

aggregates, group, having

These forms control aggregation. As an example, this SPARQL query: 

select ?account (count(?order) as ?count) (sum(?total) as ?grand) {  
  ?account :hasOrder ?order .  
  ?order :hasTotal ?total .  
} group by ?account  
having (?grand > 10000)

could be expressed as: 

(sparql.parser:sparql :select :vars (?account ?count ?grand)  
  :where  
  (:bgp  
   #(?account !<eh://hasOrder> ?order)  
   #(?order !<eh://hasTotal> ?total))  
  :aggregates  
  ((?count (:count ?order))  
   (?grand (:sum ?total))  
   (?account (:identity ?account)))  
  :group (?account)  
  :having ((> ?grand !"10000"^^<http://www.w3.org/2001/XMLSchema#integer>)))
:aggregates aggregate-expression-list
Aggregation is described by an ordered list of instructions. Each instruction consists of a list in the form (variable-name (instruction expression &key distinct)). The instruction can be one of :_assign, :avg, :count, :group_concat, :identity, :max, :min, :sample, or :sum. Most of these correspond to the obvious SPARQL aggregation operator. :identity is used for variables from the group by expression and means that the value will come from the group. :_assign is used to handle temporary aggregation variables introduced by the parser.

Consider a modified version of the query above where we now use an aggregate expression in the having clause: 

select (sum(?total) as ?grand) {  
  ?account :hasOrder ?order .  
  ?order :hasTotal ?total .  
} group by ?account  
having (sum(?total) > 10000)

This will parse as: 

(sparql.parser:sparql :select :vars (?grand) :where  
  (:bgp  
   #(?account !<eh://hasOrder> ?order)  
   #(?order !<eh://hasTotal> ?total))  
  :aggregates  
  ((?@aggregate-1 (:sum ?total))  
   (?account (:identity ?account))  
   (?grand (:_assign ?@aggregate-1)))  
  :group (?account)  
  :having ((> ?@aggregate-1 !"10000"^^<http://www.w3.org/2001/XMLSchema#integer>))

Where the expression in the having clause has been replaced by a newly introduced variable and :_assign is used to bind that variable to the query variable ?grand.

:group expression-list
This is simply a list of expressions that will be evaluated to determine the grouping.
:having filter-expression
This is a filter expression (which can include aggregation operators) used to filter out solutions.

construct-pattern, targets

:construct-pattern triple-pattern
construct-pattern is used only by SPARQL CONSTRUCT queries and is a list of triple-pattern arrays (as in a BGP). For example, this construct query: 
construct {  
  ?s rdf:type <eh://fruit> .  
  ?s ?p ?o }  
where {  
  ?s a <eh://fruityThing> .  
  ?s ?p ?o .  
}

would be equivalent to the following s-expression: 

(sparql.parser:sparql :construct :vars  
 (?o ?p ?s)  
 :where  
 (:bgp #(?s !rdf:type !<eh://fruityThing>) #(?s ?p ?o))  
 :construct-pattern  
 (#(?s !rdf:type !<eh://fruit>)  
  #(?s ?p ?o)))
:targets list
targets is used only by SPARQL DESCRIBE. It must be a list of IRIs (strings, UPIs or future-parts). This s-expression, for example, would be used to DESCRIBE two URIs: 
(sparql.parser:sparql :describe :vars nil :where (:bgp)  
  :targets (!<http://www.example.com#one> !<http://www.example.com#two>)

in-line-data

The in-line-data parameter provides the data from the outermost VALUES expression. It consists of two lists. The first list is an ordered list of the variables involved and the second is a list of lists of the values for these variables. nil is used for an undefined value. For example, the values clause: 

values (?account ?order) { (undef 2) (3 4) }

would be represented by the s-expression: 

:in-line-data  
 ((?account ?order)  
  ((nil  
    !"2"^^<http://www.w3.org/2001/XMLSchema#integer>)  
   (!"3"^^<http://www.w3.org/2001/XMLSchema#integer>  
    !"4"^^<http://www.w3.org/2001/XMLSchema#integer>)))

query-options

query-options is a association-list of values that control various switches during query execution. These include the query engine, whether or not to use Chunk-at-a-Time processing and more. The reference guide includes more information on the available query options.

vars

A list of the variables projected by the query. If omitted, then all variables will be returned in an unspecified order 3 .

default-base

Used as the default-base by the SPARQL uri function when constructing URIs.

where

The where clause describes the body of the query and is the most complex parameter. It is a tree of commands where each command is named by the head of its list and obtains its parameters from the tail. The commands are:

SPARQL queries work by obtaining bindings for variables and then combining these bindings in different ways. The :bgp, :bind, and :in-line-data clauses generate bindings and all the rest combine them or remove them from consideration.

:bgp triple-pattern
The BGP (or basic graph pattern) form has a single argument which must be a list of arrays where each array has three elements corresponding to a subject, predicate, object pattern. Each element in the array can be a symbol denoting a variable, a gensym denoting a blank node or a UPI or future-part denoting a constant. For example: 
(:bgp  
  #(?inproc !rdf:type !bench:Inproceedings)  
  #(?inproc !dc:creator #:?bnode_0)  
  #(?inproc !bench:booktitle ?booktitle)  
  #(#:?bnode_0 !foaf:name ?name))
:bind assignments query
The :bind form has two arguments: a list of assignments and the rest of the query. Each assignment in the assignments list is a list of two elements whose first element is the variable being assigned and whose second element is the expression to evaluate in order to make the assignment. For example: 
(:bind  
  ((?b !"2"^^<http://www.w3.org/2001/XMLSchema#integer>)  
   (?a !"1"^^<http://www.w3.org/2001/XMLSchema#integer>))  
 (:bgp #(?d ?e ?f))))

query is evaluated before the assignments are made so the above would first evaluate the BGP and then bind ?b to 2, ?a to 1.

:in-line-data data query
This is the in-query counterpart to the :in-line-data described above. The syntax is the same: it consists of two lists. The first is an ordered list of the variables involved and the second is a list of lists of the values for these variables. nil is used for an undefined value. For example, a query like 
select ((?total * 1.056) as ?markup) {  
  values (?account ?order) { (undef 2) (3 4) }  
  ?account :hasOrder ?order .  
  ?order :hasTotal ?total .  
}

would produce an :in-line-data clause like this: 

(sparql.parser:sparql :select :vars (?markup) :where  
 (:bind  
  ((?markup  
    (* ?total !"1.056"^^<http://www.w3.org/2001/XMLSchema#decimal>)))  
  (:join  
   (:join nil  
    (:in-line-data :data  
     ((?account ?order)  
      ((nil !"2"^^<http://www.w3.org/2001/XMLSchema#integer>)  
       (!"3"^^<http://www.w3.org/2001/XMLSchema#integer>  
        !"4"^^<http://www.w3.org/2001/XMLSchema#integer>)))  
     :rhs (:bgp)))  
   (:bgp  
    #(?account !<eh://hasOrder> ?order)  
    #(?order !<eh://hasTotal> ?total)))))

In this case, the query portion of the :in-line-data clause is the empty :bgp. Note that the positioning of the :in-line-data clause is significant as it alters how the join is processed.

joining groups of bindings

You can combine two sets of bindings (created from other query terms) using :join, :left-join and :union. You can remove a set bindings based on the contents of another set using :exists, :minus, and :not-exists. In the descriptions below, left-query and right-query are sub-trees with the same structure as the main :where clause and filter is a filter expression.

:exists left-query right-query
Evaluates left-query and right-query and keeps bindings in left-query if and only if there is a corresponding binding in right-query.
:join left-query right-query &optional filter
join evaluates left-query and right-query and then joins the two results using the variables that the two sides have in common. If there are no common variables, then join becomes a cross-product (which is usually a bad thing) 4 . The optional filter is evaluated for each row produced by the join and can be used as short-hand for 
(:filter (:join A B) filter)
:left-join left-query right-query &optional filter
left-join evaluates left-query and right-query and performs a left join using the optional filter to further remove results.
:minus left-query right-query
Evaluates left-query and right-query and removes any bindings in left-query if and only if there is a corresponding binding in right-query.
:not-exists left-query right-query
Evaluates left-query and right-query and keeps bindings in left-query if and only if there is not a corresponding binding in right-query.
:union left-query right-query
union evaluates each query piece and joins then with the current set of bindings in turn.
Miscelaneous operations
:filter query filter
The :filter clause removes bindings based on an expression. It first evaluates query and then iterates over the bindings applying the filter to each one.
:graph graph query
The graph clause changes the current graph used during query evaluation and then evaluates query. graph can be a variable, an IRI (string, UPI or future-part), or one of the keywords: :from, :from-named, or :default. These last three mean:

  • :from - iterate over the graphs in the default-graph portion of the dataset. I.e., the ones that come from the FROM clauses in the SPARQL query.

  • :from-named - iterate over the named graphs in the dataset. I.e., the ones that come from the FROM NAMED clauses in the SPARQL query.

  • :default - use the current graph.

:select query
:select is used for SPARQL 1.1 sub-query. The :select clause takes all of the arguments that the main SELECT query supports except for :from (sub-queries cannot change the dataset). For example, the (rather contrived) query: 
select * { { select * { ?a ?b ?c } order by ?c limit 1000 }}

is equivalent to the s-expression 

(sparql.parser:sparql :select  
  :vars (?a ?b ?c)  
  :where (:join nil  
           (:select :vars (?a ?b ?c)  
              :where (:bgp #(?a ?b ?c))  
              :order ((:asc ?c))  
              :limit 1000)))

(In passing, note that the inner ORDER BY will be used to order the results of the sub-query before applying the limit but there is no guarantee that the final result set will retain the ordering).

:service &key host query-string silent
The service clause sends the query-string to host and merges the result into the rest of the query. If there is an error and silent is false, then the entire query will fail. 5

Update

A SPARQL update s-expression has the following keywords: steps, default-base and query-options 6.

As in the other SPARQL query forms, default-base is used by the SPARQL uri function when constructing URIs and query-options is a association-list of values that control various switches during query execution. These include the query engine, whether or not to use Chunk-at-a-time processing and more. They are described in more detail in the reference guide.

steps is a list of actions to take. Each action is a list whose head names the action and whose tail specifies any parameters. The actions are broken down into graph manipulation, data manipulation and loading from external data sources.

All of the non data-manipulation update steps use keyword arguments in their specification. They all have a silent argument which controls whether or not errors are signaled back to the caller.

Graph manipulation

The graph manipulation commands allow you to alter sets of triples contained in particular named graphs.

:add &key first second silent
This is the add command. 
ADD ( SILENT )? ( ( GRAPH )? IRIref_from | DEFAULT) TO ( ( GRAPH )? IRIref_to | DEFAULT)

first and second correspond to from and to and can be an IRI or the keyword :default. Unless silent is true, :add will fail if the first graph is not present in the triple-store.

:clear &key graph silent
This is the clear command. 
CLEAR ( SILENT )? (GRAPH IRIref | DEFAULT | NAMED | ALL )

graph can be an IRI (UPI or future-part) or one of keywords: :default, :named or :all. Obviously, care should be taken before executing this command! The :clear command cannot fail (i.e., it is not an error to clear a graph that does not exist) so the silent parameter is ignored.

:copy &key first second silent
This is the copy command. 
COPY ( SILENT )? ( ( GRAPH )? IRIref_from | DEFAULT) TO ( ( GRAPH )? IRIref_to | DEFAULT )

first and second correspond to from and to and can be an IRI or the keyword :default. If the source graph does not exist, then the :copy command will fail unless silent is true.

:create &key graph silent
This is the create command. 
CREATE ( SILENT )? GRAPH IRIref

Where graph must be an IRI (string, future-part or UPI). AllegroGraph does not record the existence of empty graphs so the create operation will never change the contents of a triple-store. If the graph already exists in the store, however, then it will fail (unless silent is specified).

:drop &key graph silent
This is the drop command. 
DROP  ( SILENT )? (GRAPH IRIref | DEFAULT | NAMED | ALL )

graph can be an IRI (string, future-part or UPI) or one of the keywords :default, :named, or :all. Because AllegroGraph does not keep track of empty graphs, the :drop command is equivalent to the :clear command.

:move first second silent
This is the move command. 
MOVE (SILENT)? ( ( GRAPH )? IRIref_from | DEFAULT) TO ( ( GRAPH )? IRIref_to | DEFAULT)

first and second correspond to from and to and can be an IRI or the keyword :default. As in :copy, if the source graph does not exist, then the command will fail unless silent is true.

Data manipulation

The data manipulation commands let you add and remove triples from the triple-store. Each uses quad-templates to specify what to add and remove. A quad-template is like a bgp (i.e., a list of arrays) extended with additional syntax to allow you to specify the graph(s) that should be altered. I.e., it looks like: 

(#(s1 p1 o1)  
 #(s2 p2 o2)  
 (:graph g1 (#(s3 p3 o3)))  
 #(s4 p4 o4))

Note that the logic for determining the graph(s) to alter is complex and depends on graphs specified in the templates, the graphs specified in the USING or WITH clauses and various parameters passed to the run-sparql function. Please contact support if you need additional information.

:delete quad-template
This corresponds to the DELETE DATA command. Note that quad-template needs to be in an additional list. The template used is not allowed to contain either variables nor blank nodes 7 .
:insert quad-template
This corresponds to the INSERT DATA command. Note that quad-template needs to be in an additional list. Note that this template is not allowed to include variables 8 .
:modify &key graph delete insert where using
This corresponds to the DELETE/INSERT command. The delete and insert parameters must be quad-templates (or nil). The where clause uses the same grammar as it does in the query language above.

using is like from (above). I.e., it is a dataset-list and it should be a list of pairs where each pair has the car :from or :from-named and the cdr is the name of the graph to include in the dataset (either a UPI or a future-part).

The graph parameter represents the graph specified in the WITH clause of the SPARQL grammar.

Loading from external sources

:load &key from graph silent
The :load command tries to read data from the URI from (which can be a file://, http://, or https:// URI) and load it into the graph graph. graph can be nil or a URI. If it is nil, the data is loaded into the default-graph of the destination triple-store. The format of the data at the other end of the from URI is determined via its extension (treating it as if it names a file). If the type cannot be guessed, then RDf/XML is assumed. If there is a problem during loading, then an error will be signaled unless silent is true.

Filter Summary

Filter expressions can be combined with the boolean operators and and or and negated with not. We support all of the functions in the SPARQL standard. We will augment this document with a complete listing in a future version. In the meantime, please contact support if you need additional information.

Footnotes

  1. The query can be further simplified which occurs during the query planning process. ↩
  2. Note that the previously deprecated :geo clause has been removed. All SPARQL Geospatial queries must now be done using the 2D and nD magic properties. ↩
  3. You can use the third return value of run-sparql to obtain a list of the variables in the order they occur in the result set. ↩
  4. Cross products almost always indicate that a query is asking two different and unrelated things. Because there is no join, the answer will contain a row for every pair of elements on the right-hand side and the left-hand. ↩
  5. Note that the service keyword requires the query to be specified as a string. At some point, AllegroGraph may allow s-expressions to be used here too. ↩
  6. Note that this document describes the s-expression syntax for the SPARQL 1.1 engine; the SPARQL 1.0 engine supports an older version of the UPDATE language and is not compatible with the newer syntax. ↩
  7. At this time the check for variables and blank nodes happens in the parser and the behavior is unspecified if templates using them are passed in as parameters to the delete data command. ↩
  8. At this time the check for this happens in the parser and the behavior is unspecified if templates with variables are used ↩