Table of Contents

Release 4.11

Major Changes in release 4.11

Major Changes in release 4.10

Major Changes in release 4.9

Changes in earlier releases

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

AllegroGraph 4.9 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

AllegroGraph 4.8 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

Release 4.11

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 4.11.

Major Changes in release 4.11

The changes in the previous major release, version 4.10, are listed just below.

Major Changes in release 4.10

Along with bug fixes and other improvements, the major changes in release 4.10 are:

The changes in the previous major release, version 4.9, are listed just below.

Major Changes in release 4.9

Changes in earlier releases

This document lists changes in the current release and several previous releases. See the Change History for a list of all changes in earlier releases.

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

Rfe12309 - Improve algebraic query manipulation

AllegroGraph now notices patterns of the form 

(:join (:join A (:bgp B)) (:bgp C))

and transforms them into the equivalent 

(:join A (:bgp B C))

(I.e., it merges the two BGPs). This transform brings more triple-patterns together which lets AllegroGraph order the query operations better.

Rfe12305 - Relative paths in the configuration file

Relative file or directory paths in the configuration file were always interpreted relative to the directory containing the configuration file. In this rfe, a BaseDir configuration directive was added that allows another path to be specified for this purpose. See the documentation in Server Configuration for more.

Rfe12280: Simplify geospatial type-mapping API

Geospatial subtype mappings can now be set up with the regular datatype and predicate mappings rather than needing to use the specialized functions. The old functions still work but are now unnecessary. They will be removed in some future release.

Rfe12260 - Automatically map temporal predicate data to xsd:dateTimes

Triple-stores created in this version of AllegroGraph will include the three AllegroGraph temporal predicates in the list of automatically mapped predicates. I.e., the objects in triples with these predicates:

will be converted into encoded xsd:dateTime literals. This encoding makes range queries against them faster and makes it easier to use AllegroGraph's temporal functionality.

Rfe12259 - Optimize certain combinations of JOINs and FILTERs

The query engine now recognizes certain patterns of JOINs and FILTERs and rewrites them to be more efficient. For example, a pattern like 

(:join A (:filter B f))

can often be better evaluated as 

(:filter (:join A B) f)

Rfe12182 - More restrictive umask

With this change, users who aren't the owner of the file, or a member of the group of the file (that is, "other" users in chmod manpage terms) have no read, write or execute permission on files created by AllegroGraph.

On existing installations, it's recommended, but in no way enforced, to change permissions the same way with 'chmod -R o-rwx <directories>' where <directories> is the list of directories holding AllegroGraph data (LogDir, SettingsDirectory, catalog directories, etc).

Rfe12175 - Audit log

When the 'Auditing' global configuration directive is set to 'yes' (it defaults to 'no'), events that have implications on security or performance will be logged to the new system database in the new system catalog. Note that regardless of permissions only superuser can read or write the system catalog.

See Auditing for more information.

Rfe12151: Remove support for loading N-Triples into a federated triple-store

AllegroGraph 3 introduced some techniques that optimized loading of N-Triples into a federated-triple-store. These have been superseded in AllegroGraph 4 because agload is significantly faster and easy to use.

Rfe12147: Better manage memory especially for large solutions

AllegroGraph now manages large query result sets (with or without DISTINCT or ORDER BY) more efficiently.

Rfe12138 - Agload: reduced memory footprint and other enhancements

Reduced memory consumption of agload processes. Particularly in the case where there are many input sources.

Agload now analyzes input sources on the fly. This means you will not necessarily encounter errors about invalid input sources prior to modification of a triple-store. Use --dry-run command line argument if you wish to pre-analyze an agload job.

Rfe12125 - Improve performance of unordered queries with small limits

AllegroGraph now uses chunk-at-a-time query processing for queries with small LIMITs that do not use ORDER BY more frequently. In particular, the chunkProcessingAllowed query option can now take three values:

The default value for chunkProcessingAllowed has been changed from no to possibly.

Rfe12111 - Improve memory checking efficiency

Improve the efficiency of AllegroGraph's memory tracking code slightly.

Rfe12089 - Improve cursor resource management

Improve AllegroGraph's low-level cursor memory management.

Rfe12063 - Allow in-memory-triple-stores to use blank nodes

The in-memory-triple-store implementation for the remote-lisp-client was missing several methods required for full blank node support. This has been corrected.

Rfe11973: Improve usage of UPI-map

Improved the way AllegroGraph manages the use of secondary indexes (UPI-maps).

Rfe11955 - :commit argument added to delete-duplicate-triples

The :commit argument was added to delete-duplicate-triples to allow for the deleting a very large number of duplicates.

Rfe11886 - Use statistics more often when re-ordering query clauses

The query engine now uses store statistics in more places when working to reorder the clauses in a query's basic graph patterns (BGPs). This should produce better plans more often with less need for customer query tweaking.

Rfe11585 - Some queries unnecessarily accumulated partial results

AllegroGraph was sometimes accumulating results unnecessarily. For example, the query 

select (count(*) as ?total) where {?s ?p ?o}

was accumulating the actual results even though only the summary was necessary.

Rfe11511 - Encode xsd:decimals

xsd:decimals used to be stored as strings. With this change, they are automatically encoded directly in the UPI which potentially brings significant savings in storage space and computation in queries. The most significant 20 digits are preserved by the encoding.

Note that this change only affects the encoding of new triples. To convert old triples and take advantage of the more efficient storage, the database contents must be reloaded from a serialized format. We recommend exporting in nquad format and then reimporting it with agload. Restoring from a backup is not sufficient.

Rfe11197 - Serialize triple IDs as resources rather than literals

Previously, a triple ID would have been serialized as a typed literal like 

"13"^^<http://franz.com/ns/allegrograph/4.0/triple-id>

This format prevented the use of triple IDs in the subject of a triple so we have changed the serialization to a URI like 

<http://franz.com/ns/allegrograph/4.0/tripleId#13>

Any triple IDs already in a triple-store will remain the same (though they will print differently). This change does mean, however, than any files that contain triple IDs in the old format will need to be converted in order to be re-loaded (note, however, that the IDs assigned to triples are not part of any serialization format which means that they cannot reliably round-trip in any case).

Rfe10292 - Handle filename limit for unix domain sockets

Linux has a limit of 107 characters for filenames of UNIX domain sockets. With this change, when bumping into that limit, a meaningful error message is printed and the workaround of setting the UnixDomainSocketDirectory or the SettingsDirectory configuration directive to a shorter path is suggested.

Bug21793 - Fix race in inter process communication

Under extremely rare circumstances, multiple threads could end up listening on the same communication channel which can lead to errors and deadlocks. This was duly detected by an assertion that appeared in the log as: 

the assertion (endp (intersection  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::r)  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::receiver)))  
failed.

This bug has been fixed.

Bug21782 - Potential double close of cursors

An inadvertent double close of cursors was discovered that could potentially result in improper sharing of a data structure. This has been corrected.

Bug21779 - Fix low-level cursor resourcing bug

Under heavy load, the secondary index (UPI-map) creation function could leak a cursor leading to potential resource problems. This has been corrected.

Bug21761 - Prolog q- functor could be treated as q

If a reasoning triple-store can use secondary indexes (UPI-maps) then it was possible for AllegroGraph to treat some q- functors as q functors in a given Prolog select query. This has been corrected.

Bug21745 - Fix crash related to aggregate functions

Under rare circumstances, queries involving aggregate functions may cause the backend to crash. This bug has been fixed.

Bug21744 - Some Prolog select queries could return duplicate results

If using secondary indices (i.e., UPI-maps), it was possible for Prolog select queries that returned raw UPIs to overwrite one solution with another. This problem could only occur with UPI only queries and never with queries that returned strings. The problem is now fixed.

Bug21740 - Rare hangs under high load

The message broker (called the "Hub" process) may get into a state where forwarding a message blocks, preventing all other messages from being handled. This bug has been fixed.

Bug21735 - Corrupted distinct results

Under rare circumstances, queries involving DISTINCT could produce results with the bindings assigned to the wrong variables. This bug has been fixed.

Bug21702 - XSD casting in a BIND could convert to xsd:integer incorrectly

Using an XSD cast in a BIND could loss the datatype and switch it to xsd:integer. E.g., 

BIND(xs:long(?test) AS ?testLong)

would bind ?testLong to a value whose datatype was xsd:integer rather than xsd:long. This has been corrected.

Bug21693 - Avoid forking too many blank node servers in agload.

Improves agload performance under default conditions when loading ntriples or nquads files which are small and numerous.

Bug21691 - Reasoning over subproperties of symmetric properties

AllegroGraph's RDFS++ reasoner partially failed to infer triples from sub-properties of a symmetric property. This has been corrected.

Bug21680 - Some BIND forms could be evaluated incorrectly.

A BIND form at the start of a BGP could fail to get values from earlier in the query evaluation.

For example, this form 

?a a ?b .  
{ BIND( ?b as ?otherB  
  ?otherB a ?c .  
}

could lose the binding for ?otherB. This has been corrected.

Bug21670 - FILTER and EXISTS could interact incorrectly.

Depending on the shape of the rest of the query, it was possible for a FILTER clause to be applied to the inside of an EXISTS or NOT EXISTS filter rather than being applied to the main body of the query. This has been corrected.

Bug21663 - Redo of new index creation confuses instance process

If a database with a newly-created index has been shut down uncleanly, restart of the database may crash. This has been corrected.

Bug21651 - Prolog ?? and reasoning

On a reasoning triple store, the prolog ?? construct in the predictate position used to fail with a type error. This bug has been fixed.

Bug21642: Agload buffer overflow

Due to a buffer overflow, agload could corrupt memory and potentially crash. This has been fixed.

Bug21640: Race condition when reaping abandoned sessions

Previously, if concurrent processes performed the operation to reap abandoned sessions, shared memory state could be corrupted, resulting in hangs or crashes. This has been fixed. (This bug was actually fixed in version 4.10.)

Bug21625 - Large result sets could produce bogus triples with INSERT and CONSTRUCT

In cases where there were more than 100,000 results, the machinery that prevents INSERT and CONSTRUCT queries from producing duplicate results could create invalid answers. This has been corrected.

Bug21596 - Serialize-rdf/xml could fail on some predicates

AllegroGraph's RDF/XML serializer would fail if a predicate ended with something other than a letter, a digit, an underscore or a dash. This conformed to an older version of the Extensible Markup Language but not to the most recent version. This has been corrected.

Bug21586 - Possible divide by zero error in index optimizer

Under some circumstances an index optimizer could attempt to perform a division by zero when logging statistics to agraph.log. This would result in a non-fatal error message being logged instead of the statistics. This has been corrected.

Bug21580 - Improve external variable binding bookkeeping

The values for externally supplied variables could be lost for some complex interactions of EXISTS (or NOT EXISTS) filters and UNIONS.

This has been corrected.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Also, the http error code was changed from 400 ("Bad request") to 503 ("Service unavailable") to indicate a temporary condition.

Bug21553 - Fix MemoryCheckWhen without MemoryReleaseThreshold

If the memoryReleaseThreshold configuration directive was not specified but memoryCheckWhen was, then query execution failed with type errors trying to compare memory footprint to NIL. With this fix, AllegroGraph refuses to start in these circumstances.

HTTP Client

Rfe12136 - Optimize front-end http server

The timeout for reading headers was lowered to 5s which makes the front-end http server make better use of its available resources and more resilent when clients with http keep-alive don't close the connection in a timely manner.

Also, a warning is logged if the number of idle http worker threads falls below 2, because it can easily cause a performance drop.

SPARQL

Rfe12296 - Support AllegroGraph direct reification in SPARQL

AllegroGraph now supports direct reification in SPARQL queries using the <http://franz.com/ns/allegrograph/4.0/tripleId> magic property. This property allows you to bind the ID of a triple to a SPARQL variable which can then be used in reification. For example, the following query first adds two triples and then adds four additional triples that describe the first two: 

prefix : <eh://>  
prefix franz: <http://franz.com/ns/allegrograph/4.0/>  
insert data {  
  :gary :likes :dogs .  
  :gary :likes :cats .  
} ;  
 
insert {  
  ?id :mentioned ?now .  
  :gary :believes ?id .  
} where {  
  ?id franz:tripleId (:gary :likes ?what) .  
  bind( now() as ?now )  
}

See the documentation for additional details on direct reification and its advantages and limitations.

Rfe12246: Enhancements to SPARQL freetext queries

The fti:match and fti:matchExpression predicates can now optionally retrieve the object of the matching triples and select the freetext index to use. (These enhancements are currently available only for AllegroGraph's native freetext indexes. Support for Solr and MongoDB will be forthcoming).

The new features rely on SPARQL magic properties to provide a syntactic shortcut for list patterns. As an example, this query retrieves the subject (into ?subject) and text (into ?text) for the freetext query that matches "nepal": 

select ?subject ?text {  
  (?subject ?text) fti:match 'nepal' .  
}

This query retrieves only subjects and uses only the freetext index named comments. 

select ?item {  
  ?item fti:match ('upgrade' 'comments')  
}

More examples and information can be found in the documentation.

Rfe12243 - Support AllegroGraph Social Networking Analysis in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable Social Networking Analysis including centrality measures, path finding and cliques. See the documentation for more details.

Rfe12231 - Improve SPARQL geospatial integration

AllegroGraph now supports several magic properties to make SPARQL / geospatial integration easier. For example, this query finds all of the points within a circle centered at 10, 2: 

select ?who ?where {  
  (?who ?where) geo:inCircleXY ( ex:location 10 2) .  
}

The following magic properties are provided: 

geo:inBoundingBoxXY  
geo:inBoundingBox  
geo:inCircle  
geo:inCircleKilometers  
geo:inCircleMiles

(where geo is the usual Geospatial namespace abbreviation for <http://franz.com/ns/allegrograph/3.0/geospatial/>). See the documentation for additional details on using these properties.

Rfe12171 - Improve variable GRAPH clause handling in some situations

SPARQL specifies that a graph clause like 

 GRAPH ?g { ... }

must be implemented as if it iterated over each graph in the dataset and took the union of the resulting bindings. There are sometimes more efficient ways to get the same results and AllegroGraph uses these when it can. AllegroGraph was not, however, using information about any constraints on the variable ?g that were provided by other parts of the query. For example, a query like: 

?s :p ?g .  
GRAPH ?g { ... }

might produce a single binding for ?g but AllegroGraph was still iterating over all of the named graphs of the dataset. This has been optimized so that AllegroGraph will use the additional information when it can determine that the number of bindings for ?g are smaller than the number of graphs in the dataset.

Rfe12057 - Manage disk and memory resources better for CONSTRUCT queries

When there are many results, SPARQL CONSTRUCT queries buffer their results to disk to conserve memory. The cutoff value for when to buffer to disk was unnecessarily small which meant that CONSTRUCT went to disk more often then necessary. This has been corrected.

Rfe12035 - Improve handling of some SPARQL queries with a GRAPH clause

By definition, SPARQL must handle a GRAPH clause as if the query engine evaluated the clause once for each graph in the dataset. However, it is often possible to evaluate the clause more efficiently.

This change improves the heuristics AllegroGraph uses to determine when the more efficient approach will achieve the correct results which means that some queries with a GRAPH clause will be executed more quickly.

Rfe11319 - Support SPARQL federated queries (the SERVICE clause)

AllegroGraph now supports SPARQL federated query using the SERVICE clause. The initial implementation is functional but not highly optimized.

Rfe8063 - Support AllegroGraph temporal reasoning in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable temporal reasoning. See the documentation for more details.

Bug21757 - SPARQL geospatial queries could fail

Depending on the set up of a triple-store's geospatial subtype mappings, it was possible for a SPARQL query using geospatial extensions to fail to find results. This has been corrected.

Bug21742 - SPARQL parser failed on quad patterns with collections

The SPARQL parser could lose track of triple-patterns that arose from abbreviations using either nested blank nodes or collections. This has been corrected.

Bug21725 - Correct potential error when SPARQL Update touches too many rows

If a SPARQL DELETE ... INSERT ... WHERE request changes many rows, then AllegroGraph caches the results of the WHERE expression to disk to reduce memory pressure. In some cases, this could result in a error when the cached results were re-read. This only occurred for Update requests that touched more than 100,000 rows. This has been corrected.

Bug21718 - Unused variable optimization could produce incorrect results

AllegroGraph analyzes SPARQL queries that use DISTINCT in order to remove the collection of bindings for variables that are not needed to produce the final result. In some cases, it was possible for this analysis to cause the execution of a code path that added solutions incorrectly. This has been corrected.

Bug21573 - SPARQL queries could intern strings unnecessarily

SPARQL queries that used the FROM or FROM NAMED clause always tried to intern the graphs into the triple-store's string table. This was both unnecessary and could cause an error if the triple-store was read-only or a federation.

This has been corrected.

Bug21011, Bug21571 - handle nested SPARQL BGP syntactic sugar correctly

Some SPARQL triple-patterns that involved nested brackets and object lists were being parsed incorrectly. This has been corrected. Some examples of queries that work correctly now but that failed without this change include: 

PREFIX : <ex:>  
SELECT * {  
    :x :y [:z (:b [:d (:e :f),(:g :h)])]  
}  
 
PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c :d], [:e :f] .  
}

and 

PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c [:d :e]] .  
}

Improve Pattern estimation where there is no :p index

Improved query pattern estimators for triple-stores with no predicate index (e.g., :posgi, :psogi, etc.). This will help re-order query clauses more intelligently.

AGWebView

Rfe12325 - Add audit log query interface to AGWebView

There is now an audit menu item in the admin menu. This provides access to the audit log query page which lets you filter audit log events by user, class and date.

Changes to the Lisp API

Bug21708 - Fix some low-level Lisp API geospatial functions

Some low-level Lisp API functions had not been enhanced to account for AllegroGraph's geospatial encoding. This has been corrected.

Bug21689 - Lisp client printer variable leakage

On a remote store, the run-sparql function failed with an 'Illegal sharp character' error if print-length was set to a small value. This bug and more generally the leakage of printer variables into run-sparql has been fixed.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe12118 - AGRepository with connection pooling

With this change, an AGRepository can now be configured to use a connection pool via the setConnPool method, enabling Sesame apps to transparently benefit from connection pooling. See the javadoc for setConnPool for details.

Rfe10513 - Configurable blankNodesPerRequest

With this change, the Java client supports configuring the number of blank nodes automatically fetched per request to the server by the AGRepository's AGValueFactory. The default remains at 100, but it can be configured by setting the property:

com.franz.agraph.repository.blankNodesPerRequest

Applications can also configure it directly, see the javadoc for AGValueFactory#setBlankNodesPerRequest for details.

Rfe9869: Support SPARQL/Update from Jena

With this change the Jena adapter supports an execUpdate method in AGQueryExecution for executing SPARQL Updates. Jena Tutorial example13 has been augmented to demonstrate its use.

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

Rfe12004 - Ag4 plugins for TBC 4.0.0

With this change, the AG plugins have been updated to work with TBC 4.0.0. For installation details, see Top Braid Composer Plugin.

Rfe11986 - Improve efficiency of some property path queries

AllegroGraph now evaluates property path queries whose initial or final endpoints come from a small set of values more efficiently. For example, a query like 

prefix gnO:<http://www.geonames.org/ontology#>  
select ?id ?par ?parname {  
  bind (URI('http://sws.geonames.org/104515/') as ?id)  
 
  ?id rdf:type gnO:Feature .  
  ?id gnO:parentFeature+ ?par .  
  optional {  
    ?par gnO:name ?parname  
  }  
}

will be evaluated much more quickly than before because the subject of the path query is fixed.

Rfe11937: Agload: remove blank node cache size guessing code

In agload, blank node cache size guessing code did not work very well. It was removed and the blank node cache size defaults to 1,000,000 which works reasonably well on small and large memory machines.

This change will only be noticed when loading ntriple or nquad files with blank nodes.

Rfe11839 - Improve eager evaluation of impossible query constraints

AllegroGraph tries to evaluate FILTER constraints as early as possible. In particular, a constraint may become impossible or a tautology before the query even runs if external variable bindings are supplied. AllegroGraph was already handling tautologies eagerly but was not doing the same for impossible constraints. This has now been remedied.

Rfe11827 - Improved memory footprint of joins with unbound variables

Previously, queries involving joins had a worst case memory footprint of O(2^N) where N is the number of join variables. This is now O(1), but the time complexity is O(N). In practice, this means that the client gets a timeout instead of a memory exhaustion warning.

Rfe11233 - Simplify property path queries when possible

Property path queries that do not use the zero or more or one or more operators and that do not have complex alternation can be expanded directly into simpler forms. This lets them be executed using the regular query machinery which can often be faster than the specialized path query machinery.

Rfe9699 - Dynamically grow string table files

Prior to this change string tables files were large sparse files. The apparent size of these files frequently led to user confusion. In addition, full filesystem errors were not handled gracefully when modifying the sparse files. Now string table files start at a minimum size and grow as needed. If growth of a string table fails due to a full filesystem error, a complaint will be logged to agraph.log and the operation will be retried periodically until it succeeds.

Bug21592: Improve cursor resource management

Some cursors were failing to close all of their file handles which could lead to errors. This has been corrected.

Bug21583 - see Bug21575 below.

Bug21582 - Complex nested unions could fail to return some bindings

Given the right data, a complex query involving multiple nested unions could misapply an optimization meant for one part of the union to a different part. This could lead to some variables in the query being unbound.

This has been corrected.

Bug21575, Bug21583 - property path optimizations and better bookkeeping

Improve the way that AllegroGraph implements some property path queries.

Correct a problem with property path queries and over-zealous resource management.

Improve the bookkeeping of unused variables when using external bindings.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Bug21550 - Complex FILTER EXISTS expressions could cause parsing errors

Complex FILTERs that used EXISTS (or NOT EXISTS) could signal errors during parsing. This has been corrected.

Bug21549 - Nested sub-queries could fail to project variables correctly

A query with nested sub-queries that used * to return all variables in some of the sub-queries could incorrectly reason about variable scope which could cause queries to fail. This has been corrected.

Bug21534 - Garbled error message on memoryCheckWhen parsing failure

When the memoryCheckWhen configuration directive had an unsyntactic value ("memoryCheckWhen junk:1"), the error message was wrong. This bug has been fixed.

Bug21506 - Correct possible error when mixing UNION and GRAPH

Under some circumstances, a query with both UNION and GRAPH clauses could signal an error during execution. This has been corrected.

Bug21505 - Unused variables from a sub-query could confuse AllegroGraph

AllegroGraph removes query variables that are not used if a query has either the DISTINCT or REDUCED modifiers. The unused variables logic was not correctly handling the case of a sub-query whose projected variables were not used elsewhere. This could lead to an error during query processing. This has been corrected.

Bug21498: Do not signal error when all ORDER BY expressions are redundant

AllegroGraph removes ORDER BY expressions that do not depend on the bindings of a row because they cannot have any bearing on the outcome. For example, the query: 

SELECT * { ?s a ?o } ORDER BY ?pineapple

is equivalent to the query 

SELECT * { ?s a ?o }

Unfortunately, AllegroGraph was incorrectly signaling an error when all of the expressions in an ORDER BY clause were irrelevant and therefore removed.

This has been corrected.

Bug21497 - Filters could be misapplied in some situations

AllegroGraph attempts to move FILTERS closer to where they apply in order to increase query speed. If a FILTER was applicable on both sides of a UNION and also applied to single triple-patterns then it was possible for AllegroGraph to think that the FILTER only needed to be used on the left-hand side of the UNION. For example, a query like this could fail to apply the FILTER to ?c on the right-hand side: 

select * {  
  { ?a ex:b ?c . } union { ?a ex:c ?c . }  
  filter (?c*2 = 2)  
 }

This has been corrected.

Bug21496 - Improve memory management for CONSTRUCT queries

AllegroGraph now tracks memory more accurately and manages it more efficiently for CONSTRUCT queries.

Bug21493 - Bind is too aggressive in assuming single values

AllegroGraph was over-optimizing assignment (BIND) such that assignments to the same variable name in different pieces of a UNION could fail to work correctly. For example, a query like 

SELECT * {  
  { BIND( :person1 as ?person ) }  
  UNION  
  { BIND( :person2 as ?person ) }  
}

would return only one binding for ?person rather than two.

This has been corrected.

Bug21486 - Correct handling of some IN LINE data syntax

AllegroGraph was not correctly reading all of the supplied values when using the abbreviated simple-variable syntax. E.g., a query like 

select ?person (concat(?first, ' ', ?last) as ?name) {  
  values ?person { ex:person1 ex:person2 ex:person10 }  
  ?person ex:first-name ?first .  
  ?person ex:last-name ?last .  
}

was processing only the first supplied value. This has been corrected.

Bug21476 - Some blank nodes could fail to print in a 32-bit Lisp client

It was possible for blank-nodes to raise an error when AllegroGraph tried to print them in N-Triples format. This has been corrected.

Bug21470 - Corrected Turtle parser relative URI normalization

AllegroGraph was incorrectly applying sections 6.2.2 and 6.2.3 of RFC3986 when it merged a base URI with a relative one. This has been corrected.

Bug21458 - Standardize AllegroGraph exit code

AllegroGraph's license check can be resolved in three ways:

  1. Normal start up - a license is present and correct.

  2. Free start up - no license is present so the free limit is used.

  3. Limited start up - only partial license information is present so AllegroGraph runs in free mode.

In all of these cases, the exit code returned will be zero (0).

If the license code is invalid or if there are any other problems with server start up, then AllegroGraph will fail to start and return a non-zero exit code.

Bug21454 - URI parsing was too strict

AllegroGraph was being stricter than necessary when parsing BASE URIs such as foo:bar:/baz. This has been corrected.

Bug21379 - Reasoning triple-store confusion about unspecified graph fixed

A reasoning triple-store failed to distinguish between a query where the graph was left off (i.e., left as a wild card) and a query where the graph was explicitly set to the default-graph of the current triple-store. This could cause it to infer incorrect triples.

This has been corrected.

Bug21365 - Fix lookup of user access rights when authorizing access to repositories in the root catalog.

AllegroGraph would deny authorization requests on repositories in the root catalog under some conditions, despite specific user access right allowing such (read/write access on catalog "/", repository ""), or global rights (read/write access on catalog "", repository "*"). This has been corrected.

Bug21353 - Correctly handle aggregation on empty datasets

AllegroGraph was failing the Data Access Working Group (DAWG) "agg empty group" test because it was not returning a row for a query asking for aggregation on an empty dataset. This has been corrected.

Bug21158 - Catch errors during filter analysis

AllegroGraph was signaling an error during query planning if the eager analysis of a FILTER expression resulted in a error. The correct behavior is to treat the FILTER as an impossible constraint.

This is now handled correctly.

HTTP Client

Rfe11609 - Support text/table content-type in the HTTP protocol

AllegroGraph will now return a simple ASCII table if an HTTP request uses the content-type text/table. For example, a query might return a table like: 

  -----------------------------------  
  | person    | name     | location |  
  ===================================  
  | mo:Bob    | "Bob"    | ---      |  
  | mo:Robert | "Robert" | 23       |  
  -----------------------------------

Bug20622: "Mode" was not optional for DELETE /statements/duplicates

The mode parameter, which is used to indicate which components of each triple must be equivalent to count as duplicates of each other, is now optional and defaults to "spog" for the DELETE /repositories/[name]/statements/duplicates HTTP protocol request.

SPARQL

Rfe11920 - Signal error when aggregate expressions are used in the wrong place

AllegroGraph did not signal a useful error when aggregate expressions were used incorrectly in a SPARQL query. E.g., 

SELECT * {  
  ?a ex:foo ?value .  
  BIND( SUM(?value) AS ?sumValue )  
}

This has been corrected.

Rfe11494 - Update semantics of SPARQL property path queries

AllegroGraph's property path queries now match the updated semantics of the SPARQL 1.1 specification. In particular, wild-card operators like * and + no longer count their matches. AllegroGraph still supports the now deprecated counting operations, e.g., 

{2,4}

but no longer counts the unique paths found. I.e., you can use these non-standard operators to find if there is a path but not to count the number of paths.

Note that AllegroGraph now passes all of the Data Access Working Group (DAWG) tests with the exception of 

 (pp37) Nested (*)*

This test uses the path 

((:p)*)*

and AllegroGraph incorrectly returns some duplicate results.

Bug21599 - Variable GRAPH queries that used the graph could lose results

A SPARQL GRAPH query that used a variable graph and included the graph variable in the body of the query could under some circumstances lose results. For example, a query like 

SELECT * {  
  GRAPH ?g {  
    ?g a <FoodGroup> .  
    ?g <covers> ?r .  
  }  
}

could possibly have returned too few results depending on the data in the store.

This has been corrected.

Bug21581 - Improve bookkeeping when using property paths

If AllegroGraph can determine that a variable in a SPARQL query is not used, then it will enable certain optimizations. For example, if a query is returning DISTINCT results and a variable is not used in any joins nor returned by the query, then its values do not matter.

This logic was not correctly rewriting property paths to account for the unused varaibles.

This has been corrected.

Bug21562: Insert & remove-graph-uri(s) should not override WITH

If insert-graph-uri(s) or remove-graph-uri(s) were passed as query parameters to a SPARQL DELETE...INSERT...WHERE statement, then AllegroGraph was using these parameters even if the statement used WITH to specify a GRAPH. This is now corrected. SPARQL Update statements that use WITH will not be overridden via insert or remove graph URI(s).

Bug21551 - Deletes via SPARQL Update did not correctly handle encoded UPIs

SPARQL DELETE DATA and DELETE...INSERT...WHERE commands could fail to remove some triples due to AllegroGraph's internal representation of typed literals. The failure would occur only if the literal was specified directly (i.e., when the literal was a variable, AllegroGraph behaved correctly).

Bug21548 - Nested unions with fixed patterns could miss results

If a SPARQL query contained nested unions and these unions contained fixed triple-patterns (i.e., patterns with no variables or patterns where all of the variables are bound via some external binding), then some results from these unions could be lost. This has been corrected.

Bug21546 - Correct off-by-one error in some SPARQL constraints

Depending on the data and the query, some SPARQL constraint analysis could lead to a range error. This has been corrected.

Bug21515 - Improve SPARQL parser speed

Compliance changes in the SPARQL parser of AllegroGraph v4.9 reduced the speed of parsing significantly for very large queries and update requests.

This has been corrected so that parse speed is now equivalent to what it was in AllegroGraph v4.8.

Bug21514 - Fix possible error caused by nested sub-query

The data structures used for SPARQL sub-queries were sometimes being cleaned up too early which could lead to a bus error. This has been corrected.

Bug21507 - Fix problem with property path query clause estimation

AllegroGraph was incorrectly estimating the size of clauses involving property paths in SPARQL queries which could lead to poor plans. This has been corrected.

Bug21478: Fix spurious parsing error when mixing sub-query and aggregation

AllegroGraph's SPARQL parser could become confused if a query mixed sub-query and aggregation and the sub-query used * to refer to all variables.

This has been corrected.

Bug21469 - Relative URIs were not being correctly normalized by the SPARQL parser

Section 5.2 of RFC3986 specifies how "." and ".." path components of relative URIs should be normalized when merging with a BASE URI. The original SPARQL parser implementation skipped this step in the interests of efficiency. This has now been corrected.

Bug21464: Some combinations of nested UNION and OPTIONAL could fail

The SPARQL 1.1 engine could fail to correctly keep track of solution bindings for queries with certain patterns of UNIONs and OPTIONALs. This has been corrected. #### Bug21501 - Queries with cross-products and empty RHS could emit incorrect solutions

If a query had a cross-product and the right-hand side of the cross-product was empty, then it was possible for AllegroGraph to not discard the solutions on the left-hand side. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug21535 - Db registry memory leak in the Lisp client

If a triple store object isn't closed properly and becomes garbage, the finalizer is supposed to close it. Fixed a bug that rendered this mechanism ineffective, causing leakage of memory.

Bug21512 - With-variables not handled consistently with sub-queries

In some cases, external bindings imposed from with-variables could be lost when using sub-queries. This has been corrected.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe11527: Support sessions that use the main server port

With this change, the Java client can now be configured to have sessions use the main server port rather than a dedicated port. By default, a dedicated port will be used. Setting the system property com.franz.agraph.http.useMainPortForSessions=true will configure the client to use the main port for sessions.

Rfe8384: Limiting the duration of a query

With this change, the Java client now supports limiting the duration of a query. When Sesame applications use the AGQuery#setMaxQueryTime method, the AG server will timeout the query appropriately during evaluation and clean up server resources, and the client library will throw a QueryInterruptedException back to the client application.

AllegroGraph 4.9 user-visible changes

AllegroGraph Server: General Changes

Rfe11922 - Add materiazer. See Materializer for details.

Rfe11879 - Improve error message on invalid license

AllegroGraph now prints information to the console when there are problems with the license settings in the configuration file.

For example, if the configuration file is missing the LicenseExpires parameter, then AllegroGraph will print: 

AllegroGraph Server Edition 4.9, built on October 01, 2012 10:04:33 GMT-0700  
Copyright (c) 2005-2012 Franz Inc.  All Rights Reserved.  
current-time: Monday, October 01, 2012 10:43:09 AM  
 
Daemonizing...  
Server started: LicenseExpires parameter missing. Running with triple-limit of 5,000,000.

Rfe11845 - Optimize blank node printing

Blank nodes now print slightly faster and more efficiently than they did before.

Rfe11827 - Reduce memory footprint a bit

Result arrays are now freed right when they become unused. Previously, garbage accumulated until a certain limit was reached. This should have a small positive effect on high water-mark memory footprint.

Rfe11784: Release memory back to the operating system after query failure

Prior to this change, AllegroGraph would continue to claim memory used for queries even after the query would complete. AllegroGraph will now return unused memory back to the OS when a query memory failure occurs.

Rfe11762: Improve the efficiency of queries with unbound results

Queries that use UNION or OPTIONAL can have unbound columns in the result set. This change improves the way AllegroGraph does joins when there are unbound columns in a result set.

Rfe11756: Improve the chunk processing behavior of ORDER BY and LIMIT queries

The chunk-at-a-time processor automatically reduces the size of the chunks being processed based on the limit (it makes no sense to process a chunk of 500,000 rows if you are only interested in the first one!). In the case of ORDER BY queries, however, it makes sense to keep the larger limit since all of the results must be examined. This is now the case.

Rfe11728: Improve handling of large DISTINCT or ORDERed results

This enhancement improves the efficiency of the on-disk merging operations so that the size of ORDERed or DISTINCT result sets is limited only by disk space and not by available RAM.

Bug21531 - Non-simple EXISTS FILTERs could cause query failure

AllegroGraph was incorrectly processing non-simple graph patterns in an EXISTS or NOT EXISTS FILTER. This meant that queries like 

SELECT * {  
   ...  
   FILTER( EXISTS { ... } ||  
           EXISTS { ... FILTER( ... ) } )  
}

could cause an error rather than being correctly processing. This has been corrected.

Bug21415 - Queries could return invalid solutions

It was possible for solutions with no bindings for some of their variables to escape from the right-hand side of a join. This would cause too many solutions to be returned from the query.

This has been corrected.

Bug21410 - Duplicate suppression not handling deleted triples correctly

AllegroGraph was handling deleted triples improperly when duplicate suppression was enabled. This has been corrected.

Bug21407 - An error in a SELECT expression prevented processing of subsequent ones

An error in a SELECT expression would prevent processing of the rest the expressions in the query for the current solution. For example, depending on the data, in a query like: 

SELECT  
        ?s  
        ?str  
        (STRAFTER(?str,"b"@cy) AS ?abcy)    # could cause a type error  
        (STRAFTER(?str,"ab") AS ?aab)  
WHERE {  
        ?s :str ?str  
}

The evaluation of STRAFTER() with a second argument of "b"@cy could signal a type-error. AllegroGraph was stopping further computation at that point which meant that the binding for ?aab could be left empty when it should not have been.

This has been corrected: now, each expression is computed separately.

Bug21404 - Config file EOL format

Previously, non-unix newlines (CR+LF, LF+CR) in the license section of the config file could prevent the server from starting up. This has been fixed and any mix of Unix, Mac and DOS style newlines is accepted.

Bug21398 - Variables in EXISTS filter could be incorrectly bound

It was possible for variables that appeared only in an EXISTS (or NOT EXISTS) filter to be incorrectly bound during query evaluation which could lead to incorrect results.

This has been corrected.

Bug21390: N-Triples serializer does not handle embedded angle brackets

Even though valid URIs are not allowed to contain embedded angle brackets, it is still possible to add them to a triple-store. For example, the valid N-Triples file: 

<ex://h> <ex://p> <ex://invalid\u003E>

would add a triple whose object is ex://invalid>.

AllegroGraph was serializing such URIs without performing the necessary \u escaping which meant that the data exported could not be re-read by AllegroGraph's parsers.

This has been corrected.

Note that this was only a problem for data with invalid URIs.

Bug21377 - Updates with both DELETE and INSERT

Due to a race condition in a shared backends, updates that involved both DELETE and INSERT could produce bogus results. This bug has been fixed.

Bug21376 - Wrong results with ORDER BY

Due to a rare race, in shared backends, queries with ORDER BY could produce bogus results. This has been fixed.

Bug21367 - Expressions in ORDER BY could cause query failure

Queries with expressions in an ORDER BY could cause query failure during string look up. This bug was introduced in AllegroGraph v4.8 and is now corrected.

Bug21349 - Eager evaluation of boolean FILTERS could lead to failures

If a query used FILTERS consisting of boolean combinations of simpler filters and also used external variable bindings such that some of the FILTER clauses could be evaluated immediately, then it was possible for the FILTER evaluator to signal an error.

This has been corrected.

Note that bug21342 corrected this problem for the simpler case of negation. This bug fixes the more general case.

Bug21345 - Possible problem with BIND inside of an OPTIONAL

If a query included a BIND clause that stood by itself and there was a join, it was possible to lose the value of the binding. In particular, a query like 

SELECT * { ?a ?b ?c . OPTIONAL { bind('hello' as ?x } }

could leave ?x unbound. This has been corrected.

Bug21344 - Bind could fail to use external variable bindings

If a query used the BIND form and the value of the expression depended on variables set via an external binding (e.g., via the with-variables argument in the Lisp API or via HTTP parameters in the HTTP interface), then it was possible for the BIND expression to ignore those values. This has been corrected.

Bug21342 - Eager evaluation in FILTERs can lead to problems using variables

If a query uses a FILTER and also variable bindings such that an expression in the FILTER could be evaluated before actually running the query and if the expression was used in other boolean combinations (e.g., with not, and, or or), then it was possible for AllegroGraph to signal an error. For example, if the query was 

select * { ?a ?b ?c . FILTER( ! bound( ?d )) }

and if ?d was bound using a variable binding, then AllegroGraph would signal an error instead of correctly realizing that the FILTER could never succeed.

This has been corrected.

Bug21341 - Non-simple IF statements could cause an error

It was possible for non-simple IF statements in an expression to lose track of their variables which led to an error at query planning time.

This has been corrected.

Bug21326 - Geospatial query failure

Fixed a bug that, under rare circumstances, caused geospatial queries to fail with bus error.

Bug21325 - Lost results in sub-queries and some joins

When an internal buffer holding intermediate results was grown in a sub-query or in some joins, results could have been lost. This bug has been fixed.

Bug21272 - No results with BIND

Sometimes, as in the query below, instead of no results, a single result was returned when BIND was involved. 

select * {  
  bind('o' as ?c)  
  ?a ?b ?c .  
  ?d ?e ?f .  
  filter( ?a < ?d && ?d < ?a )  
}

This bug has been fixed.

Bug21270 - Sub-queries with ORDER BY can fail to bind results

Depending on the data and what other queries have been evaluated, a query that used an ORDER BY clause in a sub-query could fail to correctly bind the result variables. For example, a query like: 

select ?age {  
  { select ?age {  
        ?who <http://people.example/age> ?age .  
    } order by ?age  
  }  
}

could return results but leave the bindings for ?age unbound.

This has been corrected.

Bug21260: Xpath Constructor Functions fail on strings to numbers

The XPath Constructor Functions should convert typed string literals into the requested numeric type when the value of the string can be so coerced. For example: 

xs:int('2'^^xsd:string) ==> 2

AllegroGraph was raising a type error in this case. This has been corrected.

Bug21256: Parser problem with some queries that mixed aggregation and sub-query

AllegroGraph was incorrectly signaling a parser error on some queries that mixed aggregation and sub-query. For example, 

select (max(?a) as ?foo) {  
  select distinct ?a ?b {  
    ?a a ?c .  
    ?c a ?b .  
  }  
}

This has been corrected.

Bug21245: Allegrograph's turtle parser performs URI normalization incorrectly

Percent encoded URIs should not be unescaped when loading data in the Turtle format. AllegroGraph was incorrectly doing so. This has been corrected.

Bug21243 - Non-QNameable URIs cause turtle serializer error

If a triple contained a URI that could not be expressed as a QName, then the turtle serializer would signal an error that it could not split the URI rather than just emitting the entire URI and continuing. For example, a triple containing the URI <<ttp://example/> would cause this error: 

Cannot split <http://example/> into XML QName.

This has been corrected.

Bug21226: Agload max blank node cache size not obeyed in all cases

Agload could over-allocate memory in a certain case. This has been fixed.

Bug21215 - Improve cursor resource usage of Prolog SELECT queries

AllegroGraph previously closed some of the cursors created while running a Prolog query lazily. It now closes them when the query completes which improves overall resource utilization.

Bug20906 - Remove duplicate triples fails on a store with zero triples

An error would be raised if duplicate deletion was attempted on a store with zero triples.

This has been corrected.

HTTP Client

Rfe11755 - Add support for insert-graph-uri, remove-graph-uri, etc.

Version 6 of the Sesame HTTP Protocol includes several graph parameters that can be used to control the behavior of SPARQL Update statements. These are:

These four parameters are now supported by AllegroGraph

Rfe11743: Add baseURI to the SPARQL HTTP Protocol

The SPARQL HTTP protocol now accepts baseURI as an HTTP query request parameter. This can be used to shorten the rest of a SPARQL query.

Bug21408 - Retrieving JSON statements over HTTP

Fixed a bug that caused the statement list to be truncated when an 'application/json' encoded response was requested via the HTTP protocol.

Bug21280 - Query failures cause spurious errors

AllegroGraph 4.8 modified the internal API used to signal query failures (e.g., due to timeout or memory limits). Unfortunately, the code in the HTTP layer of AllegroGraph was not modified to accommodate the change which led to spurious bus errors when a query failed. This has been corrected.

SPARQL

Rfe11862 - Match default dataset behavior for SPARQL Update and Query

SPARQL Update was using a default dataset that contained only triples with unnamed graphs (the default graph in AllegroGraph terminology) in the FROM portion and put triples with graphs into each of their graphs in the FROM NAMED. This was different from the default dataset for SPARQL Query which put all of the triples into the FROM portion and treated the FROM NAMED portion similarly.

AllegroGraph now uses the same default dataset behavior for both Query and Update. This dataset is as described for query above. Note that this means that an Update command like 

DELETE DATA { <eh://a> <eh://b> <eh://c> }

will delete the triples 

{ <eh://a> <eh://b> <eh://c> }

from every graph in which it appears whereas the previous behavior would have been to delete it only from the default graph.

Note that triples added during an Update command which do not specify a graph will be added to the triple-store's default graph.

Rfe11817 - SPARQL parser should signal an error on badly formed requests

Previously, the SPARQL parser would silently accept nonsense queries like: 

<http://www.foo.com> <http://www.bar.com> 'banana' .

because it would silently ignore tokens that came before the SPARQL prologue (i.e., before PREFIX and BASE) and convert such a query into an empty UPDATE request (i.e., a non-operation).

These queries will now signal a SPARQL parse error. Because the parser is now more strict, some queries that previously parsed may now signal an error. For example, this invalid query was previously accepted and now will not be: 

# WRONG: bad prefix because it ends with a period (".")  
PREFIX ex: <http://example.com> .  
SELECT * { ?s ex:testsFor ?o . }

Rfe11793 - Improve handling of numeric and date literal equality filters

Improve the SPARQL 1.1 query planner's handling of numeric and date literal equality filters. In particular, AllegroGraph will now optimize queries with FILTER clauses like: 

?s ex:p1 ?o .  
FILTER( ?o = 3 )

which would previously have required a scan of all triples with predicate ex:p1.

Rfe11787 - Support SPARQL 1.1 UUID and STRUUID functions

Adds support for the SPARQL 1.1 UUID() and STRUUID() functions.

Rfe11779 - Support SPARQL 1.1 change to in-line data syntax

SPARQL 1.1 recently switched from a syntax for in-line data that used BINDINGS to one that uses VALUES. The standard also made it possible to use in-line data in more places. AllegroGraph now fully supports this feature and passes all of the Data Access Working Group (DAWG) tests for it.

Note that the BINDINGS syntax is still supported in this version of AllegroGraph. It will be deprecated and removed in some future release.

Rfe11757 - Provide select-N strategy for ordered results with small limit

Typically, a SPARQL query with an ORDER BY clause must accumulate all of the results before sorting them. If, however, the query has a small LIMIT, then it can be more efficient to retain the top LIMIT results and only sort those.

Rfe11555: Manage memory used to ensure unique results from CONSTRUCT and UPDATE

The data structures AllegroGraph uses to ensure that results of a SPARQL CONSTRUCT or UPDATE are unique have been modified so that result sets can be processed correctly regardless of size (up to the amount of available disk space).

Bug21401 - SPARQL Update must intern strings from variable bindings

If variable bindings are used in a SPARQL Update query and these bindings are part of the template being used to insert new triples into the store, then the strings from the variable values must be added to the triple-store's string table. This was not being done which could lead to a UPI not in string table error.

This has been corrected.

Bug21391 - Queries against in-memory-triple-stores could return incorrect results

The Lisp SPARQL API allows for the creation of in-memory-triple-stores from CONSTRUCT and DESCRIBE queries. These suffered from a bug such that queries that specified more than two of subject, predicate, object and graph fields could return incorrect triples. As examples, a query like: 

ASK { ex:foo ex:bar ?o }

would work correctly whereas a query like 

ASK { ex:foo ex:bar ex:baz }

could return yes when it should have returned no.

This has been corrected.

Bug21372 - Possible error when requesting sparql-xml output for Prolog select

An error could be signaled if the default query engine was sparql-1.0 and a results format of sparql-xml was requested for a Prolog select query.

This has been corrected.

Bug21371: IF confusion with free variables

SPARQL queries with free variables could produce incorrect results when using IF. For example, the following query 

SELECT ?result {  
  BIND( IF(!bound(?dataType), "first", "second") as ?result)  
}

would return second instead of first. This occurred because the query engine was not correctly analyzing the free variable ?dataType (this variable is free because the query provides not binding for it).

This has been corrected.

Bug21361 - Improve SPARQL BASE URI handling

AllegroGraph was not always correctly merging base and relative URIs. For example, a query like 

BASE <http://localhost:8080/>  
SELECT * { ?a ?b </> }

was being incorrectly run as if it was 

SELECT * { ?a ?b <http://localhost:8080//> }

This and other similar issues have been corrected.

Bug21357 - SPARQL blank-node parsing was too strict

The SPARQL parser was not allowing the full range of blank node names to be used. Thus, queries like: 

SELECT * {  
  ?a ?b _:123 .  
  _:123 ?c _:fv-0 .  
}

would fail to parse correctly. This has been corrected.

Bug21354 - Update STRAFTER() and STRBEFORE() to match changes in the standard

Recent changes to the SPARQL 1.1 standard changed the output of STRAFTER() and STRBEFORE() when no match is found. Previously, they were specified to return an empty literal with language or datatype that matched the first argument. Now they should produce an empty literal with neither language nor datatype. AllegroGraph now matches the standard.

Bug21294 - Sparql XML results for ASK queries missing head clause

The SPARQL XML generated by AllegroGraph was missing the <head/> clause. Since this clause is generally empty, only systems that validate the results against an external DTD were effected by the missing clause.

This has been corrected.

Bug21277 - Improve cursor resource usage of SPARQL 1.1 queries

AllegroGraph previously closed some of the cursors created while running SPARQL 1.1 queries lazily. It now closes them when the query completes which improves overall resource utilization.

Bug21266: Queries with only a geospatial clause could fail to gather solutions

If a SPARQL query contained only a geospatial clause, then the SPARQL 1.1 engine could fail to add the solutions it found to the answer set. This has been corrected.

Bug21249: Incorrect return value of COPY GRAPH <X> TO GRAPH <X>

A SPARQL Update statement like 

COPY DEFAULT TO DEFAULT

should have no effect on the triple-store and should succeed. AllegroGraph implemented the behavior correctly but was returning failure instead of success. This has been corrected.

Bug21239 - Concurrent queries in shared backends

With the sparql-1.1 engine, queries running in the same shared backend concurrently had unexpected interactions that lead to various errors. This has been fixed.

Bug21237: Computations in aggregate expressions could fail

If a complex expression was used inside of a SPARQL aggregation function, the value could be left unbound. A work-around was to express 

(min(2 * ?x) as ?smallestDoubled)

as 

(min(?x) as ?smallest) (2*?x as ?smallestDoubled)

This is now corrected and the work-around is no longer necessary.

Bug21233: SPIN functions could fail when used with constants

SPARQL queries that used SPIN functions could fail if the arguments to the functions were constants not already in a triple-store's string-table. This has been corrected.

Bug21222: Failing SPARQL UPDATE HTTP request should error

Previously SPARQL UPDATE requests that failed would return false in SPARQL/XML format. In order to be better conform with other SPARQL endpoints, AllegroGraph now returns a 412 status code (precondition failed) instead.

Bug21163: Add new default-dataset-behavior: rdf

SPARQL leaves things up to the implementation when no explicit dataset is specified (i.e., when no FROM or FROM NAMED clauses are part of the query). AllegroGraph now provides three choices for dataset behavior:

AllegroGraph's standard behavior continues to be :all. You can change this on a per-query basis using the defaultDatasetBehavior query option: 

PREFIX franzOption_defaultDatasetBehavior: <franz:rdf>

AGWebView

Rfe11910 - Add HTTP handlers for materialization

Added the materializeEntailed service.

A PUT request will materialize entailed triples based on the with, without, useTypeSubproperty, and commit query parameters.

A DELETE request will remove the materialized triples.

Rfe11856 - WebView: new UI Beta for Graph

In WebView next to the Documentation menu is a link to "WebView Beta" a new experimental UI for Query and Graph. A link at the top will returns to the classic WebView.

A link at the top will open the new Graph View UI.

Also Node queries have a View menu in the results to open the node in Graph view.

Administration features of WebView are not yet included within the new UI.

Rfe11676: Webview: in Scripts editor, use CodeMirror JavaScript

The WebView Scripts editor now supports syntax highlighting and indentation by including the CodeMirror JavaScript mode.

rfe11596: WebView: new UI Beta for Query

In WebView next to the Documentation menu is a link to WebView Beta a new experimental User Interface (UI) for Query. The new UI provides a modern style and easier navigation. A link at the top will returns to the classic WebView.

All existing features of WebView Query are implemented:

Administration features of WebView are not yet included within the new UI.

Rfe11582: Webview: upgrade Query Editor to CodeMirror v2.32

The query editor for SPARQL and Prolog, implemented by CodeMirror, has been upgraded from v2.24 to v2.32.

Bug21250: Webview: remove limit from download

The download link for Query results no longer uses the limit parameter in the query UI.

Changes to the Lisp API

Rfe11454: get-triples-list (Lisp API) returns cursor only on request

The Lisp get-triples-list function previously returned the cursor it created as a second value. Now, however, it returns a cursor if either one was passed in initially (using the cursor keyword argument) or if the new keyword argument return-cursor is true.

Bug21381 - (Lisp API) SPARQL introspection functions required engine argument

The functions get-allowed-results-formats and get-allowed-rdf-formats returned nil if no query engine was provided. This has been corrected.

Bug21378 - (Lisp API) sparql-1.1 engine does not implement destination-db

The Lisp API allows SPARQL Update commands to specify the triple-store that they modify. This feature was not included in the sparql-1.1 engine. This has been corrected.

Documentation

No significant changes.

Python Client

Rfe11911: Add materializer calls to python client

Added materializeEntailed and deleteMaterialized calls to RepositoryConnection.

Java Client

Rfe11906 - Java support for the materializer

With this change, the Java client now supports materialization, allowing inferences to be generated and added to a repository in a transactional fashion. See the Javadoc for AGMaterializer#newInstance() to acquire a configurable materializer, and AGRepositoryConnection#materialize to run the materializer.

Rfe11771: Upgrade embedded JavaScript implementation

Fixes several small bugs in the way uploaded JavaScript scripts are interpreted. The most visible one was that the !== operator was acting like the < operator.

Rfe11241 - Support more query/update protocol params

With this change, the Java client now sends HTTP protocol params for update, baseURI, timeout, remove-graph-uri, insert-graph-uri, using-graph-uri, using-named-graph-uri, and logQuery.

AllegroGraph 4.8 user-visible changes

AllegroGraph Server: General Changes

Rfe11711 - Improve cursor resource usage of Geospatial Prolog queries

AllegroGraph previously closed cursors created while running a Prolog geospatial query lazily. It now closes them when the query completes which improves overall resource utilization.

Rfe11653 - Rm requests can be processed out of order

The order in which file management requests were processed caused unexpected errors under heavy load. The bug is now fixed.

Rfe11649 - Manage memory for ORDER BY and DISTINCT queries

Previously AllegroGraph needed to accumulate all results in memory for queries that used DISTINCT or ORDER BY. This could greatly increase the overall memory footprint. AllegroGraph now handles large result sets in available memory.

Rfe11631 - Improve query optimizer when inference is enabled

One part of query optimization is reordering the triple-patterns in each BGP. This reordering is based on the statistics collected by the store and these statistics do not take inference into account. This meant that the plan produced could be quite bad.

This has been corrected by modifying the estimation code to take RDFS++ into account.

Rfe11416 - Provide greater control of query memory usage

Added a memoryLimit query option that controls how much memory a query can use. If a query needs to allocate more memory than this value, it will fail.

Bug21235 - Resource manager sync-files effectively ignores msync errors

If the filesystem on which a database resided filled up during a checkpoint, this fact was logged but the checkpoint process would continue running as if the error didn't happen, which would result in database corruption. This bug has been fixed. Now when a full filesystem is detected during a checkpoint, AllegroGraph stops operating in order to protect committed data.

Bug21224 - Crashes while processing queries with DISTINCT

Under rare circumstances, the internal data structures used while processing DISTINCT queries could get corrupted leading to segmentation faults. This bug has been fixed.

Bug21214 - Correct scope of BINDings introduced on the LHS of union

The binding of a variable on the left-hand side of a UNION should not influence its value on the right-hand side. For example, the following query should produce two rows with ?a bound to 1 in one of them and left unbound in the other. 

select ?a {  
  { bind(1 as ?a) } union { bind(2 as ?b) }  
}

AllegroGraph was retaining the binding on the LHS across to the RHS of the union and so was answering with two rows each of which had ?a bound to 1. This has been corrected.

Bug21187 - Stack exhaustion with shared cursor

With lots of deleted triples a query could run out of stack when creating cursors. This has been fixed.

Bug21182 - Unbound values in distinct aggregrates could lose results

If a variable binding used in a distinct aggregate was unbound, it could cause other aggregrates in the same row to fail to update properly. For example, a query like: 

SELECT (count( distinct ?taste) as ?tasteCount ) (count(?who) as ?whoCount) {  
	   ?who <http://people.example/age> ?age .  
	   OPTIONAL { ?who <http://people.example/taste> ?taste . }  
}

could lose data for the ?whoCount bindings if the ?taste binding from the OPTIONAL was unbound. This has been corrected.

Bug21145 - agload: message corrected for insufficient buffer size

The error message for insufficient ntriples/nquads buffer size was reporting buffer as size -1. This message has been corrected and now reports actual buffer size in order to abate confusion about the condition.

Note: This condition is usually triggered when an ntriple of greater than 4MB is encountered while using the default dispatch strategy.

Bug21136 - Free BGP patterns can lead to very large memory allocation requests

Queries with free patterns (like ?s ?p ?o) in a BGP on the right-hand side of a query could lead the query size estimator to believe that it should reserve a very large space to save the putative result set. For example, a query like 

SELECT ?s WHERE {  
    { ?s ?p ?o }  
    MINUS  
    { ?other ?p ?o . ?other ?someProp ?s }  
}

would lead the query engine to try and allocate 1024-Petabytes. This has been corrected.

Bug21124 - Zero length path between non-matching constants

AllegroGraph was incorrectly finding a path between two different constants which led to queries like the following to return a single (empty) result when they should have returned no results at all. 

select * {  
  ex:one ex:connectsTo{0} ex:two .  
}

This has been corrected. Other zero-length path queries were behaving correctly.

Bug20853 - Updated the soundex algorithm

AllegroGraph's freetext indexing word filter was using an older variant of the Soundex algorithm that did not match the latest version (cf. http://www.archives.gov/research/census/soundex.html). The algorithm has now been updated.

Freetext indexes created with the soundex word-filter should be dropped and re-built.

HTTP Client

Rfe11681 - HTTP: Allow statements queries using the POST method

Add a [store]/statements/query URL that corresponds to /statements, except that it isn't limited to the GET method, and thus can be used to do complex queries without building up overlong URLs.

Rfe11680 - HTTP: Add /unique/[column] service

Add a [store]/unique/[column] URL that allows clients to fetch the set of unique parts that occur in a column in the database, optionally restricting other columns.

SPARQL

Rfe11682 - Improve performance of SPARQL OPTIONAL queries

If a query has the form A OPTIONAL B (where A and B share variables so that it is a join) and the A part has no rows, then it is not necessary to evaluate the B part at all.

The SPARQL 1.1 query engine now recognizes this situation and skips the evaluation of the right-hand side.

Rfe11591 - Modified the Lisp run-sparql API to better handle query failure (e.g., for when a query runs out of time or other system resources).

Previously run-sparql would return (as multiple values):

Now run-sparql returns four values. In normal execution, the first three values are the same and the fourth value will be nil. When a query fails, the results will be:

Code that requests a cursor should be prepared to handle a condition of type db.agraph.query.base:cancel-query-condition.

Rfe11581 - handle repeated Prologues in SPARQL UPDATE queries

SPARQL 1.1 UPDATE allows the Prologue (BASE and PREFIX definitions) to be repeated between each UPDATE command. Previously, AllegroGraph only allowed BASE and PREFIX to be used at the beginning of a SPARQL statement. This is now corrected.

Rfe11559 - Add logQuery parameter to the HTTP SPARQL protocol

SPARQL queries can now use the logQuery parameter in their HTTP requests. The parameter can be set to one of:

The log output can be cryptic; please contact Franz Inc. for support in interpreting the messages.

Bug21220: SPARQL Update LOAD should require only write permission (not eval)

A user would need to have evaluate permission on a triple-store in order to use SPARQL Update's LOAD command to load data from a file. This has been changed so that having write permission is sufficient.

Bug21203 - SPARQL Update insert data fails to treat blank nodes correctly

The INSERT DATA and MODIFY SPARQL Update commands failed to use the small blank node identifier in the INSERT template. This caused multiple blank nodes to be created when there should have been only one.

Bug21192 - Upi not in string table error when mixing BIND, SPARQL Update and FTI

SPARQL Update queries that produced new strings (e.g., using BIND) on a triple-store with free-text indexing turned would lead to UPI not in string table errors. Note that in this case a triple would have been added to the store but its strings would not be added to the string table which could cause subsequent queries to fail until the errant triple was removed.

This has been corrected.

Bug21168 - Support SPARQL 1.1 COUNT(DISTINCT *)

The SPARQL 1.1 query engine was not correctly handling the COUNT(DISTINCT *) form. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

Rfe11697 - Added RepositoryConnection.prepareUpdate

Added RepositoryConnection.prepareUpdate for SPARQL 1.1 Update requests. The call to evaluate on the returned UpdateQuery object returns True or False.

Java Client

Rfe11748 - Upgrade to sesame 2.6.8

With this change, the Java client has been upgraded to use openrdf sesame 2.6.8 jars.

Rfe11243 - Support X.509 authentication

With this change, the Java client now supports X.509 server and client authentication. For set up details, see the javadoc for AGServer.

Bug21225 - With no requestEntity, post using setRequestBody

Previously, AGHTTPClient post method used setQueryString to pass params when there was no requestEntity provided. This burdened the server when a param became very large. Now params are passed using setRequestBody when no requestEntity has been provided.

See Change History for user-visible changes to earlier releases.