Table of Contents

Release 4.13.1

Major Changes in release 4.13.1

Special Heartbleed OpenSSL bug Update Notice

Major Changes in release 4.13

Major Changes in release 4.12.2

Major Changes in release 4.12/4.12.1

Major Changes in release 4.11

Changes in earlier releases

AllegroGraph 4.13.1 user-visible changes

AllegroGraph Server: General Changes

AllegroGraph 4.13 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Prolog

Documentation

Python Client

Java Client

AllegroGraph 4.12.2 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Prolog

Documentation

Python Client

Java Client

AllegroGraph 4.12 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Prolog

Documentation

Python Client

Java Client

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

Release 4.13.1

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 4.13.1.

Major Changes in release 4.13.1

Version 4.13.1 is a maintenance release which fixes a bug having to do with index optimization which was revealed after the release of version 4.13. That is the only change in this release compared to version 4.13.

Special Heartbleed OpenSSL bug Update Notice

Because AllegroGraph uses OpenSSL, the Heartbleed OpenSSL bug can introduce a security leak. See this Allegro CL Tech Corner article at http://franz.com/support/tech_corner/heartbleed040914.lhtml for important information on updating AllegroGraph with the fix to the OpenSSL Heartbleed bug.

Major Changes in release 4.13

Version 4.13 has fixes and enhancements in the following areas:

Major Changes in release 4.12.2

Release 4.12.2 is in part maintenance release, providing fixes and enhancements, but also includes the following significant changes:

Major Changes in release 4.12/4.12.1

The 4.12.1 release provided a fix to a startup problem experienced by some 4.12 users. It contained no other changes. The 4.12 release includes many bug fixes and incremental improvements and also

Major Changes in release 4.11

Changes in earlier releases

This document lists changes in the current release and several previous releases. See the Change History for a list of all changes in earlier releases.

AllegroGraph 4.13.1 user-visible changes

AllegroGraph Server: General Changes

Bug22255 - Concurrent full index optimization requests

Previously, if index optimization was requested with a full index optimization already in progress, the database instance would log an error and hang. This bug has been fixed.

AllegroGraph 4.13 user-visible changes

AllegroGraph Server: General Changes

Rfe12645 - Add a command line interface to AllegroGraph's materializer

AllegroGraph now includes a command line tool to materialize the inferred triples in a triple-store. The tool is named agmaterialize. It can currently be run only on the same machine where the AllegroGraph server is running. The command line interface is described in the Materializer documentation.

Rfe12642: Improve handling of datafile free space information

Previously, datafile free space information was maintained in a fixed sized data structure which could run out of room after a certain level of datafile fragmentation was reached. This has been corrected.

Rfe12590 - Shared backend scheduling improvements

Previously, when all shared backends were busy, further requests were held until a backend became available. Such pending requests were scheduled for execution in basically random order with latency up to 1s.

With this change, pending requests are scheduled rather than essentially random. The scheduling is basically FIFO (first in, first out, meaning the oldest request is processed first), but FIFO may be violated for efficiency reasons. (Strict FIFO behavior is undesirable, because it greatly reduces AllegroGraph's ability to schedule requests optimally under stress.)

Also with this change, scheduling latency is now minuscule and shared backends are now allowed to execute slightly more queries in parallel if all backends are busy, which allows greater efficiency in most cases.

Rfe9523 - Logging improvements

This change adds a centralized logging facility to the AllegroGraph daemon. The new logger guarantees no interleaving of messages and is able to deal with full filesystem conditions gracefully. Lisp direct clients have now the option of sending log messages to an AllegroGraph server with the with-log-routed-to-server macro. See the documentation for more. Previously, log messages were sent to the AllegroGraph server belonging to the first direct connection opened in the client.

Bug22220 - Race condition while determining last written tlog id

Previously, a rare error could be encountered when starting replication or while switching roles, indicating that replication failed due to a missing transaction log that had not yet been written to disk. This problem has been fixed. Now, only transaction logs on disk are used to start replication.

Bug22184 - Instance shutdown and replication

In rare circumstances, when the instance belonging to a replicated repository was shut down, writing the transaction log could fail. This bug has been fixed.

Bug22156 - date-string-to-upi doesn't handle zero time zones consistently

The date-string-to-upi functor was not treating a Zulu time zone specified as "Z" consistently with the rest of AllegroGraph's date and time parsers. This has been corrected.

Bug22111 - Database names may not contain special characters

Because it is impossible to reliably use databases (also called repositories) in session specifications if their names contained characters that are used in those specifications, the list of characters which must not be used in names has been expanded. The following characters are now disallowed in database (repository) names:

\ / ~ : Space $ { } < > * + [ ] | 

Previously, only the following four characters were disallowed in database (repository) names:

\ ~ / : 

Note that this change only effects new databases (repositories); existing databases (repositories) with any of the newly invalid characters in their names will continue to be accessible. It may, however, result in difficulties when attempting to use them in more complex session specifications.

Bug22100 - More strict, faster parsing of some xsd datetimes

The parsers for xsd:date, xsd:dateTime and xsd:time used to be lenient and accept things like:

20131118         # missing separator  
2013-11-18T12:27 # missing seconds 

and many other convenient but non-standard formats. The new parser only accepts what the XML Schema specification mandates which also allows it to be faster. See XSD Date and Time datatypes for more information on these datatypes.

Bug22086 - Materializer deletes triples with the same subject, predicate and object

The initial version of AllegroGraph's materializer deleted all triples with the same subject, predicate and object. This limitation has now been lifted: the materializer now only deletes triples with matching subject, predicate, object and graph.

Bug22032 - Control whether or not external references are followed during RDF/XML import

Previously, AllegroGraph ignored external references in RDF/XML source files. The Lisp API now includes a new keyword argument to load-rdf/xml:

agload also has a new command-line switch. Supplying either of -x or --external-references will cause agload to follow external references. See Data Loading.

Bug21953 - Allow import of triples to reasoning-triple-store

A bug was introduced in v4.12 that prevented import of triples via load-* routines (e.g. load-rdf/xml) on reasoning-triple-stores, when the source argument was passed a URI. This problem has been fixed. This change also provides a better error message when attempting to import statements to a federated-triple-store, which is not allowed.

Bug21656 - Cleaner daemon shutdown

When the AllegroGraph daemon was stopped, despite the claim of doing a 'clean shutdown' in agraph.log, the shutdown of the instances was not clean and the corresponding repositories would potentially require crash recovery the next time they were opened. Also, such a shutdown lost any index optimization work performed since the most recent checkpoint.

With this change, instance processes perform a checkpoint if necessary which makes shutdown slower, but startup faster and it makes sure the index optimization results persist.

Bug20773 - Support federations of RDFS++ stores

Previously, queries against a federation of RDFS++ stores would cause an error during query planning. This has been corrected.

HTTP Client

Rfe12763 - Update HTTP API for the materializer

The REST API for materializeEntailed now includes an inferredGraph argument that can be used to specify which graph newly entailed triples are added to (or deleted from). The materializer is described in Materializer.

SPARQL

Rfe12767 - Improved the efficiency of single group aggregate queries

The code generated for single group aggregate queries is now slightly more efficient.

Rfe12762 - Optimizing level 2 index optimization

With this change, level 2 index optimization (the default) no longer tries to optimize access for triples added since optimization was started. This allows index optimization to finish even if triples are added continuously at a high rate. The new code is faster on average, has the same worst case disk usage (about two times the original size), but it has a higher average disk usage.

Rfe12760 - Improve efficiency of some SPARQL Property Path queries

AllegroGraph now builds some property path data structures as needed rather than computing all of them up front. Depending on the data, this can greatly enhance the efficiency of some queries.

Rfe12757 - Improve efficiency of some property path queries

In some circumstances, AllegroGraph could choose the less efficient of two strategies when evaluating a property path query. AllegroGraph now ensures that it makes the better choice.

Rfe12717 - Handle RDFterm-equality of strings consistently

Previously, AllegroGraph implemented sameTerm() using a strict interpretation of RDF literal equality when comparing plain literals and typed literal strings but implemented = using a more relaxed interpretation. These tests are now done consistently using the relaxed interpretation. In particular,

'hello' is RDFterm-equal to 'hello'^^xsd:string 

This is in accordance with both common usage and the RDF 1.1 specification.

Rfe12699 - Improve efficiency of SPARQL equality tests on resources

The efficiency of equivalency tests in SPARQL filter expressions has been improved.

Rfe12681 - Optimize interprocess communication

The performance of message passing between different AllegroGraph server processes has been slightly improved.

Rfe12679 - Use query local namespace abbreviations in result serialization

AllegroGraph now uses the namespaces defined by a SPARQL query's prefixes when serializing output in RDF/XML. Previously, temporary namespaces were used instead.

Rfe12670 - Allow /SPARQL access to AllegroGraph as a SPARQL endpoint

Previously, access to an AllegroGraph triple-store as a SPARQL endpoint was via sending a query to

SERVER:PORT/respositories/NAME?query=... 

This REST API has been augmented so that sending a query to the triple-store's URL with a trailing /sparql also works. E.g.,

SERVER:PORT/respositories/NAME/sparql?query=... 

This is not required by the SPARQL standard but is in conformance with many other existing endpoints.

Rfe12657 - More efficient handling for the VALUES clause

AllegrGraph now rearranges queries so that VALUES clauses are evaluated first when possible. This improves query efficiency as VALUES clauses are generally small.

Rfe12656 - Improve execution of some nested joins

AllegroGraph now evaluates nested joins more efficiently.

Rfe12498 - Improve full-scan warning heuristics

AllegroGraph warns when a SPARQL query executes a full-scan. Heretofore, this meant that a query like

select * { ?s ?p ?o } limit 100 

could have produced a full-scan warning even though it is benign. AllegroGraph now saves its warnings for when they may actually indicate a problem with the query.

Rfe11981 - New query option chunkProcessingMemory

Previously when chunk-at-a-time processing was enabled (see query option chunkProcessingAllowed), the query option chunkProcessingSize controlled memory consumption by limiting the number of intermediate results stored in memory. The total memory consumption depended on the number of variables and the complexity of the query, so it was rather difficult to know how much a query will actually use. With this change, chunkProcessingSize is deprecated and chunkProcessingMemory can be specified instead. The unit of chunkProcessingMemory is a byte but the usual abbreviations are also accepted ("8G" (the default), "2M").

Rfe11515 - Fully support the SPARQL 1.1 HTTP Protocol

AllegroGraph now fully supports the five different HTTP request types specified in the SPARQL 1.1 HTTP Protocol. These are:

In addition, AllegroGraph continues to support SPARQL update using POST where the update command is encoded in the query portion of the HTTP request.

Bug22214 - Encoded IDs failed to compare properly in SPARQL queries

Inequality comparisons between encoded IDs were always returning true. This has been corrected.

Bug22204 - decimal to decimal cast loses precision

When in a SPARQL query an xsd:decimal was cast to xsd:decimal it lost precision as it was converted to a single float internally. This bug has been fixed.

Bug22203 - Fix decimal parsing in SPARQL queries

In SPARQL query strings, decimals in the (-1,0) interval were parsed with the wrong sign. For example, -0.9 was parsed as 0.9. This bug has been fixed.

Bug22153 - Some wild card property path queries could generate incorrect paths

In rare circumstances, the data structures used to cache property path information could be invalidated leading to too many path results. This has been corrected.

Bug22147 - Add support for ObjectId's in the _id field.

The SPARQL MongoDB magic predicate now works with documents containing ObjectId's in the _id field.

Bug22142 - Expressions on aggregates fail for empty group

If a query wrapped an aggregate with an expression like

select (xsd:float(avg(?age)) as ?avgAge) ... 

and the result set contained an empty group (i.e., a group with no members), then AllegroGraph would signal error. This has been corrected.

Bug22134 - SPARQL aggregates on expressions with errors could compute incorrect results

Other than COUNT, a SPARQL aggregate on an expression should return unbound if that expression evaluates to a type error during computation. AllegroGraph was incorrectly returning the aggregate of any bindings produced after the last type error. This has been corrected.

As an example, if AllegroGraph needed to compute the MIN( ?p * 2 ) and ?p was bound to 4, 7, 15, "hello", 6, and 1, then AllegroGraph would previously have returned 2 (the minimum of 12 and 2). Now, it will return no binding for that expressions.

Bug22123: Empty construct template will cause an error

If a SPARQL CONSTRUCT had an empty template like

CONSTRUCT {} WHERE { ?s a ?o } 

then AllegroGraph would signal an error. This has been corrected.

Bug22094 - Correct sparql< and sparql> for date and dateTime comparisons

Because there is no defined operator mapping for inequality comparisons between xsd:date and xsd:dateTime, comparing them should throw a SPARQL type-error. AllegroGraph was incorrectly returning nil instead. This has been corrected. See XSD Date and Time datatypes for more information on these datatypes.

Bug22074 - DELETE templates with unknown variables caused an error

AllegroGraph would signal an error if a SPARQL DELETE/WHERE command's DELETE clause contained variables that were not bound by the WHERE clause. This has been corrected. An example of such a command would be:

DELETE { graph ?unknownGraph { ?s ?p ?o } }  
WHERE { ?s ?p ?o }  
 

Bug21561 - Remote-triple-stores do not support the table results format

Remote-triple-stores did not support returning tables from SPARQL queries. This has been corrected.

AGWebView

Bug22161 - Adding security filters in Firefox

Previously, with Firefox, clicking the 'add' button in the security filters table of the user management page (Admin/users in the menu) of AGWebView did not submit the form. This bug has been fixed.

Changes to the Lisp API

Rfe12632: Improvements in Lisp API to AllegroGraph materializer

Materialization previously occurred only for triples in the default-graph of the triple-store. It now occurs for all of the graphs in the store. Materialization on subsets of the graphs can be accomplished by creating a graph-filtered triple-store and calling materialize-entailed-triples on it.

Materialization previously placed inferred triples into a special graph that could not be utilized by the rest of AllegroGraph (the special graph behaved similarly to but not exactly like the usual AllegroGraph default-graph). It now places inferred triples into a graph that can be specified in the call to materialize-triples using the materialized-graph argument. It defaults to .

Two new arguments were added to delete-materialized-triples. These are:

Materialization now logs its start and end to the AllegroGraph log file. The overall efficiently of the materializer has also been improved.

Bug22132 - calling run-sparql with the :with-variable argument with a timezoned upi

When the Lisp function run-sparql was used in conjunction with a :with-variable argument that bound a variable to an encoded xsd:dateTime or xsd:time, then in some cases comparisons involving that variable failed to match equivalent representations of the same xsd:dateTime or xsd:time with a different timezone. This bug has been fixed.

Prolog

Bug22171: Select0 should return UPIs when it can

Previously, the Prolog select0 query macro could return future-parts rather than UPIs. For example, this query would return a future-part.

(select0 (?a) (= ?a !rdf:type)) 

This could cause problems in the unlikely event that namespace abbreviations changed when running the same query multiple times. select0 now returns UPIs so that the above behavior is not possible. Note that if a future-part represents a triple that is not currently in the triple-store, then a query can return UPIs that cannot be printed (because the UPI will not be in the string dictionary).

Bug22157 - Temporal functors could signal an error when used with all bound values

Previously, AllegroGraph used real numbers to represent times, dates and dateTimes. Now, the internal representation includes time-zone information which means that comparing two values can no longer use simple inequalities. The Prolog temporal functors like point-before-datetime had not been updated to reflect these changes. This oversight has been corrected. This change means that the Prolog functors will use the same XSD logic used by the SPARQL date and time functions.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

AllegroGraph 4.12.2 user-visible changes

AllegroGraph Server: General Changes

Rfe12734 - More aggressive level 2 index optimization

In some cases, level 2 index optimization (the default) left a small part of the index unoptimized. With this change, the entire index is optimized in all cases.

Rfe12726 - Index optimization increasing database size on disk

Space in some garbage on-disk data structures is now reclaimed in a more timely manner which makes the increase in database size during index optimization less severe.

Rfe12725 - Faster level 2 index optimization

Level 2 index optimization (which is the default level) has been made faster.

Rfe12636 - Make config-file argument to agraph-backup backup-settings optional.

Previously, the agraph-backup backup-settings command required a config-file argument. This argument has been eliminated in favor of an optional --config option that can be used in place of the --port argument. If both --port and --config are supplied, then the port argument takes precedence. Also, agraph-backup could fail in the past if there were discrepencies between a --port argument specified on the command line, and a Port directive contained in a config file. Now, if the --port argument is specified, it will take precedence over any Port directives read from a config file. See Backup and Restore.

Rfe12635 - Major revision to agraph-backup operating modes.

The various backup and restore modes of agraph-backup have been overhauled.

Previously, backup mode created an archive file containing only repository data. Now, it creates an archive directory that contains both repository data and settings. This archive can be restored via restore or restore-all. The --ext-backup-command option is supported. Previously, restore mode operated on an archive file. Now, it can also operate on an archive directory created by backup or backup-all. In either case, it will restore a single repository (data and settings) from the archive.

The layout of archive directories created by backup-all are slightly changed, and support for the --ext-backup-command option is added. The restore-all mode will now fail if repositories being restored to already exist. Use the --supersede option to force an overwrite of any such repositories.

Previously, the backup-settings mode created an archive containing both repository and system settings. It also supported the --ext-backup-command option. Now, it only archives system settings and the --ext-backup-command option is no longer supported. Adds a restore-settings option that restores system settings, overwriting any existing settings. It operates on any valid archive directry (though ones created by backup will likely not have any system settings data).

Please refer to the updated Backup and Restore documentation for full details.

Rfe12621 - xsd:time improvements

Zoneless xsd:times are no longer converted to zero timezone when added, and timezone offsets roundtrip to and from the database. Also, with this change xsd:times and xsd:dateTimes are no longer comparable. This puts xsd:time on equal footing with xsd:dateTime. See the date and time documentation for more information.

Rfe12580 - Don't create repo if client/server uids don't match

Previously, if the uid of the direct Lisp client and the AllegroGraph server differed, then a create-triple-store call would fail with a `server-pid-and-client-pid-must-match-error` indicating that the creation failed. However, the repository would actually be created, possibly overwriting an existing repository. Now, the uid check is performed earlier in the create repository process, so that the repository is not created when the uids do not match.

Rfe12456 - Time zone roundtrip for xsd:dateTime and xsd:time

AllegroGraph previously represented xsd:times and xsd:dateTimes with Zulu (i.e. Greenwich Mean) time internally and converted data to that time zone as it was imported. On output, any xsd:time or xsd:dateTimes would be displayed in Zulu time and would lose the original time zone information. For example,

"2014-02-23T08:18:59+05:00"^^xsd:dateTime 

would become

"2014-02-23T03:18:59Z"^^xsd:dateTime 

AllegroGraph now keeps the original data's time zone information. Note that the additional computation involved with coercing time information during query means that queries involving xsd:dateTimes or xsd:times may run slightly (a few percentage points) more slowly. There is also a new query option, presentationTimeZone, that controls the display of time zones in the output of SPARQL queries. For example, adding

PREFIX franzOption_presentationTimeZone: <franz:-04:00> 

will cause all xsd:times and xsd:dateTimes to be printed at the -04:00 time zone. See the documentation for more information. Since the database format has been changed, the usual upgrade process involving a backup/restore cycle (see Backup and Restore) is required. Please note the following:

1: There are no changes for xsd:dates. They still support what the XML Schema describes as 'recoverable timezone'.

2: "2014-02-23T08:18:59+08:00"^^xsd:dateTime is DISTINCT from "2014-02-23T00:18:59Z"^^xsd:dateTime.

3: Joins can be made over dates that represent the same point on the same timeline even if they have different timezones. Which (equivalent) representation is returned by the join is unspecified.

See the date and time documentation for more information.

Rfe11303 - Fix partial repo creation on agraph-backup restore operations

When performing an agraph-backup restore operation, if you passed a directory name as the archive argument, the restore would fail, but not before a repository was created. Further, the repository would remain in 'restore' mode, so it could not be opened, and its existence would prevent a subsequent valid agraph-backup restore operation from succeeding. agraph-backup now first checks that the argument filename names an existing file which is not a directory before creating any repositories. Also, error messages when agraph-backup** is passed invalid archive file arguments have been improved. See Backup and Restore.

Rfe11044 - Add -p short form of --port to agraph-backup.

With agraph-backup, users may use either -p or --port to specify the port on which an AllegroGraph server is running. See Backup and Restore.

Rfe10629 - Add --supersede argument to agraph-backup

agraph-backup now accepts a --supersede option, which (if specfied) allows overwriting of existing archives (for the backup command) and directories (for the backup-all command), and databases (for the restore and restore-all commands). See Backup and Restore.

Bug22099 - Ensure UUIDs are unique during agraph-backup restores.

Previously, when restoring archive backups to an AllegroGraph server, it was trivially possible to create multiple repositories with the same uuid. Now, agraph-backup will error if a restore operation will cause a uuid conflict. See Backup and Restore.

Bug22090 - Timezone minutes in xsd types

Specifying timezone minutes is no longer optional in xsd:dateTimes and xsd:times, so the following examples are invalid:

"1978-01-03T00:00:00+05"^^xsd:dateTime INVALID  
   Correct form: "1978-01-03T00:00:00+05:00"^^xsd:dateTime  
"00:00:00+05"^^xsd:time INVALID  
   Correct for: "00:00:00+05:00"^^xsd:time 

Previously, only xsd:dates checked for the presence of timezone minutes. See the documentation for more information.

Bug22083 - xsd:date and xsd:dateTime years

AllegroGraph now correctly parses both positive and negative dates with more than four digit years. It also signals an error if the date is too large to be encoded into a UPI. Previously, dates like these were silently truncated. See the date and time documentation for more information.

Bug22073 - ORDER BY and non-encoded decimals

When non-encoded decimals were being ordered, they were first converted to IEEE 54 double floats (which have a precision of 15-17 significant digits) which potentially caused them to be sorted incorrectly. This bug has been fixed.

Bug22072 - Validate xsd:decimals

AllegroGraph now signals an error if an xsd:decimal's exponent in scientific notation is outside of the range [-63, 63] because these values cannot be encoded into a UPI. Previously, xsd:decimals like these were silently truncated.

Bug22068 - Reasoning and constant triple patterns

With reasoning enabled, some queries involving triple patterns without variables failed with 'Bus error'. This has been corrected.

Bug22066 - Printing of NaN, INF and -INF

Serialization of xsd single and double float NaN, INF and -INF was broken (for instance, INF became "#.inEinity-double"). This bug has been fixed.

Bug22057 and Bug22054- agraph-backup backup-settings works as documented.

The agraph-backup backup-settings command was incorrectly backing up data as well as settings rather than just backing up settings. Now it just backs up settings. Also, the backup-settings command now can work by finding the settings location from the config file (so the server need not be running). See Backup and Restore where a note describes all the agraph-backup revisions.

Bug22045 - 14 hour datetime comparison logic

For two xsd:dateTimes, one with and one without a timezone, all comparisons must return false if they differ by less than 14 hours. Similar logic applies to xsd:dates and xsd:times. Previously, support for this 14 hour comparison logic was spotty. It worked for some combinations (such as comparisons in a FILTER with a literal) but didn't work for others. With this fix, comparisons for all three types follow the XML Schema specification. See the date and time documentation for more information.

Bug22044 - Fix xsd:date xsd:dateTime comparisons

AllegroGraph previously compared xsd:dates and xsd:dateTimes by converting the date to a dateTime at its first instance. Because dates are actually 24-hour intervals, this logic is suspect so AllegroGraph now signals a type error if this comparison is made (i.e., FILTERs will not longer match and BIND would leave its variable unbound). The previous behavior can still be had by using an explicit cast from date to dateTime. See the date and time documentation for more information.

Bug22041 - Comparisons to decimal zero

Due to a bug in the encoding, decimal 0 appeared to be greater than all decimals less than 1.0. The usual upgrade process involving a backup/restore cycle (see Backup and Restore) is required. All misencoded decimal 0s will be converted to the correct value. If any such corrections are made during the upgrade, then indices are resorted and background index optimization is begun. The index optimization may continue even after the upgrade process has completed. The resulting indices are then as optimized as if all the triples were just imported into the database. This means if the backup was of a fully optimized repository, then after the upgrade running level 2 (the default) index optimization is necessary to get back to that state.

Bug22039 - agload character encoding fixes

Previously, the --encoding argument to agload was ignored. This has been corrected. See Data Loading for information on agload.

Bug22034 - part->concise on encoded dates and decimals

For xsd:dates, xsd:dateTimes, xsd:times and xsd:decimals, the Lisp functions part->concise and part->terse returned the stringified internal representation (like "172839/500" for decimals, and "( . )" for datetimes) which didn't fulfill the promise of being in a "human readable form". With this change, the above functions will return the value in a readable string format without the type. An example:

(part->concise (value->upi "2013-01-03T10:34:00-05:00" :date-time))  
=> "2013-01-03T10:34:00-05:00" 

See XSD Date and Time datatypes.

HTTP Client

No significant changes.

SPARQL

Rfe12624 - Standardize some low-level SPARQL equality tests

The efficiency and conformance of low-level tests used to compute RDFterm-equality and SPARQL ordering has been improved. One non-backwards compatible change is that AllegroGraph previously allowed ordered comparisons between plain literals that looked like numbers and typed numeric literals (e.g., "6.21" and "6.23"^^xsd:integer). This comparison now correctly throws a SPARQL type error.

Bug22092 - Fix for SPARQL queries that use the STR() function with a constant

If a SPARQL query contained a FILTER that used the STR() function applied to a constant, then the query could signal an error at planning time. For example, a FILTER like

FILTER( STR( <http:://example.com> ) = ?var ) 

would not succeed. This has been corrected.

Bug22078 - SPARQL insert did not always encode typed literals

Triples added to a triple-store using SPARQL inserts that used computed values did not always have their typed literals encoded into UPIs. For example, an insert like:

insert { <ex:a> <ex:b> ?c . } where { bind( '1'^^xsd:byte as ?c ) } 

would store the literal in string form rather than encoding it directly. This has been corrected.

Bug22077 - Improve efficiency of certain SPARQL sub-queries

Bound variable information could be lost while processing certain SPARQL sub-query forms which would then result in less efficient query execution. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug21608 - Include ag-call in remote Lisp client.

Previously ag-call was omitted from the remote Lisp client. This has been corrected.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

AllegroGraph 4.12 user-visible changes

AllegroGraph Server: General Changes

Rfe12576 - Improve RDF/XML parser efficiency

Improve the lower-level efficiency of AllegroGraph's RDF/XML parser so that it is between 5 and 10% faster.

Rfe12522 - Global configuration directive for query options

A new global configuration directive named QueryOption was added. It can be used multiple times. Each specification is equivalent to a query prefix option. The global configuration directive "QueryOption NAME=VALUE" is the same as the query prefix option "PREFIX franzOption_NAME: ". See Server Configuration for information on configuration directives.

Rfe12429 - More flexible parsing for InstanceTimeout

With this change, InstanceTimeout configuration parameter values in the configuration file can be specified as times rather than only as a number of seconds. For example, you can use 12s, 2m or 3h to specify 12, 120 or 10,800 seconds. The configuration file is described in Server Configuration.

Rfe11010 - Require the i index for freetext queries

AllegroGraph's native freetext index requires the presence of the triple-id (i) index in order to operate efficiently. A common problem was to try to use freetext queries when there was no i index. This change automatically adds an i index whenever a freetext index is created and also signals an error if a freetext query is attempted when there is no such index.

Bug22024 - Serialization of decimals

xsd:decimals with zeroes right before the decimal point were serialized with the decimal point misplaced. This bug has been fixed. The database contents were not affected.

Bug21978 - serialize-* functions in Lisp client default to UTF-8.

Use UTF-8 as the external-format when a filename is passed to one of the serialize-* functions in the Lisp client instead of the default external format, which can vary depending on the system's locale.

Bug21976 - Complex GRAPH queries could produce incorrect graph bindings

Depending on the dataset, queries with deeply nested structure that used EXISTS or NOT EXISTS FILTERS with embedded GRAPH clauses could generate incorrect bindings for the graph variable. This has been corrected.

Bug21975 - Increase AllegroGraph Load speed

Low-level efficiency enhancements improve AllegroGraph load speed by roughly 10%.

Bug21966 - Possible "UPI not in string table" problem with remote-triple-stores

It was a possible for a remote-triple-store to lose track of the strings cached on the client after a rollback operation. This could lead to a "UPI not in string table" error. This has been corrected.

Bug21956 - Improve memory usage for zero-length property path queries

A zero-length property path query connects all subjects and objects on the right and left hand sides of the path to themselves. If the store is large and the path produces many matches on one or both sides, AllegroGraph was using more memory than necessary. This has been greatly improved though some queries may experience a slight performance degradation.

Bug21954 - Problem generating soundexes for phrases in Callimachus

If a store was operating in Callimachus mode and a phrase was soundexed that had words in it with no soundex (e.g., a number) then AllegroGraph would generate garbage soundex data. This has been corrected

Bug21949 - Improve bookkeeping of already processed variables during query execution

AllegroGraph was not updating the list of bound variables when evaluating a BIND clause during query execution. This did not change the final results of a query but did mean that AllegroGraph would not always evaluate the query in the most efficient manner. This has been corrected.

Bug21937 - Zoneless xsd:dateTime roundtrip

Adding a triple with an xsd:dateTime without a timezone (e.g. "2013-07-30T23:59:59") was stored as if it had a timezone of zero. With this bug fix, zoneless dateTimes are stored and serialized without timezone. This is in agreement with the XML schema specification and similar to how xsd:dates behave.

Bug21934 - String table mutex not reinitialized after recovery

A mutex used in the string table data structure was not being reinitialized after database restart. Under certain circumstances this could result in a database hanging after restart. This problem has been corrected.

Bug21920 - Soundex should ignore punctuation

AllegroGraph's soundex function (which is used by Callimachus) was not ignoring punctuation properly. This has been corrected.

Bug21915 - Make upi->value on xsd:dates return integers

Due to an undocumented change in v4.11, upi->value on a upi encoded as :date returned a cons of the integer representing the instant and a flag indicating whether the timezone was present. With this change only the integer part is returned (like before v4.11). The flag is returned as the second value by upi->values.

Bug21914 - xsd:dates on the first day of the month

Adding a triple with an xsd:date with a positive timezone that falls on the first day of a month resulted in error "illegal date 0". This bug has been fixed.

Bug21909 - agraph-backup could fail to generate unique UUID on restore

When using the --newuuid option, it was possible for agraph-backup to create multiple triple-stores with the same UUID. This has been corrected. See Backup and Restore for information on agraph-backup.

Bug21903 - Free text indexing with chunk-at-a-time processing

Depending on the execution plan, a query that used both fti:match and chunk-at-a-time processing (see franzOption_chunkProcessingAllowed) could produce too few results. This bug has been fixed.

Bug21899 - Shared memory server fails to clean up

Under low memory conditions, the shared memory server can fail to allocate blocks of shared memory on behalf of a database instance. Under certain circumstances the shared memory server would fail to clean up from partial allocations, thereby exacerbating the low memory condition. This problem has been fixed.

Bug21873 - Link to docs in rpm install

When AllegroGraph was installed from an rpm package, the documentation links in AGWebView were broken. This bug has been fixed.

Bug21872 - agload will write output to agload.log when in --debug mode

Along with the other effects of using the --debug command line argument to agload, all output will be logged to agload.log. See Data Loading.

Bug21869 - Fix problem with registering cartesian predicate mappings

There was a problem which prevented the shortened form of cartesian predicate mappings from working. This has been corrected.

Bug21853 - Connection Pool session port connection failure

Under certain circumstances, the AllegroGraph Java client would throw an exception like the following:

com.franz.agraph.http.exception.AGHttpException:  
Possible session port connection failure 

Setting the SessionPorts configuration parameter is discussed here in Server Installation and also in Server Configuration. But this exception could occur even with a proper SessionPorts setting. This problem has been resolved.

Bug21846 - Long running federated queries could fail

The query canceling interface introduced in AllegroGraph 4.11 could lead to query failures when using federated triple-stores. This has been corrected. Note that it is not currently possible to cancel queries running on a federated store. This will be improved upon in a future release of AllegroGraph.

Bug21816 - Fix session multiplexing file descriptor leak

When requests to sessions are multiplexed through the server's main port, the frontend didn't close the http socket used to communicate with the session process. This bug has been fixed. Note that session multiplexing can be enabled for the java client by setting the system property:

com.franz.agraph.http.useMainPortForSessions=true  
 
 

Bug21784 - Safer auto-close for garbage db's in the Lisp direct client

The db's are now marked for closure when it is noticed they are no longer used and closed at a later specific time.

Bug18813 - Reasoning and subclasses of owl:TransitiveProperty

With this fix the reasoner is able to infer that a property whose type is a subclass of owl:TransitiveProperty is a transitive property. That is,

:p rdfs:subClassOf owl:TransitiveProperty .  
:q rdf:type :p .  
:a :q :b .  
:b :q :c . 

entails

:a :q :c .  

HTTP Client

No significant changes.

SPARQL

Rfe12537 - Improve zero length property path clause estimation

This change improves the heuristics AllegroGraph uses to estimate the number of rows that a zero length property path query can generate which improves the way AllegroGraph processes a BGP.

Rfe12520 - The SNA members and size magic properties work with RDF lists

You can now use the and Magic Properties to query RDF lists stored in your triple-store. For example, if points to the head of the RDF list "1", "2", "3", then this query would return the three elements:

prefix sna: <http://franz.com/ns/allegrograph/4.11/sna/>  
select ?item {  
  <ex:list> <ex:pointsTo> ?list .  
  ?item sna:members ?list .  
} 

Would return "1", "2" and "3". See SPARQL Magic Properties for SNA.

Rfe12506 - Improve handling of some queries that use GRAPH clauses

SPARQL is defined such that a query like

GRAPH ?g { A } 

should behave as if A was evaluated with ?g bound to each graph in the dataset in turn and then the results of all of these evaluations were unioned together. While correct, this implementation can be very inefficient. AllegroGraph uses various heuristics to determine if a given GRAPH clause can be evaluated more directly and does so when it can. This change improves these heuristics so that more queries will be able to use the more efficient implementation.

Rfe12481 - Improve SPARQL DELETE efficiency

This change greatly increases the efficiency of the DELETE portion of a SPARQL DELETE... INSERT... WHERE... command. The actual increase is dependent on the data and the delete template but it can be more than 40 times faster.

Rfe12442 - Efficiency improvements for SPARQL aggregation

Optimize the code used for SPARQL aggregate queries in general and for count aggregates in particular.

Rfe12400 - Minor property-path query optimization

Improve the underlying efficiency of some property-path queries.

Rfe12398 - Clarify semantics of the identity clause reorderer

AllegroGraph has two clause reorderers which try to preserve the ordering of triple-patterns in a BGP (basic graph pattern). Previously, these were named:

To avoid future confusion, these options are being renamed such that the first method is called :weak-identity and the second is called :identity.

The default query clause reorderer continues to be statistical so queries that do not specify a reorderer will behave exactly as they did before this change.

Rfe12392 - Optimize certain complex join combinations

The query optimizer now converts a query with an algebra like:

(:join (:join A :bgp1) :bgp2) 

into the equivalent:

 (:join A :bgp1-and-bgp2) 

This BGP (basic graph pattern) merger improves AllegroGraph's ability to reorder queries for efficient execution. AllegroGraph also converts the expression:

(:join (:join :bgp1 A) :bgp2) 

into

(:join :bgp1-and-bgp2 A) 

unless the two BGPs do not overlap (in which case the transform would produce a cross-product).

Rfe12385 - Improve BGP reordering in some cases

In some cases, the SPARQL query planner now does a better job reordering the triple-patterns in a BGP (basic graph pattern).

Rfe12380 - Improve efficiency of some DISTINCT or REDUCED queries

AllegroGraph now collects fewer intermediate results for some DISTINCT or REDUCED queries.

Rfe12377 - Add intervalContainsTime magic property

Added an additional temporal magic property that can be used to determine if a time is contained within an interval. See SPARQL Magic Properties.

Rfe12372 - Make BGP reordering more intuitive

Previously, AllegroGraph would switch the execution of patterns in a BGP when the BGP contained only two patterns and these patterns had the same statistical clause estimates. E.g., in a query like:

?a ?b ?c .  
?c ?d ?e . 

AllegroGraph would execute the second pattern first. AllegroGraph now executes the first pattern first in the case of ties which is more intuitive.

Rfe12368 - Use constraints to limit bindings for GRAPH queries

Improve AllegroGraph's use of FILTERs in limiting the graphs considered in GRAPH clauses with variable graphs. For example,

GRAPH ?g { A }  
FILTER( ?g = <ex:test> ) 

will now be evaluated more efficiently since AllegroGraph will only consider the graph <ex:test> rather than every graph in the dataset.

Rfe12352 - Provide a sort order for SNA path and group gensyms

Social Network Analysis (SNA) Magic Properties can create node identifiers to keep track of paths and groups. These act like blank nodes and so had no sort order defined for them. This was inconvenient because it made it impossible to, for example, list all of the nodes of a path together. This change adds a sort order to these node identifiers so that they are easier to use in queries.

Rfe12192 - Optimize some wild-card property-path queries

AllegroGraph now computes property-path queries that use the * or + operators more efficiently.

Rfe12032 - Improve property-path expansion

AllegroGraph now expands more property-path queries into their equivalent BGPs (basic graph patterns). This can improve the performance of the query optimizer. For example, AllegroGraph previously expanded

?s :a/:b ?o .  
into 

?s :a :b0 . :b0 :b ?o .

but did not expand paths with non-simple operators like wild-cards or alternation. It now expands these as well so that a path like

?s :a/:b/(:c|:d)/:e/:f ?o 

becomes

?s   :a _:b0 .  
_:b0 :b _:b1 .  
_:b1 (:c|:d) _:b2 .  
_:b2 :e _:b3 .  
_:b3 :f _?o .  

Rfe11948 - Add PREFIX option for enabling reasoning in a SPARQL query

AllegroGraph's SPARQL 1.1 engine now supports the inferenceMode query option to control the kind of inference used by an individual query. The mode is selected using the query option PREFIX notation. For example, the following PREFIX would cause a query to be evaluated using AllegroGraph's RDFS++ reasoner:

PREFIX franzOption_inferenceMode: <franz:rdfs++> 

The following modes can be supplied:

Rfe11625 - Improve algebraic optimizations of joins and left-joins

Improve the logic used in the algebraic transform of

(:join (:left-join A B) C) into (:left-join (:join A B) C) 

Which is the textual equivalent of moving the optional clause down in a query like

A  
optional { B }  
C 

so that it becomes

A  
C  
optional { B } 

which can be evaluated more efficiently unless A and C share no bindings.

Rfe11146 - Handle simple OPTIONAL clauses more efficiently

AllegroGraph now handles OPTIONAL clauses that contain only a single triple-pattern more efficiently. This means that queries with clauses that look like

 ?s :predicateA ?o .  
 OPTIONAL { ?s :predicateB ?b }  
 OPTIONAL { ?s :predicateC ?c } 

will be processed more efficiently.

Rfe11068 - Improve literal equality constraint handling

AllegroGraph now handles filter expressions involving literal equality more efficiently. For example, in a query like

select * {  
  ?a :predicate ?b .  
  filter( ?b = 'literal' )  
} 

AllegroGraph previously would scan all triples with the given predicate and apply the filter afterwards. Now AllegroGraph determines the set of possible matches for 'literal' and queries for them directly.

In addition, AllegroGraph now performs a similar operation if the filter is a literal equality on an expression using the SPARQL str function. For example:

filter( str(?variable) = 'searching' ) 

As with regular literal equality constraints, such an expression is converted into the equivalent tests for specific values.

Bug22030 - ORDER BY expressions with non-query variables could cause errors

If a SPARQL query used an ORDER BY expression that contained variables that were not found anywhere in the query, then an error could result. An example of this would be a query like:

select * {  
  ?s a ?o .  
} order by desc( str( ?notFoundInQuery )) 

This issue has been corrected.

Bug22021: Correct SPARQL TSV printing problem for encoded UPIs

When encoded UPIs were serialized from a SPARQL query using the SPARQL TSV (tab sepated values) output format, they were printing without the enclosing quotes. This has been corrected.

Bug22016 - Correct SPARQL 1.1 bug with IN FILTERs

It was possible for the query engine to produce incorrect results when using FILTER with either the SPARQL IN operator or multiple equality filters on the same variable. This has been corrected.

Bug22015 - Fix possible error when using the SPOGI cache and SPARQL 1.1

In rare circumstance it was possible for a SPARQL 1.1 query to signal an error when the SPOGI cache was enabled. This has been corrected.

Bug21986 - Using distinct aggregation with GROUP_CONCAT and a separator could fail

If a SPARQL query used the GROUP_CONCAT aggregator along with both the DISTINCT and SEPARATOR modifiers, then the solution bindings for that variable could be left empty. This has been corrected.

Bug21980 - Too many results in DISTINCT and REDUCED queries

Under extremely rare circumstances, queries involving DISTINCT or REDUCED could return too many results. This bug has been fixed.

Bug21974 - Assertion failure involving 'n-valid-partials'

When chunk-at-a-time processing was enabled (see query prefix franzOption_chunkProcessingAllowed, described here in the SPARQL Reference), the code keeping track of the number of intermediate results used to run into an assertion failure under very rare circumstances (when two consecutive intermediate results were the same). This bug has been fixed.

Bug21962 - Improve estimates for certain query patterns

AllegroGraph was mis-estimating the sizes of query patterns that resulted from the inlining of filters that used the IN expression. For example,

?s ?predicate ?object .  
FILTER( ?predicate IN ( ex:a, ex:b, ex:c ) 

This could result in sub-optimal BGP re-ordering. This has been improved.

Bug21961 - Incorrect results with EXISTS and NOT EXISTS

Some queries involving EXISTS or NOT EXISTS filter expressions returned too few results. This bug has been fixed.

Bug21955 - Sorting on unbound variables could lead to a string dictionary error

If a query used an ORDER BY expression involving a computation and some of the variables in the expression were unbound, then the expression could signal a UPI not in string dictionary error. This has been corrected.

Bug21948 - Improve bookkeeping of bound variables during sub-query

When executing a sub-query with a limit, AllegroGraph failed to update the list of variables known to be bound. This prevented some queries from executing as efficiently as possible. This has been corrected.

Bug21947 - Improve efficiency of VALUES in some cases

AllegroGraph now does a better job of using the information in a VALUES clause to restrict alternate solutions. This can greatly increase the speed of some queries.

Bug21941 - Queries with magic properties could fail in a remote-triple-store

Queries that used magic properties could cause an error when run from a remote-triple-store. This has been corrected.

Bug21940 - Using magic properties with filters could lose solutions

If the circumstances were just right, a query that used a magic property that produced multiple bindings and that used a filter on those bindings could fail to return all solutions. This has been corrected.

Bug21933 - Externally introduced variables could fail in nested sub-queries

If a query used externally introduced variables (e.g., via the Lisp APIS's with-variables argument or via the variable bindings in an HTTP REST request), and these variables were used in FILTER clauses in a nested sub-query, then the query could fail to find any matches. This has been corrected.

Bug21921 - BGP triple filters with Chunk-at-a-Time processing

In rare cases, a filtering triple-pattern could fail to remove some solutions when using Chunk-at-a-Time query processing. This has been fixed.

Bug21902 - Fix incompatibility between magic predicates and the identity reorderer

If the identity BGP clause reorderer was being used in a query with magic predicates, then the magic predicates were being sent to the end of the BGP. This has been corrected.

Bug21901 - Minor improvements to some range query clause orderings

Improve the statistical estimates used to reorder BGP patterns that use range queries.

Bug21892 - Query engine selection failed to take OFFSET into account

AllegroGraph's decision on whether to use the single-set or the chunk-at-a-time query execution engine relied only on the LIMIT of the query rather than the LIMIT and the OFFSET. This has been corrected.

Bug21880 - Using with-variables in SPARQL describe caused an error

Using SPARQL describe with an external variable binding would lead to an error at query execution time. For example, this query would fail:

 describe ?x 

(where ?x was bound externally). This has been corrected.

Bug21865 - Comparisons of xsd:dateTimes with fractional seconds

Comparisons (including ORDER BY) of xsd:dateTimes with fractional seconds were broken. This bug has been fixed.

Bug21862 - Some FILTERs and BGP orderings could lead to empty bindings

In rare cases, AllegroGraph could fail in its bookkeeping of which variables were bound while it executed a query. This could lead to the introduction of spurious rows with null bindings. This has been corrected.

Bug21858 - Distinct, SERVICE and sub-query could lead to planning failure

If a query used a sub-query in a SERVICE clause and asked for unique results, then AllegroGraph would signal an error during query planning. This has been corrected.

Bug21852 - Improve bound variable tracking

AllegroGraph should assume that any variable supplied via external settings (e.g., via the with-variables parameter to run-sparql) is bound. Failure to do so could cause some queries to fail. This has been corrected.

Bug21828 - VALUES could fail to correctly handle OPTIONAL bindings

Using the VALUES clause could incorrectly drop solutions whose rows contained unbound values. This has been corrected.

Bug21815 - Fix property-path memory leak

Some queries involving property-path expressions could run out of memory, because the intermediate result set was not freed as soon as it was consumed. This bug has been fixed.

Bug21803 - Exclusive ranges of encoded non-integer types

Previously, filter expressions on all non-integer types (floats, decimals, etc, excluding xsd:date and xsd:dateTime) involving exclusive ranges ('<' and '> ') behaved as if an inclusive range ('<=', '> =') had been specified. This bug has been fixed.

Rfe12591: Improve efficiency of some Basic Graph Pattern evaluations

AllegroGraph now makes better use of intermediate result sets when evaluating basic graph patterns (BGPs) in nested groups of joins and left-joins.

AGWebView

Bug21997 - Agwebview and sessions

Due to a thread safety issue, AGWebView's 'Create Session' or any subsequent action in that session could produce garbage results ranging from 'Not found' to garbled output. This bug has been fixed.

Changes to the Lisp API

Rfe12562 - upi->value round-trip

For dates and date-times a complete representation is a (<universal-time> . <timezone-information>) cons. While this would make a good candidate for the return value of upi->value, to maintain backward compatibility by default upi->value must return a single number representing the universal time.

However, this means that during the upi->value/value->upi round-trip the timezone information is lost.

This change adds a new keyword argument :complete? (defaults to nil). With :complete? t, upi->value returns the complete representation and the roundtrip works in the sense of:

(upi= (value->upi (upi->value upi :complete? t) (upi-type-code upi))  
      upi) 

Currently, for UPIs other than dates and date-times, :complete? has no effect.

Rfe12557 - run-sparql can now use a UPI or future-part for the default base

Previously the Lisp function run-sparql only allowed a string to specify the default-base. It can now take a string or a UPI or future-part.

Rfe12508 - normalizing with-variables should convert $var to ?var

The with-variables parameter in the Lisp API to run-sparql allows for variable names to be identified as strings or symbols. The name can be a plain string like "person" or a string whose first character is #\? or #\$ like "?person" or "$person". All three of these representations refer to the same query variable: ?person. Prior to this change, AllegroGraph could misinterpret variable names that used $ as the first character. This has been corrected.

Bug21953 - Support server-side loading via the Lisp client.

When using the Lisp client and connected to a remote-triple-store, passing a URI or URI-like string to one of the bulk-load functions (e.g. load-ntriples) will cause the resource to be loaded directly by the server instead of being delivered via the client.

Bug21891 - db.agraph.log:log-output captured in dumplisp

If dumplisp (a function used to save a running image) was used after opening a triple store, the stream used for logging would be captured. Upon restore of the dumped image, subsequent logging attempts (performed by internal AllegroGraph code) could result in confusing errors or file corruption. This has been fixed.

Bug21860 - Make Lisp clients find bundled liblzo2.so.2

Previously, Lisp clients resorted to using the liblzo library located in the acl directory ("sys:") or somewhere the OS dynamic library loader can find it. With this change the bundled liblzo2.so.2 is loaded.

Bug21841 - db-get-triple leaks sockets

The Lisp function db-get-triple didn't close the socket it used for communication with a remote triple store which resulted in leaking sockets in CLOSE_WAIT state. This bug has been fixed.

Bug21848 - load-turtle could use the wrong triple-store

The load-turtle command could fail to respect the value of the db parameter and instead load triples into the store currently bound to db. This has been corrected.

Bug21810 - Polygon-vertexes did not work with future-parts

If called with a future-part rather than a UPI, the polygon-vertexes function would return no results. This has been corrected.

Bug21611 - Load patches in Lisp remote client

With this fix the Lisp remote client loads patches and enables pretty printing of parts and triples just as the direct client does.

Bug21070 - Fix corruption hazard in direct Lisp client.

Prevent possible database corruption that could occur when interrupting a direct Lisp client app that is modifying the transaction log.

Bug21045 - create-freetext-index function failed when called with UPIs

The create-freetext-index function (in the Lisp API) would fail if called with UPIs (rather than, e.g., future-parts). This has been corrected.

Prolog

Rfe12222 - AllegroGraph Prolog unifies UPIs and future-parts

When AllegroGraph is loaded, the semantics of Prolog unification is extended such that:

This enhancement does not change the capabilities of Prolog queries in AllegroGraph, but allows simpler, less-verbose coding that looks a little more declarative and less procedural. For example, the following

(select (?manager ?employee)  
  (member ?employee (!acme:joe !acme:jill !acme:jim !acme:jerry))  
  (q ?employee !acme:hasManager ?manager)  
  (member ?manager (!acme:bill !acme:bob !acme:betty !acme:benjamin))) 

would not work as intended before this change because ?manager would be unified with a UPI, and a UPI would not unify the future-parts in the last clause. The last clause would have had to be coded as something like

(member ?manager1 (!acme:bill !acme:bob !acme:betty !acme:benjamin))  
(part= ?manager manager1) 

Using Prolog is AllegroGraph is documented here in the Lisp Reference.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug21953. Fix generation of some POST method requests.

AGHTTPClient would write query parameters to the body of POST requests when the content-type was not application/x-www-form-urlencoded. This can result in http servers not finding the query parameters and processing the request incorrectly. This bug corrects the problem by writing POST request query parameters to the URL, unless the request has no body and the content type is application/x-www-form/urlencoded. AGRepositoryConnection.load() (see Javadocs) now works for importing server-side statements.

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

Rfe12309 - Improve algebraic query manipulation

AllegroGraph now notices patterns of the form

(:join (:join A (:bgp B)) (:bgp C)) 

and transforms them into the equivalent

(:join A (:bgp B C)) 

(I.e., it merges the two BGPs). This transform brings more triple-patterns together which lets AllegroGraph order the query operations better.

Rfe12305 - Relative paths in the configuration file

Relative file or directory paths in the configuration file were always interpreted relative to the directory containing the configuration file. In this rfe, a BaseDir configuration directive was added that allows another path to be specified for this purpose. See the documentation in Server Configuration for more.

Rfe12280 - Simplify geospatial type-mapping API

Geospatial subtype mappings can now be set up with the regular datatype and predicate mappings rather than needing to use the specialized functions. The old functions still work but are now unnecessary. They will be removed in some future release.

Rfe12260 - Automatically map temporal predicate data to xsd:dateTimes

Triple-stores created in this version of AllegroGraph will include the three AllegroGraph temporal predicates in the list of automatically mapped predicates. I.e., the objects in triples with these predicates:

will be converted into encoded xsd:dateTime literals. This encoding makes range queries against them faster and makes it easier to use AllegroGraph's temporal functionality.

Rfe12259 - Optimize certain combinations of JOINs and FILTERs

The query engine now recognizes certain patterns of JOINs and FILTERs and rewrites them to be more efficient. For example, a pattern like

(:join A (:filter B f)) 

can often be better evaluated as

(:filter (:join A B) f)  

Rfe12182 - More restrictive umask

With this change, users who aren't the owner of the file, or a member of the group of the file (that is, "other" users in chmod manpage terms) have no read, write or execute permission on files created by AllegroGraph.

On existing installations, it's recommended, but in no way enforced, to change permissions the same way with 'chmod -R o-rwx <directories>' where <directories> is the list of directories holding AllegroGraph data (LogDir, SettingsDirectory, catalog directories, etc).

Rfe12175 - Audit log

When the 'Auditing' global configuration directive is set to 'yes' (it defaults to 'no'), events that have implications on security or performance will be logged to the new system database in the new system catalog. Note that regardless of permissions only superuser can read or write the system catalog.

See Auditing for more information.

Rfe12151 - Remove support for loading N-Triples into a federated triple-store

AllegroGraph 3 introduced some techniques that optimized loading of N-Triples into a federated-triple-store. These have been superseded in AllegroGraph 4 because agload is significantly faster and easy to use.

Rfe12147 - Better manage memory especially for large solutions

AllegroGraph now manages large query result sets (with or without DISTINCT or ORDER BY) more efficiently.

Rfe12138 - Agload: reduced memory footprint and other enhancements

Reduced memory consumption of agload processes. Particularly in the case where there are many input sources.

Agload now analyzes input sources on the fly. This means you will not necessarily encounter errors about invalid input sources prior to modification of a triple-store. Use --dry-run command line argument if you wish to pre-analyze an agload job.

Rfe12125 - Improve performance of unordered queries with small limits

AllegroGraph now uses chunk-at-a-time query processing for queries with small LIMITs that do not use ORDER BY more frequently. In particular, the chunkProcessingAllowed query option can now take three values:

The default value for chunkProcessingAllowed has been changed from no to possibly.

Rfe12111 - Improve memory checking efficiency

Improve the efficiency of AllegroGraph's memory tracking code slightly.

Rfe12089 - Improve cursor resource management

Improve AllegroGraph's low-level cursor memory management.

Rfe12063 - Allow in-memory-triple-stores to use blank nodes

The in-memory-triple-store implementation for the remote-lisp-client was missing several methods required for full blank node support. This has been corrected.

Rfe11973 - Improve usage of UPI-map

Improved the way AllegroGraph manages the use of secondary indexes (UPI-maps).

Rfe11955 - :commit argument added to delete-duplicate-triples

The :commit argument was added to delete-duplicate-triples to allow for the deleting a very large number of duplicates.

Rfe11886 - Use statistics more often when reordering query clauses

The query engine now uses store statistics in more places when working to reorder the clauses in a query's basic graph patterns (BGPs). This should produce better plans more often with less need for customer query tweaking.

Rfe11585 - Some queries unnecessarily accumulated partial results

AllegroGraph was sometimes accumulating results unnecessarily. For example, the query

select (count(*) as ?total) where {?s ?p ?o} 

was accumulating the actual results even though only the summary was necessary.

Rfe11511 - Encode xsd:decimals

xsd:decimals used to be stored as strings. With this change, they are automatically encoded directly in the UPI which potentially brings significant savings in storage space and computation in queries. The most significant 20 digits are preserved by the encoding.

Note that this change only affects the encoding of new triples. To convert old triples and take advantage of the more efficient storage, the database contents must be reloaded from a serialized format. We recommend exporting in nquad format and then reimporting it with agload. Restoring from a backup is not sufficient.

Rfe11197 - Serialize triple IDs as resources rather than literals

Previously, a triple ID would have been serialized as a typed literal like

"13"^^<http://franz.com/ns/allegrograph/4.0/triple-id> 

This format prevented the use of triple IDs in the subject of a triple so we have changed the serialization to a URI like

<http://franz.com/ns/allegrograph/4.0/tripleId#13> 

Any triple IDs already in a triple-store will remain the same (though they will print differently). This change does mean, however, than any files that contain triple IDs in the old format will need to be converted in order to be re-loaded (note, however, that the IDs assigned to triples are not part of any serialization format which means that they cannot reliably round-trip in any case).

Rfe10292 - Handle filename limit for unix domain sockets

Linux has a limit of 107 characters for filenames of UNIX domain sockets. With this change, when bumping into that limit, a meaningful error message is printed and the workaround of setting the UnixDomainSocketDirectory or the SettingsDirectory configuration directive to a shorter path is suggested.

Bug21793 - Fix race in inter process communication

Under extremely rare circumstances, multiple threads could end up listening on the same communication channel which can lead to errors and deadlocks. This was duly detected by an assertion that appeared in the log as:

the assertion (endp (intersection  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::r)  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::receiver)))  
failed. 

This bug has been fixed.

Bug21782 - Potential double close of cursors

An inadvertent double close of cursors was discovered that could potentially result in improper sharing of a data structure. This has been corrected.

Bug21779 - Fix low-level cursor resourcing bug

Under heavy load, the secondary index (UPI-map) creation function could leak a cursor leading to potential resource problems. This has been corrected.

Bug21761 - Prolog q- functor could be treated as q

If a reasoning triple-store can use secondary indexes (UPI-maps) then it was possible for AllegroGraph to treat some q- functors as q functors in a given Prolog select query. This has been corrected.

Bug21745 - Fix crash related to aggregate functions

Under rare circumstances, queries involving aggregate functions may cause the backend to crash. This bug has been fixed.

Bug21744 - Some Prolog select queries could return duplicate results

If using secondary indices (i.e., UPI-maps), it was possible for Prolog select queries that returned raw UPIs to overwrite one solution with another. This problem could only occur with UPI only queries and never with queries that returned strings. The problem is now fixed.

Bug21740 - Rare hangs under high load

The message broker (called the "Hub" process) may get into a state where forwarding a message blocks, preventing all other messages from being handled. This bug has been fixed.

Bug21735 - Corrupted distinct results

Under rare circumstances, queries involving DISTINCT could produce results with the bindings assigned to the wrong variables. This bug has been fixed.

Bug21702 - XSD casting in a BIND could convert to xsd:integer incorrectly

Using an XSD cast in a BIND could loss the datatype and switch it to xsd:integer. E.g.,

BIND(xs:long(?test) AS ?testLong) 

would bind ?testLong to a value whose datatype was xsd:integer rather than xsd:long. This has been corrected.

Bug21693 - Avoid forking too many blank node servers in agload.

Improves agload performance under default conditions when loading ntriples or nquads files which are small and numerous.

Bug21691 - Reasoning over subproperties of symmetric properties

AllegroGraph's RDFS++ reasoner partially failed to infer triples from sub-properties of a symmetric property. This has been corrected.

Bug21680 - Some BIND forms could be evaluated incorrectly.

A BIND form at the start of a BGP could fail to get values from earlier in the query evaluation.

For example, this form

?a a ?b .  
{ BIND( ?b as ?otherB  
  ?otherB a ?c .  
} 

could lose the binding for ?otherB. This has been corrected.

Bug21670 - FILTER and EXISTS could interact incorrectly.

Depending on the shape of the rest of the query, it was possible for a FILTER clause to be applied to the inside of an EXISTS or NOT EXISTS filter rather than being applied to the main body of the query. This has been corrected.

Bug21663 - Redo of new index creation confuses instance process

If a database with a newly-created index has been shut down uncleanly, restart of the database may crash. This has been corrected.

Bug21651 - Prolog ?? and reasoning

On a reasoning triple store, the prolog ?? construct in the predictate position used to fail with a type error. This bug has been fixed.

Bug21642 - Agload buffer overflow

Due to a buffer overflow, agload could corrupt memory and potentially crash. This has been fixed.

Bug21640 - Race condition when reaping abandoned sessions

Previously, if concurrent processes performed the operation to reap abandoned sessions, shared memory state could be corrupted, resulting in hangs or crashes. This has been fixed. (This bug was actually fixed in version 4.10.)

Bug21625 - Large result sets could produce bogus triples with INSERT and CONSTRUCT

In cases where there were more than 100,000 results, the machinery that prevents INSERT and CONSTRUCT queries from producing duplicate results could create invalid answers. This has been corrected.

Bug21596 - Serialize-rdf/xml could fail on some predicates

AllegroGraph's RDF/XML serializer would fail if a predicate ended with something other than a letter, a digit, an underscore or a dash. This conformed to an older version of the Extensible Markup Language but not to the most recent version. This has been corrected.

Bug21586 - Possible divide by zero error in index optimizer

Under some circumstances an index optimizer could attempt to perform a division by zero when logging statistics to agraph.log. This would result in a non-fatal error message being logged instead of the statistics. This has been corrected.

Bug21580 - Improve external variable binding bookkeeping

The values for externally supplied variables could be lost for some complex interactions of EXISTS (or NOT EXISTS) filters and UNIONS.

This has been corrected.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Also, the http error code was changed from 400 ("Bad request") to 503 ("Service unavailable") to indicate a temporary condition.

Bug21553 - Fix MemoryCheckWhen without MemoryReleaseThreshold

If the memoryReleaseThreshold configuration directive was not specified but memoryCheckWhen was, then query execution failed with type errors trying to compare memory footprint to NIL. With this fix, AllegroGraph refuses to start in these circumstances.

HTTP Client

Rfe12136 - Optimize front-end http server

The timeout for reading headers was lowered to 5s which makes the front-end http server make better use of its available resources and more resilent when clients with http keep-alive don't close the connection in a timely manner.

Also, a warning is logged if the number of idle http worker threads falls below 2, because it can easily cause a performance drop.

SPARQL

Rfe12296 - Support AllegroGraph direct reification in SPARQL

AllegroGraph now supports direct reification in SPARQL queries using the <http://franz.com/ns/allegrograph/4.0/tripleId> magic property. This property allows you to bind the ID of a triple to a SPARQL variable which can then be used in reification. For example, the following query first adds two triples and then adds four additional triples that describe the first two:

prefix : <eh://>  
prefix franz: <http://franz.com/ns/allegrograph/4.0/>  
insert data {  
  :gary :likes :dogs .  
  :gary :likes :cats .  
} ;  
 
insert {  
  ?id :mentioned ?now .  
  :gary :believes ?id .  
} where {  
  ?id franz:tripleId (:gary :likes ?what) .  
  bind( now() as ?now )  
} 

See the documentation for additional details on direct reification and its advantages and limitations.

Rfe12246 - Enhancements to SPARQL freetext queries

The fti:match and fti:matchExpression predicates can now optionally retrieve the object of the matching triples and select the freetext index to use. (These enhancements are currently available only for AllegroGraph's native freetext indexes. Support for Solr and MongoDB will be forthcoming).

The new features rely on SPARQL magic properties to provide a syntactic shortcut for list patterns. As an example, this query retrieves the subject (into ?subject) and text (into ?text) for the freetext query that matches "nepal":

select ?subject ?text {  
  (?subject ?text) fti:match 'nepal' .  
} 

This query retrieves only subjects and uses only the freetext index named comments.

select ?item {  
  ?item fti:match ('upgrade' 'comments')  
} 

More examples and information can be found in the documentation.

Rfe12243 - Support AllegroGraph Social Networking Analysis in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable Social Networking Analysis including centrality measures, path finding and cliques. See SPARQL Magic Properties for more details.

Rfe12231 - Improve SPARQL geospatial integration

AllegroGraph now supports several magic properties to make SPARQL / geospatial integration easier. For example, this query finds all of the points within a circle centered at 10, 2:

select ?who ?where {  
  (?who ?where) geo:inCircleXY ( ex:location 10 2) .  
} 

The following magic properties are provided:

geo:inBoundingBoxXY  
geo:inBoundingBox  
geo:inCircle  
geo:inCircleKilometers  
geo:inCircleMiles 

(where geo is the usual Geospatial namespace abbreviation for <http://franz.com/ns/allegrograph/3.0/geospatial/>). See SPARQL Magic Properties for additional details on using these properties.

Rfe12171 - Improve variable GRAPH clause handling in some situations

SPARQL specifies that a graph clause like

 GRAPH ?g { ... } 

must be implemented as if it iterated over each graph in the dataset and took the union of the resulting bindings. There are sometimes more efficient ways to get the same results and AllegroGraph uses these when it can. AllegroGraph was not, however, using information about any constraints on the variable ?g that were provided by other parts of the query. For example, a query like:

?s :p ?g .  
GRAPH ?g { ... } 

might produce a single binding for ?g but AllegroGraph was still iterating over all of the named graphs of the dataset. This has been optimized so that AllegroGraph will use the additional information when it can determine that the number of bindings for ?g are smaller than the number of graphs in the dataset.

Rfe12057 - Manage disk and memory resources better for CONSTRUCT queries

When there are many results, SPARQL CONSTRUCT queries buffer their results to disk to conserve memory. The cutoff value for when to buffer to disk was unnecessarily small which meant that CONSTRUCT went to disk more often then necessary. This has been corrected.

Rfe12035 - Improve handling of some SPARQL queries with a GRAPH clause

By definition, SPARQL must handle a GRAPH clause as if the query engine evaluated the clause once for each graph in the dataset. However, it is often possible to evaluate the clause more efficiently.

This change improves the heuristics AllegroGraph uses to determine when the more efficient approach will achieve the correct results which means that some queries with a GRAPH clause will be executed more quickly.

Rfe11319 - Support SPARQL federated queries (the SERVICE clause)

AllegroGraph now supports SPARQL federated query using the SERVICE clause. The initial implementation is functional but not highly optimized.

Rfe8063 - Support AllegroGraph temporal reasoning in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable temporal reasoning. See SPARQL Magic Properties for more details.

Bug21757 - SPARQL geospatial queries could fail

Depending on the set up of a triple-store's geospatial subtype mappings, it was possible for a SPARQL query using geospatial extensions to fail to find results. This has been corrected.

Bug21742 - SPARQL parser failed on quad patterns with collections

The SPARQL parser could lose track of triple-patterns that arose from abbreviations using either nested blank nodes or collections. This has been corrected.

Bug21725 - Correct potential error when SPARQL Update touches too many rows

If a SPARQL DELETE ... INSERT ... WHERE request changes many rows, then AllegroGraph caches the results of the WHERE expression to disk to reduce memory pressure. In some cases, this could result in a error when the cached results were re-read. This only occurred for Update requests that touched more than 100,000 rows. This has been corrected.

Bug21718 - Unused variable optimization could produce incorrect results

AllegroGraph analyzes SPARQL queries that use DISTINCT in order to remove the collection of bindings for variables that are not needed to produce the final result. In some cases, it was possible for this analysis to cause the execution of a code path that added solutions incorrectly. This has been corrected.

Bug21573 - SPARQL queries could intern strings unnecessarily

SPARQL queries that used the FROM or FROM NAMED clause always tried to intern the graphs into the triple-store's string table. This was both unnecessary and could cause an error if the triple-store was read-only or a federation.

This has been corrected.

Bug21011, Bug21571 - handle nested SPARQL BGP syntactic sugar correctly

Some SPARQL triple-patterns that involved nested brackets and object lists were being parsed incorrectly. This has been corrected. Some examples of queries that work correctly now but that failed without this change include:

PREFIX : <ex:>  
SELECT * {  
    :x :y [:z (:b [:d (:e :f),(:g :h)])]  
}  
 
PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c :d], [:e :f] .  
} 

and

PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c [:d :e]] .  
} 

Improve Pattern estimation where there is no :p index

Improved query pattern estimators for triple-stores with no predicate index (e.g., :posgi, :psogi, etc.). This will help reorder query clauses more intelligently.

AGWebView

Rfe12325 - Add audit log query interface to AGWebView

There is now an audit menu item in the admin menu. This provides access to the audit log query page which lets you filter audit log events by user, class and date.

Changes to the Lisp API

Bug21708 - Fix some low-level Lisp API geospatial functions

Some low-level Lisp API functions had not been enhanced to account for AllegroGraph's geospatial encoding. This has been corrected.

Bug21689 - Lisp client printer variable leakage

On a remote store, the run-sparql function failed with an 'Illegal sharp character' error if print-length was set to a small value. This bug and more generally the leakage of printer variables into run-sparql has been fixed.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe12118 - AGRepository with connection pooling

With this change, an AGRepository can now be configured to use a connection pool via the setConnPool method, enabling Sesame apps to transparently benefit from connection pooling. See the javadoc for setConnPool for details.

Rfe10513 - Configurable blankNodesPerRequest

With this change, the Java client supports configuring the number of blank nodes automatically fetched per request to the server by the AGRepository's AGValueFactory. The default remains at 100, but it can be configured by setting the property:

com.franz.agraph.repository.blankNodesPerRequest

Applications can also configure it directly, see the javadoc for AGValueFactory#setBlankNodesPerRequest for details.

Rfe9869 - Support SPARQL/Update from Jena

With this change the Jena adapter supports an execUpdate method in AGQueryExecution for executing SPARQL Updates. Jena Tutorial example13 has been augmented to demonstrate its use.

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

Rfe12004 - Ag4 plugins for TBC 4.0.0

With this change, the AG plugins have been updated to work with TBC 4.0.0. For installation details, see Top Braid Composer Plugin.

Rfe11986 - Improve efficiency of some property-path queries

AllegroGraph now evaluates property path queries whose initial or final endpoints come from a small set of values more efficiently. For example, a query like

prefix gnO:<http://www.geonames.org/ontology#>  
select ?id ?par ?parname {  
  bind (URI('http://sws.geonames.org/104515/') as ?id)  
 
  ?id rdf:type gnO:Feature .  
  ?id gnO:parentFeature+ ?par .  
  optional {  
    ?par gnO:name ?parname  
  }  
} 

will be evaluated much more quickly than before because the subject of the path query is fixed.

Rfe11937 - Agload: remove blank node cache size guessing code

In agload, blank node cache size guessing code did not work very well. It was removed and the blank node cache size defaults to 1,000,000 which works reasonably well on small and large memory machines.

This change will only be noticed when loading ntriple or nquad files with blank nodes.

Rfe11839 - Improve eager evaluation of impossible query constraints

AllegroGraph tries to evaluate FILTER constraints as early as possible. In particular, a constraint may become impossible or a tautology before the query even runs if external variable bindings are supplied. AllegroGraph was already handling tautologies eagerly but was not doing the same for impossible constraints. This has now been remedied.

Rfe11827 - Improved memory footprint of joins with unbound variables

Previously, queries involving joins had a worst case memory footprint of O(2^N) where N is the number of join variables. This is now O(1), but the time complexity is O(N). In practice, this means that the client gets a timeout instead of a memory exhaustion warning.

Rfe11233 - Simplify property path queries when possible

Property path queries that do not use the zero or more or one or more operators and that do not have complex alternation can be expanded directly into simpler forms. This lets them be executed using the regular query machinery which can often be faster than the specialized path query machinery.

Rfe9699 - Dynamically grow string table files

Prior to this change string tables files were large sparse files. The apparent size of these files frequently led to user confusion. In addition, full filesystem errors were not handled gracefully when modifying the sparse files. Now string table files start at a minimum size and grow as needed. If growth of a string table fails due to a full filesystem error, a complaint will be logged to agraph.log and the operation will be retried periodically until it succeeds.

Bug21592 - Improve cursor resource management

Some cursors were failing to close all of their file handles which could lead to errors. This has been corrected.

Bug21583 - see Bug21575 below.

Bug21582 - Complex nested unions could fail to return some bindings

Given the right data, a complex query involving multiple nested unions could misapply an optimization meant for one part of the union to a different part. This could lead to some variables in the query being unbound.

This has been corrected.

Bug21575, Bug21583 - property path optimizations and better bookkeeping

Improve the way that AllegroGraph implements some property path queries.

Correct a problem with property path queries and over-zealous resource management.

Improve the bookkeeping of unused variables when using external bindings.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Bug21550 - Complex FILTER EXISTS expressions could cause parsing errors

Complex FILTERs that used EXISTS (or NOT EXISTS) could signal errors during parsing. This has been corrected.

Bug21549 - Nested sub-queries could fail to project variables correctly

A query with nested sub-queries that used * to return all variables in some of the sub-queries could incorrectly reason about variable scope which could cause queries to fail. This has been corrected.

Bug21534 - Garbled error message on memoryCheckWhen parsing failure

When the memoryCheckWhen configuration directive had an unsyntactic value ("memoryCheckWhen junk:1"), the error message was wrong. This bug has been fixed.

Bug21506 - Correct possible error when mixing UNION and GRAPH

Under some circumstances, a query with both UNION and GRAPH clauses could signal an error during execution. This has been corrected.

Bug21505 - Unused variables from a sub-query could confuse AllegroGraph

AllegroGraph removes query variables that are not used if a query has either the DISTINCT or REDUCED modifiers. The unused variables logic was not correctly handling the case of a sub-query whose projected variables were not used elsewhere. This could lead to an error during query processing. This has been corrected.

Bug21498 - Do not signal error when all ORDER BY expressions are redundant

AllegroGraph removes ORDER BY expressions that do not depend on the bindings of a row because they cannot have any bearing on the outcome. For example, the query:

SELECT * { ?s a ?o } ORDER BY ?pineapple 

is equivalent to the query

SELECT * { ?s a ?o } 

Unfortunately, AllegroGraph was incorrectly signaling an error when all of the expressions in an ORDER BY clause were irrelevant and therefore removed.

This has been corrected.

Bug21497 - Filters could be misapplied in some situations

AllegroGraph attempts to move FILTERS closer to where they apply in order to increase query speed. If a FILTER was applicable on both sides of a UNION and also applied to single triple-patterns then it was possible for AllegroGraph to think that the FILTER only needed to be used on the left-hand side of the UNION. For example, a query like this could fail to apply the FILTER to ?c on the right-hand side:

select * {  
  { ?a ex:b ?c . } union { ?a ex:c ?c . }  
  filter (?c*2 = 2)  
 } 

This has been corrected.

Bug21496 - Improve memory management for CONSTRUCT queries

AllegroGraph now tracks memory more accurately and manages it more efficiently for CONSTRUCT queries.

Bug21493 - Bind is too aggressive in assuming single values

AllegroGraph was over-optimizing assignment (BIND) such that assignments to the same variable name in different pieces of a UNION could fail to work correctly. For example, a query like

SELECT * {  
  { BIND( :person1 as ?person ) }  
  UNION  
  { BIND( :person2 as ?person ) }  
} 

would return only one binding for ?person rather than two.

This has been corrected.

Bug21486 - Correct handling of some IN LINE data syntax

AllegroGraph was not correctly reading all of the supplied values when using the abbreviated simple-variable syntax. E.g., a query like

select ?person (concat(?first, ' ', ?last) as ?name) {  
  values ?person { ex:person1 ex:person2 ex:person10 }  
  ?person ex:first-name ?first .  
  ?person ex:last-name ?last .  
} 

was processing only the first supplied value. This has been corrected.

Bug21476 - Some blank nodes could fail to print in a 32-bit Lisp client

It was possible for blank-nodes to raise an error when AllegroGraph tried to print them in N-Triples format. This has been corrected.

Bug21470 - Corrected Turtle parser relative URI normalization

AllegroGraph was incorrectly applying sections 6.2.2 and 6.2.3 of RFC3986 when it merged a base URI with a relative one. This has been corrected.

Bug21458 - Standardize AllegroGraph exit code

AllegroGraph's license check can be resolved in three ways:

  1. Normal start up - a license is present and correct.

  2. Free start up - no license is present so the free limit is used.

  3. Limited start up - only partial license information is present so AllegroGraph runs in free mode.

In all of these cases, the exit code returned will be zero (0).

If the license code is invalid or if there are any other problems with server start up, then AllegroGraph will fail to start and return a non-zero exit code.

Bug21454 - URI parsing was too strict

AllegroGraph was being stricter than necessary when parsing BASE URIs such as foo:bar:/baz. This has been corrected.

Bug21379 - Reasoning triple-store confusion about unspecified graph fixed

A reasoning triple-store failed to distinguish between a query where the graph was left off (i.e., left as a wild card) and a query where the graph was explicitly set to the default-graph of the current triple-store. This could cause it to infer incorrect triples.

This has been corrected.

Bug21365 - Fix lookup of user access rights when authorizing access to repositories in the root catalog.

AllegroGraph would deny authorization requests on repositories in the root catalog under some conditions, despite specific user access right allowing such (read/write access on catalog "/", repository ""), or global rights (read/write access on catalog "", repository "*"). This has been corrected.

Bug21353 - Correctly handle aggregation on empty datasets

AllegroGraph was failing the Data Access Working Group (DAWG) "agg empty group" test because it was not returning a row for a query asking for aggregation on an empty dataset. This has been corrected.

Bug21158 - Catch errors during filter analysis

AllegroGraph was signaling an error during query planning if the eager analysis of a FILTER expression resulted in a error. The correct behavior is to treat the FILTER as an impossible constraint.

This is now handled correctly.

HTTP Client

Rfe11609 - Support text/table content-type in the HTTP protocol

AllegroGraph will now return a simple ASCII table if an HTTP request uses the content-type text/table. For example, a query might return a table like:

  -----------------------------------  
  | person    | name     | location |  
  ===================================  
  | mo:Bob    | "Bob"    | ---      |  
  | mo:Robert | "Robert" | 23       |  
  -----------------------------------  
 

Bug20622 - "Mode" was not optional for DELETE /statements/duplicates

The mode parameter, which is used to indicate which components of each triple must be equivalent to count as duplicates of each other, is now optional and defaults to "spog" for the DELETE /repositories/[name]/statements/duplicates HTTP protocol request.

SPARQL

Rfe11920 - Signal error when aggregate expressions are used in the wrong place

AllegroGraph did not signal a useful error when aggregate expressions were used incorrectly in a SPARQL query. E.g.,

SELECT * {  
  ?a ex:foo ?value .  
  BIND( SUM(?value) AS ?sumValue )  
} 

This has been corrected.

Rfe11494 - Update semantics of SPARQL property path queries

AllegroGraph's property path queries now match the updated semantics of the SPARQL 1.1 specification. In particular, wild-card operators like * and + no longer count their matches. AllegroGraph still supports the now deprecated counting operations, e.g.,

{2,4} 

but no longer counts the unique paths found. I.e., you can use these non-standard operators to find if there is a path but not to count the number of paths.

Note that AllegroGraph now passes all of the Data Access Working Group (DAWG) tests with the exception of

 (pp37) Nested (*)* 

This test uses the path

((:p)*)* 

and AllegroGraph incorrectly returns some duplicate results.

Bug21599 - Variable GRAPH queries that used the graph could lose results

A SPARQL GRAPH query that used a variable graph and included the graph variable in the body of the query could under some circumstances lose results. For example, a query like

SELECT * {  
  GRAPH ?g {  
    ?g a <FoodGroup> .  
    ?g <covers> ?r .  
  }  
} 

could possibly have returned too few results depending on the data in the store.

This has been corrected.

Bug21581 - Improve bookkeeping when using property paths

If AllegroGraph can determine that a variable in a SPARQL query is not used, then it will enable certain optimizations. For example, if a query is returning DISTINCT results and a variable is not used in any joins nor returned by the query, then its values do not matter.

This logic was not correctly rewriting property paths to account for the unused varaibles.

This has been corrected.

Bug21562 - Insert & remove-graph-uri(s) should not override WITH

If insert-graph-uri(s) or remove-graph-uri(s) were passed as query parameters to a SPARQL DELETE...INSERT...WHERE statement, then AllegroGraph was using these parameters even if the statement used WITH to specify a GRAPH. This is now corrected. SPARQL Update statements that use WITH will not be overridden via insert or remove graph URI(s).

Bug21551 - Deletes via SPARQL Update did not correctly handle encoded UPIs

SPARQL DELETE DATA and DELETE...INSERT...WHERE commands could fail to remove some triples due to AllegroGraph's internal representation of typed literals. The failure would occur only if the literal was specified directly (i.e., when the literal was a variable, AllegroGraph behaved correctly).

Bug21548 - Nested unions with fixed patterns could miss results

If a SPARQL query contained nested unions and these unions contained fixed triple-patterns (i.e., patterns with no variables or patterns where all of the variables are bound via some external binding), then some results from these unions could be lost. This has been corrected.

Bug21546 - Correct off-by-one error in some SPARQL constraints

Depending on the data and the query, some SPARQL constraint analysis could lead to a range error. This has been corrected.

Bug21515 - Improve SPARQL parser speed

Compliance changes in the SPARQL parser of AllegroGraph v4.9 reduced the speed of parsing significantly for very large queries and update requests.

This has been corrected so that parse speed is now equivalent to what it was in AllegroGraph v4.8.

Bug21514 - Fix possible error caused by nested sub-query

The data structures used for SPARQL sub-queries were sometimes being cleaned up too early which could lead to a bus error. This has been corrected.

Bug21507 - Fix problem with property path query clause estimation

AllegroGraph was incorrectly estimating the size of clauses involving property paths in SPARQL queries which could lead to poor plans. This has been corrected.

Bug21478 - Fix spurious parsing error when mixing sub-query and aggregation

AllegroGraph's SPARQL parser could become confused if a query mixed sub-query and aggregation and the sub-query used * to refer to all variables.

This has been corrected.

Bug21469 - Relative URIs were not being correctly normalized by the SPARQL parser

Section 5.2 of RFC3986 specifies how "." and ".." path components of relative URIs should be normalized when merging with a BASE URI. The original SPARQL parser implementation skipped this step in the interests of efficiency. This has now been corrected.

Bug21464 - Some combinations of nested UNION and OPTIONAL could fail

The SPARQL 1.1 engine could fail to correctly keep track of solution bindings for queries with certain patterns of UNIONs and OPTIONALs. This has been corrected. #### Bug21501 - Queries with cross-products and empty RHS could emit incorrect solutions

If a query had a cross-product and the right-hand side of the cross-product was empty, then it was possible for AllegroGraph to not discard the solutions on the left-hand side. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug21535 - Db registry memory leak in the Lisp client

If a triple store object isn't closed properly and becomes garbage, the finalizer is supposed to close it. Fixed a bug that rendered this mechanism ineffective, causing leakage of memory.

Bug21512 - With-variables not handled consistently with sub-queries

In some cases, external bindings imposed from with-variables could be lost when using sub-queries. This has been corrected.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe11527 - Support sessions that use the main server port

With this change, the Java client can now be configured to have sessions use the main server port rather than a dedicated port. By default, a dedicated port will be used. Setting the system property com.franz.agraph.http.useMainPortForSessions=true will configure the client to use the main port for sessions.

Rfe8384 - Limiting the duration of a query

With this change, the Java client now supports limiting the duration of a query. When Sesame applications use the AGQuery#setMaxQueryTime method, the AG server will timeout the query appropriately during evaluation and clean up server resources, and the client library will throw a QueryInterruptedException back to the client application.

See Change History for user-visible changes to earlier releases.