Introduction

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 6.0.

Release 6.0

Major Changes

Database internal format changed from 5.1/5.1.1/5.1.2

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.0 is changed from the format in versions 5.1, 5.1.1, AND 5.1.2 (it was previously changed in version 5.0). Therefore all databases must be upgraded before use in 6.0. Note that backing data up before using in a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

Programmer notes

Release 5.1.2

Release 5.1.2 is a maintenance release which contains the bug fix noted below.

Release 5.1.2

Release 5.1.2 is a maintenance release which contains the bug fix noted below.

Release 5.1.1

Release 5.1.1 is a special release which adds support for loading files from Hadoop Distributed File System; see Loading files from HDFS filesystems in Data Loading.

Release 5.1

Major Changes

Version 5.1 is a maintenance release with fixes and general improvements.

Programmer notes

User visible changes

This document lists changes in the current release and several previous releases. See the Change History for a list of changes in all earlier releases.

AllegroGraph 6.0

AllegroGraph Server: General Changes

Use agraph for the Lisp client fasl name

AllegroGraph 6.0 now uses agraph.fasl instead of agraph[major-version-number].fasl for the name of the Lisp fasl file. Both the direct Lisp client fasl file and the various remote Lisp client fasl files (on the various platforms suitable for use as remote clients) are named agraph.fasl. Users need to manage the locations of the various individual agraph.fasl files to be sure the correct one is loaded when used.

Rfe13969 - Add HDFS input support to agload

agload now supports loading files from HDFS sources. For this to work, there must be a working "hdfs" program in your PATH and HDFS input files must be specified on the agload command line using the hdfs:// prefix. For example: 

agload repo hdfs:///user/bruce/lubm-50.nt.gz

agload is described in the Data Loading document. (This change was also in release 5.1.1, as noted below, but we repeat it here as some users may not have installed version 5.1.1.)

Rfe13826 - Improve estimates on federated remote stores

Cardinality estimation of query result sets on remote stores has been greatly improved which leads to more efficient query plans on federated remote stores.

Rfe13807 - Added deleted triple purge operation

When triples are deleted they are not immediately removed from indices. They must remain in all indices until all transactions which may potentially need to see those triples (such as transactions that started before the triples were deleted) have completed (committed or rolled back). A triple that cannot possibly be accessed by any live transaction is referred to as an inaccessible triple. The existence of inaccessible triples in indices wastes space and impacts query performance. Therefore AllegroGraph now provides a facility to purge these triples from indices. See Purging Deleted Triples.

Rfe13806 - Configuration option to control index style

AllegroGraph v6.0 adds a new style of index. The old style is called style 1 and the new style is called style 2. Style 2 indices are faster than style 1 when performing point queries where all of s/p/o or s/p/o/g are fixed (where the result is typically a single triple), but these indices take longer to construct and to optimize. Style 1 indices are better for queries which end up scanning many triples and typically produce results with many triples.

The Style2Indices catalog configuration directive can be used to control which index flavors should use style 2. The directive value can be a comma- or space-separated list of index flavor names, or the value none. The default value is none. Configuration options are discussed in Server Configuration and Control.

Examples 

Style2Indices none  
Style2Indices posgi  
Style2Indices spogi, posgi

The Lisp API function add-index and the REST interface command PUT /repositories/[name]/indices/[type] allow for specifying the index style as well.

Rfe13641 - Change default for temporaryFileDiskSpace

AllegroGraph now defaults to using no more than 25% of the available filesystem space for temporary query files (at the time the query begins). This setting can be changed using the temporaryFilesystemSpaceLimit query option either in the AllegroGraph configuration file or via a PREFIX option to a SPARQL query. If a query attempts to use more space than this limit allows, it will be canceled. temporaryFilesystemSpaceLimit is a new name for the temporaryFileDiskSpace option. The old name has been deprecated. See agcontrol.

Rfe12000 - Minor improvements in how typed string literals are stored

AllegroGraph now stores typed string literals like "hello"^^xsd:string more efficiently.

Rfe11116 - Support SHA384 digest function in SPARQL

In addition to MD5, SHA1, SHA256, and SHA512, AllegroGraph's SPARQL engine now supports SHA384.

Rfe11078 - Improve agload verbosity behavior

Previously, passing -v or --verbose to agload (see Data Loading) could actually decrease its level of verbosity. This has been fixed. The new verbosity levels are: 

0 (-v not supplied): Only report periodic load rate information.  
1 (-v)             : As above, plus print the job option summary  
		 before starting the operation.  
2 (-v -v)          : As above, plus print the name of every file  
		 that has been processed.

(This change was also in release 5.1.1, as noted below, but we repeat it here as some users may not have installed version 5.1.1.)

Bug23660 - AuditEventsToEmail parser too strict

Previously the events list in an AuditEventsToEmail configuration directive had to be comma-separated with no intervening whitespace. This has been relaxed.

Bug23654 - agraph-backup restore fails if StringTableSize specified

An error would occur when trying to restore a database if its StringTableSize did not match the one in the catalog into which it was being restored. The error was: 

Fetching user information failed: String table configuration mismatch:  
Configured number of slots (X) does not match recovered number  
of slots (Y).

This has been corrected. agraph-backup is described in Backup and Restore.

Bug23604 - Improve bidirectional search speed when no paths are found

AllegroGraph's bidirectional search algorithm was expanding more nodes than necessary when no paths between the start and end were found. This has been corrected.

Bug23598 - Error when accessing a suspended account

Previously, if an account was suspended but AccountUnsuspendTimeout wasn't configured, the following error would be signaled when accessing the suspended account: 

nil is not of the expected type number

This has been corrected.

Bug23585 - Audit dialog not showing any results

It was possible for the Audit Log query in AGWebView to fail to display results. This has been corrected.

Bug23526 - Fixed size deleted triple bitmap

Previously an error would occur if a triple with id greater than 34,359,738,367 was deleted. This has been fixed.

Bug23512 - Signal an error when attempting a freetext query and no freetext indices exist

Previously, AllegroGraph would simply return no results if a freetext query was executed and there were no freetext indices defined. Now, AllegroGraph will signal an error. This also modifies the behavior of list-freetext-indices for federated triple-stores. Previously, a federated store would return the intersection of its freetext indices. Now, it returns the union. This makes it easier to use the freetext indexing facility with them.

Bug23442 - Full filesystems can lead to truncated user files

In some cases a full filesystem could cause AllegroGraph user preference files to be truncated. This has been corrected.

Bug23441 - Retry tlog creation if filesystem is full

Previously, if the filesystem ran out of space while allocating a new tranasction log file during a checkpoint or commit, the instance would crash. Now the affected process will block, log a message to agraph.log, and periodically retry the operation until it succeeds. To avoid confusing hangs, failure to preallocate transaction log files on database creation will still result in an immediate error.

Bug23370 - N-Triple, N-Quad and Turtle parsers should handle UTF8 files with a BOM

AllegroGraph's parsers did not correctly parse UTF8 files that started with a byte order marker (BOM). This has been corrected.

Bug23270 - flush-in-memory-chunks deadlock

Previously, when performing duplicate triple deletion, it was possible for the AllegroGraph instance to become deadlocked and hang. This has been fixed.

Bug23361 - N-Triples parser could incorrectly import non 7-bit ASCII data

Unless a UTF-8 encoding was explicitly specified, the AllegroGraph N-Triples parser could fail to handle Unicode N-Triples data correctly. This has been corrected.

Bug23238 - Fix 'UPI has not been added to string dictionary' bug

Previously, certain queries involving large intermediate result sets could fail with a UPI has not been added to string dictionary error. This bug has been fixed.

Bug23233 - Fix bug in agraph-backup progress output.

Some garbled output showing a "Printer error" could be produced when running a backup operation using agraph-backup. This has been corrected.

Bug23231 - Prevent spurious output from agraph-replicate

Previously, agraph-replicate would emit spurious output when exiting. This output has been suppressed.

Bug23229 - agraph-replicate --stop would not stop replication.

Previously, agraph-replicate --stop would not actually cause replication to stop. This has been corrected. See Replication for information on agraph-replicate.

Bug22490 - Fix potential deadlock when creating/modifying freetext indexes.

With proper timing, it was possible hang the server when adding or updating a freetext index. This has been corrected.

Bug22489 - agraph-backup fails due to missing tlog file

Under certain circumstances agraph-backup could fail to back up a database due to not finding a transaction log file. This error has been corrected.

Bug22165 - commit-time duplicate suppression does not operate under lock

Previously, commit-time duplicate triple suppression did not properly consider triples that were added in a concurrent transaction, possibly resulting in duplicate triples being added. This has been corrected.

Bug19831 - xsd:booleans are now stored as encoded UPIs

Previously, xsd:booleans were stored as strings. They are now stored as encoded UPIs. This will make some queries slightly more efficient.

HTTP Client

Rfe13941 - Allow specification of chunk style when adding an index

The PUT command 

PUT /repositories/[name]/indices/[type]

now accepts a style query parameter which may be 0, 1, or 2. See AllegroGraph Indices for a discussion of index styles.

Rfe13750 - Log errors during writing the HTTP response

While using the REST API, which most clients do, serving a request usually has a computation phase that's followed by an output phase. If an error happens in the computation phase, the appropriate HTTP status code is returned. The tricky case is with chunked encoding (for example, streaming query results back) when some results were already written to the socket. Then upon encountering an error, it is not possible to change the HTTP status code and return a meaningful error message. For this reason, errors should normally be detected during the computation phase.

With this change, any errors (except those caused by the client end of the socket being closed) encountered during the output phase are logged to make debugging easier.

Rfe13711 - Add query PREFIX option to cancel queries when there are warnings

AllegroGraph has a new query PREFIX option: cancelQueryOnWarnings. If true, then a SPARQL query will cancel itself if it encounters any warnings during query planning or execution. Typical warnings include:

If the option is set to no, then warnings will be logged but query execution will continue.

Bug23546 - Problems adding very long objects with JSON upload

An error could occur if triples with very long strings were imported using the REST API and the application/json content-type. This has been corrected. Other content-types were not affected by this bug.

Bug23324 - Some text index properties could not be viewed

If an AllegroGraph freetext index specified that it should only index literals with one or more particular datatypes, then that index would signal an error when its properties were read using the REST API. This has been corrected.

SPARQL

Rfe14076 - Optimize RDF serialization of CONSTRUCT and DESCRIBE queries

RDF serialization formats like RDF/XML or Turtle were spending more time than necessary trying to find namespace abbreviations. This has been improved.

Rfe13998 - agquery can now interact with SPARQL endpoints

AllegroGraph's command-line agquery tool can now send queries to SPARQL endpoints and retrieve the results. agquery is described in the Querying document.

Rfe13975 - Streamline internal AllegroGraph merge operations

Query operations that require merging of multiple result streams have been optimized. Examples include the inference of types and properties in the dynamic reasoner and computing distinct SPARQL results.

Rfe13519 - General low-level SPARQL query optimizations

Modified the low-level cursor operations in the SPARQL query engine so that they perform between 3 and 5% better for most queries.

Rfe13387 - Improve performance of SPARQL replace function

The SPARQL replace function is now significantly faster when called with a constant regular expression.

Rfe13327: Improve parsing of diskChunkRowCount and other query parameters

It is now possible to specify query parameters like diskChunkRowCount and other query parameters using sizes like 100m or 1g rather than needing to write out the numeric value as an integer.

Rfe12143 - Improve analysis of constant BIND and VALUES forms

AllegroGraph now does a better job of analyzing and using constant BIND values and single variable VALUES forms. When possible, these forms are pulled into the triple-patterns in the BGPs rather than being evaluated at run-time. For example, both of these queries will now run much more efficiently: 

select * {  
    bind( 'test' as ?test )  
    ?s :predicate ?test .  
}

and 

select * {  
  values ?test { 'test' 'two' 'three' }  
  ?s :predicate ?test .  
}

Rfe12131 - Remove support for the old SPARQL 1.0 query engine

AllegroGraph no longer supports the older and long deprecated SPARQL 1.0 query engine. The newer 1.1 engine is to be preferred in all cases. In particular, this means that you can no longer force the use of the 1.0 engine via the queryEngine PREFIX.

Bug23696 - Improve query plan for some property path queries in sub-queries

AllegroGraph could fail to optimize a simple alternation property path query when it appeared inside of a sub-query. This has been corrected.

Bug23653 - Remove spurious cross-product warning for (optional identity A)

AllegroGraph was signaling a cross-product warning for SPARQL queries that contained an OPTIONAL with no left hand side. For example: 

select * {  
  optional { ?s a :Animal }  
}

has a query algebra like: 

(left-join identity BGP)

Though the warning is technically correct, the construct above is harmless and the warning is unnecessary.

Bug23632 - SPARQL parser has trouble with escaped backslashes at the end of a literal

The SPARQL parser would fail to correctly interpret literals with an escaped backslash as their final character, e.g., 'abc\'. This has been corrected.

Bug23616 - COUNT aggregation counts unbound values from unknown variables

If a COUNT aggregration expression included variables that were not projected from the underlying query, then AllegroGraph was treating them as if they were bound and returning a count. This has been corrected.

Bug23582 - SPARQL could lose results when a sub-query used limit and distinct

It was possible for solutions to be overwritten and lost when a sub-query used both LIMIT and DISTINCT. This has been corrected.

Bug23581 - Optional mis-analysis in sub-queries could drop solutions

It was possible for AllegroGraph to mis-analyze whether a variable in a SPARQL sub-query was optional or not and therefore to lose solutions. This has been corrected.

Bug23524 - Property path queries could lose results when there is circularity

Under some circumstances, a SPARQL query that used property paths could lose results that arose from circular paths through the triple-store. This has been corrected.

Bug23519 - SPARQL sub-query and GRAPH clauses could interact badly

It was possible for a query that used sub-query and GRAPH clauses to lose solutions. This has been corrected.

Bug23489 - Queries that started with a SERVICE clause could lose bindings

If a SPARQL query started with a SERVICE clause and contained a cross-product then it was possible for some bindings to be lost. This has been corrected.

Bug23452 - Some temporal magic properties could fail to produce results

Some of AllegroGraph's temporal magic properties could produce plans that introduced filters that could not succeed. This has been corrected.

Bug23448 - Fix problem with SPARQL t:pointBeforeDatetime and t:pointAfterDatetime

The two SPARQL magic properties t:pointBeforeDatetime and t:pointAfterDatetime could fail to find bindings if the dateTime supplied was a constant not present in the triple-store. This has been corrected. SPARQL magic properties are listed here in SPARQL Magic Properties.

Bug23443 - Remote SPARQL queries cannot handle really large results

Returning very large query result sets (i.e., Gigabytes) from remote SPARQL endpoints could lead to resource exhaustion. This has been corrected.

Bug23373 - Improve SPARQL efficiency for small ORDERed or DISTINCT result sets

SPARQL queries with small result sets that use ORDER BY or DISTINCT will now be slightly faster.

Bug23344 - SPIN magic properties don't work if they use other SPARQL functions

A query-time error would be signaled if a SPIN magic property used other SPARQL functions like CONCAT or SUBSTR in its definition. This has been corrected.

Bug23341 - remote-triple-stores handle limit and offset in SPARQL queries incorrectly

If a query specified an OFFSET as part of the query string, then any externally imposed OFFSET (i.e., from the HTTP query parameters) was ignored. This has been corrected. If a query specified a LIMIT as part of the query string, then the smaller of that limit and the externally imposed one would be used instead of just using the external one. This has been corrected. Note that this change means that specifying a limit in a query in AGWebView will have no effect as the limit is controlled via the dropbox selection box.

Bug23329 - Use SPARQL/JSON for SERVICE calls to avoid problems with embedded control characters

AllegroGraph now uses SPARQL/JSON rather than SPARQL/XML to receive results from SPARQL endpoints. This makes it possible to return results that contain control characters like ^Z. The format is also more concise and easier to parse which makes processing the transmissions approximately twice as fast.

Bug23321 - Nested sub-queries and BIND forms can fail to compute correct results

In rare cases, it was possible for a query involving nested SPARQL sub-queries and BIND expressions to fail. This has been corrected.

Bug23311 - Requesting application/nquads from a CONSTRUCT query leads to error

If an application requested application/nquads from a CONSTRUCT query via the HTTP interface, a string-table lookup error would result. This has been corrected. Although version 1.1 of SPARQL CONSTRUCT can only return triples, it still makes sense to ask for N-Quads. In this case, the N-Quads will simply have the graph place left off.

Bug22674 - DESCRIBE and CONSTRUCT queries on end points could fail

SPARQL CONSTRUCT and DESCRIBE queries against remote SPARQL end points could fail when trying to parse the returned results. This has been corrected.

Bug22672 - Queries where every FROM NAMED graph did not exist could signal an error

If every FROM NAMED graph in the dataset did not exist, then some queries that used the GRAPH clause in their body could signal an error rather than returning results. This has been corrected.

Bug22515 - Increase responsiveness of query canceling during result generation

AllegroGraph did not check for query canceling or timeouts during result post-processing (e.g., while sorting the final result or removing duplicate entries from a DISTINCT result set) or result generation. This has been corrected.

Bug22292 - tripleid magic property should handle fixed triple IDs

The SPARQL magic property for AllegroGraph tripleId-based reification would not function for fixed triple IDs. This has been corrected.

Bug22268 - Sub-queries that used Aggregation and limit could lose solutions

If a sub-query had a limit and used aggregation, then it was possible for some solutions to be lost during processing. This has been corrected.

AGWebView

Bug23690 - Fixed bug breaking the AGWebView View Statements dialog in IE.

A bug in AGWebView that prevented the View Statements dialog from fetching and displaying statements when brought up in Internet Explorer has been fixed. The symptom was a partially loaded page with the spinning "Loading..." icon remaining permanently on the page.

Bug23521 - More... button in AGWebView is broken for Prolog select queries

The button to show additional results in the AGWebView query window was broken for Prolog select queries. This has been corrected.

Bug23358 - AGWebView doesn't always display Unicode consistently

Under some conditions, AGWebView would display literals and resources containing Unicode using the \u escaping mechanism of the N-Triples syntax rather than just displaying the characters directly. This has been corrected.

Bug23315 - Running backup from within AGWebView does not work

The Backup this Store dialog in AGWebView was broken. This has been corrected.

Changes to the Lisp API

Rfe13941 - Allow specification of chunk style when adding an index

add-index now accepts a style keyword argument which can be 0 or nil (either meaning use the default chunk style for the flavor based on the Style2Indices configuration parameter), 1, or 2. See AllegroGraph Indices for a discussion of index styles.

Bug23600 - remote-triple-stores CONSTRUCT queries now correctly returns cursors

In the Lisp client, a SPARQL CONSTRUCT query should return a cursor that yields arrays of bindings but the remote-triple-store was instead yielding lists of bindings. This has been corrected.

Bug23597 - remote-triple-store CONSTRUCT queries return nil bindings

Many of the programmatic RDF formats (such as :lists or :arrays) for SPARQL CONSTRUCT queries on remote-triple-stores returned nil bindings rather than the actual values. This has been corrected. Note that this bug did not effect text based output formats like N-Triples or Turtle.

Bug23510 - remote-triple-stores did not support load-trig

Lisp remote-triple-stores did not support loading the TriG format. This has been corrected.

Bug23509 - Altering restriction-reasoning-enabled-p could lead to an internal error

Altering the value of restriction-reasoning-enabled-p using the setf method after the reasoner had been applied and prepared could lead to an error. This has been corrected.

Bug23404 - It was impossible to create an anonymous remote session

AllegroGraph would signal an error that This session is owned by another user when trying to create an anonymous remote session in the Lisp client. For example, the following would fail: 

 (open-triple-store "remote"  
    :triple-store-class 'remote-triple-store  
    :server "www.example.com"  
    :specification "<sp2b-5e4>[rdfs++]"  
    :port 9002)

This has been corrected.

Bug23320 - The Lisp direct client now ensures that UPIs are interned when calling add-triple

The Lisp direct client was not checking that UPIs were interned which could lead to a UPI-not-in-string-table error when there were bugs in customer code. This has been corrected. The check does add a small cost to programmatic triple insertion. The while-not-validating-that-upis-are-in-string-table macro can be used to temporarily disable the checks once you are sure that the code is correct.

Bug22844 - The SPARQL datatype function failed to handle nD-geospatial literals

The SPARQL datatype function was unable to handle nD-geospatial literals. This has been corrected.

Bug22793 - Problem with remote-reasoning-triple-stores and SNA

On the Lisp client only, calling Social Networking (SNA) functions on a remote-reasoning-triple-store instance could lead to a stack overflow. This has been corrected.

Bug22791 - get-triple was broken for remote-reasoning-triple-stores

On the Lisp client only, calls to get-triple on a remote-reasoning-triple-store could fail to return results. This has been corrected.

Bug22587 - copy-triples doesn't support remote triple-stores

The copy-triples function did not work on remote-triple-stores. This has been corrected. Note that copy-triples on remote-triple-stores requires that

Bug22586 - Improve copy-triples

The copy-triples function was not behaving correctly with respect to the source-db argument. This has been corrected.

Bug22447 - Fix cursor memory management

Under rare circumstamces, especially in a multi-threaded environment, cursors used in the Lisp client could end up being used in an unsafe way leading to various errors. With the AllegroGraph server, shared backends experiencing high load were at the risk of such failures. This bug has been fixed.

Also, in the Lisp client with a direct connection, any code that involved the creation of cursors (such as queries and calls to get-triples) had to be wrapped in a with-query-environment form. While that's still the recommended approach to take in production, with this change when there is no enclosing with-query-environment, cursors holding critical resources get finalizers which makes it more convenient to experiment at the REPL by reducing the risk of resource exhaustion.

Bug20852 - remote-triple-stores do not support SPARQL update

Lisp remote-triple-stores now support SPARQL update.

Prolog

Rfe13968 - Add :offset to Prolog select

AllegroGraph's Prolog based select query language now supports an :offset option. This can be used to skip the first offset results. For example, 

(select (?class)  
  (:offset 10)  
  (q ?s !rdf:type ?class))

would skip the first 10 results and return the rest of the bindings for ?class.

Documentation

No significant changes.

Python Client

Rfe12828 - Support Python 3

AllegroGraph now supports both Python 2 and Python 3. The sources for Python 2 continue to be in the src2 directory whereas those for Python 3 are in the new src3 directory.

Java Client

No significant changes.

nD Geospatial

Rfe13978 - Validate arguments to ndfn:ordinateValue at parse time when possible

AllegroGraph now validates constant arguments to ndfn:ordinateValue at parse time. See the N-dimensional Geospatial Usage Guide and Example document.

AllegroGraph 5.1.2

AllegroGraph Server: General Changes

Bug19296 - AllegroGraph can stop responding because all threads seem busy

Under rare circumstances it was possible for AllegroGraph to stop responding. Such a hang was accompanied by "All threads busy, pause" log messages. This has been corrected.

AllegroGraph 5.1.1

Because there are only a few changes in release 5.1.1, only headings with actual changes are listed.

AllegroGraph Server: General Changes

Rfe13969 - Add HDFS input support to agload

agload now supports loading files from HDFS sources. For this to work, there must be a working "hdfs" program in your PATH and HDFS input files must be specified on the agload command line using the hdfs:// prefix. For example: 

agload repo hdfs:///user/bruce/lubm-50.nt.gz

agload is described in the Data Loading document. HDFS loading is described in the section Loading files from HDFS filesystems in that document.

Rfe11078 - Improve agload verbosity behavior

Previously, passing -v or --verbose to agload could actually decrease its level of verbosity. This has been fixed.

The new verbosity levels are:

0 (-v not supplied): Only report periodic load rate information. 1 (-v) : As above, plus print the job option summary before starting the operation. 2 (-v -v) : As above, plus print the name of every file that has been processed.w

agload is described in the Data Loading document.

AllegroGraph 5.1

AllegroGraph Server: General Changes

Rfe13570 - Support the TriG format for serialization

AllegroGraph can now serialize data using the TriG format. For example, you can use the output format trig in agexport or an Accept type of application/trig from the REST API.

Rfe13501 - Implement HTTPNoProxy configuration directive

This change adds a new global configuration directive, HTTPNoProxy. HTTP requests to domains that match one of the suffixes specified with HTTPNoProxy are never proxied. HTTPNoProxy can be specified multiple times. 

HTTPNoProxy localhost  
HTTPNoProxy mydomain.com

Requests to 'localhost' and '127.0.0.1' are never proxied. Top level directives are described in Server Configuration and Control.

Rfe12010 - Support the TriG format for data import

AllegroGraph now supports the TriG data format for loading data. See http://www.w3.org/TR/trig/ for details on the format. See Data Loading.

Rfe11730 - Update Turtle parser to conform to the specification

AllegroGraph's turtle parser now passes the W3C conformance tests (http://www.w3.org/TR/turtle/). The parser is also significantly faster.

Bug23077 - Rare thread safety issue in shared back-ends

Previously, when multiple requests for the same repository were being executed concurrently on the same shared back-end a very rare thread safety issue could cause the backend to die. This bug has been fixed.

Bug22944 - Freetext indexer could mishandle encoded literals

AllegroGraph's built-in freetext indexer was trying to index encoded literals (numbers, dates, times and dateTimes) and would sometimes misinterpret them leading to extremely slow indexing. This has been corrected by preventing the indexing of these literals. This means that freetext indices will not include values for encoded literals.

Bug20556: Double-fault of trie-full condition

Previously, when adding a word with many hits to a text index a "trie-full" error could occur. This has been fixed.

HTTP Client

Bug23064 - Post requests to the statements REST API could fail

It was possible for POST requests to the statements REST API to corrupt the stream of bytes being sent. This has been corrected.

SPARQL

Rfe13683 - Improve variable count tracking during query execution

AllegroGraph keeps better track of the estimated cardinality of bindings for variables during query execution which improves the execution engine's overall efficiency.

Rfe13601 - Improve handling of property path queries that use simple alternation

AllegroGraph can now apply its constraint machinery to simple property path queries that use only alternation. For example: 

select ?o {  
  ?s :a | :b | :c ?o .  
  filter( ?o > 10 )  
}  
This means that queries like the above can operate significantly faster.
Rfe13599 - Improve query pattern constraint logic

AllegroGraph now does a better job in choosing how to constrain variables in SPARQL triple-patterns during execution.

Rfe13476 - Add more logging information to SPARQL SERVICE queries

AllegroGraph now includes more logging during the execution of SPARQL SERVICE queries.

Rfe13426 - Provide more control over federated SPARQL queries

The new query option serviceRequestSize provides control over the size of the POST requests used during SPARQL federated query. The default size is one megabyte. A smaller size may cause additional trips to the remote server. Some servers may reject requests that are too large. SPARQL query options are listed in the SPARQL Query Options of the SPARQL Reference.

Rfe13213 - Allow BIND to float during query analysis

AllegroGraph's SPARQL query analyzer now does a better job rewriting query patterns involving BIND. In particular, the BIND clause now floats around JOIN, GRAPH and FILTER clauses. This can help make these queries perform significantly better.

Rfe12269 - Improve query planning with respect to cross products

AllegroGraph's SPARQL query planner now does a better job of detecting cross products at plan time and modifying the query plan to avoid the cross product when this is possible.

Bug23125 - Improve query planning for some property path queries

SPARQL Property Path queries using the zero-or-more operator (i.e., *) could sometimes choose a sub-optimal plan because AllegroGraph was misestimating the size of the triple-patterns involved. This has been corrected.

Bug23095 - Correct internal error that occurred during some Property Path queries

Some Property Path queries could fail to run to completion because of an internal bookkeeping error. This has been corrected.

Bug23081 - Certain SPARQL queries could cause cursor data-structure leakage

Certain SPARQL queries could fail to correctly clean up all of the cursors that they used which could lead to backends becoming unresponsive if they were under heavy load. This has been corrected.

Bug23027 - SPARQL was not ordering reified triple IDs correctly

The SPARQL query engine was not correctly sorting URIs representing triple IDs. This has been corrected.

Bug23026 - Some SPARQL SERVICE calls could mangle their query

It was possible for a SPARQL SERVICE call to lose space characters and thereby send the wrong query to the remote end point. This has been corrected.

Bug23005 - RDF/XML serializer could emit an invalid namespace abbreviation

If there was a namespace mapping between the empty string and some URI, then the RDF/XML serializer would emit an invalid namespace abbreviation. This could occur, for example, when running a CONSTRUCT query using a PREFIX abbreivation like 

PREFIX : <http://example.org#>

This problem has been corrected.

Bug23004 - Sparql XML parser could fail to parse some literals

AllegroGraph's SPARQL XML parser is used when making federated queries against remote endpoints. This parser was incorrectly handling literals with ampersands (or other escaped characters) in them. This has been corrected.

Bug22999 - When embedded in a GRAPH clause freetext SPARQL queries could fail

Depending on the data and the query, a freetext match or matchExpression magic predicate could fail to find all results when it was embedded inside of a GRAPH clause. This has been corrected.

Bug22994: Fix bad interaction between SPARQL sub-query and federation

If a SPARQL query used a sub-query inside of a federated query, AllegroGraph could lose track of the variables involved and create inadvertant cross-products. This has been corrected.

Bug22966 - SERVICE clause in a sub-query could fail

Depending on the variables projected, a SPARQL query that used a SERVICE clause inside of a sub-query could fail during execution.

Bug22937 - GROUP BY returned an empty row when there are no aggregates projected

If a SPARQL aggregation query projects only variables (i.e., it does not use any aggregation functions) and the query has no results, then AllegroGraph would return a single row of unbound bindings rather than returning no results at all. This has been corrected. An example of the query in question would be 

select ?anything {  
  ?s <ex://missingPredicate> ?anything .  
} group by ?anything

Bug22915 - SPARQL STR function is incompatible with aggregation

Queries that used the SPARQL STR function on the result of an aggregation would fail. This has been corrected. An example of such a query would be: 

select ( str( count(1) ) as ?count) {  
  ?s :testPredicate ?o .  
}
Bug22904 - Bad query bookkeeping could cause performance problems

Given the right data and the right query, the SPARQL query engine could be led to evaluate triple patterns twice: once with constraints and once without which would lead to severe performance problems. This has been corrected.

Bug22899 - Query options could not always be used with SPARQL SERVICE

When using the SERVICE clause, AllegroGraph's query PREFIX options were not always being sent to endpoints in a way that they could be read. This has been corrected.

Bug22874 - N-Dimensional magic property queries could fail

Depending on the data and the query, AllegroGraph could choose an incorrect plan ordering for an n-Dimensional SPARQL magic property. This would lead to the UPI not in string table error. This has been corrected.

Bug22845 - A query timeout of zero now means "no timeout"

Previously, a query timeout of zero was interpreted literally to mean that the query should terminate immediately. Now, a timeout of zero means that the query should run as if there is no timeout specified. This is especially helpful when a default timeout is included in the AllegroGraph configuration file.

Bug22751 - Error using one-or-more property paths in SERVICE clause

AllegroGraph was signaling an error if a query tried to use a one-or-more property path query inside of a SERVICE clause. This has been corrected.

Bug22680 - Correct SPARQL parsing problem for property paths in object lists

The SPARQL Parser was not correctly handling property paths embedded inside object lists like ?s <ex:predicate> [ <ex:Path>* ?o ]. This has been corrected.

Bug22593 - Use the correct count estimates during query execution

Depending on the query and the store, AllegroGraph could sometimes use the wrong variable count estimates which would lead to sub-optimial performance. This has been greatly improved.

Bug22388: Chunkprocessingmemory defaults to a larger value than memoryLimit

The chunkProcessingMemory size defaulted to a larger size than the memoryLimit which meant that queries could fail when using chunk processing. This has been corrected.

Bug22212 - Magic property for t:intervalStarts logic was wrong

The magic property for t:intervalStarts was incorrectly behaving the same as t:intervalCotemporal. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Rfe13593 - The compiler is needed in direct lisp clients that perform queries

This change will likely not affect any users since the compiler is normally in Lisp images, but it is possible to build an image without the compiler and without the compiler being available to load (though, again, it is unusual to do so). Such images serving as direct Lisp clients cannot perform queries.

Bug22973 - Fix Lisp direct client sigsegv when rolling back a closed store

Previously, in the Lisp client when using a direct connection to a triple store, calling rollback on a closed store would produce a SIGSEGV error. With this fix, an informative error condition is signaled.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

nD Geospatial

Bug22786, Bug22850: improve nD string writers and parsers

nD data encoding and decoding supports various human-readable formats (ISO6709 and ISO8601) when such a format can be applied unambiguously to a particular nD encoding. If a human-readable format applies, it will be preferred on decoding (output) and accepted on input (encoding). Previously some correct externalization formats were rejected on input or not used on output.

nD now correctly supports encodings with multiple ordinates of the same specialized type, i.e. lat, lon, alt, and time. Ordinates must of course all have different names. Use of such encodings will necessarily suppress human-readable externalizations since they are ambiguous.

Error detection during nD encoding/decoding is improved along with error message text.

AllegroGraph 5.0.1

AllegroGraph Server: General Changes

Bug23016 - Incorrect triple-count and garbage triples

Under certain circumstances (particularly when delete-duplicate-triples is used), crash recovery or replication can result in an incorrect triple count and garbage triples being added to the database. This has been corrected. We recommend that users using replication and duplicate triple deletion recreate their replicas from a fresh backup. Replication is described in the Replication document.

HTTP Client

Bug23040 - Performing the same ASK query more than once under REST could error

Under certain circumstances, if an ASK query that used joins was evaluated more than once under the HTTP REST API, then AllegroGraph would signal an error. This has been corrected.

AllegroGraph 5.0

AllegroGraph Server: General Changes

Rfe13506 - Minor query optimization for triple pattern filters

AllegroGraph now performs triple pattern filtering slightly more efficiently when the current set of solutions has only a single member.

Rfe13405 - Warn if transparent hugepages are enabled

Due to the potential for negative performace impact, AllegroGraph now logs a warning into agraph.log on startup if Transparent Hugepages are enabled. See this section of Performance Tuning for information on the transparent hugepage issue and its effect on performance.

Rfe13376 - Improve hashing performance

AllegroGraph's internal hashing functions have been further optimized leading to small general performance improvements.

Rfe13346 - configure-agraph should allow for specification of port

The configure-agraph script now prompts for the Port configuration parameter and accepts the --port command line argument. See Running the configuration script in Server Installation.

Rfe13265 - Update N-Triple and N-Quad parsers

AllegroGraph's N-Triple and N-Quad parsers now support the full W3C specifiation and pass all tests. The performance is also slightly improved. Note that this change means that AllegroGraph will no longer load invalid N-Triple and N-Quad source data unless the relax-syntax argument is used. See Data Loading, the Lisp API functions load-ntriples and load-nquads, and [PUT/POST /repositories/name]/statements in REST/HTTP interface.

Rfe13172: Allow setting size of strings hash table at database creation time

A new database configuration parameter StringTableSize is now available. It can be used to configure the number of hash table slots used for the string storage hash table and setting it properly can enhance performance. See Catalog Parameters that Improve Performance in Performance Tuning for more information on performance-enhancing options for details.

Rfe13146 - Improve suboptimal index scan performance

The efficiency of low-level cursor operations has been improved when an optimal index for the cursor is not available.

Rfe12872 - Remove ag3-export / ag4-import

The obsolete ag3-export and ag4-import utilities have been removed. Serializing data to nquads in AllegroGraph 3 and loading it into AllegroGraph 4 is the recommended alternative.

Rfe11580: Added --no-daemonize flag to agraph binary

The agraph binary is the actual program which starts AllegroGraph. It is not typically invoked directly, but rather using the agraph-control script, but it can be called directly. When it is and --no-daemonize is supplied, the service daemon will remain in the foreground. See agcontrol.

Bug22837 - Text index incorrectly indexing the string "nil"

Since version 4.13 the text indexer was incorrectly indexing the word "nil" for triples that were ineligible for text indexing. This resulted in unnecessary work being performed and space being consumed. This has been fixed. We recommend that you drop and recreate any existing text indexes.

Bug22835 - Fix numa info parsing failure

At startup, information about numa (non-uniform memory access) topology is collected and logged. Previously, parsing failed with an error if a cpumap file in /sys/ contained a comma separated list of numbers instead of just a single number. This bug has been fixed.

Bug22767 - Use agraph5 for the Lisp client fasl name

AllegroGraph 5.0 now uses agraph5.fasl instead of agraph4.fasl for the name of the Lisp fasl file.

Bug22754 - Concurrent duplicate triple deletion can result in inaccurate triple count

Under some circumstances duplicate triple deletion transactions which occur concurrently with other deletion transactions could result in an inaccurate triple count. This has been corrected.

Notes:

Bug22566 - Encoded ID validation was too weak

In some cases, a URI representing an invalid encoded ID could be accepted by AllegroGraph instead of signaling an error. This has been corrected.

Bug22564 - Data import could fail to determine the correct base URI

When importing a compressed RDF/XML file, AllegroGraph could fail to determine the correct base URI. This has been corrected.

Bug22559 - RDF/XML parser should validate the base URI if supplied

Previously, the RDF/XML parser failed to validate the base URI it was given which could lead to invalid data. This has been corrected.

Bug21867 - Ensure text index queries are called using strings

AllegroGraph sometimes called lower level text index query functions with invalid arguments. This has been corrected.

Bug21460 - Polygons data cannot be exported and imported

AllegroGraph now converts resources of the form 

<http://franz.com/ns/allegrograph/4.14/index#_N>

(where N is replaced by a whole number) into encoded UPIs rather than hashed ones. These URIs are used to serialize AllegroGraph polygon data which means that polygons can be efficiently exported and imported. Note that the Lisp API upi->value will return the numeric representation for the UPI rather than the string representation. Use part->string if the string representation is needed.

Bug20497, Rfe11047 - Fix fti too many open files error

Previously, when constructing a new text index for a database with existing triples, there was potential for the operation to fail with a "Too many open files" error. This has been fixed.

HTTP Client

Rfe13304 - Support relaxed load syntax for remote triple-stores

The HTTP REST API for loading statements now includes the relaxSyntax argument. When true, N-Triple and N-Quad data are not validated as stringently. For example, blank node names may include underscores. See PUT/POST statements in REST/HTTP interface.

Rfe13249 - Augment optimize-indices API to allow for single index optimization

The HTTP API's indices/optimize service now takes an additional index query parameter which can be specified multiple times. If omitted, then all of a store's indices will be optimized. If supplied, each parameter value should be the name of an index whose optimization is being requested. See [POST /repositories/name]/indices/optimize in REST/HTTP interface.

Rfe11547 - Support Sesame query timeout parameter in HTTP

AllegroGraph's SPARQL HTTP REST interface now supports an optional timeout parameter. If specified, it must be a number greater than zero which denotes the number of seconds to limit SPARQL query execution time. If unspecified, then there is no limit on the length of a time a query may run. See [GET/POST /repositories/name in REST/HTTP interface.

Rfe11221 - General type-mapping improvements

The interface for setting up predicate- and datatype-mappings has been simplified. The same HTTP REST API calls can be used to setup regular XSD datatype mappings and geospatial ones (including cartesian, spherical and n-Dimensional). In each case, the datatype must be specified using its URL. E.g., the 10x10 cartesian mapping with resolution 1 would use the URL: 

<http://franz.com/ns/allegrograph/3.0/geospatial/cartesian/0.0/10.0/0.0/10.0/1.0>

See Type mappings in REST/HTTP interface.

Bug22576 - POST|PUT statements service should not require eval permission

AllegroGraph would not allow client-side files to be imported into a store unless the user had eval permission. This has been corrected. Of course, eval permission is still required to import server-side files.

SPARQL

Rfe13510 - Improve performance of the SPARQL query REST API

AllegroGraph now automatically caches query information across REST API calls to improve overall query performance.

Rfe13323 - Provide API to retrieve only the ID part of an encoded ID

There is now a SPARQL function, , that retrieves the non-prefix portion of an encoded ID. For example, if the encoded ID has IRI , then the function will return "3489a". In a query, this would look like: 

select * {  
  _:b ex:identifier ?id .  
  bind( <http://franz.com/ns/allegrograph/fn#encodedIdId>(?id) as ?value )  
}
Rfe13305 - Deprecate old-style SPARQL GEO syntax

Because AllegroGraph's magic properties support a richer range of geospatial features than the old-style SPARQL GEO syntax, the old syntax is being deprecated and will now signal a warning when it is used. See N-dimensional Geospatial Overview.

Rfe13297 - Reduce memory consumption for distinct aggregation in SPARQL queries

AllegroGraph now better manages memory when evaluating distinct aggregation queries like: 

select ?dept (count(distinct ?student) as ?sc) where {  
  ?dept a :Department .  
  ?student :memberOf ?dept .  
} group by ?dept
Rfe13289 - When logging, use query supplied namespaces for abbreviations

Previously, AllegroGraph would ignore SPARQL query supplied namespaces when displaying resources during query logging. It now uses these namespaces to improve the readability of the display.

Rfe13171 - Improve query checks for unknown variables

AllegroGraph now warns more consistently when a valid SPARQL expression uses variables in unreasonable ways. For example, this query is perfectly valid but the ORDER BY variables are not bound by the query so it is unlikely to be what was intended: 

select ?a ?year ?label {  
  ?a a :Car .  
  ?a :modelYear ?year .  
  ?a rdfs:label ?label .  
} order by ?autoYear
Rfe12470 - Apply multiple constraints to the same triple pattern

Previously, AllegroGraph could only apply a single logical FILTER expression to any given triple pattern. This meant that queries like: 

select * {  
  ?s ?p ?o .  
  filter( ?p = ex:pred1 && ?o > 10 )  
}  
would optimize the query plan with either the ?p constraint or the ?o  
constraint but not both. This has now been improved so that queries  
like the above will use both constraints simultaneously and thereby  
run much more efficiently.
Bug22803 - Fix bug in authorizationBasic query option

The authorizationBasic query option was not being parsed correctly which meant that queries that tried to use it would fail. This has been corrected.

See Change History for user-visible changes to earlier releases.