Introduction

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 6.1.

Release 6.1

Major Changes

Database internal format changed from versions 6.0/6.0.1/6.0.2

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.1 is changed from versions 6.0.2/6.0.1/6.0. Therefore it is necessary to upgrade all databases created by versions prior to 6.1 for them to be used with 6.1. Note that backing data up before using in a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

6.1 Programmer notes

Release 6.0.2

Version 6.0.2 is a maintenance release which fixes a security hole associated with Prolog and AllegroGraph. The problem is described below, in the entry for bug23877.

Release 6.0.1

Major Changes

Version 6.0.1 is a maintenance release with some v6.0 bug fixes. See below for information on the fixes.

Database internal format unchanged from version 6.0

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.0.1 is unchanged from version 6.0. (The format in version 6.0.1, like that version 6.0, is changed from the format in versions 5.1, 5.1.1, and 5.1.2.) Therefore it is not necessary to upgrade 6.0 databases for them to be used with 6.0.1. Note that backing data up before using in a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

6.0.1 Programmer notes

Release 6.0

Major Changes

Database internal format changed from 5.1/5.1.1/5.1.2

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.0 is changed from the format in versions 5.1, 5.1.1, AND 5.1.2 (it was previously changed in version 5.0). Therefore all databases must be upgraded before use in 6.0. Note that backing data up before using in a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

Programmer notes

Release 5.1.2

Release 5.1.2 is a maintenance release which contains the bug fix noted below.

Release 5.1.2

Release 5.1.2 is a maintenance release which contains the bug fix noted below.

Release 5.1.1

Release 5.1.1 is a special release which adds support for loading files from Hadoop Distributed File System; see Loading files from HDFS filesystems in Data Loading.

Release 5.1

Major Changes

Version 5.1 is a maintenance release with fixes and general improvements.

Programmer notes

User visible changes

This document lists changes in the current release and several previous releases. See the Change History for a list of changes in all earlier releases.

AllegroGraph 6.1

Server Changes

Rfe14282 TBC 5.1.4 plugin

AllegroGraph plugin for TopBraid Composer has been updated to be compatible with TBC 5.1.4. See TopBraid Composer Plugin for more information.

Rfe14279 - agraph-backup should shut down instances

Previously, agraph-backup would leave instances open and lingering after performing a backup or restore operation. This could result in hangs due to memory exhaustion when running backup-all or restore-all operations across many repositories. The likelihood of hang would depend on a combination of repository size and the value of the InstanceTimeout directive.

Now, agraph-backup will attempt to shut down the instance process of all repositories it operates on, as soon as the operation has completed. agraph-backup is described in the Backup and Restore document.

Rfe14273 - Reduce repeated query log messages

AllegroGraph was repeating some messages unnecessarily when logging query execution. This repetition has been reduced.

Bug24130 - Access to the system initfile should be protected

It was possible for any user to access the contents of the AllegroGraph init file. This has been corrected. Now only users with superuser privileges will be allowed to view the file.

Bug24128 - Blank nodes in RDF/XML collections

When parsing an RDF/XML file containing a collection with a blank node as one of its elements, AllegroGraph used to generate an invalid triple (using "li" as the predicate, instead of one of 1, 2, _3, ...).

This has been corrected.

Bug24126 - RDF/XML parser incorrectly handles empty elements.

The RDF/XML parser used to incorrectly handle empty literals encoded as property elements with empty bodies. The parser would generate a blank node instead of an empty literal.

This has been corrected.

Bug24079 - agload with '--error-strategy ignore' and unrecognized file types

agload will now print a warning before ignoring a file of an unrecognized type regardless of the value of the --error-strategy argument. agload is described in the document Data Loading. Example:

agload --error-strategy ignore repo wrong-file  
Cannot guess the format of wrong-file. Use --input to specify it.  
Load finished 0 sources in 00:00:01 (1.00 seconds).  No triples added.  
Terminating agload processes, please wait... 

Compare with previous behavior:

agload --error-strategy ignore repo wrong-file  
Load finished 0 sources in 00:00:01 (1.00 seconds).  No triples added.  
Terminating agload processes, please wait... 

Bug24066 - Sessions did not close their databases after idle timeout expires

When a session expired after a given timeout, it would not close its database until another session was started. This has been corrected.

Bug23964 - Incorrect escaping of IRIs in Turtle output

AllegroGraph used to incorrectly escape some characters when using the Turtle format. Furthermore, spaces and a few other characters that are not legal in IRIs were not escaped at all. Other characters, such as double quotes, were escaped using an invalid syntax (\" instead of \u0022).

Qualified names were affected by a similar problem.

This has been corrected - all characters are now properly escaped in both IRIs and qualified names.

Bug23960 - gz file import is broken

Importing .gz or .bz2 files using single-threaded import used to cause an error as the file was not correctly uncompressed.

This has been corrected.

Bug23828 - CSRF vulnerability in file upload

The import dialog in AGWebView was not adequately protected against Cross-Site Request Forgery (CSRF) attacks. When a logged-in user visited a malicious web site, an attacker could import arbitrary triples into any store the user can access.

This has been corrected - a CSRF attempt will now result in a password prompt.

Bug23820: Prevent repository access when a restore is in progress

Previously, an attempt to access a repository while it was in the process of being restored could result in random errors and problems, including being unable to close a repository. This change improves the way in which AllegroGraph notes that a repository is being restored, and improves the check for restore-mode so that these issues are prevented.

Bug23705 - A triple-store must be open to define SPIN magic properties

Changes in AllegroGraph 6.0 required that a triple-store be open before any SPIN magic properties could be defined. This has been corrected.

Bug23339 - Canonical escaping in N-Triples

AllegroGraph used to produce invalid escape sequences in URIs when printing triples in the N-Triples format. Some reserved characters, such as double quotes, were escaped with a backslash. Others were not escaped at all.

This has been corrected. All reserved characters are now properly escaped using the \u notation.

In addition, escaping of strings in N-Triples has been modified to make the output match the canonical form described in the standard. The most notable change is that non-ascii characters are no longer escaped.

Bug23181 - Consistency bug when backing up deleted bitmap

If a triple was deleted while backup was in progress, that triple would also be marked as deleted in the restored store. This has been fixed, backup now represents a consistent state of the store at the time it was started.

Bug22115: Fix configure-agraph handling of passwords.

configure-agraph could crash when entering passwords that contain certain non-alphanumeric characters in them. This bug has now been fixed.

Bug21655 - serialize-rdf/xml can emit illegal xml characters

In XML, control characters (e.g. NUL, BEL, STX, ...) cannot be represented in any form. The RDF/XML serializer (with the Lisp function serialize-rdf/xml, used by agexport when passed --output rdfxml) used to silently output such characters, producing files that could not be parsed.

This has been corrected - if the serialized data contains any invalid characters an error will be reported, as in the following example:

bash-4.2$ agexport -o rdfxml store output.rdf  
open the triple-store store on port 10035 and export to ...  
Warming up the store  
Error Unable to escape character U+0008 (Backspace). occurred during processing  
logging error to /.../agexport.log  

Bug20220 - Fix bug when validating the --loaders argument to agload

The --loaders argument to agload accepts as a value a positive integer between 1 and 32. On machines with a large number of cores, agload would default to a number of loaders greater than 32 and then error due to the value being too large. This validation bug is now fixed.

HTTP Client

Rfe14445 - Add REST API for [catalogs/CATNAME]/respositories/REPONAME/exists

AllegroGraph's REST API now includes

[catalogs/CATNAME]/respositories/REPONAME/exists 

which can be used to determine whether or not the named repository exists. It will return a 404 code if the repository does not exist and a 200 response with the body true if it does.

Bug23942 - Handle HTTPS scheme for SPARQL UPDATE LOAD command

The SPARQL UPDATE LOAD command would fail when using the HTTPS URL scheme. This has been corrected.

Bug23817 - Running a query that used MINUS repeatedly via REST could error

Under certain circumstances, if a SPARQL query that used MINUS was evaluated more than once using the HTTP REST API, AllegroGraph could signal an error. This has been corrected.

SPARQL

Rfe14278 - Improve SPARQL SERVICE efficiency for AllegroGraph endpoints

AllegroGraph now uses a more efficient binary data format when communicating with AllegroGraph-based SPARQL endpoints. This can lead to large performance improvements for SPARQL SERVICE calls.

Rfe14257 - DISTINCT|REDUCED mode should be passed on to SERVICE calls

When the main query uses DISTINCT or REDUCED, any use of SPARQL federation will pass on the modifier to endpoints. This can greatly reduce overall IO.

Rfe14254 - SERVICE should remove duplicates from any VALUES it passes along

If the main query is using the DISTINCT or REDUCED modifier, then the SERVICE requests should remove duplicate rows from the list of VALUES that is passed to the endpoint. This can make the query much more efficient.

Rfe14086 - New SPARQL functions for performing haversine calculations.

These SPARQL functions accept various combinations of latitude/longitude coordinates and nD encoded literals. They compute distances on the Earth's surface.

The individual names indicate the required argument types: a Lat or Lon must be a number interpreted as degrees. A Loc must be an nD literal that encodes a latitude and a longitude. Its nD encoding must be registered in the open database, but is not needed in any of the function calls themselves.

Each of these functions accepts an optional additional argument specifying the units of the computed result. It may be :km (default) or :miles expressed as a resource in the keyword namespace, i.e., <http://franz.com/ns/keyword#km> or <http://franz.com/ns/keyword#miles>.

See the SPARQL functions list in the SPARQL Magic Properties document for a list of SPARQL function with links to further documentation.

Rfe11390 - Modify SPARQL 1.1 lexer/parser to match changes to standard

The SPARQL parser used to fail when the input contained any:

All mentioned cases are now parsed correctly, in accordance with the SPARQL grammar.

Bug24161 - Correct possible segmentation faults during SPARQL DELETE operations

Under some circumstances, a SPARQL DELETE command could cause a segmentation fault. This has been corrected.

Bug24103 - Correct problem in nD magic property SPARQL queries

It was possible to get an error when executing a query that combined nD magic properties and other patterns. This has been corrected.

Bug24030 - An empty IN filter is ignored if there are any other IN filters

If a SPARQL query contained two or more IN filters that were associated with the same triple pattern and one of the IN filters was empty, then both IN filters were ignored. This has been corrected.

Bug24026 - Fix query failure when using DISTINCT, SERVICE and magic predicates

It was possible for a query to fail during planning if it used multiple SERVICE clauses, the service clauses used magic predicates, and the query asked for DISTINCT results. This has been corrected.

Bug24020 - Property path queries in saved plans with Chunk-at-a-Time engine can lose solutions

It was possible for some solutions to be lost on subsequent runs of a saved query plan using the Chunk-at-a-Time execution engine if the query used SPARQL OneOrMorePath or ZeroOrMorePath property path expressions. This bug sometimes appeared when running queries in AGWebView because it directs AllegroGraph to use saved plans. This has been corrected.

Bug23988 - Fix query failure in federations of reasoning and non-reasoning stores

Some queries could fail with an error in a federated triple-store composed of a mix of reasoning and non-reasoning leaf stores. This has been corrected.

Bug23949 - Filtering on multiple language tags could fail

The FILTERing machinery in AllegroGraph's SPARQL engine could fail to correctly process collections of language tag checks. For example, a query with a filter like:

FILTER( lang(?x) = 'fr' || lang(?x) = 'en' ) 

could fail. This has been corrected.

Bug23897 - Query warnings are incorrectly cached

Query warnings saved in SPARQL plans were not being updated to take account of changes in the underlying store.

Bug23893 - Federated-triple-stores and statistics caching could lead to poor plans

The caching of store statistics could lead to incorrect plans when querying a federated-triple-store. This has been corrected.

Bug23889 - Correct breakage in solR magic property queries

The SPARQL magic properties for solR queries were broken in AllegroGraph 6.0. This has been corrected.

Bug23884 - Constant pattern only CONSTRUCT could fail when executed via a saved plan

If a CONSTRUCT query template contained only constant patterns (i.e., no variables), and there was a limit on the query, and the query was being executed via a saved plan, then only the first execution would return results and subsequent executions would return nothing. This has been corrected.

Bug23882 - Algebraic query combines some BGPs too aggressively

The query planner was sometimes combining BGPs and inadvertently producing cross-products which could lead to slower query performance. This has been corrected.

Bug23881 - FILTER clause could be lost in complex SPARQL queries

It was possible for a FILTER clause to be lost during query planning. This has been corrected.

Bug23873 - Neither limit nor planner should require eval permission

Users without 'eval' permissions were unable to choose a planner or limit for a query. This has been corrected.

Bug23870 - Restriction reasoning could cause error during SPARQL query

Depending on the way it was invoked, a SPAQRL query could signal a error when restriction reasoning was used. This has been corrected.

Bug23841 - Do not allow triples in query results to be deleted

It was possible to remove triples from the database by clicking the X icon next to CONSTRUCT or DESCRIBE query results. This possibility has been disabled. CONSTRUCT and DESCRIBE queries do not return enough information to uniquely identify a triple in the triple-store. Offering to delete them is error prone and ambiguous. To delete triples, either use a SPARQL DELETE statement or the node view (click any part of a triple in the results).

Bug23755 - Problem with alternating property path in sub-queries

A sub-query that used a property path pattern with alternation could incorrectly bind the predicate of matched triples to output variables. An example of the problem would have been:

select ?s ?o {  
  { select ?s ?o {  
     ?s :hasChild | :hasParent ?o .  
    } }  
} 

This query would bind the predicate of matched triples to ?o. This has been corrected.

AGWebView

Rfe14371 - Allow the number of loaders to be selected in AGWebView

It is now possible to specify the number of loaders when invoking agload through AGWebView.

Rfe13428 - Display query information for CONSTRUCT, DESCRIBE and ASK queries

The query information tab is now displayed alongside results of CONSTRUCT, DESCRIBE and ASK queries.

Bug24047 - WebView: download duplicates ignores mode

The Export duplicate statement command in Store Control ignored the choice of SPO/SPOG. This has been corrected.

Bug23948: Fixed paged behavior after using the browser's back button

Previously, the Download and Load more buttons would function incorrectly if used after clicking the browser's back button. This has been corrected.

Bug23888 - Upload progress indicator not showing

AGWebView will now display progress information when uploading a file. This was previously not visible due to a bug. Progress is reported as a text line similar to this:

Uploading file triples.nt(1475 Kb uploaded) 

Bug23887 - Automatic scrolling in the AGWebView view server log pane broken

The server log in AGWebView would not auto-scroll when new log data appeared. This has been corrected.

Bug23857 - Load more does not work in Free Text search

Loading the next page of Free Text search results did not work. This has been corrected.

Bug23856 - Repeated text search breaks navigation

Page navigation would sometimes break when a user entered the same query twice in the Free Text search field. This has been corrected.

Bug23840 - Wrong even/odd row coloring when odd limit chosen

Row coloring could appear incorrect when using an odd limit and paging in the query view. This has been corrected.

Bug23839 - Planner selector grayed-out after language detection

It was possible to disable the planner selector in the query view if language autodetection was used. This has been corrected.

Bug23827 - Process list in WebView crashes on Firefox

The process and job list views of AllegroGraph WebView did not display any processes or jobs on Firefox. This has been corrected.

Bug23811 - AGWebView import fails for RDF/XML

Multi-core file import in AGWebView would fail if the chosen format was RDF/XML. This has been corrected.

Bug23810 - Importing via file upload no longer works in AGWebView

The multi-core file import feature in AGWebView was not working unless the file type was set directly. This has been corrected. In addition, AllegroGraph now recognizes application/n-triples and application/n-quads as media-types for the import of N-Triples and N-Quads (respectively).

Bug23800 - Query progress spinner only appears on every other query in Agwebview

The "Loading..." indicator was not displaying the first time that a new SPARQL query was executed via AGWebView's query editor. Now the loading indicator is shown every time.

Bug23774 - CONSTRUCT queries not displaying results in AGWebView

Results from CONSTRUCT queries were not being displayed in Safari under OS X. This has been corrected.

Bug23752 - Agwebview Lisp editor fails when digits are at the end of a line

The Lisp and Prolog script editors would become unresponsive if the last token on a line was a number. The JavaScript console showed the following error:

TypeError: undefined is not an object (evaluating 'stream.peek().toUpperCase') 

This has been fixed.

Bug23141 - AGWebView's CONSTRUCT doesn't show the MORE button

Previously, if the number of results of a CONSTRUCT query exceeded the limit set by the user, there was no way to show more results. Now, there is a MORE button at the bottom of the results which functions similarly to the one used for SELECT queries.

Bug22314 - Federated-triples-stores, encoded IDs and AGWebview don't mix

Attempting to use AGWebView to query a federated triple-store whose leaf nodes used encoded IDs led to an error. This has been corrected. Note: Correct behavior of a federation of stores that use encoded IDs assumes that each leaf node has the same definitions for encoded ID prefixes.

Changes to the Lisp API

Rfe11567 - Support store specifications in the Lisp client

In addition to using the catalog keyword argument, AllegroGraph now accepts store names in the form CATALOG-NAME/STORE-NAME. E.g.,

(open-triple-store "general/test-data") 

would open the store test-data in the catalog general.

You can also supply a triple-store specification in calls to open-triple-store. This will build a store instance from the specification, bind it to *db* and return it. In this case, the store instance will be given a name like store12341.

Finally, you can query store specifications directly using any of the HTTP clients by using a URL encoded version of the specification in place of a simple store name. E.g.,

curl -X GET -v "http://localhost:11040/repositories/%3Ca%3E%2B%3Cb%3E?query=select%20*%20%7B%20%3Fs%20%3Fp%20%3Fo%20%7D%20limit%205" 

would send the query

select * { ?s ?p ?o } limit 5 

to the federation of triples stores a and b (as specified by <a>+<b>).

Note that in this case, AllegroGraph will not start a new session so Prolog functors or other things defined in the servers initialization file will not necessarily be present.

Bug24105 - Deprecate serialize-rdf-n3 in favor of serialize-turtle

AllegroGraph can serialize data in turtle format but not in N3 so it is better to use the correct name. The now deprecated function serialize-n3-rdf will be removed in a future release and should no longer be used. serialize-turtle should be used instead.

Bug24006 - pcache Lisp module conflicts with AllegroGraph

A package conflict made it impossible to load AllegroCache and pcache in the same Lisp execution as AllegroGraph (without tiresome package manipulation). The problem has been fixed in this release.

Bug23846 - warmup-triple-store does not work in remote-lisp-client

Calling warmup-triple-store on a remote store used to have no effect. The call did not fail, but no operation was performed. The remote client will now correctly pass the request to the server it is connected to, preparing it for subsequent operations.

Prolog

Bug23877 - Prolog select exposes a security hole

There was a remote code execution vulnerability in Prolog query handling code where users without eval permissions could execute arbitrary code on the server. Executing Prolog queries now requires eval permissions. (This bug was fixed in various maintenance releases including version 6.0.2. We repeat it here because of its importance.)

Documentation

Bug23874 - Documentation search in local AllegroGraph docs doesn't work

The documentation search field in the upper left corner of AllegroGraph documentation pages now uses Google to search for the given phrase within:

franz.com/agraph/support/documentation/<version>/ 

where <version> is vX.Y[.Z] (v6.1 in the case of the current release). This means that even if you have a local copy of the documentation the search is done in the web copy on the Franz Inc. website.

Python Client

RepositoryConnection.createCoordinate arguments renamed

The lat and long arguments of RepositoryConnection.createCoordinate have been renamed latitude and longitude.

Bug23901 - RepositoryResult.asList() always returns None

The asList method in the RepositoryResult class was buggy and always returned None. It now correctly returns all statements from the result object.

Java Client

No significant changes.

Geospatial

Rfe12345 - Unnecessary geospatial functions removed

The functions geospatial-subtype-index-in-this-image and geospatial-subtype-index-in-db are neither useful nor necessary in the geospatial API. They have been unexported and removed from the documentation.

Bug23933 - Geospatial subtypes could be mistranslated on remote stores

It was possible for encoded geospatial data to be mistranslated when connecting to AllegroGraph via a remote-triple-store in the Lisp environment. This has been corrected.

Bug23848 - nd:inCircle :units incorrectly defaults to :miles, not :km

The :units argument to get-triples-nd-within-radius in Lisp and the similarly named functor in Prolog, and nd:inCircle in SPARQL, incorrectly defaulted to :miles instead of :km. This has been fixed so the behavior now agrees with the documentation. Since a kilometer is less than a mile, the change may cause previous queries that did not specify :units now to return fewer results. The extra results before the change were bogus.

Bug23767 - automate-nd-datatype-mappings could fail under parallel load

Spurious metadata conflict errors could be signaled if a store using automatic nD-datatype mappings was being accessed from multiple clients (for example, via agload). This has been corrected.

Bug23584 - nD query support for ISO6709 time

In an nD query, the min and max limits for a :time type ordinate may be expressed as a real number, as a dateTime UPI, or as a string literal in ISO6709 syntax (either as an xsd:dateTime typed string or an untyped string). These are supported in both SPARQL and Lisp/Prolog queries. Previously ISO6709 was not accepted in all contexts.

AllegroGraph 6.0.2 user-visible changes

Prolog

Bug23877 - Prolog select exposes a security hole

There was a remote code execution vulnerability in Prolog query handling code where users without eval permissions could execute arbitrary code on the server. Executing Prolog queries now requires eval permission. User permissions are discussed in the section Managing Users in the WebView document.

AllegroGraph 6.0.1

Server Changes

Bug23796: Disk-chunks-search.*.tmp files leaked when closing database

Starting in v6.0, files matching the pattern disk-chunks-search.*.tmp would remain open after closing a database. If a database was opened and closed frequently, this would eventually exhaust all available file descriptors in the process. This has been fixed.

Bug23715 and Bug23748 - FTI bugs

Bug23715: Creating a text index in which a word occurred in more than two million triples could sometimes crash the server.

This resulted in a log message similar to the following:

Process 'db-name FTI Merger' raised: opening #<...> resulted  
in error (code 22): Invalid argument. 

Bug23748: Indexing a very long document might cause a crash.

An error would sometimes occur when trying to add a triple with very long text to a text index. The error looked like this:

Error: opening #<mapped-file-simple-stream ...>  
       resulted in error (code 22): Invalid argument. 

Both bugs have been corrected.

Bug23010 - Rolling back a freshly staged text index leaves a file around

When a text index was deleted or its creation rolled back, a relatively large (~5MB) file used to be left behind and only deleted at the next checkpoint. This has been changed. The file is now deleted immediately as long as there are no live sessions that could still "see" the index before deletion.

HTTP Client

Bug23589 - Sending a SPARQL Update command to the Prolog REST API generates a backtrace

Sending a Prolog query that is not a valid list (e.g. because it is actually a SPARQL query) to the Prolog REST API caused the system to produce a backtrace and return error 500 with the following message:

Attempt to take the cdr of db.agraph.query.prolog:select  
which is not listp. 

This has been corrected: the server now answers with code 400 and the following message:

MALFORMED QUERY: Prolog queries must be lists.  

SPARQL

Bug23789, Bug23788 - issues with nD geospatial SPARQL queries

SPARQL n-Dimensional geospatial magic properties were signaling an error when used inside of a GRAPH clause. They were also signaling an error when the magic property was used with no variable bindings (i.e., when all of the arguments in the pattern were fixed). Both of these issues have been corrected. (Repeated under nD Geospatial below.)

AGWebView

Bug22920 - Agwebview result list has dynamic height

The window with SPARQL query results in AGWebView used to have a height of 300 pixels. The height is now dynamically adjusted to fill the available viewport.

Bug22604 - Improve handling of invalid user files

Previously, an invalid user file (e.g., one that was zero length or corrupted) would cause AGWebView to become unusable. This has been corrected. Now, invalid files will be ignored. Note that this means they will not be displayed in AGWebView's user list.

Changes to the Lisp API

Bug21770 - Some old SPARQL functionality now deprecated

Using either of the extendedp and permitted-verbs arguments to run-sparql will now signal a warning. These parameters will be dropped completely in a future release of AllegroGraph. This change also removes the special variable sparql:*use-extended-sparql-verbs-p*. See 6.0.1 Programmer Notes for more details.

Prolog

Bug23745 - Improve future-part handling in Prolog select queries

It was possible for a Prolog select query to fail to find results if future-parts were used inside lisp expressions in the query. This has been corrected.

Bug23726 - Prolog select queries that involved type mapping could fail

Under some circumstances, a Prolog select query that used constants requiring type mapping (e.g., '1234'^^xsd:integer) could fail to return results. This has been corrected.

Documentation

No significant changes.

Python Client

Bug22303 - python-client: missing not-equal method for URIs

Trying to test if two URI objects in the Python API are different used to be impossible (i.e. code such as uri1 != uri2 raised an exception). The missing comparison method has been implemented. In addition, it is now possible to order URIs, literals and BNodes. Comparing values of two different types yields an arbitrary but consistent result under Python 2 and causes an exception under Python 3. This follows conventions established by Python's built-in types.

Java Client

No significant changes.

nD Geospatial

Bug23789, Bug23788 - issues with nD geospatial SPARQL queries

SPARQL n-Dimensional geospatial magic properties were signaling an error when used inside of a GRAPH clause. They were also signaling an error when the magic property was used with no variable bindings (i.e., when all of the arguments in the pattern were fixed). Both of these issues have been corrected. (Repeated under SPARQL above.)

AllegroGraph 6.0

AllegroGraph Server: General Changes

Use agraph for the Lisp client fasl name

AllegroGraph 6.0 now uses agraph.fasl instead of agraph[major-version-number].fasl for the name of the Lisp fasl file. Both the direct Lisp client fasl file and the various remote Lisp client fasl files (on the various platforms suitable for use as remote clients) are named agraph.fasl. Users need to manage the locations of the various individual agraph.fasl files to be sure the correct one is loaded when used.

Rfe13969 - Add HDFS input support to agload

agload now supports loading files from HDFS sources. For this to work, there must be a working "hdfs" program in your PATH and HDFS input files must be specified on the agload command line using the hdfs:// prefix. For example:

agload repo hdfs:///user/bruce/lubm-50.nt.gz 

agload is described in the Data Loading document. (This change was also in release 5.1.1, as noted below, but we repeat it here as some users may not have installed version 5.1.1.)

Rfe13826 - Improve estimates on federated remote stores

Cardinality estimation of query result sets on remote stores has been greatly improved which leads to more efficient query plans on federated remote stores.

Rfe13807 - Added deleted triple purge operation

When triples are deleted they are not immediately removed from indices. They must remain in all indices until all transactions which may potentially need to see those triples (such as transactions that started before the triples were deleted) have completed (committed or rolled back). A triple that cannot possibly be accessed by any live transaction is referred to as an inaccessible triple. The existence of inaccessible triples in indices wastes space and impacts query performance. Therefore AllegroGraph now provides a facility to purge these triples from indices. See Purging Deleted Triples.

Rfe13806 - Configuration option to control index style

AllegroGraph v6.0 adds a new style of index. The old style is called style 1 and the new style is called style 2. Style 2 indices are faster than style 1 when performing point queries where all of s/p/o or s/p/o/g are fixed (where the result is typically a single triple), but these indices take longer to construct and to optimize. Style 1 indices are better for queries which end up scanning many triples and typically produce results with many triples.

The Style2Indices catalog configuration directive can be used to control which index flavors should use style 2. The directive value can be a comma- or space-separated list of index flavor names, or the value none. The default value is none. Configuration options are discussed in Server Configuration and Control.

Examples

Style2Indices none  
Style2Indices posgi  
Style2Indices spogi, posgi 

The Lisp API function add-index and the REST interface command PUT /repositories/[name]/indices/[type] allow for specifying the index style as well.

Rfe13641 - Change default for temporaryFileDiskSpace

AllegroGraph now defaults to using no more than 25% of the available filesystem space for temporary query files (at the time the query begins). This setting can be changed using the temporaryFilesystemSpaceLimit query option either in the AllegroGraph configuration file or via a PREFIX option to a SPARQL query. If a query attempts to use more space than this limit allows, it will be canceled. temporaryFilesystemSpaceLimit is a new name for the temporaryFileDiskSpace option. The old name has been deprecated. See agcontrol.

Rfe12000 - Minor improvements in how typed string literals are stored

AllegroGraph now stores typed string literals like "hello"^^xsd:string more efficiently.

Rfe11116 - Support SHA384 digest function in SPARQL

In addition to MD5, SHA1, SHA256, and SHA512, AllegroGraph's SPARQL engine now supports SHA384.

Rfe11078 - Improve agload verbosity behavior

Previously, passing -v or --verbose to agload (see Data Loading) could actually decrease its level of verbosity. This has been fixed. The new verbosity levels are:

0 (-v not supplied): Only report periodic load rate information.  
1 (-v)             : As above, plus print the job option summary  
		 before starting the operation.  
2 (-v -v)          : As above, plus print the name of every file  
		 that has been processed. 

(This change was also in release 5.1.1, as noted below, but we repeat it here as some users may not have installed version 5.1.1.)

Bug23660 - AuditEventsToEmail parser too strict

Previously the events list in an AuditEventsToEmail configuration directive had to be comma-separated with no intervening whitespace. This has been relaxed.

Bug23654 - agraph-backup restore fails if StringTableSize specified

An error would occur when trying to restore a database if its StringTableSize did not match the one in the catalog into which it was being restored. The error was:

Fetching user information failed: String table configuration mismatch:  
Configured number of slots (X) does not match recovered number  
of slots (Y). 

This has been corrected. agraph-backup is described in Backup and Restore.

Bug23604 - Improve bidirectional search speed when no paths are found

AllegroGraph's bidirectional search algorithm was expanding more nodes than necessary when no paths between the start and end were found. This has been corrected.

Bug23598 - Error when accessing a suspended account

Previously, if an account was suspended but AccountUnsuspendTimeout wasn't configured, the following error would be signaled when accessing the suspended account:

nil is not of the expected type number 

This has been corrected.

Bug23585 - Audit dialog not showing any results

It was possible for the Audit Log query in AGWebView to fail to display results. This has been corrected.

Bug23526 - Fixed size deleted triple bitmap

Previously an error would occur if a triple with id greater than 34,359,738,367 was deleted. This has been fixed.

Bug23512 - Signal an error when attempting a freetext query and no freetext indices exist

Previously, AllegroGraph would simply return no results if a freetext query was executed and there were no freetext indices defined. Now, AllegroGraph will signal an error. This also modifies the behavior of list-freetext-indices for federated triple-stores. Previously, a federated store would return the intersection of its freetext indices. Now, it returns the union. This makes it easier to use the freetext indexing facility with them.

Bug23442 - Full filesystems can lead to truncated user files

In some cases a full filesystem could cause AllegroGraph user preference files to be truncated. This has been corrected.

Bug23441 - Retry tlog creation if filesystem is full

Previously, if the filesystem ran out of space while allocating a new tranasction log file during a checkpoint or commit, the instance would crash. Now the affected process will block, log a message to agraph.log, and periodically retry the operation until it succeeds. To avoid confusing hangs, failure to preallocate transaction log files on database creation will still result in an immediate error.

Bug23370 - N-Triple, N-Quad and Turtle parsers should handle UTF8 files with a BOM

AllegroGraph's parsers did not correctly parse UTF8 files that started with a byte order marker (BOM). This has been corrected.

Bug23270 - flush-in-memory-chunks deadlock

Previously, when performing duplicate triple deletion, it was possible for the AllegroGraph instance to become deadlocked and hang. This has been fixed.

Bug23361 - N-Triples parser could incorrectly import non 7-bit ASCII data

Unless a UTF-8 encoding was explicitly specified, the AllegroGraph N-Triples parser could fail to handle Unicode N-Triples data correctly. This has been corrected.

Bug23238 - Fix 'UPI has not been added to string dictionary' bug

Previously, certain queries involving large intermediate result sets could fail with a UPI has not been added to string dictionary error. This bug has been fixed.

Bug23233 - Fix bug in agraph-backup progress output.

Some garbled output showing a "Printer error" could be produced when running a backup operation using agraph-backup. This has been corrected.

Bug23231 - Prevent spurious output from agraph-replicate

Previously, agraph-replicate would emit spurious output when exiting. This output has been suppressed.

Bug23229 - agraph-replicate --stop would not stop replication.

Previously, agraph-replicate --stop would not actually cause replication to stop. This has been corrected. See Replication for information on agraph-replicate.

Bug22490 - Fix potential deadlock when creating/modifying freetext indexes.

With proper timing, it was possible hang the server when adding or updating a freetext index. This has been corrected.

Bug22489 - agraph-backup fails due to missing tlog file

Under certain circumstances agraph-backup could fail to back up a database due to not finding a transaction log file. This error has been corrected.

Bug22165 - commit-time duplicate suppression does not operate under lock

Previously, commit-time duplicate triple suppression did not properly consider triples that were added in a concurrent transaction, possibly resulting in duplicate triples being added. This has been corrected.

Bug19831 - xsd:booleans are now stored as encoded UPIs

Previously, xsd:booleans were stored as strings. They are now stored as encoded UPIs. This will make some queries slightly more efficient.

HTTP Client

Rfe13941 - Allow specification of chunk style when adding an index

The PUT command

PUT /repositories/[name]/indices/[type] 

now accepts a style query parameter which may be 0, 1, or 2. See AllegroGraph Indices for a discussion of index styles.

Rfe13750 - Log errors during writing the HTTP response

While using the REST API, which most clients do, serving a request usually has a computation phase that's followed by an output phase. If an error happens in the computation phase, the appropriate HTTP status code is returned. The tricky case is with chunked encoding (for example, streaming query results back) when some results were already written to the socket. Then upon encountering an error, it is not possible to change the HTTP status code and return a meaningful error message. For this reason, errors should normally be detected during the computation phase.

With this change, any errors (except those caused by the client end of the socket being closed) encountered during the output phase are logged to make debugging easier.

Rfe13711 - Add query PREFIX option to cancel queries when there are warnings

AllegroGraph has a new query PREFIX option: cancelQueryOnWarnings. If true, then a SPARQL query will cancel itself if it encounters any warnings during query planning or execution. Typical warnings include:

If the option is set to no, then warnings will be logged but query execution will continue.

Bug23546 - Problems adding very long objects with JSON upload

An error could occur if triples with very long strings were imported using the REST API and the application/json content-type. This has been corrected. Other content-types were not affected by this bug.

Bug23324 - Some text index properties could not be viewed

If an AllegroGraph freetext index specified that it should only index literals with one or more particular datatypes, then that index would signal an error when its properties were read using the REST API. This has been corrected.

SPARQL

Rfe14076 - Optimize RDF serialization of CONSTRUCT and DESCRIBE queries

RDF serialization formats like RDF/XML or Turtle were spending more time than necessary trying to find namespace abbreviations. This has been improved.

Rfe13998 - agquery can now interact with SPARQL endpoints

AllegroGraph's command-line agquery tool can now send queries to SPARQL endpoints and retrieve the results. agquery is described in the Querying document.

Rfe13975 - Streamline internal AllegroGraph merge operations

Query operations that require merging of multiple result streams have been optimized. Examples include the inference of types and properties in the dynamic reasoner and computing distinct SPARQL results.

Rfe13519 - General low-level SPARQL query optimizations

Modified the low-level cursor operations in the SPARQL query engine so that they perform between 3 and 5% better for most queries.

Rfe13387 - Improve performance of SPARQL replace function

The SPARQL replace function is now significantly faster when called with a constant regular expression.

Rfe13327: Improve parsing of diskChunkRowCount and other query parameters

It is now possible to specify query parameters like diskChunkRowCount and other query parameters using sizes like 100m or 1g rather than needing to write out the numeric value as an integer.

Rfe12143 - Improve analysis of constant BIND and VALUES forms

AllegroGraph now does a better job of analyzing and using constant BIND values and single variable VALUES forms. When possible, these forms are pulled into the triple-patterns in the BGPs rather than being evaluated at run-time. For example, both of these queries will now run much more efficiently:

select * {  
    bind( 'test' as ?test )  
    ?s :predicate ?test .  
} 

and

select * {  
  values ?test { 'test' 'two' 'three' }  
  ?s :predicate ?test .  
} 

Rfe12131 - Remove support for the old SPARQL 1.0 query engine

AllegroGraph no longer supports the older and long deprecated SPARQL 1.0 query engine. The newer 1.1 engine is to be preferred in all cases. In particular, this means that you can no longer force the use of the 1.0 engine via the queryEngine PREFIX.

Bug23696 - Improve query plan for some property path queries in sub-queries

AllegroGraph could fail to optimize a simple alternation property path query when it appeared inside of a sub-query. This has been corrected.

Bug23653 - Remove spurious cross-product warning for (optional identity A)

AllegroGraph was signaling a cross-product warning for SPARQL queries that contained an OPTIONAL with no left hand side. For example:

select * {  
  optional { ?s a :Animal }  
} 

has a query algebra like:

(left-join identity BGP) 

Though the warning is technically correct, the construct above is harmless and the warning is unnecessary.

Bug23632 - SPARQL parser has trouble with escaped backslashes at the end of a literal

The SPARQL parser would fail to correctly interpret literals with an escaped backslash as their final character, e.g., 'abc\'. This has been corrected.

Bug23616 - COUNT aggregation counts unbound values from unknown variables

If a COUNT aggregration expression included variables that were not projected from the underlying query, then AllegroGraph was treating them as if they were bound and returning a count. This has been corrected.

Bug23582 - SPARQL could lose results when a sub-query used limit and distinct

It was possible for solutions to be overwritten and lost when a sub-query used both LIMIT and DISTINCT. This has been corrected.

Bug23581 - Optional mis-analysis in sub-queries could drop solutions

It was possible for AllegroGraph to mis-analyze whether a variable in a SPARQL sub-query was optional or not and therefore to lose solutions. This has been corrected.

Bug23524 - Property path queries could lose results when there is circularity

Under some circumstances, a SPARQL query that used property paths could lose results that arose from circular paths through the triple-store. This has been corrected.

Bug23519 - SPARQL sub-query and GRAPH clauses could interact badly

It was possible for a query that used sub-query and GRAPH clauses to lose solutions. This has been corrected.

Bug23489 - Queries that started with a SERVICE clause could lose bindings

If a SPARQL query started with a SERVICE clause and contained a cross-product then it was possible for some bindings to be lost. This has been corrected.

Bug23452 - Some temporal magic properties could fail to produce results

Some of AllegroGraph's temporal magic properties could produce plans that introduced filters that could not succeed. This has been corrected.

Bug23448 - Fix problem with SPARQL t:pointBeforeDatetime and t:pointAfterDatetime

The two SPARQL magic properties t:pointBeforeDatetime and t:pointAfterDatetime could fail to find bindings if the dateTime supplied was a constant not present in the triple-store. This has been corrected. SPARQL magic properties are listed here in SPARQL Magic Properties.

Bug23443 - Remote SPARQL queries cannot handle really large results

Returning very large query result sets (i.e., Gigabytes) from remote SPARQL endpoints could lead to resource exhaustion. This has been corrected.

Bug23373 - Improve SPARQL efficiency for small ORDERed or DISTINCT result sets

SPARQL queries with small result sets that use ORDER BY or DISTINCT will now be slightly faster.

Bug23344 - SPIN magic properties don't work if they use other SPARQL functions

A query-time error would be signaled if a SPIN magic property used other SPARQL functions like CONCAT or SUBSTR in its definition. This has been corrected.

Bug23341 - remote-triple-stores handle limit and offset in SPARQL queries incorrectly

If a query specified an OFFSET as part of the query string, then any externally imposed OFFSET (i.e., from the HTTP query parameters) was ignored. This has been corrected. If a query specified a LIMIT as part of the query string, then the smaller of that limit and the externally imposed one would be used instead of just using the external one. This has been corrected. Note that this change means that specifying a limit in a query in AGWebView will have no effect as the limit is controlled via the dropbox selection box.

Bug23329 - Use SPARQL/JSON for SERVICE calls to avoid problems with embedded control characters

AllegroGraph now uses SPARQL/JSON rather than SPARQL/XML to receive results from SPARQL endpoints. This makes it possible to return results that contain control characters like ^Z. The format is also more concise and easier to parse which makes processing the transmissions approximately twice as fast.

Bug23321 - Nested sub-queries and BIND forms can fail to compute correct results

In rare cases, it was possible for a query involving nested SPARQL sub-queries and BIND expressions to fail. This has been corrected.

Bug23311 - Requesting application/nquads from a CONSTRUCT query leads to error

If an application requested application/nquads from a CONSTRUCT query via the HTTP interface, a string-table lookup error would result. This has been corrected. Although version 1.1 of SPARQL CONSTRUCT can only return triples, it still makes sense to ask for N-Quads. In this case, the N-Quads will simply have the graph place left off.

Bug22674 - DESCRIBE and CONSTRUCT queries on end points could fail

SPARQL CONSTRUCT and DESCRIBE queries against remote SPARQL end points could fail when trying to parse the returned results. This has been corrected.

Bug22672 - Queries where every FROM NAMED graph did not exist could signal an error

If every FROM NAMED graph in the dataset did not exist, then some queries that used the GRAPH clause in their body could signal an error rather than returning results. This has been corrected.

Bug22515 - Increase responsiveness of query canceling during result generation

AllegroGraph did not check for query canceling or timeouts during result post-processing (e.g., while sorting the final result or removing duplicate entries from a DISTINCT result set) or result generation. This has been corrected.

Bug22292 - tripleid magic property should handle fixed triple IDs

The SPARQL magic property for AllegroGraph tripleId-based reification would not function for fixed triple IDs. This has been corrected.

Bug22268 - Sub-queries that used Aggregation and limit could lose solutions

If a sub-query had a limit and used aggregation, then it was possible for some solutions to be lost during processing. This has been corrected.

AGWebView

Bug23690 - Fixed bug breaking the AGWebView View Statements dialog in IE.

A bug in AGWebView that prevented the View Statements dialog from fetching and displaying statements when brought up in Internet Explorer has been fixed. The symptom was a partially loaded page with the spinning "Loading..." icon remaining permanently on the page.

Bug23521 - More... button in AGWebView is broken for Prolog select queries

The button to show additional results in the AGWebView query window was broken for Prolog select queries. This has been corrected.

Bug23358 - AGWebView doesn't always display Unicode consistently

Under some conditions, AGWebView would display literals and resources containing Unicode using the \u escaping mechanism of the N-Triples syntax rather than just displaying the characters directly. This has been corrected.

Bug23315 - Running backup from within AGWebView does not work

The Backup this Store dialog in AGWebView was broken. This has been corrected.

Changes to the Lisp API

Rfe13941 - Allow specification of chunk style when adding an index

add-index now accepts a style keyword argument which can be 0 or nil (either meaning use the default chunk style for the flavor based on the Style2Indices configuration parameter), 1, or 2. See AllegroGraph Indices for a discussion of index styles.

Bug23600 - remote-triple-stores CONSTRUCT queries now correctly returns cursors

In the Lisp client, a SPARQL CONSTRUCT query should return a cursor that yields arrays of bindings but the remote-triple-store was instead yielding lists of bindings. This has been corrected.

Bug23597 - remote-triple-store CONSTRUCT queries return nil bindings

Many of the programmatic RDF formats (such as :lists or :arrays) for SPARQL CONSTRUCT queries on remote-triple-stores returned nil bindings rather than the actual values. This has been corrected. Note that this bug did not effect text based output formats like N-Triples or Turtle.

Bug23510 - remote-triple-stores did not support load-trig

Lisp remote-triple-stores did not support loading the TriG format. This has been corrected.

Bug23509 - Altering restriction-reasoning-enabled-p could lead to an internal error

Altering the value of restriction-reasoning-enabled-p using the setf method after the reasoner had been applied and prepared could lead to an error. This has been corrected.

Bug23404 - It was impossible to create an anonymous remote session

AllegroGraph would signal an error that This session is owned by another user when trying to create an anonymous remote session in the Lisp client. For example, the following would fail:

 (open-triple-store "remote"  
    :triple-store-class 'remote-triple-store  
    :server "www.example.com"  
    :specification "<sp2b-5e4>[rdfs++]"  
    :port 9002) 

This has been corrected.

Bug23320 - The Lisp direct client now ensures that UPIs are interned when calling add-triple

The Lisp direct client was not checking that UPIs were interned which could lead to a UPI-not-in-string-table error when there were bugs in customer code. This has been corrected. The check does add a small cost to programmatic triple insertion. The while-not-validating-that-upis-are-in-string-table macro can be used to temporarily disable the checks once you are sure that the code is correct.

Bug22844 - The SPARQL datatype function failed to handle nD-geospatial literals

The SPARQL datatype function was unable to handle nD-geospatial literals. This has been corrected.

Bug22793 - Problem with remote-reasoning-triple-stores and SNA

On the Lisp client only, calling Social Networking (SNA) functions on a remote-reasoning-triple-store instance could lead to a stack overflow. This has been corrected.

Bug22791 - get-triple was broken for remote-reasoning-triple-stores

On the Lisp client only, calls to get-triple on a remote-reasoning-triple-store could fail to return results. This has been corrected.

Bug22587 - copy-triples doesn't support remote triple-stores

The copy-triples function did not work on remote-triple-stores. This has been corrected. Note that copy-triples on remote-triple-stores requires that

Bug22586 - Improve copy-triples

The copy-triples function was not behaving correctly with respect to the source-db argument. This has been corrected.

Bug22447 - Fix cursor memory management

Under rare circumstamces, especially in a multi-threaded environment, cursors used in the Lisp client could end up being used in an unsafe way leading to various errors. With the AllegroGraph server, shared backends experiencing high load were at the risk of such failures. This bug has been fixed.

Also, in the Lisp client with a direct connection, any code that involved the creation of cursors (such as queries and calls to get-triples) had to be wrapped in a with-query-environment form. While that's still the recommended approach to take in production, with this change when there is no enclosing with-query-environment, cursors holding critical resources get finalizers which makes it more convenient to experiment at the REPL by reducing the risk of resource exhaustion.

Bug20852 - remote-triple-stores do not support SPARQL update

Lisp remote-triple-stores now support SPARQL update.

Prolog

Rfe13968 - Add :offset to Prolog select

AllegroGraph's Prolog based select query language now supports an :offset option. This can be used to skip the first offset results. For example,

(select (?class)  
  (:offset 10)  
  (q ?s !rdf:type ?class)) 

would skip the first 10 results and return the rest of the bindings for ?class.

Documentation

No significant changes.

Python Client

Rfe12828 - Support Python 3

AllegroGraph now supports both Python 2 and Python 3. The sources for Python 2 continue to be in the src2 directory whereas those for Python 3 are in the new src3 directory.

Java Client

No significant changes.

nD Geospatial

Rfe13978 - Validate arguments to ndfn:ordinateValue at parse time when possible

AllegroGraph now validates constant arguments to ndfn:ordinateValue at parse time. See the N-dimensional Geospatial Usage Guide and Example document.

AllegroGraph 5.1.2

AllegroGraph Server: General Changes

Bug19296 - AllegroGraph can stop responding because all threads seem busy

Under rare circumstances it was possible for AllegroGraph to stop responding. Such a hang was accompanied by "All threads busy, pause" log messages. This has been corrected.

AllegroGraph 5.1.1

Because there are only a few changes in release 5.1.1, only headings with actual changes are listed.

AllegroGraph Server: General Changes

Rfe13969 - Add HDFS input support to agload

agload now supports loading files from HDFS sources. For this to work, there must be a working "hdfs" program in your PATH and HDFS input files must be specified on the agload command line using the hdfs:// prefix. For example:

agload repo hdfs:///user/bruce/lubm-50.nt.gz 

agload is described in the Data Loading document. HDFS loading is described in the section Loading files from HDFS filesystems in that document.

Rfe11078 - Improve agload verbosity behavior

Previously, passing -v or --verbose to agload could actually decrease its level of verbosity. This has been fixed.

The new verbosity levels are:

0 (-v not supplied): Only report periodic load rate information. 1 (-v) : As above, plus print the job option summary before starting the operation. 2 (-v -v) : As above, plus print the name of every file that has been processed.w

agload is described in the Data Loading document.

AllegroGraph 5.1

AllegroGraph Server: General Changes

Rfe13570 - Support the TriG format for serialization

AllegroGraph can now serialize data using the TriG format. For example, you can use the output format trig in agexport or an Accept type of application/trig from the REST API.

Rfe13501 - Implement HTTPNoProxy configuration directive

This change adds a new global configuration directive, HTTPNoProxy. HTTP requests to domains that match one of the suffixes specified with HTTPNoProxy are never proxied. HTTPNoProxy can be specified multiple times.

HTTPNoProxy localhost  
HTTPNoProxy mydomain.com 

Requests to 'localhost' and '127.0.0.1' are never proxied. Top level directives are described in Server Configuration and Control.

Rfe12010 - Support the TriG format for data import

AllegroGraph now supports the TriG data format for loading data. See http://www.w3.org/TR/trig/ for details on the format. See Data Loading.

Rfe11730 - Update Turtle parser to conform to the specification

AllegroGraph's turtle parser now passes the W3C conformance tests (http://www.w3.org/TR/turtle/). The parser is also significantly faster.

Bug23077 - Rare thread safety issue in shared back-ends

Previously, when multiple requests for the same repository were being executed concurrently on the same shared back-end a very rare thread safety issue could cause the backend to die. This bug has been fixed.

Bug22944 - Freetext indexer could mishandle encoded literals

AllegroGraph's built-in freetext indexer was trying to index encoded literals (numbers, dates, times and dateTimes) and would sometimes misinterpret them leading to extremely slow indexing. This has been corrected by preventing the indexing of these literals. This means that freetext indices will not include values for encoded literals.

Bug20556: Double-fault of trie-full condition

Previously, when adding a word with many hits to a text index a "trie-full" error could occur. This has been fixed.

HTTP Client

Bug23064 - Post requests to the statements REST API could fail

It was possible for POST requests to the statements REST API to corrupt the stream of bytes being sent. This has been corrected.

SPARQL

Rfe13683 - Improve variable count tracking during query execution

AllegroGraph keeps better track of the estimated cardinality of bindings for variables during query execution which improves the execution engine's overall efficiency.

Rfe13601 - Improve handling of property path queries that use simple alternation

AllegroGraph can now apply its constraint machinery to simple property path queries that use only alternation. For example:

select ?o {  
  ?s :a | :b | :c ?o .  
  filter( ?o > 10 )  
}  
This means that queries like the above can operate significantly faster. 
Rfe13599 - Improve query pattern constraint logic

AllegroGraph now does a better job in choosing how to constrain variables in SPARQL triple-patterns during execution.

Rfe13476 - Add more logging information to SPARQL SERVICE queries

AllegroGraph now includes more logging during the execution of SPARQL SERVICE queries.

Rfe13426 - Provide more control over federated SPARQL queries

The new query option serviceRequestSize provides control over the size of the POST requests used during SPARQL federated query. The default size is one megabyte. A smaller size may cause additional trips to the remote server. Some servers may reject requests that are too large. SPARQL query options are listed in the SPARQL Query Options of the SPARQL Reference.

Rfe13213 - Allow BIND to float during query analysis

AllegroGraph's SPARQL query analyzer now does a better job rewriting query patterns involving BIND. In particular, the BIND clause now floats around JOIN, GRAPH and FILTER clauses. This can help make these queries perform significantly better.

Rfe12269 - Improve query planning with respect to cross products

AllegroGraph's SPARQL query planner now does a better job of detecting cross products at plan time and modifying the query plan to avoid the cross product when this is possible.

Bug23125 - Improve query planning for some property path queries

SPARQL Property Path queries using the zero-or-more operator (i.e., *) could sometimes choose a sub-optimal plan because AllegroGraph was misestimating the size of the triple-patterns involved. This has been corrected.

Bug23095 - Correct internal error that occurred during some Property Path queries

Some Property Path queries could fail to run to completion because of an internal bookkeeping error. This has been corrected.

Bug23081 - Certain SPARQL queries could cause cursor data-structure leakage

Certain SPARQL queries could fail to correctly clean up all of the cursors that they used which could lead to backends becoming unresponsive if they were under heavy load. This has been corrected.

Bug23027 - SPARQL was not ordering reified triple IDs correctly

The SPARQL query engine was not correctly sorting URIs representing triple IDs. This has been corrected.

Bug23026 - Some SPARQL SERVICE calls could mangle their query

It was possible for a SPARQL SERVICE call to lose space characters and thereby send the wrong query to the remote end point. This has been corrected.

Bug23005 - RDF/XML serializer could emit an invalid namespace abbreviation

If there was a namespace mapping between the empty string and some URI, then the RDF/XML serializer would emit an invalid namespace abbreviation. This could occur, for example, when running a CONSTRUCT query using a PREFIX abbreivation like

PREFIX : <http://example.org#> 

This problem has been corrected.

Bug23004 - Sparql XML parser could fail to parse some literals

AllegroGraph's SPARQL XML parser is used when making federated queries against remote endpoints. This parser was incorrectly handling literals with ampersands (or other escaped characters) in them. This has been corrected.

Bug22999 - When embedded in a GRAPH clause freetext SPARQL queries could fail

Depending on the data and the query, a freetext match or matchExpression magic predicate could fail to find all results when it was embedded inside of a GRAPH clause. This has been corrected.

Bug22994: Fix bad interaction between SPARQL sub-query and federation

If a SPARQL query used a sub-query inside of a federated query, AllegroGraph could lose track of the variables involved and create inadvertant cross-products. This has been corrected.

Bug22966 - SERVICE clause in a sub-query could fail

Depending on the variables projected, a SPARQL query that used a SERVICE clause inside of a sub-query could fail during execution.

Bug22937 - GROUP BY returned an empty row when there are no aggregates projected

If a SPARQL aggregation query projects only variables (i.e., it does not use any aggregation functions) and the query has no results, then AllegroGraph would return a single row of unbound bindings rather than returning no results at all. This has been corrected. An example of the query in question would be

select ?anything {  
  ?s <ex://missingPredicate> ?anything .  
} group by ?anything 

Bug22915 - SPARQL STR function is incompatible with aggregation

Queries that used the SPARQL STR function on the result of an aggregation would fail. This has been corrected. An example of such a query would be:

select ( str( count(1) ) as ?count) {  
  ?s :testPredicate ?o .  
}  
Bug22904 - Bad query bookkeeping could cause performance problems

Given the right data and the right query, the SPARQL query engine could be led to evaluate triple patterns twice: once with constraints and once without which would lead to severe performance problems. This has been corrected.

Bug22899 - Query options could not always be used with SPARQL SERVICE

When using the SERVICE clause, AllegroGraph's query PREFIX options were not always being sent to endpoints in a way that they could be read. This has been corrected.

Bug22874 - N-Dimensional magic property queries could fail

Depending on the data and the query, AllegroGraph could choose an incorrect plan ordering for an n-Dimensional SPARQL magic property. This would lead to the UPI not in string table error. This has been corrected.

Bug22845 - A query timeout of zero now means "no timeout"

Previously, a query timeout of zero was interpreted literally to mean that the query should terminate immediately. Now, a timeout of zero means that the query should run as if there is no timeout specified. This is especially helpful when a default timeout is included in the AllegroGraph configuration file.

Bug22751 - Error using one-or-more property paths in SERVICE clause

AllegroGraph was signaling an error if a query tried to use a one-or-more property path query inside of a SERVICE clause. This has been corrected.

Bug22680 - Correct SPARQL parsing problem for property paths in object lists

The SPARQL Parser was not correctly handling property paths embedded inside object lists like ?s <ex:predicate> [ <ex:Path>* ?o ]. This has been corrected.

Bug22593 - Use the correct count estimates during query execution

Depending on the query and the store, AllegroGraph could sometimes use the wrong variable count estimates which would lead to sub-optimial performance. This has been greatly improved.

Bug22388: Chunkprocessingmemory defaults to a larger value than memoryLimit

The chunkProcessingMemory size defaulted to a larger size than the memoryLimit which meant that queries could fail when using chunk processing. This has been corrected.

Bug22212 - Magic property for t:intervalStarts logic was wrong

The magic property for t:intervalStarts was incorrectly behaving the same as t:intervalCotemporal. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Rfe13593 - The compiler is needed in direct lisp clients that perform queries

This change will likely not affect any users since the compiler is normally in Lisp images, but it is possible to build an image without the compiler and without the compiler being available to load (though, again, it is unusual to do so). Such images serving as direct Lisp clients cannot perform queries.

Bug22973 - Fix Lisp direct client sigsegv when rolling back a closed store

Previously, in the Lisp client when using a direct connection to a triple store, calling rollback on a closed store would produce a SIGSEGV error. With this fix, an informative error condition is signaled.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

nD Geospatial

Bug22786, Bug22850: improve nD string writers and parsers

nD data encoding and decoding supports various human-readable formats (ISO6709 and ISO8601) when such a format can be applied unambiguously to a particular nD encoding. If a human-readable format applies, it will be preferred on decoding (output) and accepted on input (encoding). Previously some correct externalization formats were rejected on input or not used on output.

nD now correctly supports encodings with multiple ordinates of the same specialized type, i.e. lat, lon, alt, and time. Ordinates must of course all have different names. Use of such encodings will necessarily suppress human-readable externalizations since they are ambiguous.

Error detection during nD encoding/decoding is improved along with error message text.

AllegroGraph 5.0.1

AllegroGraph Server: General Changes

Bug23016 - Incorrect triple-count and garbage triples

Under certain circumstances (particularly when delete-duplicate-triples is used), crash recovery or replication can result in an incorrect triple count and garbage triples being added to the database. This has been corrected. We recommend that users using replication and duplicate triple deletion recreate their replicas from a fresh backup. Replication is described in the Replication document.

HTTP Client

Bug23040 - Performing the same ASK query more than once under REST could error

Under certain circumstances, if an ASK query that used joins was evaluated more than once under the HTTP REST API, then AllegroGraph would signal an error. This has been corrected.

AllegroGraph 5.0

AllegroGraph Server: General Changes

Rfe13506 - Minor query optimization for triple pattern filters

AllegroGraph now performs triple pattern filtering slightly more efficiently when the current set of solutions has only a single member.

Rfe13405 - Warn if transparent hugepages are enabled

Due to the potential for negative performace impact, AllegroGraph now logs a warning into agraph.log on startup if Transparent Hugepages are enabled. See this section of Performance Tuning for information on the transparent hugepage issue and its effect on performance.

Rfe13376 - Improve hashing performance

AllegroGraph's internal hashing functions have been further optimized leading to small general performance improvements.

Rfe13346 - configure-agraph should allow for specification of port

The configure-agraph script now prompts for the Port configuration parameter and accepts the --port command line argument. See Running the configuration script in Server Installation.

Rfe13265 - Update N-Triple and N-Quad parsers

AllegroGraph's N-Triple and N-Quad parsers now support the full W3C specifiation and pass all tests. The performance is also slightly improved. Note that this change means that AllegroGraph will no longer load invalid N-Triple and N-Quad source data unless the relax-syntax argument is used. See Data Loading, the Lisp API functions load-ntriples and load-nquads, and [PUT/POST /repositories/name]/statements in REST/HTTP interface.

Rfe13172: Allow setting size of strings hash table at database creation time

A new database configuration parameter StringTableSize is now available. It can be used to configure the number of hash table slots used for the string storage hash table and setting it properly can enhance performance. See Catalog Parameters that Improve Performance in Performance Tuning for more information on performance-enhancing options for details.

Rfe13146 - Improve suboptimal index scan performance

The efficiency of low-level cursor operations has been improved when an optimal index for the cursor is not available.

Rfe12872 - Remove ag3-export / ag4-import

The obsolete ag3-export and ag4-import utilities have been removed. Serializing data to nquads in AllegroGraph 3 and loading it into AllegroGraph 4 is the recommended alternative.

Rfe11580: Added --no-daemonize flag to agraph binary

The agraph binary is the actual program which starts AllegroGraph. It is not typically invoked directly, but rather using the agraph-control script, but it can be called directly. When it is and --no-daemonize is supplied, the service daemon will remain in the foreground. See agcontrol.

Bug22837 - Text index incorrectly indexing the string "nil"

Since version 4.13 the text indexer was incorrectly indexing the word "nil" for triples that were ineligible for text indexing. This resulted in unnecessary work being performed and space being consumed. This has been fixed. We recommend that you drop and recreate any existing text indexes.

Bug22835 - Fix numa info parsing failure

At startup, information about numa (non-uniform memory access) topology is collected and logged. Previously, parsing failed with an error if a cpumap file in /sys/ contained a comma separated list of numbers instead of just a single number. This bug has been fixed.

Bug22767 - Use agraph5 for the Lisp client fasl name

AllegroGraph 5.0 now uses agraph5.fasl instead of agraph4.fasl for the name of the Lisp fasl file.

Bug22754 - Concurrent duplicate triple deletion can result in inaccurate triple count

Under some circumstances duplicate triple deletion transactions which occur concurrently with other deletion transactions could result in an inaccurate triple count. This has been corrected.

Notes:

Bug22566 - Encoded ID validation was too weak

In some cases, a URI representing an invalid encoded ID could be accepted by AllegroGraph instead of signaling an error. This has been corrected.

Bug22564 - Data import could fail to determine the correct base URI

When importing a compressed RDF/XML file, AllegroGraph could fail to determine the correct base URI. This has been corrected.

Bug22559 - RDF/XML parser should validate the base URI if supplied

Previously, the RDF/XML parser failed to validate the base URI it was given which could lead to invalid data. This has been corrected.

Bug21867 - Ensure text index queries are called using strings

AllegroGraph sometimes called lower level text index query functions with invalid arguments. This has been corrected.

Bug21460 - Polygons data cannot be exported and imported

AllegroGraph now converts resources of the form

<http://franz.com/ns/allegrograph/4.14/index#_N> 

(where N is replaced by a whole number) into encoded UPIs rather than hashed ones. These URIs are used to serialize AllegroGraph polygon data which means that polygons can be efficiently exported and imported. Note that the Lisp API upi->value will return the numeric representation for the UPI rather than the string representation. Use part->string if the string representation is needed.

Bug20497, Rfe11047 - Fix fti too many open files error

Previously, when constructing a new text index for a database with existing triples, there was potential for the operation to fail with a "Too many open files" error. This has been fixed.

HTTP Client

Rfe13304 - Support relaxed load syntax for remote triple-stores

The HTTP REST API for loading statements now includes the relaxSyntax argument. When true, N-Triple and N-Quad data are not validated as stringently. For example, blank node names may include underscores. See PUT/POST statements in REST/HTTP interface.

Rfe13249 - Augment optimize-indices API to allow for single index optimization

The HTTP API's indices/optimize service now takes an additional index query parameter which can be specified multiple times. If omitted, then all of a store's indices will be optimized. If supplied, each parameter value should be the name of an index whose optimization is being requested. See [POST /repositories/name]/indices/optimize in REST/HTTP interface.

Rfe11547 - Support Sesame query timeout parameter in HTTP

AllegroGraph's SPARQL HTTP REST interface now supports an optional timeout parameter. If specified, it must be a number greater than zero which denotes the number of seconds to limit SPARQL query execution time. If unspecified, then there is no limit on the length of a time a query may run. See [GET/POST /repositories/name in REST/HTTP interface.

Rfe11221 - General type-mapping improvements

The interface for setting up predicate- and datatype-mappings has been simplified. The same HTTP REST API calls can be used to setup regular XSD datatype mappings and geospatial ones (including cartesian, spherical and n-Dimensional). In each case, the datatype must be specified using its URL. E.g., the 10x10 cartesian mapping with resolution 1 would use the URL:

<http://franz.com/ns/allegrograph/3.0/geospatial/cartesian/0.0/10.0/0.0/10.0/1.0> 

See Type mappings in REST/HTTP interface.

Bug22576 - POST|PUT statements service should not require eval permission

AllegroGraph would not allow client-side files to be imported into a store unless the user had eval permission. This has been corrected. Of course, eval permission is still required to import server-side files.

SPARQL

Rfe13510 - Improve performance of the SPARQL query REST API

AllegroGraph now automatically caches query information across REST API calls to improve overall query performance.

Rfe13323 - Provide API to retrieve only the ID part of an encoded ID

There is now a SPARQL function, , that retrieves the non-prefix portion of an encoded ID. For example, if the encoded ID has IRI , then the function will return "3489a". In a query, this would look like:

select * {  
  _:b ex:identifier ?id .  
  bind( <http://franz.com/ns/allegrograph/fn#encodedIdId>(?id) as ?value )  
} 
Rfe13305 - Deprecate old-style SPARQL GEO syntax

Because AllegroGraph's magic properties support a richer range of geospatial features than the old-style SPARQL GEO syntax, the old syntax is being deprecated and will now signal a warning when it is used. See N-dimensional Geospatial Overview.

Rfe13297 - Reduce memory consumption for distinct aggregation in SPARQL queries

AllegroGraph now better manages memory when evaluating distinct aggregation queries like:

select ?dept (count(distinct ?student) as ?sc) where {  
  ?dept a :Department .  
  ?student :memberOf ?dept .  
} group by ?dept  
Rfe13289 - When logging, use query supplied namespaces for abbreviations

Previously, AllegroGraph would ignore SPARQL query supplied namespaces when displaying resources during query logging. It now uses these namespaces to improve the readability of the display.

Rfe13171 - Improve query checks for unknown variables

AllegroGraph now warns more consistently when a valid SPARQL expression uses variables in unreasonable ways. For example, this query is perfectly valid but the ORDER BY variables are not bound by the query so it is unlikely to be what was intended:

select ?a ?year ?label {  
  ?a a :Car .  
  ?a :modelYear ?year .  
  ?a rdfs:label ?label .  
} order by ?autoYear 
Rfe12470 - Apply multiple constraints to the same triple pattern

Previously, AllegroGraph could only apply a single logical FILTER expression to any given triple pattern. This meant that queries like:

select * {  
  ?s ?p ?o .  
  filter( ?p = ex:pred1 && ?o > 10 )  
}  
would optimize the query plan with either the ?p constraint or the ?o  
constraint but not both. This has now been improved so that queries  
like the above will use both constraints simultaneously and thereby  
run much more efficiently. 
Bug22803 - Fix bug in authorizationBasic query option

The authorizationBasic query option was not being parsed correctly which meant that queries that tried to use it would fail. This has been corrected.

See Change History for user-visible changes to earlier releases.