Introduction

This document has release notes for several releases prior to the current release. The release notes for the current release are in the Release Notes document.

Releases are numbered X.Y.Z. A major release has a number X.Y.0 or X.0.0. Usually a change to the first digit (from, say, 6.5.4 to 7.0.0) indicates a major enhancement or revision, a change to the second digit (7.0.4 to 7.1.0) indicates important changes and new features, and a change to the third digit (7.1.0 to 7.1.1) indicates bug fixes and minor enhancements. The Change History document repeats the list of user-visible modifications for every AllegroGraph release.

Database internal format

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. The last time the internal format was changed was in version 6.5.0. Thus all datbases from version 6.5.0 can be loaded into later versions without upgrading. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 8.2.0

See the current release notes.

Release 8.1.0

Release 8.1.1

This is a maintenance release, with SPARQL fixes, changes to the New WebView interface, and improved documentation.

Release 8.1.0

This release contains fixes and enhancements as described below. There are no programmer or admin notes.

New in Release 8.1.0

Release 8.1.0 is a major release with important new features.

Database internal format is unchanged from version the previous version (8.0.x)

Release 6.4.6 had a different internal format. All releases since then have the same internal format. This means version 6.5.0 through 7.3.1 repositories can be directly loaded into version 8.1.0 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Version 8.1.1 Admin notes

There are no additional admin motes for release 8.1.1.

Version 8.1.0 Admin notes

These three items were actually introduced in version 8.0.0 but the changes are important enough to repeat.

This sections notes changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

Version 8.1.1 Programmer notes

There are no programmer nore for release 8.1.1.

Version 8.1.0 Programmer notes

Version 8.1.0 Programmer notes

This section notes changes which may affect how certain code or operations work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

User-visible changes version 8.1.1 and 8.1.0

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

No significant changes.

SPARQL

AG-643 - incorrect results when using complex FILTER [NOT] EXIST clauses

Due to a regression introduced in AllegroGraph v8.0.0, queries with complex FILTER [NOT] EXISTS clauses (e.g. with inlined VALUES and inner FILTER) could potentially produce fewer results because of a join operation misconfiguration. This has been fixed.

AG-737 - SPARQL parser fails on space-separated + property path modifiers

AllegroGraph's SPARQL parser used to try to interpret + character after the predicate part of a triple pattern as an explicit numeric sign of the object part if it was separated from the predicate by a space: 

?s :p + ?o  ->  ?s :p (+ ?o)  
?s :p +?o   ->  ?s :p (+ ?o)  
?s :p+ ?o   ->  ?s (:p +) ?o

This has been fixed and now the correct grouping is used: 

?s :p + ?o  ->  ?s (:p +) ?o  
?s :p +?o   ->  ?s :p (+ ?o)  
?s :p+ ?o   ->  ?s (:p +) ?o

The + character is grouped with the object part of the pattern as its explicit numeric sign only if there is no space between it and the object, and as a path modifier in all other cases.

AG-770 - inconsistent results of joins with UNION on SBQE

SPARQL queries with UNION operations could lose some of the results on SBQE if the operands had differing sets of bound variables, because the result of the UNION would incorrectly inherit bound variables of its left-hand side operand. For example the result of the following operation 

{ VALUES ?s { :person1 :person2 } }  
UNION  
{ VALUES ?o { :person1 :person2 :person3 } }

would have ?s marked as bound, while it is only bound in 2 out of 5 results, causing a subsequent join operation to lose results. This has been fixed by correctly computing the set of bound variables for the result of the UNION.

AGWebView

AG-354 - Fix losing content of query tabs in New WebView after reload

Before 8.1.1, in New WebView unsaved changes in queries were lost after page reload, page closing, and user logout (explicit or implicit due to inactivity). Now, in New WebView all query changes persist on the server side per user for a repository. In other words, even unsaved queries are saved automatically. The difference between saved and unsaved queries is the following:

  1. Unsaved queries don't have persistent names.
  2. Unsaved queries aren't listed in the right sidebar in the "Saved queries" group, only in the query tab.
  3. Unsaved queries have user scope. Saved queries can be saved per repo, e.g. any user with read access could see them.
  4. Natural language queries were saved in browser's local storage which is limited to 5MB only. Now, New WebView stores results on the server side which is limited only by server's storage.

AG-367 - Make the query language option local to a query tab instead of being global for a user

Before 8.1.1, the query language was part of "In-Browser query option" group and global for a user. Now, New WebView has language attribute for each query (saved or unsaved). The language is displayed as an icon in the query tab. Additionally, rename the "Free text" to "Text Search", and "Query documents" to "ChatStream - Natural Language" queries.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

User-visible changes version 8.1.0

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

No significant changes.

SPARQL

No significant changes.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

Releases 8.0.0 and 8.0.1

Release 8.0.0 is a major release with many significant new features as well as bug fixes and enhancements. Release 8.0.1 is a maintenance release.

New in Release 8.0.0/8.0.1

Version 8.0.0 Admin notes

This sections notes changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

Version 8.0.0 Programmer notes

This section notes changes which may affect how certain code or operations work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

Version 8.0.0 user visible changes

This section lists changes in the current release. See the Change History for a list of changes in all earlier releases.

AllegroGraph Server: General Changes

Some changes to install-agraph

You run the install-agraph program to install the tar file distribution of AllegroGraph. Relative pathnames are now supported and there are new arguments including --help (describe how the program works), --license-file (specify a file containing the AllegroGraph license saving you from having to edit the config file after creation) and --super-password (actually not new but now works more intuitively) and --super-password-file (the latter in particular allowing you to avoid entering the password in the transcript which might be visible to others).

Users will be logged out after 30 minutes of idle time

Logging back in will return to the state when the logout occurred. If you have multiple browser tabs open to AllegroGraph, all will be logged out and logged back in simultaneously.

Two new AuthPolicy values external-token and token-external

See AuthPolicy in the Server Configuration and Control document.

Further allowed abbreviations of server specs

Now allowed are:

user:password@ meaning user:[email protected]:10035 user:password@:1234 meaning user:[email protected]:1234

127.1 is the same as localhost. 10035 is the default port number which is used unless another port is specified. See the Repository Specification document where server specs and associated repository specs are discussed.

Some changes when configuring AllegroGraph

User password is by default handled within AllegroGraph, but users may wish to use some external authentication via LDAP in order to authorize users. In that case supplying passwords within AllegroGraph is neither necessary or desired. During installation if using the install-agraph script, supplying a blank superuser password can be accepted but only after confirming you wish to disable password authentication: 

SuperUser account password: [user simply enters Return]  
 
Password can't be blank!  
Would you like to disable authentication via password?:  
[n]: y

Also the default instance timeout is now 1 hour (3600 seconds).

Exiting shared backend processes

Shared Backend processes (not session backend processes) will now exit if they are haven't been used for the last ten minutes. This will free up system resources and allow repos to close if the only reason they are open is that a backend process has them open. See the agraph.cfg parameter BackendMaxIdle to control how long a backend must be idle before it is eliminated.

SHACL now allows specifying all graphs

Previously you could specify default or a list of graphs. Now all also works for specifying the union of all graphs in the triple-store. This works with:

Text starting with a # is a comment in agraph.cfg

Previously, # had to be the first character of a line to make the following text a comment. Now everything after # on a line is a comment regardless of where the # appears. So the line 

Auditing no   # may enable if needed

suppresses auditing and ignores # may enable later.

Names of Fedshard shards now end in .fshard

The Fedshard utility (see the Fedshard-def document) requires that names or shard repos end in .shard, in part to distinguish them from normal repos. Some users may have used earlier experimental versions of Fedshard available in earlier releases where that was not a requirement. They must change names to conform with the new specification.

Backing up settings now backs up attributes data

See the description of backing up settings in Repository Backup and Restore.

Rfe17027: Add delete-duplicate-triples and get-dupliate-triples support to agtool

Duplicate triples can now be obtained with agtool get-duplicate-triples ... and deleted using agtool delete-duplicate-triples .... See agtool.

Bug26968: Occasional segmentation faults when using user attributes

User attributes could cause segmentation faults on repositories that used advanced features like reasoning or security filters. This has been fixed.

Bug26926 - Bogus results produces by joins on optional variables

When joining on a variable that is bound in some results and unbound in others, like the variable ?x3 in the following query 

SELECT ?x0 ?x1 ?x2 ?x3 ?x4 {  
  ?x0 :pred1 ?x1 .  
  ?x0 :pred2 ?x2 .  
  OPTIONAL { ?x0 :pred3 ?x3 }  
  OPTIONAL { ?x3 :pred4 ?x4 }  
}

the correct behavior for the rows where ?x3 is unbound is to compute a cross-product of the triples with predicate :pred4. Due to a bug in the internal join machinery, AllegroGraph computed cross-products for incorrect rows instead, producing bogus results. This has been fixed.

Bug26397 - login failures caused by AGWebView tabs of different versions

WebView login used to fail when there were multiple browser tabs with different WebView versions. This has been fixed.

HTTP Client

Rfe17140 - support prefixed names in HTTP API endpoints accepting parts

Many HTTP API endpoints taking triple parts as arguments used to only allow complete IRIs to denote resources. This is no longer the case and now prefixed names like rdf:type are also allowed, but only if the given namespace is defined in that context (e.g. only server-level namespaces are used for security filters, while user namespaces are used for API calls <repository>/statements; see Namespaces hierarchy for more details).

Rfe17040 Add RedirectHTTPOnHTTPS AG directive

When set, HTTP requests sent to the AG SSLPORT will result in a 301 Moved Permanently response with the location set to the HTTPS version of the URL.

SPARQL and GRAPHQL

Single quotes work like double quotes to designate strings in GraphQL

Strings in GraphQL can be written 'foo' as well as "foo". Also '''string''' is supported just like """string"""

This is an AllegroGraph extension to the GraphQL spec. See the GraphQL document.

New configuration and query options for query caching

The query option allowCachingResults can cause query results (unaffected by LIMIT or OFFSET) to be cached so running the same query repeatedly (perhaps with different OFFSETs and LIMITs) can simply access the chached results. New configuration options QueryResultsCacheSize and QueryResultsCacheStorageSize allow specifying the maximum number of query caches and space used by query caching or to disable cahcing altogether.

Rfe17214 - Opt-in unification of owl:sameAs duplicates in SPARQL queries

If multiple items are declared owl:sameAs, you may wish only one returned in response to a query. This can be done with the new query option unifyOwlSameAsDuplicates. See its documentation in the SPARQL Reference for further details and an example.

Rfe17183 - support for owl:SymmetricProperty in materializer

The materializer now supports OWL 2 prp-symp (owl:SymmetricProperty) axiom as called for by the dynamic RDFS++ reasoner. (It was not supported in earlier versions of the materializer.)

Rfe16975 - Use the same default base URI for all backends/sessions

A base URI for the SPARQL query can be supplied via one of the two query parameters (uri or baseURI), but if neither is present, AllegroGraph falls back to using the URL of the query request itself. This could result in different base URIs for different SPARQL query invocations, because the request URL would contain a port of the actual AllegroGraph backend or session that was handling the request, for example the following query 

SELECT ?v { VALUES ?v { <> <v0> </v1> <#v2> } }

would return 

| v                                                  |  
|----------------------------------------------------|  
| http://127.0.0.1:40123/repositories/test           |  
| http://127.0.0.1:40123/repositories/v0             |  
| http://127.0.0.1:40123/v1                          |  
| http://127.0.0.1:40123/repositories/test#v2        |

when run through the main port on the repository test and 

| v                                                  |  
|----------------------------------------------------|  
| https://127.0.0.1:50456/sessions/UUID              |  
| https://127.0.0.1:50456/sessions/v0                |  
| https://127.0.0.1:50456/v1                         |  
| https://127.0.0.1:50456/sessions/UUID#v2           |

when run through a session (actual UUID replaced with a placeholder).

Staring with AG v8.0.0, the default base URI is defined as 

http://<host>:<port>/[catalogs/<cat>/]repositories/<repo>/sparql

for queries handled by backends and sessions that wrap a single concrete triple store and 

http://<host>:<port>/sparql

for all other sessions. For example the query above will now return 

| v                                                  |  
|----------------------------------------------------|  
| http://127.0.0.1:10035/repositories/test/sparql    |  
| http://127.0.0.1:10035/repositories/test/v0        |  
| http://127.0.0.1:10035/v1                          |  
| http://127.0.0.1:10035/repositories/test/sparql#v2 |

for backends and sessions wrapping a single concrete triple stores, and 

| v                                                  |  
|----------------------------------------------------|  
| http://127.0.0.1:10035/sparql                      |  
| http://127.0.0.1:10035/v0                          |  
| http://127.0.0.1:10035/v1                          |  
| http://127.0.0.1:10035/sparql#v2                   |

for all other sessions.

Rfe16328: Warn about unknown constants in SPARQL DESCRIBE clauses

AllegroGraph used to be silent about non-existent IRIs in the DESCRIBE clauses of simple SPARQL DESCRIBE queries like the following 

DESCRIBE <http://non-existent/resource>

This has been changed and now, just as for SELECT queries, an unknown-constants query warning is included in the response.

Bug27048 - query parsing failures with aggregates and outer VALUES

SPARQL queries containing both aggregates and a VALUES clause after the WHERE clause, like 

SELECT (COUNT(?o) AS ?count)  
WHERE { ?s ?p ?o }  
VALUES ?s { <http://example.org/s0> }

used to fail during parsing. This has been fixed.

Bug27090 - SPARQL SERVICE calls do not follow HTTP redirects

SPARQL SERVICE calls used to only perform a single HTTP request and fail if a redirect response was received. This has been fixed and HTTP calls in SERVICE clauses now follow redirects up to 5 times.

Bug26880 - Saved queries HTTP API ignores uri/baseURI arguments

When using session's saved queries API to save a named query that contains a relative IRI, it was impossible to supply a base URI via the uri/baseURI query arguments, as they would be ignored and the operation would fail with something like 

MALFORMED QUERY: No base URI for URI ...

The only way to supply a base URI was through the BASE declaration in the query itself. This has been fixed and the uri/baseURI parameters now work as expected.

Bug26772: Obscure errors in patterns with repeated variables on MJQE

When using MJQE, if a query contains a pattern with repeating variables or blank nodes, and either a VALUES clause that injects bindings into that pattern, for example 

SELECT * {  
  VALUES ?o { :o1 :o2 }  
  ?a ?a ?o.  
}

or a numerical range FILTER clause on one of the variables in that pattern, for example 

SELECT * {  
  ?a ?a ?o.  
  FILTER (?o > 10)  
}

it might fail with an incomprehensible error. This has been fixed.

Bug26675: Possible incorrect bindings order in SPARQL solutions

Queries with OPTIONAL clauses in rare cases can return solutions with wrong binding order. For example, the following rather synthetic query 

PREFIX : <http://example.org/>  
 
SELECT ?p ?o {  
  VALUES ?g { :g0 }  
  VALUES ?b { :b }  
  GRAPH ?g {  
    :a :next ?b.  
    ?b :next ?c.  
    OPTIONAL { ?c ?p ?o }  
  }  
}

could return a solution with values for variables ?p and ?o switched. This has been fixed.

Bug25697 - Wrong results with OPTIONAL clauses sharing a variable

SPARQL queries that perform a join on variable that might be unbound, like, for example, variable ?x2 in the following query 

SELECT ?x0 ?x1 ?x2 {  
  ?x0 <http://example.org/p1> ?x1 .            # 1  
  OPTIONAL { ?x1 <http://example.org/p2> ?x2 } # 2  
  OPTIONAL { ?x2 <http://example.org/p3> ?x3 } # 3  
}

might return incorrect results, because patterns (2) and (3) get inlined into BGP (1) during optimization, but the proper join is not introduced, which results in a cross product between patterns (2) and (3). This optimization can turned off by setting 

PREFIX franzOption_inlineSingleBGPOptionals: <franz:no>

but the query might still return incorrect results due to bug26926. Both issues have been fixed.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes other than new files for new features.

Distributed AllegroGraph

No significant changes.

Release 7.3.1

Release 7.3.1 is a minor release with bug fixes and various speed and interface enhancements but no major new features.

Release 7.3.0

Release 7.3.0 is a major release with important new features.

Release 7.2.0

Release 7.2.0 is a major release with important new features.

Release 7.1.0

Release 7.1.0 is a major release with important new features.

Release 7.0.4

Release 7.0.4 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.3

Release 7.0.3 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.2

Release 7.0.2 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.1

Release 7.0.1 is a maintenance release fixing bugs and providing minor enhancements. There are no major new features.

Release 7.0.0

Release 7.0.0 is a major release with significant new features.

The most important new feature is distributed repositories.

Distributed repositories (see the Distributed Repositories Using Shards and Federation Tutorial) allow data to be distributed over several repositories, called shards. Each shard runs on its own server separately from other shards (a single server may support more than one shard). Queries then are run by each server on each shard, with results combined after all individual queries have completed. With very large databases, use of distributed repositories can significantly speed up certain kinds of queries. See the documents Distributed Repositories Using Shards and Federation Tutorial and Distributed Repositories Using Shards and Federation Setup for more information.

Gruff in a browser is available in AllegroGraph 7.0.0. Gruff is a graph visualization and graphical query builder designed to work with AllegroGraph. It is available as a standalone application (see the Gruff page on the AllegroGraph website) and it will run in a browser tab. See Gruff in AllegroGraph document for more information. Documentation for Gruff itself is in the Gruff document.

We have added to the AllegroGraph examples on Github repo. There are examples of using Multi-master replication to allow for scaling out query load. These examples use Kubernetes, Terraform, load balancing with NGINX, and so on.

Admin notes

These sections note changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

Version 7.3.1 Admin notes

No admin notes for 7.3.1.

Version 7.3.0 Admin notes

No admin notes for 7.3.0.

7.2.0 Admin notes

7.1.0 Admin notes

There are no admin notes for release 7.1.0.

7.0.3 Admin notes

There are no admin notes for v7.0.3.

7.0.2 Admin notes

Licenses may restrict the number of cores which can be used by AllegroGraph. The restriction is indicated by the license. If the system detects more core available, AllegroGraph startup may fail with an error message noting that excess cores are available. The new configuration directive UseLicensedCores described in the Server Configuration and Control document) can be set in the agraph.cfg to ensure that only the licensed number of cores will be used.

A problem with expired passwords unable to be reset in AGWebView has been fixed. Note that agtool user set-password (see the Managing Users document) can always be used to change a password and a superuser can change any user's password.

7.0.1 Admin notes

There are no admin notes for v7.0.1.

7.0.0 Admin notes

Programmer notes

Version 7.3.1 Programmer notes

No programmer notes for version 7.3.1

Version 7.3.0 Programmer notes

Version 7.2.0 Programmer notes

This section notes changes which may affect how certain code or operations work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

7.1.0 Programmer notes

7.0.4 Programmer notes

An unexplained problem with Internet Explorer (IE) 11 makes it unable to display AGWebView. All other important browsers, including Edge, work fine. The problem seems to be related to running JavaScript.

7.0.3 Programmer notes

There are no programmer note for version 7.0.3.

7.0.2 Programmer notes

There are no programmer note for version 7.0.2.

7.0.1 Programmer notes

SPARQL queries applied to distributed repositories (see Distributed Repositories Using Shards and Federation Setup) always work on the federations of the distributed shards and all knowledge base (kb) repos. Therefore when you do View triples or View quads on the Repository page in WebView, because the information is obtained by a SPARQL query, the triples in the distributed repo and all associated kb repos will be displayed (up to the display limit). In contrast, the number of statements (triples) displayed at the top of the repository page will be the number in the distributed repo only. The Lisp function get-triples and REST interface commands such as size work on the distributed repo only, not on its federation with kb repos.

7.0.0 Programmer notes

See the current release notes.

Version 7.3.1 user visible changes

AllegroGraph Server: General Changes

spr45740 - SHACL: errors in franzOption_profileQuery and resultMessage

These valid SHACL requests might signal the following errors:

These have been fixed. There have also other been SHACL speedups.

Bug26839 - Fix SSLCipherSuite directive

A bug fetching the value of the SSLCipherSuite config directive (see Server Configuration and Control) at startup caused it to be ignored. This is now fixed.

Bug26806 - Turtle parser fails on prefixes starting with base/prefix

AllegroGraph's Turtle/TriG parser used to fail on prefixes that start with directive keywords (base and prefix). For example, all the statements in the following Turtle file 

 @prefix : <http://example.org/>.  
 @prefix base: <http://example.org/base/>.  
 @prefix basenames: <http://example.org/basenames/>.  
 @prefix prefix: <http://example.org/prefix/>.  
 @prefix prefixes: <http://example.org/prefixes/>.  
 base:s      :p :o.  
 basenames:s :p :o.  
 prefix:s    :p :o.  
 prefixes:s  :p :o.

would have failed to parse. This has been fixed.

Bug26775 - Turtle parser fails on boolean literals followed by dots

Turtle parser used to signal an error when a boolean literal was used in an object position of a statement and immediately followed by the dot character, for example 

@prefix : <http://example.org/>.  
:s0 :p0 true.  
:s1 :p1 false.

bug26765 - Problem with Admin/Audit log page

The audit log was not recording audit events. This is now fixed.

Bug26754 - Load-transform fails on floating point numbers in JSON

Floating point numbers in JSON documents used to produce errors of the form 

`0.0d0` is not of type xsd:double

due to conversion operation malfunction. This has been fixed.

Bug26747 - Allow using transform templates with JSON list elements

It is now possible to use tranform templates with elements of JSON lists. For example, assuming JSON source 

{"foo": "1", "bar": ["2", "3"]}

command 

agtool load <repo> \  
    --tr-prefix 'http://example.org/' \  
    --tr-transform 'bar=http://example.org/bar${foo}-${bar}' \  
    <source.json>

will create two objects for property bar: 

_:e0 ex:bar ex:bar1-2, ex:bar1-3.

bug26135 - change in stored queries format in 7.3.0

Stored SPARQL should be preserved when backed up and restored but this failed in 7.3.0 because the name of one of the fields was change from "text" to "query". Now either label can be used.

Bug26755 - Errors when using numeric values in load-transform templates

Populating templates with numeric values (as opposed to strings) would cause an error during load-transform operations, for example, assuming the following JSON document 

{ "foo": 1 }

the following load-transform 

agtool load <repo> -i json \  
       --tr-transform 'foo=http://example.org/Foo/${foo}' \  
       <source.json>

would signal an obscure error 

excl::string2 argument should be a string

because of an internal type mismatch. This has been fixed.

HTTP Client

No significant changes.

SPARQL

Bug26872 - "Not CaaT state" errors with large OFFSET and DISTINCT

It was possible to get a confusing error of the form 

Not CaaT state: nil within set

when running a SPARQL query with DISTINCT modifier and a large OFFSET, e.g. 

SELECT DISTINCT ?s ?p ?o { ?s ?p ?o }  
OFFSET 500000  
LIMIT 1000

when the actual number of solutions iterated through exceeded one of the internal thresholds. This has been fixed.

Bug26739 - Type errors on language-tagged literals in SPARQL's REGEX

SPARQL REGEX function used to report a type error when its first argument was a language-tagged string, for example 

SELECT (REGEX('Blade Runner'@en, 'Blade') AS ?r) {}

This has been fixed.

AGWebView

bug26762 - cancel query button didn't work in AG WebView

This has been fixed.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

Version 7.3.0 user visible changes

This section lists changes in the current release. See the Change History for a list of changes in all earlier releases.

AllegroGraph Server: General Changes

agtool --output option replaced by --output-format.

--output-format is more descripting. The affected agtool commands include:

--output can still be used but will trigger a deprecation warning.

Rfe16785 - Deprecate --results-format in favor of --output-format

The command-line interface for agtool query has a new argument, --output-format for specifying the desired output format, replacing --results-format. --output-format is the argument used by all agtool commands. Here is a sample call: 

agtool query <repo> query.rq --output-format sparql-json

The old argument can still be used but will trigger a deprecation warning.

Rfe16780 - Support for using AWS session tokens with agtool

New --aws-session-token command-line argument argument was added to the agtool commands that support interacting with AWS S3 (agtool export, agtool load and agtool archive). The session token value can also be read automatically from the configuration files. See Temporary credentials in the agtool document.

HTTP Client

No significant changes.

SPARQL

Rfe16766 - New query option to print query execution error backtraces

The logBacktraceOnQueryFailure causes debugging information to be printed to the log when errors occur when executing a query. While this information is rarely useful to the user, it may be useful when contacting AllegroGraph support.

Rfe16761 - New query options for profiling query execution

New query options profileQuery and profileOutputFormat can cause profile data to be written the the AllegroGraph log. This is useful when a query seems to take longer than expected. See Query execution options in SPARQL Reference.

Bug26683 - Incorrect results with duplicate blank nodes in patterns

SPARQL queries with patterns containing duplicate blank nodes, for example 

SELECT * { _:x ?p _:x }

could return incorrect solutions where the values corresponding to the blank nodes were not the same. This has been fixed.

Bug26629 - Errors in GROUP expressions with unknown variables on MJQE

SPARQL queries with unknown variables in a singleton complex GROUP expression, for example 

PREFIX franzOption_engine: <franz:mjqe>  
 
SELECT (SAMPLE(1) AS ?foo) { ?s ?p ?o }  
GROUP BY CONCAT(\"bar\", ?bar)

used to fail with the error message 

Received signal number 7 (Bus error)

This has been fixed and now the result of evaluating such an expression is assumed to be UNDEF, and the type error warning is reported alongside the unknown variables warning.

Bug26627 - Some SPARQL query warnings may cause obscure print errors

Certain query warnings, for example repeated projected variables like in the following query 

SELECT ?s ?s ?o { ?s ?p ?o }

are generated during parsing and trying to send such a query in a parsed form to a remote triple store or a distributed triple store shard may cause errors like 

Unable to print  
#<db.agraph.sparql.parser:warn-repeated-projected-variables-ignored  
  @ #x10012f443b2>  
readably and *print-readably* is true.

This has been fixed.

Bug26621 - SPARQL ORDER clause might not work when GROUP present

SPARQL queries with both GROUP and ORDER clauses could return results in an incorrect order because ORDER clause expressions used to have no access to bindings in GROUP and SELECT clauses that defined during aggregation. This has been fixed.

Additionally, ordering now also works with non-aggregate variables: in such cases the order expression is wrapped in an implicit SAMPLE operation and a corresponding warn-implicit-sample-introduced query warning is reported.

Bug26616 - Allow using constant SPARQL expressions as aggregates

It should be possible in SPARQL to use constant expression assignments alongside aggregated variables, for example 

SELECT ?s ("constant" AS ?c)  
WHERE { ?s ?p ?o }  
GROUP BY ?s

but assignments likes these used to cause queries to fail with errors like 

All selected variables must be aggregates but ?c is not.

This has been fixed.

Changes to the Lisp API

Rfe16175: Unify run-sparql's rdf-format/results-format arguments

The run-sparql function's arguments rdf-format and results-format are deprecated in favor of the new output-format argument. The old arguments can still be used but will signal a deprecation warning.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

Version 7.2.0 user visible changes

This section lists changes in the current release. See the Change History for a list of changes in all earlier releases.

AllegroGraph Server: General Changes

Some TopBraid Composer (TBC) examples no longer work

The document TopBraid Composer Plugin has examples showing how you can connect TBC to an AllegroGraph repository and then directly add triples created by TBC. This worked by using the Sesame2 Remote Repository export format from TBC. Recent versions of TBC (which is now at version 7.x) no longer support that format. To copy triples from TBC to AllegroGraph you must save the triples to a file and load the file into AllegroGraph. The TopBraid Composer Plugin document has been updated to not this change.

New agtool vload command loads materialized triples from a relational database

See agtool and the Creating Virtualized Graphs document for more information. There is an example using the Ontop utility.

New data streaming example

See Graph streaming for an example connecting AllegroGraph a data streaming tool such as Apache Kafka.

New example using Apache Spark

An example of graph analytics using AllegroGraph and Apache Spark can be found here in the franzinc/agraph-examples repo. See the Examples on Github document, which includes examples using Apache Spark, Apache Kafka, Kubernetes, Terraform, load balancing with NGINX, and so on.

New agtool commands query-options and namespaces

The command agtool query-options can be used to add, remove, and list default query options for the user and repo specified in the command line. The command agtool namespaces can be used to add, remove, and list default user namespaces for the user and repo specified in the command line.

New --port and --sslport control options to agraph-control

The agraph-control program now accept --port and --sslport control options, which, if specified, override the values specified for Port and SSLPort in the AllegroGraph configuration file (usually named agraph.cfg). This is convenient if the values specified in the configuration file are already in use. The agraph-control is documented here in the Server Configuration and Control document. The associated agraph also now accepts --port and --sslport arguments.

Handling of numbers in JSON-LD code fixed

Things that looked like numbers in JSON-LD code, including the string ".", were converted to typed numbers, even when that was not wanted. Strings "true" and "false" were also always converted to booleans. This has been fixed. Conversion is now performed only when the appropriate datatype is explicitly specified.

New storage-report option to agtool

The tool shows storage used by a repository. See here in the agtool General Command Utility document. The report includes, along with other information, the percentage of triples stored but deleted.

The Table in the Cluster Managment page modified

The table is illustrated in the Multi-master Replication document. The table now shows the host and port of each instance.

The maximum number of unique strings has increased

From 2^32 to 2^39. (The hash table in shared memory used for locating strings doubled in size.)

Rfe16711 - implement query-results-limit user access flag

Users can now be restricted to a maximum number of results from a query. See Limiting results of a query in SPARQL Reference and Limiting the results a user can see in Managing Users. Also see the new QueryResultsLimit configuration directive.

Rfe16709 - XML 1.1 support for rdf serializer

Make RDF serializer output XML 1.1 by default. Previously AllegroGraph wrote version 1.0 XML which is unable to express all of the characters you might find in repository data. The new XMLVersion configuration option can be used to revert back to XML 1.0 if needed.

Rfe16691 - New agtool for automating repository optimization

See the Release Notes description above. Also see agtool auto optimize, or AGWebView.

Rfe16639 - Rename agtool command token to tokens for consistency

The agtool command tree token has been renamed to tokens for consistency with other similar commands. The old token name will still work to ensure backward compatibility but it is deprecated and excluded from the help text.

Rfe16562: Support limiting agtool query output to results only

The output of agtool query can now be limited to results only by specifying the --quiet option. When --quiet is set, all log output will be written to the agquery.log file. See agtool query for more information.

Rfe16540 - Add support for creating, deleting, and listing repos and dynamic catalogs with agtool

agtool has new repos and catalogs tools. They can be used, respectively, to create and delete repos (including replicated repos) and dynamic catalogs. The tools can also list existing repos and catalogs. See the agtool document for more information.

Bug26575 - SHACL implicit class targets only worked for rdfs:Class

If a shape is also declared to be a class in the shapes graph then all SHACL instances of this class are a target for the shape (see the SHACL spec here). This only worked if the shape had a rdfs:Class type and failed if it was a subclass of rdfs:Class e.g. owl:Class. This has been fixed.

Bug26473 - Decimal production in SPARQL 1.1

DECIMAL production now uses the SPARQL 1.1 definition, meaning that at least one digit after the dot is required. Previously 1. was read as the DECIMAL 1.0, instead of being correctly read as the integer 1 followed by a period, just like 

1 .

This is an incompatible change. The SPARQL 1.0 rule is specified here and the changed SPARQL 1.1 rule is here.

Bug26414: Fix default query memory limit initialization

Default query memory was incorrectly initialized to the 13.3 GB instead of 85% of available system memory, as specified in the documentation. Because of this in some cases the users needed to increase the memory limit manually via the query option 

PREFIX franzOption_memoryLimit: <franz:NEW_LIMIT>

where NEW_LIMIT is either a number (of bytes) or a value of the form <number><unit> (e.g. 32G). This has been fixed so default query memory will start at 85% of available system memory.

Bug26221 - Speedup Freetext Index (FTI) phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Rfe16528 - Remove reorderDuringExecution query option

In practice the pattern estimates are good enough to not have to consider reordering. Using the special query prefix: 

PREFIX franzOption_reorderDuringExecution: <franz:yes>

will now give a query warning: 

obsoleteQueryOption: reorderDuringExecution

Rfe16510 - Support for SPARQL*

AllegrGraph now supports queries that use the SPARQL* syntax, like: 

select * {  
  << ?s ?p ?o >> <ex:key> ?value  
}  
 
select * {  
  bind( << ?s <ex:pred> ?o >> as ?triple )  
}

A reified triple is represented as an IRI that looks like: <http://franz.com/ns/allegrograph/4.0/tripleId#123> which means isIRI(?triple) will return true for reified triples.

There is a new SPARQL filter function: isTriple(?triple) that returns true only if ?triple is a reified triple.

SPARQL* queries are not yet supported on federated or distributed repositories.

Rfe16504: Remove "enableDuplicateReduction" query option

The SPARQL query option enableDuplicateReduction, if set to true, would behave somewhat like specifying reduced. In all cases where it would be relevant to enable this reduction, it is better to specify reduced or distinct in the query. Therefore the setting is removed.

Usage of the franzOption_enableDuplicateReduction query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_enableDuplicateReduction: <franz:yes>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: enableDuplicateReduction  
  [time] ...  
  ...>

Bug26558 - Fix zero-length property paths with non-existent S/O parts

According to the SPARQL specification, assuming an empty data set, all of these queries should return the same single result :s1: 

# query 1  
SELECT ?s {  
  ?s :p1* :s1  
}  
 
# Query 2  
SELECT ?s {  
  VALUES ?s { :s1 }  
  ?s :p1* :s1.  
}  
 
# Query 3  
SELECT ?s {  
  ?s :p1* :s1.  
  FILTER (?s = :s1)  
}

but only the first one returns the correct result, while all others return an empty result set because of a premature optimization. This has been fixed.

Bug26557 - Remove odd zero-length property paths matches on MJQE

MJQE used to return additional results for zero length paths when either subject or object of the pattern was fixed. For example, for the data set 

:s1 :p1 [ :p2 :o1 ].  
:s2 :p3 :o2.

the query 

SELECT ?o {  
  VALUES ?o { :o2 }  
  :s2 (:p1|:p2)* ?o  
}

returned 

-------  
| o   |  
=======  
| :s2 |  
-------

which is only correct when the object of the pattern is not fixed. This has been resolved.

Bug26532 - Query canceling does not work for queries run on MJQE

It was impossible to cancel a query that was being executed on MJQE because of the incorrect job initialization. For example, pressing the Cancel button in WebView had no effect on the MJQE query currently in progress. This has been fixed.

Bug26531 - Fix total query time reporting for queries run via HTTP

When running SPARQL queries through HTTP endpoint (for example, in WebView), the query information sent in x-query-final-info trailer contained incorrect total query time that was approximately twice as big as the real total time. This has been fixed.

Bug26525 - Format selection fails for CONSTRUCT queries run with agtool

When executing queries with agtool query, the --results-format argument was not working properly for DESCRIBE and CONSTRUCT queries with the keywords in upper case. This has been fixed.

Bug26515 - Restore reorderDuringExecution query option

The option was removed in v7.1.0 but has now been restored. See Query execution options in the SPARQL Reference document.

Bug26493 - Queries with magic properties may return less results

It was possible for some queries with magic properties and fixed values to return less results because of an incorrect optimization, For example, for the following dataset 

@prefix : <http://example.org#>.  
:alice :owns "Galaxy Note".  
:bob   :owns "Galaxy Note".

the following query 

PREFIX : <http://example.org#>  
PREFIX fti: <http://franz.com/ns/allegrograph/2.2/textindex/>  
SELECT ?p ?o WHERE {  
  (:bob ?o ?p) fti:match "galaxy".  
}

will return an empty result, while the query without the fixed constant 

PREFIX : <http://example.org#>  
SELECT ?s ?p ?o WHERE {  
  (?s ?o ?p) fti:match "galaxy".  
}

returns both results, as expected: 

--------------------------------  
| s      | p     | o           |  
================================  
| :alice | :owns | Galaxy Note |  
| :bob   | :owns | Galaxy Note |  
--------------------------------

This has been fixed.

Bug26460: Empty RDF graph was returned by SPARQL's DESCRIBE * WHERE ...

The following two SPARQL queries are expected to be equivalent 

DESCRIBE * WHERE { ?s ?p ?o . }  
 
DESCRIBE ?s ?p ?o WHERE { ?s ?p ?o . }

but the first one (with the wildcard *) used to return an empty result. This has been fixed.

Bug26452 - MJQE may omit bindings introduced by SPARQL VALUES clause

When more then one variable in a pattern has bindings introduced by the VALUES clause, MJQE engine may omit the bindings for one of the variables.

For example, assuming the data 

:a :p1 :b.  
:a :p2 :c.  
:b :p2 :c.

the following query 

SELECT ?s ?p ?o WHERE {  
  VALUES ?s { :a }  
  VALUES ?p { :p1 :p2 }  
  ?s ?p ?o.  
}

used to return the following result 

------------------  
| s   | p   | o  |  
==================  
| --- | :p1 | :b |  
| --- | :p2 | :c |  
------------------

when run on MJQE. This has been fixed.

Bug26451 - Fix occasional errors in queries on security-filtered stores

When security filters were used, it was possible for some queries to fail with an error message of the form 

QUERY FAILED: `#<db.agraph.cursor::filtered-cursor ...>' is not  
of the expected type `number'

indicating an internal type error. This has been fixed.

Bug26436 - Fix missing results for multiple property path matches

In cases when there are multiple zero-or-more paths between the subject and object in the restricted pattern, the query engine used to drop the result with multiple paths, for example for the dataset 

:s1 :p1 :o1.  
:s2 :p1 :o1.  
 
:s1 :p2 [ :p2 :o2 ].  
:s2 :p3 [ :p2 :o2 ].  
:s2 :p2 [ :p3 :o2 ].

the query 

SELECT ?s {  
  # this restricts the pattern in FILTER EXISTS  
  ?s :p1 :o1.  
  # this ends up return less results  
  FILTER EXISTS { ?s (:p2|:p3)* :o2 }  
}

used to return only one result :s1, while it should have returned :s1, :s2. This has been fixed.

Bug26333 - Property paths that join bindings may produce bogus results

It was possible for the SPARQL sequence property paths to produce too many results when joining bindings from VALUES clauses or subqueries. This has been fixed.

Bug26290 - Max when some variable is unbound, must give unbound result value

In AllegroGraph 6.7.0 a SPARQL regression was introduced, where SPARQL MAX aggregator would treat unbound values as bound values (lowest in ordering) and so return a bound value as result.

This was incorrect: when aggregating over an unbound value, the aggregation result should be an unbound value due to a SPARQL type error in the aggregation.

This has been fixed, so that the aggregation result is the unbound value in the described case.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

Bug26260 - Fix spurious "emptyBindClause" query warnings

SPARQL queries with a BIND clause at the start of a BGP could generate a spurious "emptyBindClause" warnings saying: 

The BINDING form for ?val cannot succeed

This can be an incorrect warning if the BIND expression does a special operation that can deal with unbound variables, like BOUND, COALESCE, IN or IF.

Examples: 

select * {  
  bind(bound(?o) as ?val) .  
}  
-> ?val = false  
 
select * {  
  bind(coalesce(?o,1,2) as ?val) .  
}  
-> val = 1

This is now fixed so that no spurious warnings are generated.

AGWebView

Bug26332: Broken bulk namespace input dialog in AGWV

The Add a namespace button in query editor shows a dialog that has a Bulk Input button which in turn should be showing a different dialog that allows to input multiple namespace declarations at once. This dialog was broken and an empty dialog box was shown instead. This has been fixed. See the New Query Page section of the WebView document for more information on the Add a namespace button.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

There is a new AllegroGraph examples document.

This document has links to example locations, either part of the distribution or on our franzinc/agraph-examples repo on Github.

Distributed AllegroGraph

No significant changes.

Release 7.1.0

Release 7.1.0 is a major release with important new features.

Release 7.0.4

Release 7.0.4 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.3

Release 7.0.3 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.2

Release 7.0.2 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Release 7.0.1

Release 7.0.1 is a maintenance release fixing bugs and providing minor enhancements. There are no major new features.

Release 7.0.0

Release 7.0.0 is a major release with significant new features.

The most important new feature is distributed repositories.

Distributed repositories (see the Distributed Repositories Tutorial) allow data to be distributed over several repositories, called shards. Each shard runs on its own server separately from other shards (a single server may support more than one shard). Queries then are run by each server on each shard, with results combined after all individual queries have completed. With very large databases, use of distributed repositories can significantly speed up certain kinds of queries. See the documents Distributed Repositories Tutorial and Distributed Repositories Setup for more information.

Gruff in a browser is available in AllegroGraph 8.2.1. Gruff is a graph visualization and graphical query builder designed to work with AllegroGraph. It is available as a standalone application (see the Gruff page on the AllegroGraph website) and it will run in a browser tab. See Gruff in AllegroGraph document for more information. Documentation for Gruff itself is in the Gruff document.

We have added to the AllegroGraph examples on Github repo. There are examples of using Multi-master replication to allow for scaling out query load. These examples use Kubernetes, Terraform, load balancing with NGINX, and so on.

Release 6.6.0

Release 6.6.0 is a major release with significant new features, including the following:

Release 6.5.0

AllegroGraph 6.5.0 was released on March 4, 2019. Release 6.5.0 is a major release with significant new features, including the following:

Admin notes

These sections note changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

7.1.0 Admin notes

There are no admin notes for release 7.1.0.

7.0.3 Admin notes

There are no admin notes for v7.0.3.

7.0.2 Admin notes

7.0.1 Admin notes

There are no admin notes for v7.0.1.

7.0.0 Admin notes

Programmer notes

These sections note changes which may affect how certain operations or code work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

7.1.0 Programmer notes

7.0.4 Programmer notes

An unexplained problem with Internet Explorer (IE) 11 makes it unable to display AGWebView. All other important browsers, including Edge, work fine. The problem seems to be related to running JavaScript.

7.0.3 Programmer notes

There are no programmer note for version 7.0.3.

7.0.2 Programmer notes

There are no programmer note for version 7.0.2.

7.0.1 Programmer notes

7.0.0 Programmer notes

6.6.0 Programmer notes

There are no programmer notes for release 6.6.0.

User visible changes

These sections list changes in some releases prior to the current release. See the Change History for a list of changes in all earlier releases.

AllegroGraph 7.1.0

AllegroGraph Server: General Changes

New storage-report option to agtool

The tool shows storage used by a repository. See here in the agtool General Command Utility document.

The Table in the Cluster Managment page modified

The table is illustrated in the Multi-master Replication document. The table now shows the host and port of each instance.

Rfe16562: Support limiting agtool query output to results only

The output of agtool query can now be limited to results only by specifying the --quiet option. When --quiet is set, all log output will be written to the agquery.log file. See agtool query for more information.

Rfe16543 - Replace default index :gospi by :psogi

The set of default triple indices has been modified: gospi has been taken out, and psogi has been added. See 7.1.0 Programmer Notes above.

Rfe16540 - Add support for creating, deleting, and listing repos and dynamic catalogs with agtool

agtool has new repos and catalogs tools. They can be used, respectively, to create and delete repos (including replicated repos) and dynamic catalogs. The tools can also list existing repos and catalogs. See the agtool document for more information.

Bug26221 - Speedup Freetext Index (FTI) phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Rfe16528 - Remove reorderDuringExecution query option

In practice the pattern estimates are good enough to not have to consider reordering. Using the special query prefix: 

PREFIX franzOption_reorderDuringExecution: <franz:yes>

will now give a query warning: 

obsoleteQueryOption: reorderDuringExecution

Rfe16510 - Support for SPARQL*

AllegrGraph now supports queries that use the SPARQL* syntax, like: 

select * {  
  << ?s ?p ?o >> <ex:key> ?value  
}  
 
select * {  
  bind( << ?s <ex:pred> ?o >> as ?triple )  
}

A reified triple is represented as an IRI that looks like: <http://franz.com/ns/allegrograph/4.0/tripleId#123> which means isIRI(?triple) will return true for reified triples.

There is a new SPARQL filter function: isTriple(?triple) that returns true only if ?triple is a reified triple.

SPARQL* queries are not yet supported on federated or distributed repositories.

Rfe16504: Remove "enableDuplicateReduction" query option

The SPARQL query option enableDuplicateReduction, if set to true, would behave somewhat like specifying reduced. In all cases where it would be relevant to enable this reduction, it is better to specify reduced or distinct in the query. Therefore the setting is removed.

Usage of the franzOption_enableDuplicateReduction query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_enableDuplicateReduction: <franz:yes>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: enableDuplicateReduction  
  [time] ...  
  ...>

Bug26333 - Property paths that join bindings may produce bogus results

It was possible for the SPARQL sequence property paths to produce too many results when joining bindings from VALUES clauses or subqueries. This has been fixed.

Bug26290 - Max when some variable is unbound, must give unbound result value

In AllegroGraph 6.7.0 a SPARQL regression was introduced, where SPARQL MAX aggregator would treat unbound values as bound values (lowest in ordering) and so return a bound value as result.

This was incorrect: when aggregating over an unbound value, the aggregation result should be an unbound value due to a SPARQL type error in the aggregation.

This has been fixed, so that the aggregation result is the unbound value in the described case.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

Bug26260 - Fix spurious "emptyBindClause" query warnings

SPARQL queries with a BIND clause at the start of a BGP could generate a spurious "emptyBindClause" warnings saying: 

The BINDING form for ?val cannot succeed

This can be an incorrect warning if the BIND expression does a special operation that can deal with unbound variables, like BOUND, COALESCE, IN or IF.

Examples: 

select * {  
  bind(bound(?o) as ?val) .  
}  
-> ?val = false  
 
select * {  
  bind(coalesce(?o,1,2) as ?val) .  
}  
-> val = 1

This is now fixed so that no spurious warnings are generated.

AGWebView

Bug26332: Broken bulk namespace input dialog in AGWV

The Add a namespace button in query editor shows a dialog that has a Bulk Input button which in turn should be showing a different dialog that allows to input multiple namespace declarations at once. This dialog was broken and an empty dialog box was shown instead. This has been fixed. See the New Query Page section of the WebView document for more information on the Add a namespace button.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

AllegroGraph 7.0.4

AllegroGraph Server: General Changes

Rfe16569 - backup-all (and restore-all) now backs up gruff options

agtool archive backup-all/backup-settings now backs up the settings/gruff/ directory.

Bug26353 - Turtle parser failure on prefixes with upper-case characters

Turtle parser was incorrectly converting prefixes of prefixed names to lower-case if they were at the beginning of the lines and started with the same (ignoring the case) characters as Turtle keywords prefix or base, which caused a namespace lookup failure.

For example, an attempt to parse the following Turtle 

 prefix Basic: <http://example.org/basic#>  
 Basic:subject Basic:predicate Basic:object.

failed with the following error 

 While parsing /path/to/file.ttl: Turtle parser  
 error on line 2: No prefix expansion defined for 'basic'

This has been fixed.

HTTP Client

No significant changes.

SPARQL

Bug26348 - Fix handling of the FROM clauses of SPARQL ASK queries

Starting with AG 7.0.0, running SPARQL ASK queries with the FROM clauses failed with a verbose error that said something like keyword list ... should only contain ... but unexpectedly found .... This has been fixed.

Bug26346 - Fix 'Funcall of nil which is a non-function' error

It was possible to get a Funcall of nil which is a non-function error when executing a SPARQL query that has a UNION of GRAPH clauses with unsatisfiable patterns. This has been fixed.

Bug26331 - Restriction reasoning did not work in remote Lisp AG client

Both SPARQL and Prolog queries run from a remote Lisp AllegroGraph client would miss inferred results when restriction reasoning is enabled. This has been fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

AllegroGraph 7.0.3

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Bug26309 - Missing zero-length property path results if both s and o are collections

It was possible for a query of the form: 

select ... {  
  values ?start { ... }  
  values ?end { ... }  
  ?start ( <ex:pred1> / <ex:pred2>* ) ?end  
}

with triple data: 

<ex:s> <ex:pred1> <ex:o>

to not return the valid solution: 

?start = <ex:a>  
?end = <ex:b>

where was followed zero times, so effectively not followed. This is corrected.

Bug26293: Property path error if object value constrained

In certain cases a property path expresion in which the object value is constrained to a literal, could lead to an error.

For example in this query ?o is constrained to value 2: 

SELECT ... {  
  bind(2 as ?o).  
  ?s <ex:p>+ ?o  
}

Depending on the active indices and the exact triples in the repository, this could give the error: 

The assertion  
  (loop for x in (list s p o g s-end p-end o-end g-end)  
    always (typep x (quote (or upi null))))  
failed

This is now fixed.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

(remark -- 7.0.2 start ; -- }

AllegroGraph 7.0.2

AllegroGraph Server: General Changes

An AllegroGraph server may restrict itself to the number of licensed cores

If the UseLicensedCores configuration directive is specified true (that is UseLicensedCores true is in agraph.cfg) then the agraph process will automatically reduce the number of cores it uses to the value given in the license file. Otherwise the server will fail to start up if there are more cores on the machine than the license specifies.

Bug26226 - Changing passwords in AGWebView could fail

When a password has expired, changing it in AGWebView could fail. This has been fixed.

Bug26221 - Speedup fti phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Bug26274 - Insert ..^^xsd:string fails

Queries that insert a triple with an XSD string as object, like: 

INSERT {  
  <ex:s> <ex:prop> "2.0.0"^^xsd:string .  
}  
WHERE { }

would fail with the error message: :string fell through a ecase form. This is fixed.

Bug26273 - Do not skip index when rewriting left-joins to joins

In some cases the system would skip the index component of a plan step when constructing a JOIN plan step. This was breaking the assumption that index is non-nil when removing equivalent variables from JOINs. This is now fixed.

Bug26272 - Count DISTINCT of UNION incorrect

It was possible for queries with COUNT DISTINCT of a UNION to give a number that was too high. For example: 

SELECT (COUNT (DISTINCT ?s) as ?s)  
{  
  { ?s <ex:p1> <ex:o1> }  
  union  
  { ?s <ex:p2> <ex:o2> }  
}

This is now fixed.

Bug26263 - Type errors when using UNION and FILTER

This is an issue relevant to queries containing UNION in which an unprojected variable occurs both inside the UNION (at most once per UNION subclause) and once outside it, like ?o in: 

SELECT DISTINCT ?s {  
{ ?s <ex:p1> ?o } UNION { ?s <ex:p2> ?o }  
FILTER (?o > 3)  
  }

In such cases it was possible for ?o to become unbound in the second occurrence (in the FILTER clause) even though the first UNION clause would bind it. This would lead to incorrect query results.

This is now fixed.

Bug26259 - Property path error: "the value of p is "

Certain property path expression would fail with the error: 

the value of db.agraph.sbqe.property-path::p is  
#<db.agraph.utility.dictionary:dictionary ...>  
which is not of type (or null upi real).

This is fixed.

Bug26240 - SPARQL property path that joins bindings misses results

It was possible for property path expressions to miss certain results if the subject or object of the property path was bound by a previous pattern. This is now fixed.

Bug26236 - Errors when using multi-match clause with multiple vars as filter-pattern

There were situations where a query containing a "VALUES" or "FILTER (.. IN ..)" clause, in combination with other patterns, would give incorrect query results. This has been fixed.

Bug26235 - Assertion error :is-ordered-cursor-for-distinct-variable

It was possible for queries with DISTINCT to fail with the error: 

the assertion  
  (null (getf (db.agraph.cursor::cursor-properties db.agraph.sbqe::cursor)  
              :is-ordered-cursor-for-distinct-variable))  
failed.

This has been fixed.

Bug26222 - SPARQL FILTER range errs if there are objects with non-overlapping type

If a SPARQL query included a numerical filter, in the presence of triples whose object value is of a numerical type that does not overlap with that range, this would signal an error at query plan time.

For example: 

SELECT * {  
  ?s ?p ?o .  
  FILTER (?o > 3 && ?o < 10)  
}

This range (3, 10) of positive integers does not overlap with type xsd:negativeInteger. If the repository contain a triple like: 

{ <ex:s> <ex:p> '-123'^^xsd:negativeInteger }

the resulting error would be: 

Error: `3` is not of type xsd:negativeInteger, exceeds the maximum: -1  
  [condition type: xsd-datatype-error]

This has been fixed

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

Bug26220 - Fedshard failure for "count distinct" with "group by"

When using FedShard, running a query that combines COUNT DISTINCT with GROUP BY could result in an error. For example: 

SELECT ?s (COUNT(DISTINCT ?o) AS ?count) {  
  ?s ?p ?o  
} GROUP BY ?s

This has been fixed.

AllegroGraph 7.0.1

AllegroGraph Server: General Changes

Rfe16481 - Increase triggered checkpoint timeout

Previously while adding a replica of a large repo on a large memory machine an error like the following could occur:

Error: instance add failed with code 400, body "failed with message: Could not trigger a checkpoint: Checkpointer process failed to respond"

This has been fixed.

Bug26176 - SHACL validation would consider shape graph triples

Under certain circumstances, SHACL validation would incorrectly consider shape graph triples for validation, resulting in reporting triples non-conforming when the data graph triples actually conformed. (Reporting triples as conforming when in fact they did not conform never occurred.) This has been fixed. SHACL and SHACL validation are described in the SHACL document.

HTTP Client

No significant changes.

SPARQL

Rfe16452 - Let GROUP BY with no result rows give zero result rows

The results of GROUP BY queries with aggregates in an edge case has been changed to closer follow the intention of the SPARQL 1.1 standard.

Take this query and assume there are no triples with predicate : 

select ?s (count(?o) AS ?count)  
where {  
  ?s <ex:p> ?o  
} group by ?s

AllegroGraph would return one result row with ?s unbound, and count zero.

An alternative behaviour that some other triple stores have, is to return zero result rows. The text of the SPARQL 1.1 standard conflicts with the "agg-empty-group2" test case, so that either behaviour is reasonable.

The current consensus is that having zero result rows was intended, and is expected by users. See e.g. https://afs.github.io/sparql-agg-group-empty.html#group-agg-no-rows

AllegroGraph has been updated to follow this consensus. So for GROUP BY queries with aggregates, in case of zero matching pattern in the body, there will be zero result rows.

Bug26190: Incorrect rewrite of OPTIONAL with non-patterns on LHS

In some cases queries of the form: 

SELECT .. {  
  VALUES ..  
  ..  
  OPTIONAL { .. }  
}

could be optimized incorrectly, resulting in missing query results. This has been fixed.

Bug26188 - Executing query failed for query with VALUES

Certain queries of the form: 

SELECT .. {  
  VALUES ?x { <a> <b> <c> }  
  .  
}

could lead to an error: 

Error: the value of ag.sbqe::index is nil,  
       which is not of type ag.sbqe::plan-index-type.

this has been fixed.

Bug26181 - Incorrect :time-total addenda value for query on remote store

When executing a SPARQL query in AGWebView the Information tab shows query execution statistics, including the time it took to plan, execute, and output query results.

It could happen that the total time would be incorrect (too low). This has been fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

Rfe16457 - improve shard-local kb support

Previously, when defining a distributed store in the cluster config file, there was limited support for shard-local kb's by including localhost in the KB declaration. That is the following declaration would define a kb on localhost and would inherit whatever scheme/port were the default at the point where the declaration was made: 

KB localhost/catalogs/tests/repositories/kb4

If these defaults match the scheme/port used by each AG shard, then the KB will be shard-local. Now, any triple-store-spec that does not include a server part can be used to define a shard-local KB. Thus 

KB tests:kb4

no longer will inherit defaults from the environment at the point of the declaration, and will inherit the scheme/port of each shard, even if scheme/port differ across all shards.

See the Distributed Repositories Using Shards and Federation Setup document.

Bug26187 - Do not track servers in KB directives

In Distributed AllegroGraph, the servers that comprise a cluster are derived from the directives that refer to servers in the agcluster.cfg file. This includes server, repo, replica, and --incorrectly-- kb directives.

The servers referred to only in KB directives are no longer considered as part of the cluster. This allows remote KBs which do not reside on a cluster host.

AllegroGraph 7.0.0

AllegroGraph Server: General Changes

agtool load changes

The (undocumented) --blank-node-cache-size argument has been removed. (Even though this argument was undocumented, some users may have been told of its existence.) Blank nodes are no longer cached during data loading.

There is a new -w/--workspace option to agtool load. The value should be an existing directory which will be used for temporary files during data loading, particularly for the temporary storage of a table of blank node information. Absent this option, temporary data is stored in /tmp and if that directory is too small, the load will fail.

agtool now supports a triple-count tool

The triple-count tool returns the number of triples in a repository. See here in the agtool document.

agraph-control now takes a --cluster-config control option

The option allows the path of the cluster configuration file to be specified. This file defines distributed repositories (see Distributed Repositories Setup). The agraph-control program is described in the agraph-control of the Server Configuration and Control document.

Adding a triple with a predicate mapping and a datetype mapping might add two triples

Before this change, when adding a triple, if both a predicate mapping and a datatype mapping applied to the object of the triple, two triples would be added. The new defined behavior is that predicate mappings take precedence over datatype mappings.

index-literals argument passed when creating/modifying a freetext index

When creating a freetext index, the index-literals argument determines if and/or under what circumstances a literal will be indexed. The value passed can be either a boolean or a list containing UPIs, future-parts, or strings.

With this change, when passing a string to index-literals, it is expected to be equivalent to the value returned when calling part->value. That is, the string should not contain any embedded angle-brackets (< or >) or double-quotes (") used to denote the string represents a resource or literal, that are not part of the actual value represented by the string.

Rfe16417 - Deprecate style 2 indices

There are two triple index styles available, "style 1" and "style 2", with different storage structures and thus performance characteristics. In practice "style 1" was generally recommended, and "style 2" has been deprecated. The latter has been removed from the documentation and might be removed in a future release.

Rfe16386 - New BriefBacktrace configuration directive.

See the [administrator note](#admin-notes-700} for more information. See Server Configuration and Control for information on configuration directives.

Rfe16364 - New LicenseWarnInAdvance configuration directive.

See the administrator note for more information. See Server Configuration and Control for information on configuration directives.

Rfe16283 - New configuration directive to control the stale DNS retain time

There is a new configuration directive StaleDNSEntryRetainTime which specifies how long AllegroGraph will retain a stale DNS entry. If the DNS mapping from name to IP address may change while the application is running you'll want to specify a very small value (number of seconds) for this directive. See StaleDNSEntryRetainTime in Server Configuration and Control.

Rfe16244 - Update PARSE-NTRIPLES to support N-Triples 1.1 blank node syntax

When importing N-Triple data, certain blank node formats with dots, dashes or underscores were unsupported. For example importing this line: 

 <http://s> <http://p> _:a-b_c.d .

would fail with: 

 Error: N-Triples parser error while parsing string at line 1  
       (last character was #\_): Illegal character

Now AllegroGraph supports the blank node syntax as defined in the N-Triples 1.1 technical recommendation.

Rfe16239 - agtool load/export/archive from/to s3

agtool can access files in Amazon S3. To do so, AWS authentication must be specified on the agtool command line. See the section Accessing and operating on files on Amazon S3 in the agtool document. Files in S3 must be prefaced by s3://, like the following: 

 s3://bucketname/a/b/c/filename

Rfe14667 - Log THP warning only (don't show in AGWV)

The THP (Transparent Hugepages) warning is now logged to agraph.log only and not the console or AGWebView.

Bug26040 - agtool archive restore to a different repo.

The documentation printed by 'agtool archive restore --help' claimed that if the archive directory contained only one repo backup then you needn't name that backup if you restored to a different repo name. This claim has been removed from the documentation because the behavior isn't supported.

Bug26017 - Literal language in rdf/xml was not properly loaded

Previously, instead of setting the language of the literal object, an additional triple with the object being the language value and the predicate http://www.w3.org/XML/1998/namespacelang was incorrectly added to the triple store.

For example, <rdfs:label xml:lang="fr">Autriche</rdfs:label> loaded as: 

<subj> <pred> "Autriche"  
<subj> <http://www.w3.org/XML/1998/namespacelang> "fr"

Bug26011 - Allegrograph licensed cores check prints wrong core count in error message

Previously if the number of CPU cores exceeded the licensed limit, AllegroGraph would generate an error message like the following:

This machine exceeds the CPU core count specified in the license. Maximum licensed cores: X. This machine has: Y.

The value of X would be the license limit but the value of Y would potentially be incorrect (it would report the number of available threads, rather than the number of cores in the machine, which will be different values if hyperthreading is enabled). This has been fixed.

Bug26000 - Fti recovery crashes if stray fti files exist from a prior instance crash

In some circumstances instance recovery could fail with an error message like the following: 

opening of database myrepo has failed (The database system tried to  
allocate a new file with the logical name fti-1-chunk-1 which already  
exists in database myrepo)

This has been fixed.

Bug25999 - Crashed instance process can leave service daemon in confused state

Under some circumstances a crashed instance process would not be automatically restarted by the service daemon. This has been fixed.

Bug25986/Bug25997 - Fix agraph-backup tool initialization issues

When the repo spec was used to specify the repository, the credentials and host values from the spec were ignored and the tool always attempted to connect to 127.0.0.1. This commend thus 

$ agtool archive backup test:xyzzy@remotehost:10035/test test.bak

could fail with a Connection refused (errno 111) message because the host is not localhost. Specifying the credentials when restoring data also could cause the tool to fail (bug25997): 

 # Fails with the message 'Invalid arguments (:params,  
 # :restore-mode-p) found while creating triple-store of class  
 # db.agraph:remote-triple-store. ...'  
 $ agtool archive restore-all test.bak test:[email protected]:10035

Both these issues have been fixed. Note that using deprecated arguments like --host and --port could cause problems if the values specified did not match the values in the REPO-SPEC, even default (and thus not-visible) values in some cases. See Older repository specification arguments in the agtool General Command Utility document for examples.

Bug25959 - Fix ACID isolation violation in some ordered cursor operations.

Fixes a bug introduced in v6.5.0 where some ordered triple cursors could produce triples that were not visible in the current transaction. The scope of the problem was limited to triples committed in concurrent transactions that had yet to be written to disk.

Bug25907 - Document how to pass args to agraph using agraph-control

The ability to use agraph-control to pass arguments to the agraph program (which is called by agraph-control) when starting or restarting AllegroGraph existed prior to this change, but was not clearly documented. How to do so is now documented in the agraph-control --help text and in the description of agraph-control in the Server Configuration and Control document.

Bug25901 - Importing triples bypassed security-filters

It was possible for users with security-filters to import triples into a db that those filters were intended to prevent.

This problem is now fixed.

Bug25880 - Remove whitespace trimming in TRIX literals parsing

To restore the behavior specified by the standard, removed whitespace trimming in the TRIX parser while parsing literals.

HTTP API

No significant changes.

SPARQL

Rfe16433 - Remove full-scan-warning

The query warning type "warn-fullscan" and the query option "fullScanWarningSize" have been removed.

In practice these warnings could occur without there being a problem with the query. It would fill agraph.log with warning lines that would be ignored, defeating the purpose.

Usage of the franzOption_fullScanWarningSize query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_fullScanWarningSize: <franz:210000000>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: fullScanWarningSize  
  [time] ...  
  ...>

Rfe16324 - Warn on ASK modifiers

An ASK query does not support LIMIT, OFFSET, or ORDER BY. Now a query warning is signaled if any of these is provided, with the exception of LIMIT 1.

Rfe16279 - Improve checking for unknown constants in the query

The check for unknown constants in the query at plan time is now more advanced, resulting in faster query execution.

For example if a resource <s> only occurs in the subject position of triples, a pattern like ?s <p> <s> will now be marked impossible at plan time.

Another example are references to standard IRIs that are present in every repository automatically, like <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>. Previously a pattern like ?s a ?foo would never be marked impossible, but now if there are no triples with such predicate, the pattern will be marked impossible.

This also has effect on magic properties like <http://franz.com/ns/allegrograph/3.0/temporal/intervalBefore> that under the hood expand into patterns with properties like <t:starttime>, <t:startpoint> and <t:time> (with http://franz.com/ns/allegrograph/3.0/temporal/ as t). If a repository only uses starttime, and never the startpoint/time combination, the plan will be optimized accordingly.

Bug26144 - CAAT ignores :pattern-filters of query patterns

Queries executed with the Chunk-at-at-Time (CAAT) strategy (so with a LIMIT, without DISTINCT or aggregators) could give incorrect results, if a specific query pattern was planned to act as filter for another pattern. This has been fixed.

Bug26143 - Graph <g> {} gives error

A query with an empty named graph clause, like: 

select ?s ?o {  
  graph <ex:g> {}  
  ?s ?p ?o .  
}

would fail with an error: Attempt to access the name field of {g} which is not a symbol. This is now fixed.

The meaning of graph <ex:g> {} as a pattern, is that some triple must exist in the given fixed graph <ex:g>. The meaning of graph ?g {} is that some triple must exist in any named graph.

Bug26137 - Spurious unknown-variables warning for VALUES in inner SELECT

A VALUES clause inside an inner SELECT, like in this query: 

SELECT ?region {  
  SELECT ?region WHERE {  
     VALUES ?type { <ex:Region> }  
     ?region a ?type .  
   }  
 }

would give a spurious warning about the VALUES variable: 

unknown-variables: Unknown variable used in VALUES clause: ?type_1_1

This is now fixed, so that there will be no more warnings.

Bug26116 - FILTER CONTAINS LCASE gives UPI-NOT-IN-STRING-TABLE-ERROR

Certain SPARQL expressions that create intermediate strings, like: 

select ?s {  
  ?s <ex:p> ?o .  
  filter ( contains(lcase(?o), 'something'))  
}

could signal the error: The UPI #(...)is not present in the repository's string table.

This has been fixed.

Bug26093 - GROUP_CONCAT interprets separator arg as format string

If a SEPARATOR argument containing a tilde was supplied to the GROUP_CONCAT aggregator, the resulting string could be incorrect. This has been fixed.

Bug26090 - Boolean value serialized as sparql-json lacks datatype

A boolean value serialized in sparql-json format would look like (here we use the Lisp function sparql:run-sparql but the results would be the same if the query was executed in AGWebView and exported in sparql-json format): 

(sparql:run-sparql "select * { bind(true as ?x) }"  
   :results-format :sparql-json)  
 
-> {"head": {"vars": ["x"]},  
    "results": {"bindings": [{"x": {"type": "literal", "value": "true"}}]}}

But the correct way to serialize a boolean is to include the datatype. Thus this: 

"x": {"type": "literal", "value": "true"}`

is incorrect and should instead be: 

"x": {"type": "literal",  
      "datatype": "http://www.w3.org/2001/XMLSchema#boolean",  
      "value": "true"}

This has been corrected.

Bug26078 - SPARQL FILTER on numeric range excludes UPI types that partially overlap

If a SPARQL query FILTER expression restricts a triple part to a numeric range, and if there is a triple whose part value falls in that range but is of a data type such that the filter range exceeds the data type range, then the triple would not pass the filter, which was incorrect.

For example: 

(add-triple "<ex:s>" "<ex:p>" (value->upi 127 :byte))

and query: 

select * {  
  ?s ?p ?o .  
  filter ((?o > 120) && (?o < 130)  
}

The query FILTER restricts the triple object to (120, 130). The ?o value 127 should pass this filter. The value 127 is of type :byte, which has range [-128, 127].

The incorrect behaviour is that the FILTER range (up to 130) exceeds the data type range (up to 127) and therefore the value 127 was incorrectly filtered out, meaning the triple would not be returned by this query.

This has been fixed.

Bug26071 - COUNT aggregate on unknown variable should return 0.

If a query aggregates a variable that is not otherwise used, like: 

SELECT (COUNT(?bogus) AS ?count) { ?s ?p ?o }

this would be evaluated as if the unknown variable ?bogus was replaced by true. The value of ?count would be the total triple count.

This has been corrected, so that the returned ?count is now zero.

Bug26020 - Invalid OPTIONAL reordering

If a query has an OPTIONAL pattern that binds a variable, followed by a non-OPTIONAL pattern that references that variable, like: 

SELECT * {  
  ?s <ex:p1> ?o .    # (1)  
  OPTIONAL {  
    ?t <ex:p2> ?o2 . # (2) possibly binds ?o2  
  }  
  ?s <ex:p3> ?o2     # (3) references (if bound already) or binds ?o2  
}

the query engine could create an incorrect query plan with evaluation order (1) (3) (2). The evaluation order of (2) and (3) would be reversed: variable ?o2 would be bound based on pattern (3). This is now corrected to have evaluation order (1) (2) (3).

Bug26005 - Incorrect SUM and GROUP_CONCAT default values for empty group

If a SPARQL query has no result rows (or if GROUP BY is used, and a specific group has no result rows) a SUM aggregation and a GROUP_CONCAT aggregation would both return an unbound value.

This was incorrect according to the SPARQL 1.1 spec. Now SUM will return 0, and GROUP_CONCAT will return the empty string.

Bug26004 - Queries that have a constantly true HAVING could fail.

A constantly true HAVING expression in a query, like: 

SELECT ?s (max(?o) as ?maxO) {  
  ?s ?p ?o  
} GROUP BY ?s  
HAVING (3 < 4)

could err with Received signal number 7 (Bus error). This has been fixed.

Bug25999 - Crashed instance process can leave service daemon in confused state

Under some circumstances a crashed instance process would not be automatically restarted by the service daemon. This has been fixed.

Bug25988 - SPARQL function REGEX should err on non-string

Previously AllegroGraph was liberal in accepting a constant resource as REGEX pattern. For example: 

SELECT (regex('ex:foo', <ex:foo>) as ?match) {}

would return "true".

This has been fixed in order to adhere to the SPARQL specification: if the target or pattern is not a string, a type-error is signaled. The above SELECT now returns an unbound value.

Bug25984 - SPARQL function LANGMATCHES should err on non-string

Previously AllegroGraph was liberal in accepting non-strings to function LANGMATCHES and converting them using STR.

For example: 

BASE <http://foo/>  
SELECT (langMatches(<ex>, <ex>) as ?a)  
   (langMatches(123, 123) as ?b) {}

was interpreted as: 

BASE <http://foo/>  
SELECT (langMatches( STR(<ex>), STR(<ex>)) as ?a)  
   (langMatches( STR(123), STR(123)) as ?b) {}

and would return "true" twice.

This has been fixed in order to adhere to the SPARQL specification: if either argument to langMatches() is not a string, the result is a type-error. The above SELECT now returns two unbound values.

Bug25982: Fix HAVING constraint on implicit single group without aggregates

If a query uses HAVING without using GROUP BY, and no aggregates (like MIN, COUNT), as for example in: 

SELECT * { ?s ?p ?o } HAVING (?s = <ex:s>)

then the HAVING clause acts like FILTER: 

SELECT ?s ?p ?o { ?s ?p ?o . FILTER (?s = <ex:s>) }

However in the first query AllegroGraph would incorrectly ignore the HAVING clause, meaning results that should have been filtered out were returned. This is now corrected.

Bug25941 - Blank node patterns not supported in subject position of BGP triples

A query pattern with a blank node pattern as subject, and a property path, like: 

select * {  
  [ <ex:p2> ?o2 ]  ( <ex:p1> | <ex:p2> )  ?o  
}

would give a parse error at the "(" character. This has been fixed.

Bug25936 - Inefficient join with UNION

The query execution planning for certain queries containing UNION has been improved, leading to faster execution. This impacts also magic properties that are implemented using UNION, like <t:intervalBefore>.

Bug25934 - Problem with BIND (EXISTS { pattern1, pattern2 } as ?x)

If a query binds a variable to an EXISTS expression, where the EXISTS contains multiple patterns, like: 

SELECT ?person ?known where {  
  ?person a <ex:Person> .  
  BIND ( EXISTS {  
           ?employee a <ex:Employee> ;  
                     <ex:knows> ?person  
         } as ?known )  
}

then the outcome could be incorrect: the second and later patterns could be ignored, or the query could result in an error like: 

Error: Bad variables for set: (?person).

This is now fixed.

Bug25933 - Duplicate patterns in blank node + property path expansion

For query triple patterns with a property path, and a blank node as object, like: 

?s  ^<ex:p>  [ <ex:p1> <ex:foo> ;  
               <ex:p2> <ex:bar> ]

the expansion into simple triple patterns could contain duplicates: 

1. ?bnode_1 ex:p ?s  
2. ?bnode_1 ex:p ?s  (duplicate)  
3. ?bnode_1 ex:p2 ex:bar  
4. ?bnode_1 ex:p1 ex:foo

The duplicated pattern resulted in superfluous work during query execution. This is now fixed.

Bug25931 - Sending SPARQL query with typed booleans to remote-triple-stores fails

Making SPARQL queries with typed booleans (e.g. true"^^xsd:boolean) would fail with an error indicating that AllegroGraph is unable to convert a boolean UPI into a future part.

This is now fixed

Bug25925 - SPARQL MAX does not handle unbound values correctly

The SPARQL MAX aggregator would return an unbound value as result, if any of the values to aggregate was unbound. This has been fixed.

Bug25920 - SPARQL SERVICE using https without chunking errs

When a SPARQL query was sent to remote SPARQL endpoint that uses HTTPS, and the endpoint sent results not using chunked encoding, then an error would be signaled: 

Error: the value of stream is #<excl::ssl-client-stream ..>,  
       which is not of type  
 (or net.aserve::unchunking-stream util.zip:inflate-stream  
           excl::hiper-socket-stream).  
  [condition type: type-error]

This could happen in a SERVICE <https://...> clause, or by proving a :HOST "https://.." argument to function run-sparql.

This has been fixed.

Bug25919 - SPARQL function STRLEN should err on non-string

Previously AllegroGraph was liberal in accepting non-strings to function STRLEN and converting them using STR.

For example: 

SELECT (STRLEN (<http://franz.com>) as ?s)  
       (STRLEN (123.45) as ?t) {}

was interpreted as: 

SELECT (STRLEN (STR (<http://franz.com>)) as ?s)  
       (STRLEN (STR(123.45)) as ?t) {}

and would return values 16 and 6.

This has been fixed in order to adhere to the SPARQL specification: taking the length of a non-string now gives a type-error. The above SELECT now returns unbound values.

Bug25916 - COUNT DISTINCT of expressions gives error

A query for COUNT DISTINCT where the value is not a variable (like ?o) but an expression (like "strlen(?o)"), for example: 

select (count(distinct(strlen(?o))) as ?olen) {  
  ?s ?p ?o  
}

used to signal an error: 

Received signal number 11 (Segmentation fault)

These queries are now supported.

Bug25898 - SPARQL error for property multimatch with filtered object range

Queries with a property path with alternatives, that have a filter on the object variable, where the object values are of a type that is hashed to a UPI, like: 

select * {  
  ?s ( <ex:p1> | <ex:p2> ) ?o .  
  filter (lang(?o) = 'en')  
}

could result in an error like: 

Error: The UPI #(0 0 0 0 0 0 0 0 0 0 0 3) (which is a literal-language)  
       is not present in the triple-store's string dictionary.

This has been fixed.

Bug25821 - SPARQL function SUBSTR should clip LENGTH to string length

If in SUBSTR the length argument is larger than then the string's length, this would result in an error: 

SELECT * { bind (substr('abc', 1, 100) as ?s) }  
 
Executing query failed:  
In subseq, `end' (100) is beyond the end of the sequence (3).

This is now fixed, so that the substring up to the end of the string is returned. In this case: abc.

Bug25806 - Fix ORDER BY on non-projected variable

For certain SPARQL SELECT queries that use ORDER BY on a non-projected variable, the results could be ordered incorrectly. This has been fixed.

Bug25694 - Fix simplifying SPARQL= into UPI=

It was possible for a SPARQL equality filter: 

FILTER (?a = ?b)

to return false if the values are of different types. For example, if ?a is the integer 2 and ?b the float 2.0. This has been fixed.

Bug25636 - Triple cursor FILTER containing [NOT] EXISTS applied incorrectly

If a query contains a FILTER clause that uses EXISTS or NOT EXISTS, there were cases where that FILTER was evaluated incorrectly and as a result the query would return incorrect results.

Concretely, if in the query plan the filter was integrated in the first triple pattern, it was possible for EXISTS to return true in certain cases where false was the correct answer; and for NOT EXISTS to return false where true was the correct answer.

This has been fixed.

Bug25585 - Support ORDER BY on an aggregate expression

Queries in which the ORDER BY expression refer to an aggregation, for example: 

SELECT ?s (MIN(?o) as ?minO) {  
  ?s ?p ?o  
}  
GROUP BY ?s  
ORDER BY (min(?o))

would fail with the error: 

:aggregate fell through a etypecase form

This is now fixed.

Bug24815 - Fix MINUS with an empty row on the RHS

A query that uses MINUS, where the second BGP contains only unbound values, like: 

SELECT ?x {  
  { VALUES (?x) { (1) (2) (3) (4) } }  
  MINUS  
  { VALUES (?x) { (UNDEF) }  
  }  
}

should have 4 results, namely ?x bound to each integer.

Previously AllegroGraph would return no bindings. This is now corrected.

AGWebView

Bug26010 - Fixed Delete statement confirmation dialog

The Delete statement confirmation dialog in AGWebView is now fixed. Instead of asking for the confirmation, if showed the message: function () { [native code] } and did not allow to confirm the operation. Now, the confirmation dialog is displayed properly and, if the operation is confirmed, it will complete correctly.

Bug25947 - AGWebView doesn't allow all session spec characters

In 6.6.0 AGWebView was too restrictive on the characters allowed in the Session Specification edit box on the main catalog page. This has been fixed.

Bug25856 - Back-up this repository on WebView repo page does not work

Fixed the inner API for remote backup request with target on the server side file system. Fixed the dialog to correctly describe what is expected from user.

Changes to the Lisp API

Improve Materializer support for remote-triple-stores

The keyword arguments :delete-first? and :delete-duplicates-first? are now supported when calling materialize-entailed-triples on a remote triple store via the AllegroGraph Lisp client, and can also be specified via the REST API as well.

Rfe16211 - add IF-DOES-NOT-EXIST parameter to OPEN-TRIPLE-STORE

This change extends the open-triple-store function with an :if-does-not-exist argument. The following values are supported:

Rfe12356 - HTTP proxy support for AllegroGraph Lisp client

New :proxy parameter for create-triple-store, open-triple-store and triple-store-exists-p functions of the AllegroGraph Lisp client API allow to specify an HTTP proxy to be used for all HTTP requests issued by client. Proxy settings can be specified in the form 

[<user>[:<password>]@]<host>[<port>]

Alternatively, components of the proxy specification can be provided explicitly by :proxy-user, :proxy-password :proxy-host :proxy-port parameters. Additionally, the list of domain name suffix strings, for which the proxy should be bypassed, can be specified via :proxy-bypass-list parameter.

Bug25974 - load-ntriples doesn't work with namestring source on remote-triple-store

Previously calling load-ntriples (or any of the other load-* functions) would fail for remote-triple-stores if the source argument was a string naming a file instead of a pathname. This has been fixed.

Prolog

No significant changes.

Documentation

rfe16275 - Storage and memory document for AllegroGraph

There is a new Memory Usage document which discusses memory and disk space requirements when using AllegroGraph.

nD Geospatial

No significant changes.

AllegroGraph 6.6.0

AllegroGraph Server: General Changes

Rfe16127 - Make AllegroGraph's scripting model permissions more strict

Superuser permission is now required to upload, list, view, or delete both sitescripts and reposcripts.

Additionally, reposcripts can no longer be loaded into shared backends.

Rfe16096 - Simplify AllegroGraph script naming.

Scripts no longer support hierarchical naming, in favor of a simple flat namespace. For example, when uploading a script: 

Legal:   http://example.com:10035/scripts/sc.cl  
Illegal: http://example.com:10035/scripts/com/app/module/sc.cl

Rfe16093 - Incoming IP restrictions mechanism for AllegroGraph

New AllowIP configuration directive provides a way to restrict access to AG instance to certain ranges of IP addresses.

Rfe15992 - Add MaximumBackupAge parameter to the agraph.cfg file

One can specify the maximum age in seconds that a backup of the multi-master controlling instance is considered acceptable to use to grow the cluster. See the Instance Settings section of the Multi-master Replication document and also the Top-level directives for multi-master replication clusters section of the Server Configuration and Control document).

Rfe15886, Rfe15470 - UI Improvements to the triple attribute pages.

Merges the 'Define Attribute' and 'Define Static Filter' pages into a single 'Triple Attributes' page that is more streamlined and user friendly. See the Attributes example in the AGWebView documentation.

Bug25833 - Using CorsAllowRegex would break session/backend creation

Setting the CorsAllowRegex directive (see Server Configuration and Control) in your agraph.cfg would cause the creation of dedicated sessions and shared backends to break with a confusing error message in the agraph.log file.

Bug25832 - Only compare CorsAllowOrigin directive against Origin host

Previously, each value specified by a CorsAllowOrigin directive was matched against the full value of the Origin header: scheme, host, and port. Now, each CorsAllowOrigin directive is matched only against the host, per the documentation.

Bug25812 - agtool query --standard-namespaces

If agtool query is given the argument '--standard-namespaces no' then the AllegroGraph set of standard namespace PREFIX definitions will not be prepended to the text of query. Furthermore if the query is going to an AllegroGraph repository that agtool will open itself then the set of namespaces known to AllegroGraph is cleared before the query is run.

Bug25805 - Fix bug in SNA path-finding functions used with remote-triple-stores.

SNA path-finding functions like depth-first-search and all-bidirectional-search-paths would fail when called on a remote-triple-store if either the START or END arguments represented literals. This is now fixed.

Bug25801 - CSV import hangs on surrogate pairs

The CSV importer has been fixed to work with UTF8 surrogate pairs.

Bug25784 - Agtool archive failed when using an https spec.

Trying to use agtool archive with an https spec would fail due to the tool attempting to make regular http requests to the ssl port. This problem is now fixed.

Bug25736 - improved handling of GRAPH in JSON-LD

A JSON-LD error is now signaled when a string which is neither a resource nor URL is passed to as the --graph argument to agtool load. :root is supported as a special value of GRAPH, resulting in using the toplevel subject as a default graph for all triples added for the JSON-LD object with that subject.

Bug25382 - Fix security flaw in AllegroGraph scripting.

Fixes a bug, introduced in v6.3.0, that allowed any file on a filesystem accessible to AllegroGraph to be loaded as a script. Scripts are now only searched for and loaded from their expected locations relative to the SettingsDir directive.

HTTP Client

Bug25820 - CORS does not work via HTTPS

Previously CORS configuration options were ignored for accesses to AllegroGraph made via HTTPS. This has been corrected.

SPARQL

Rfe16157 - Implement logging of slow queries

A new configuration directive SlowQueryLogThreshold, if set, causes SPARQL queries that have total run time in milliseconds greater or equal to the value, to be logged to agraph.log or to a log file specified by SlowQueryLogFile configuration directive.

Bug25831 - Fix optimization of GRAPH with a non-existant value in SPARQL

The query: 

select ?s ?p ?o  
{  
  VALUES ?s { <http://s> }  
  VALUES ?p { <http://p> }  
  GRAPH <http://non-existing> {  
OPTIONAL {  
  ?s ?p ?o  
}  
  }  
}

should return 1 result row with ?s, ?p bound and ?o unbound.

However due to an incorrect optimization of unreachable GRAPH clauses there would be no results. This has been fixed.

Bug25803 - Fix augmenting pattern with two IRI equivalence constraints

SPARQL queries that bind a variable multiple times, like: 

SELECT * {  
  BIND (<http://ex/p2> as ?p)  
  {  
BIND (<http://ex/p1> as ?p)  
?s ?p ?o  
  }  
}

could be optimized incorrectly, leading to the error message: 

Executing query failed:  
 nil is not of the expected type `number'

This has been fixed.

Bug25802 - Join query rewriters lose filters

Certain SPARQL queries where a join happens, and one side contains BIND and FILTER, like: 

SELECT ... {  
  ?s ?p1 ?o1  
  {  
    BIND (<http://p2> as ?p2)  
    ?s ?p2 ?o2  
    FILTER (.. ?o2 ..)  
  }  
}

could be optimized wrongly in that the FILTER would be dropped. This has been fixed.

Bug25794 - BIND variables disappear in join

Certain SPARQL queries where a variable was used in a filter and also projected could return results in which that variable was unbound due to an incorrect query optimization. For example: 

SELECT * {  
  {  
    bind (4 as ?four)  
    ?s ?p ?o .  
    filter (?o != ?four)  
  }  
  {  
    bind (3 as ?three)  
    ?s ?p ?o .  
    filter (?o != ?three)  
  }  
}

could return rows with ?three and ?four unbound. This has been fixed.

Bug25761 - Whitespace symbols are not escaped properly when writing in TSV format

According to the TSV SPARQL standard, when serializing RDF terms some whitespace symbols like tabs, newlines and carriage returns are encoded in strings as \t, \n and \r respectively. This has not been the case, as these symbols were serialized as-is, leading to some parsing problems later, specifically a tab character being treated as separator for TSV format. This has been corrected.

Bug25756 - Unused variable in complex property path gives errors

Queries of the form: 

select ?o {  
  ?s ( ^<http://foo> | <http://bar> ) ?o  
}

in which the property path was not very simple, and also one variable is not otherwise used (here ?s), could lead to an error during execution. This has been fixed.

Bug25752 - Fix missing results for DISTINCT + LIMIT

There could be missing results for queries that uses both DISTINCT and LIMIT due to the DISTINCT condition being applied incorrectly. This has been fixed.

Bug25749 - Filter (str(?var) = '..') marked as impossible

If a SPARQL query had filter pattern comparing a STR() value to a literal string, like: 

filter ('http://g' = str(?g))

this would result in no results and the warning: 

 Warning: `?g` cannot be a literal so the FILTER ?g is one of 1 thing will always fail

This has been fixed.

Bug25714 - SPARQL IRI() and URI() functions must return valid, absolute results

Previously, the SPARQL IRI() and URI() functions could return IRIs and URIs that were syntactically invalid and/or relative. This has been corrected. AllegroGraph will now signal SPARQL type errors which will be treated as UNBOUND for assignment and as FALSE for FILTER evaluation.

Bug22350 - Fix missing results when using CAAT with DISTINCT

When a SPARQL query containing DISTINCT or REDUCED was executed using the chunk-at-a-time strategy, there could be results missing due to incorrect handling of intermediate query results. This has been fixed.

AGWebView

Rfe16012 - Don't show AllegroGraph Utilities menu unless logged in.

When not logged in, AGWebview displayed the Utilities menu even though none of its menu items work without being logged in. Now, the menu is not presented until a user is logged in.

Changes to the Lisp API

Rfe16071 - Implement breadth-first-search-if functionality

The four breadth-first search SNA functions:

have all been enhanced such that the end argument can be a function of one parameter. If end is such a function, then the search code will call it on each node found during the search and will record a successful search if the function returns true.

Bug25759 - Change load-trix :default-graph argument to :graph

To ensure consistency with other load- functions, :default-graph argument to load-trix has been deprecated and will be removed in the subsequent release. :graph should be used instead.

Bug25738 - AGWebView users are notified when query results may be stale

When reloading queries via the 'Saved' or 'Recent' menus--but also when executing new queries--AGWV will now report when there's a possibility the results displayed are out of date.

Prolog

No changes

Documentation

No changes

User-visible changes in earlier releases

See Change History for user-visible changes to earlier releases.