Introduction

This document lists all changes in the most recent major release, which is 7.2.0.

See here for links to release notes for recent earlier versions.

Change History repeats the list of user-visible modifications for this release and includes similar lists for all earlier releases.

Release 7.2.0

Release 7.2.0 is a major release with important new features.

Database internal format is unchanged from version the previous version (7.1.x)

Release 6.4.6 had a different internal format. All releases since then have the same internal format. This means version 6.5.0 through 7.1.0 repositories can be directly loaded into version 7.2.0 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Version 7.2.0 Admin notes

This sections notes changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

Version 7.2.0 Programmer notes

This section notes changes which may affect how certain code or operations work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

Version 7.2.0 user visible changes

This section lists changes in the current release. See the Change History for a list of changes in all earlier releases.

AllegroGraph Server: General Changes

Some TopBraid Composer (TBC) examples no longer work

The document TopBraid Composer Plugin has examples showing how you can connect TBC to an AllegroGraph repository and then directly add triples created by TBC. This worked by using the Sesame2 Remote Repository export format from TBC. Recent versions of TBC (which is now at version 7.x) no longer support that format. To copy triples from TBC to AllegroGraph you must save the triples to a file and load the file into AllegroGraph. The TopBraid Composer Plugin document has been updated to not this change.

New agtool vload command loads materialized triples from a relational database

See agtool and the Creating Virtualized Graphs document for more information. There is an example using the Ontop utility.

New data streaming example

See Graph streaming for an example connecting AllegroGraph a data streaming tool such as Apache Kafka.

New example using Apache Spark

An example of graph analytics using AllegroGraph and Apache Spark can be found here in the franzinc/agraph-examples repo. See the Examples on Github document, which includes examples using Apache Spark, Apache Kafka, Kubernetes, Terraform, load balancing with NGINX, and so on.

New agtool commands query-options and namespaces

The command agtool query-options can be used to add, remove, and list default query options for the user and repo specified in the command line. The command agtool namespaces can be used to add, remove, and list default user namespaces for the user and repo specified in the command line.

New --port and --sslport control options to agraph-control

The agraph-control program now accept --port and --sslport control options, which, if specified, override the values specified for Port and SSLPort in the AllegroGraph configuration file (usually named agraph.cfg). This is convenient if the values specified in the configuration file are already in use. The agraph-control is documented here in the Server Configuration and Control document. The associated agraph also now accepts --port and --sslport arguments.

Handling of numbers in JSON-LD code fixed

Things that looked like numbers in JSON-LD code, including the string ".", were converted to typed numbers, even when that was not wanted. Strings "true" and "false" were also always converted to booleans. This has been fixed. Conversion is now performed only when the appropriate datatype is explicitly specified.

New storage-report option to agtool

The tool shows storage used by a repository. See here in the agtool General Command Utility document. The report includes, along with other information, the percentage of triples stored but deleted.

The Table in the Cluster Managment page modified

The table is illustrated in the Multi-master Replication document. The table now shows the host and port of each instance.

The maximum number of unique strings has increased

From 2^32 to 2^39. (The hash table in shared memory used for locating strings doubled in size.)

Rfe16711 - implement query-results-limit user access flag

Users can now be restricted to a maximum number of results from a query. See Limiting results of a query in SPARQL Reference and Limiting the results a user can see in Managing Users. Also see the new QueryResultsLimit configuration directive.

Rfe16709 - XML 1.1 support for rdf serializer

Make RDF serializer output XML 1.1 by default. Previously AllegroGraph wrote version 1.0 XML which is unable to express all of the characters you might find in repository data. The new XMLVersion configuration option can be used to revert back to XML 1.0 if needed.

Rfe16691 - New agtool for automating repository optimization

See the Release Notes description. Also see agtool auto optimize, or AGWebView.

Rfe16639 - Rename agtool command token to tokens for consistency

The agtool command tree token has been renamed to tokens for consistency with other similar commands. The old token name will still work to ensure backward compatibility but it is deprecated and excluded from the help text.

Rfe16562: Support limiting agtool query output to results only

The output of agtool query can now be limited to results only by specifying the --quiet option. When --quiet is set, all log output will be written to the agquery.log file. See agtool query for more information.

Rfe16540 - Add support for creating, deleting, and listing repos and dynamic catalogs with agtool

agtool has new repos and catalogs tools. They can be used, respectively, to create and delete repos (including replicated repos) and dynamic catalogs. The tools can also list existing repos and catalogs. See the agtool document for more information.

Bug26575 - SHACL implicit class targets only worked for rdfs:Class

If a shape is also declared to be a class in the shapes graph then all SHACL instances of this class are a target for the shape (see the SHACL spec here). This only worked if the shape had a rdfs:Class type and failed if it was a subclass of rdfs:Class e.g. owl:Class. This has been fixed.

Bug26473 - Decimal production in SPARQL 1.1

DECIMAL production now uses the SPARQL 1.1 definition, meaning that at least one digit after the dot is required. Previously 1. was read as the DECIMAL 1.0, instead of being correctly read as the integer 1 followed by a period, just like 

1 .

This is an incompatible change. The SPARQL 1.0 rule is specified here and the changed SPARQL 1.1 rule is here.

Bug26414: Fix default query memory limit initialization

Default query memory was incorrectly initialized to the 13.3 GB instead of 85% of available system memory, as specified in the documentation. Because of this in some cases the users needed to increase the memory limit manually via the query option 

PREFIX franzOption_memoryLimit: <franz:NEW_LIMIT>

where NEW_LIMIT is either a number (of bytes) or a value of the form <number><unit> (e.g. 32G). This has been fixed so default query memory will start at 85% of available system memory.

Bug26221 - Speedup Freetext Index (FTI) phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Rfe16528 - Remove reorderDuringExecution query option

In practice the pattern estimates are good enough to not have to consider reordering. Using the special query prefix: 

PREFIX franzOption_reorderDuringExecution: <franz:yes>

will now give a query warning: 

obsoleteQueryOption: reorderDuringExecution

Rfe16510 - Support for SPARQL*

AllegrGraph now supports queries that use the SPARQL* syntax, like: 

select * {  
  << ?s ?p ?o >> <ex:key> ?value  
}  
 
select * {  
  bind( << ?s <ex:pred> ?o >> as ?triple )  
}

A reified triple is represented as an IRI that looks like: <http://franz.com/ns/allegrograph/4.0/tripleId#123> which means isIRI(?triple) will return true for reified triples.

There is a new SPARQL filter function: isTriple(?triple) that returns true only if ?triple is a reified triple.

SPARQL* queries are not yet supported on federated or distributed repositories.

Rfe16504: Remove "enableDuplicateReduction" query option

The SPARQL query option enableDuplicateReduction, if set to true, would behave somewhat like specifying reduced. In all cases where it would be relevant to enable this reduction, it is better to specify reduced or distinct in the query. Therefore the setting is removed.

Usage of the franzOption_enableDuplicateReduction query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_enableDuplicateReduction: <franz:yes>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: enableDuplicateReduction  
  [time] ...  
  ...>

Bug26558 - Fix zero-length property paths with non-existent S/O parts

According to the SPARQL specification, assuming an empty data set, all of these queries should return the same single result :s1: 

# query 1  
SELECT ?s {  
  ?s :p1* :s1  
}  
 
# Query 2  
SELECT ?s {  
  VALUES ?s { :s1 }  
  ?s :p1* :s1.  
}  
 
# Query 3  
SELECT ?s {  
  ?s :p1* :s1.  
  FILTER (?s = :s1)  
}

but only the first one returns the correct result, while all others return an empty result set because of a premature optimization. This has been fixed.

Bug26557 - Remove odd zero-length property paths matches on MJQE

MJQE used to return additional results for zero length paths when either subject or object of the pattern was fixed. For example, for the data set 

:s1 :p1 [ :p2 :o1 ].  
:s2 :p3 :o2.

the query 

SELECT ?o {  
  VALUES ?o { :o2 }  
  :s2 (:p1|:p2)* ?o  
}

returned 

-------  
| o   |  
=======  
| :s2 |  
-------

which is only correct when the object of the pattern is not fixed. This has been resolved.

Bug26532 - Query canceling does not work for queries run on MJQE

It was impossible to cancel a query that was being executed on MJQE because of the incorrect job initialization. For example, pressing the Cancel button in WebView had no effect on the MJQE query currently in progress. This has been fixed.

Bug26531 - Fix total query time reporting for queries run via HTTP

When running SPARQL queries through HTTP endpoint (for example, in WebView), the query information sent in x-query-final-info trailer contained incorrect total query time that was approximately twice as big as the real total time. This has been fixed.

Bug26525 - Format selection fails for CONSTRUCT queries run with agtool

When executing queries with agtool query, the --results-format argument was not working properly for DESCRIBE and CONSTRUCT queries with the keywords in upper case. This has been fixed.

Bug26515 - Restore reorderDuringExecution query option

The option was removed in v7.1.0 but has now been restored. See Query execution options in the SPARQL Reference document.

Bug26493 - Queries with magic properties may return less results

It was possible for some queries with magic properties and fixed values to return less results because of an incorrect optimization, For example, for the following dataset 

@prefix : <http://example.org#>.  
:alice :owns "Galaxy Note".  
:bob   :owns "Galaxy Note".

the following query 

PREFIX : <http://example.org#>  
PREFIX fti: <http://franz.com/ns/allegrograph/2.2/textindex/>  
SELECT ?p ?o WHERE {  
  (:bob ?o ?p) fti:match "galaxy".  
}

will return an empty result, while the query without the fixed constant 

PREFIX : <http://example.org#>  
SELECT ?s ?p ?o WHERE {  
  (?s ?o ?p) fti:match "galaxy".  
}

returns both results, as expected: 

--------------------------------  
| s      | p     | o           |  
================================  
| :alice | :owns | Galaxy Note |  
| :bob   | :owns | Galaxy Note |  
--------------------------------

This has been fixed.

Bug26460: Empty RDF graph was returned by SPARQL's DESCRIBE * WHERE ...

The following two SPARQL queries are expected to be equivalent 

DESCRIBE * WHERE { ?s ?p ?o . }  
 
DESCRIBE ?s ?p ?o WHERE { ?s ?p ?o . }

but the first one (with the wildcard *) used to return an empty result. This has been fixed.

Bug26452 - MJQE may omit bindings introduced by SPARQL VALUES clause

When more then one variable in a pattern has bindings introduced by the VALUES clause, MJQE engine may omit the bindings for one of the variables.

For example, assuming the data 

:a :p1 :b.  
:a :p2 :c.  
:b :p2 :c.

the following query 

SELECT ?s ?p ?o WHERE {  
  VALUES ?s { :a }  
  VALUES ?p { :p1 :p2 }  
  ?s ?p ?o.  
}

used to return the following result 

------------------  
| s   | p   | o  |  
==================  
| --- | :p1 | :b |  
| --- | :p2 | :c |  
------------------

when run on MJQE. This has been fixed.

Bug26451 - Fix occasional errors in queries on security-filtered stores

When security filters were used, it was possible for some queries to fail with an error message of the form 

QUERY FAILED: `#<db.agraph.cursor::filtered-cursor ...>' is not  
of the expected type `number'

indicating an internal type error. This has been fixed.

Bug26436 - Fix missing results for multiple property path matches

In cases when there are multiple zero-or-more paths between the subject and object in the restricted pattern, the query engine used to drop the result with multiple paths, for example for the dataset 

:s1 :p1 :o1.  
:s2 :p1 :o1.  
 
:s1 :p2 [ :p2 :o2 ].  
:s2 :p3 [ :p2 :o2 ].  
:s2 :p2 [ :p3 :o2 ].

the query 

SELECT ?s {  
  # this restricts the pattern in FILTER EXISTS  
  ?s :p1 :o1.  
  # this ends up return less results  
  FILTER EXISTS { ?s (:p2|:p3)* :o2 }  
}

used to return only one result :s1, while it should have returned :s1, :s2. This has been fixed.

Bug26333 - Property paths that join bindings may produce bogus results

It was possible for the SPARQL sequence property paths to produce too many results when joining bindings from VALUES clauses or subqueries. This has been fixed.

Bug26290 - Max when some variable is unbound, must give unbound result value

In AllegroGraph 6.7.0 a SPARQL regression was introduced, where SPARQL MAX aggregator would treat unbound values as bound values (lowest in ordering) and so return a bound value as result.

This was incorrect: when aggregating over an unbound value, the aggregation result should be an unbound value due to a SPARQL type error in the aggregation.

This has been fixed, so that the aggregation result is the unbound value in the described case.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

Bug26260 - Fix spurious "emptyBindClause" query warnings

SPARQL queries with a BIND clause at the start of a BGP could generate a spurious "emptyBindClause" warnings saying: 

The BINDING form for ?val cannot succeed

This can be an incorrect warning if the BIND expression does a special operation that can deal with unbound variables, like BOUND, COALESCE, IN or IF.

Examples: 

select * {  
  bind(bound(?o) as ?val) .  
}  
-> ?val = false  
 
select * {  
  bind(coalesce(?o,1,2) as ?val) .  
}  
-> val = 1

This is now fixed so that no spurious warnings are generated.

AGWebView

Bug26332: Broken bulk namespace input dialog in AGWV

The Add a namespace button in query editor shows a dialog that has a Bulk Input button which in turn should be showing a different dialog that allows to input multiple namespace declarations at once. This dialog was broken and an empty dialog box was shown instead. This has been fixed. See the New Query Page section of the WebView document for more information on the Add a namespace button.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

There is a new AllegroGraph examples document.

This document has links to example locations, either part of the distribution or on our franzinc/agraph-examples repo on Github.

Distributed AllegroGraph

No significant changes.

User-visible changes in earlier releases

See Recent Version Release Notes for changes in recent release and Change History for user-visible changes to all earlier releases.