Introduction

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 7.1.0.

Release 7.1.0

Release 7.1.0 is a major release with important new features.

Database internal format is unchanged from version 7.0.4

This means version 7.0.4 repositories can be directly loaded into version 7.1.0 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 7.0.4

Release 7.0.4 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Database internal format is unchanged from version 7.0.3

This means version 7.0.3 repositories can be directly loaded into version 7.0.4 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 7.0.3

Release 7.0.3 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Database internal format is unchanged from version 7.0.2

This means version 7.0.2 repositories can be directly loaded into version 7.0.3 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 7.0.2

Release 7.0.2 is a maintenance release fixing bugs and providing enhancements. There are no major new features.

Database internal format is unchanged from version 7.0.1

This means version 7.0.1 repositories can be directly loaded into version 7.0.2 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 7.0.1

Release 7.0.1 is a maintenance release fixing bugs and providing minor enhancements. There are no major new features.

Database internal format is unchanged from version 7.0.0

This means version 7.0.0 repositories can be directly loaded into version 7.0.1 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 7.0.0

Release 7.0.0 is a major release with significant new features.

The most important new feature is distributed repositories.

Distributed repositories (see the Distributed Repositories Tutorial) allow data to be distributed over several repositories, called shards. Each shard runs on its own server separately from other shards (a single server may support more than one shard). Queries then are run by each server on each shard, with results combined after all individual queries have completed. With very large databases, use of distributed repositories can significantly speed up certain kinds of queries. See the documents Distributed Repositories Tutorial and Distributed Repositories Setup for more information.

Gruff in a browser is available in AllegroGraph 7.1.0. Gruff is a graph visualization and graphical query builder designed to work with AllegroGraph. It is available as a standalone application (see the Gruff page on the AllegroGraph website) and it will run in a browser tab. See Gruff in AllegroGraph document for more information. Documentation for Gruff itself is in the Gruff document.

We have added to the AllegroGraph examples on Github repo. There are examples of using Multi-master replication to allow for scaling out query load. These examples use Kubernetes, Terraform, load balancing with NGINX, and so on.

Database internal format is unchanged from version 6.6.0

This means version 6.6.0 repositories can be directly loaded into version 7.0.0 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 6.6.0

Release 6.6.0 is a major release with significant new features, including the following:

Database internal format is unchanged from version 6.5.0

This means version 6.5.0 repositories can be directly loaded into version 6.6.0 without upgrading. Note that backing up repositories before loading them into a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Release 6.5.0

AllegroGraph 6.5.0 was released on March 4, 2019. Release 6.5.0 is a major release with significant new features, including the following:

Database internal format is changed from version 6.4.6

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.5.0 is changed from version 6.4.6 and from earlier releases such as 6.4.5, 6.4.4, 6.4.3 and 6.4.2. (The format was previously changed in release 6.4.0.) Therefore it is necessary to upgrade 6.4.6 and earlier databases for them to be used with 6.5.0. Note that backing up repositories before upgrading them to a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same. Upgrading is discussed in the Repository Upgrading document.

Admin notes

These sections note changes which may affect AllegroGraph administration, which includes managing users, dealing with passwords, and so on.

7.1.0 Admin notes

There are no admin notes for release 7.1.0.

7.0.3 Admin notes

There are no admin notes for v7.0.3.

7.0.2 Admin notes

7.0.1 Admin notes

There are no admin notes for v7.0.1.

7.0.0 Admin notes

Programmer notes

These sections note changes which may affect how certain operations or code work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

7.1.0 Programmer notes

7.0.4 Programmer notes

An unexplained problem with Internet Explorer (IE) 11 makes it unable to display AGWebView. All other important browsers, including Edge, work fine. The problem seems to be related to running JavaScript.

7.0.3 Programmer notes

There are no programmer note for version 7.0.3.

7.0.2 Programmer notes

There are no programmer note for version 7.0.2.

7.0.1 Programmer notes

7.0.0 Programmer notes

6.6.0 Programmer notes

There are no programmer notes for release 6.6.0.

6.5.0 Programmer notes

6.4.6 Programmer notes

Several recent patches to Allegro CL modified the way URIs are handled in Allegro CL. These changes are incompatible with AllegroGraph 6.4.x and should not be applied to a Lisp client intended to be used with those versions of AllegroGraph. The patches are uri/uri.001 (posted in September, 2018) through uri/uri.006 (posted January 16, 2019). If you downloaded these patches, remove them from the uri/ subdirectory of the Allegro CL distribution directory and update your Lisp by executing update.sh/update.exe.

User visible changes

These sections list changes in the current release and in some earlier releases. See the Change History for a list of changes in all earlier releases.

AllegroGraph 7.1.0

AllegroGraph Server: General Changes

New storage-report option to agtool

The tool shows storage used by a repository. See here in the agtool General Command Utility document.

The Table in the Cluster Managment page modified

The table is illustrated in the Multi-master Replication document. The table now shows the host and port of each instance.

Rfe16562: Support limiting agtool query output to results only

The output of agtool query can now be limited to results only by specifying the --quiet option. When --quiet is set, all log output will be written to the agquery.log file. See agtool query for more information.

Rfe16543 - Replace default index :gospi by :psogi

The set of default triple indices has been modified: gospi has been taken out, and psogi has been added. See 7.1.0 Programmer Notes above.

Rfe16540 - Add support for creating, deleting, and listing repos and dynamic catalogs with agtool

agtool has new repos and catalogs tools. They can be used, respectively, to create and delete repos (including replicated repos) and dynamic catalogs. The tools can also list existing repos and catalogs. See the agtool document for more information.

Bug26221 - Speedup Freetext Index (FTI) phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Rfe16528 - Remove reorderDuringExecution query option

In practice the pattern estimates are good enough to not have to consider reordering. Using the special query prefix: 

PREFIX franzOption_reorderDuringExecution: <franz:yes>

will now give a query warning: 

obsoleteQueryOption: reorderDuringExecution

Rfe16510 - Support for SPARQL*

AllegrGraph now supports queries that use the SPARQL* syntax, like: 

select * {  
  << ?s ?p ?o >> <ex:key> ?value  
}  
 
select * {  
  bind( << ?s <ex:pred> ?o >> as ?triple )  
}

A reified triple is represented as an IRI that looks like: <http://franz.com/ns/allegrograph/4.0/tripleId#123> which means isIRI(?triple) will return true for reified triples.

There is a new SPARQL filter function: isTriple(?triple) that returns true only if ?triple is a reified triple.

SPARQL* queries are not yet supported on federated or distributed repositories.

Rfe16504: Remove "enableDuplicateReduction" query option

The SPARQL query option enableDuplicateReduction, if set to true, would behave somewhat like specifying reduced. In all cases where it would be relevant to enable this reduction, it is better to specify reduced or distinct in the query. Therefore the setting is removed.

Usage of the franzOption_enableDuplicateReduction query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_enableDuplicateReduction: <franz:yes>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: enableDuplicateReduction  
  [time] ...  
  ...>

Bug26333 - Property paths that join bindings may produce bogus results

It was possible for the SPARQL sequence property paths to produce too many results when joining bindings from VALUES clauses or subqueries. This has been fixed.

Bug26290 - Max when some variable is unbound, must give unbound result value

In AllegroGraph 6.7.0 a SPARQL regression was introduced, where SPARQL MAX aggregator would treat unbound values as bound values (lowest in ordering) and so return a bound value as result.

This was incorrect: when aggregating over an unbound value, the aggregation result should be an unbound value due to a SPARQL type error in the aggregation.

This has been fixed, so that the aggregation result is the unbound value in the described case.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

Bug26260 - Fix spurious "emptyBindClause" query warnings

SPARQL queries with a BIND clause at the start of a BGP could generate a spurious "emptyBindClause" warnings saying: 

The BINDING form for ?val cannot succeed

This can be an incorrect warning if the BIND expression does a special operation that can deal with unbound variables, like BOUND, COALESCE, IN or IF.

Examples: 

select * {  
  bind(bound(?o) as ?val) .  
}  
-> ?val = false  
 
select * {  
  bind(coalesce(?o,1,2) as ?val) .  
}  
-> val = 1

This is now fixed so that no spurious warnings are generated.

AGWebView

Bug26332: Broken bulk namespace input dialog in AGWV

The Add a namespace button in query editor shows a dialog that has a Bulk Input button which in turn should be showing a different dialog that allows to input multiple namespace declarations at once. This dialog was broken and an empty dialog box was shown instead. This has been fixed. See the New Query Page section of the WebView document for more information on the Add a namespace button.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

AllegroGraph 7.0.4

AllegroGraph Server: General Changes

Rfe16569 - backup-all (and restore-all) now backs up gruff options

agtool archive backup-all/backup-settings now backs up the settings/gruff/ directory.

Bug26353 - Turtle parser failure on prefixes with upper-case characters

Turtle parser was incorrectly converting prefixes of prefixed names to lower-case if they were at the beginning of the lines and started with the same (ignoring the case) characters as Turtle keywords prefix or base, which caused a namespace lookup failure.

For example, an attempt to parse the following Turtle 

 prefix Basic: <http://example.org/basic#>  
 Basic:subject Basic:predicate Basic:object.

failed with the following error 

 While parsing /path/to/file.ttl: Turtle parser  
 error on line 2: No prefix expansion defined for 'basic'

This has been fixed.

HTTP Client

No significant changes.

SPARQL

Bug26348 - Fix handling of the FROM clauses of SPARQL ASK queries

Starting with AG 7.0.0, running SPARQL ASK queries with the FROM clauses failed with a verbose error that said something like keyword list ... should only contain ... but unexpectedly found .... This has been fixed.

Bug26346 - Fix 'Funcall of nil which is a non-function' error

It was possible to get a Funcall of nil which is a non-function error when executing a SPARQL query that has a UNION of GRAPH clauses with unsatisfiable patterns. This has been fixed.

Bug26331 - Restriction reasoning did not work in remote Lisp AG client

Both SPARQL and Prolog queries run from a remote Lisp AllegroGraph client would miss inferred results when restriction reasoning is enabled. This has been fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

AllegroGraph 7.0.3

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

No significant changes.

SPARQL

Rfe16530 - Add support for xsd:dateTime subtraction in SPARQL queries

Arithmetic operations in SPARQL's BIND clauses now support xsd:date/xsd:time/xsd:dateTime subtraction that returns xsd:duration values, i.e. query like 

SELECT ?x  
WHERE {  
    BIND (("2020-07-29T17:26:08Z"^^xsd:dateTime -  
           "2020-07-28T17:26:08Z"^^xsd:dateTime) AS ?x)  
}

will return a value 

"P1D"^^xsd:duration

Note that results of date/time subtraction are restricted to xsd:dateTimeDuration subset of xsd:duration values for simplicity reasons.

Bug26309 - Missing zero-length property path results if both s and o are collections

It was possible for a query of the form: 

select ... {  
  values ?start { ... }  
  values ?end { ... }  
  ?start ( <ex:pred1> / <ex:pred2>* ) ?end  
}

with triple data: 

<ex:s> <ex:pred1> <ex:o>

to not return the valid solution: 

?start = <ex:a>  
?end = <ex:b>

where was followed zero times, so effectively not followed. This is corrected.

Bug26293: Property path error if object value constrained

In certain cases a property path expresion in which the object value is constrained to a literal, could lead to an error.

For example in this query ?o is constrained to value 2: 

SELECT ... {  
  bind(2 as ?o).  
  ?s <ex:p>+ ?o  
}

Depending on the active indices and the exact triples in the repository, this could give the error: 

The assertion  
  (loop for x in (list s p o g s-end p-end o-end g-end)  
    always (typep x (quote (or upi null))))  
failed

This is now fixed.

Bug26282 - Stack overflow during OPTIONAL query planning

Certain queries containing an OPTIONAL clause could lead to an error ("stack overflow") during query planning. This is now fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

No significant changes.

(remark -- 7.0.2 start ; -- }

AllegroGraph 7.0.2

AllegroGraph Server: General Changes

An AllegroGraph server may restrict itself to the number of licensed cores

If the UseLicensedCores configuration directive is specified true (that is UseLicensedCores true is in agraph.cfg) then the agraph process will automatically reduce the number of cores it uses to the value given in the license file. Otherwise the server will fail to start up if there are more cores on the machine than the license specifies.

Bug26226 - Changing passwords in AGWebView could fail

When a password has expired, changing it in AGWebView could fail. This has been fixed.

Bug26221 - Speedup fti phrase match searches

Fixed an inefficiency in FTI phrase matches that caused a slowdown when a word in the phrase appeared in a single triple multiple times. This is now optimized. The speedup seen will vary based on the frequency in which a word from the phrase appears in a given triple. For small string literals and resources, the improvement will be very modest, but for phrase matches in large string literals the speedup can be significant

HTTP Client

No significant changes.

SPARQL

Bug26274 - Insert ..^^xsd:string fails

Queries that insert a triple with an XSD string as object, like: 

INSERT {  
  <ex:s> <ex:prop> "2.0.0"^^xsd:string .  
}  
WHERE { }

would fail with the error message: :string fell through a ecase form. This is fixed.

Bug26273 - Do not skip index when rewriting left-joins to joins

In some cases the system would skip the index component of a plan step when constructing a JOIN plan step. This was breaking the assumption that index is non-nil when removing equivalent variables from JOINs. This is now fixed.

Bug26272 - Count DISTINCT of UNION incorrect

It was possible for queries with COUNT DISTINCT of a UNION to give a number that was too high. For example: 

SELECT (COUNT (DISTINCT ?s) as ?s)  
{  
  { ?s <ex:p1> <ex:o1> }  
  union  
  { ?s <ex:p2> <ex:o2> }  
}

This is now fixed.

Bug26263 - Type errors when using UNION and FILTER

This is an issue relevant to queries containing UNION in which an unprojected variable occurs both inside the UNION (at most once per UNION subclause) and once outside it, like ?o in: 

SELECT DISTINCT ?s {  
{ ?s <ex:p1> ?o } UNION { ?s <ex:p2> ?o }  
FILTER (?o > 3)  
  }

In such cases it was possible for ?o to become unbound in the second occurrence (in the FILTER clause) even though the first UNION clause would bind it. This would lead to incorrect query results.

This is now fixed.

Bug26259 - Property path error: "the value of p is "

Certain property path expression would fail with the error: 

the value of db.agraph.sbqe.property-path::p is  
#<db.agraph.utility.dictionary:dictionary ...>  
which is not of type (or null upi real).

This is fixed.

Bug26240 - SPARQL property path that joins bindings misses results

It was possible for property path expressions to miss certain results if the subject or object of the property path was bound by a previous pattern. This is now fixed.

Bug26236 - Errors when using multi-match clause with multiple vars as filter-pattern

There were situations where a query containing a "VALUES" or "FILTER (.. IN ..)" clause, in combination with other patterns, would give incorrect query results. This has been fixed.

Bug26235 - Assertion error :is-ordered-cursor-for-distinct-variable

It was possible for queries with DISTINCT to fail with the error: 

the assertion  
  (null (getf (db.agraph.cursor::cursor-properties db.agraph.sbqe::cursor)  
              :is-ordered-cursor-for-distinct-variable))  
failed.

This has been fixed.

Bug26222 - SPARQL FILTER range errs if there are objects with non-overlapping type

If a SPARQL query included a numerical filter, in the presence of triples whose object value is of a numerical type that does not overlap with that range, this would signal an error at query plan time.

For example: 

SELECT * {  
  ?s ?p ?o .  
  FILTER (?o > 3 && ?o < 10)  
}

This range (3, 10) of positive integers does not overlap with type xsd:negativeInteger. If the repository contain a triple like: 

{ <ex:s> <ex:p> '-123'^^xsd:negativeInteger }

the resulting error would be: 

Error: `3` is not of type xsd:negativeInteger, exceeds the maximum: -1  
  [condition type: xsd-datatype-error]

This has been fixed

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

Bug26220 - Fedshard failure for "count distinct" with "group by"

When using FedShard, running a query that combines COUNT DISTINCT with GROUP BY could result in an error. For example: 

SELECT ?s (COUNT(DISTINCT ?o) AS ?count) {  
  ?s ?p ?o  
} GROUP BY ?s

This has been fixed.

AllegroGraph 7.0.1

AllegroGraph Server: General Changes

Rfe16481 - Increase triggered checkpoint timeout

Previously while adding a replica of a large repo on a large memory machine an error like the following could occur:

Error: instance add failed with code 400, body "failed with message: Could not trigger a checkpoint: Checkpointer process failed to respond"

This has been fixed.

Bug26176 - SHACL validation would consider shape graph triples

Under certain circumstances, SHACL validation would incorrectly consider shape graph triples for validation, resulting in reporting triples non-conforming when the data graph triples actually conformed. (Reporting triples as conforming when in fact they did not conform never occurred.) This has been fixed. SHACL and SHACL validation are described in the SHACL document.

HTTP Client

No significant changes.

SPARQL

Rfe16452 - Let GROUP BY with no result rows give zero result rows

The results of GROUP BY queries with aggregates in an edge case has been changed to closer follow the intention of the SPARQL 1.1 standard.

Take this query and assume there are no triples with predicate : 

select ?s (count(?o) AS ?count)  
where {  
  ?s <ex:p> ?o  
} group by ?s

AllegroGraph would return one result row with ?s unbound, and count zero.

An alternative behaviour that some other triple stores have, is to return zero result rows. The text of the SPARQL 1.1 standard conflicts with the "agg-empty-group2" test case, so that either behaviour is reasonable.

The current consensus is that having zero result rows was intended, and is expected by users. See e.g. https://afs.github.io/sparql-agg-group-empty.html#group-agg-no-rows

AllegroGraph has been updated to follow this consensus. So for GROUP BY queries with aggregates, in case of zero matching pattern in the body, there will be zero result rows.

Bug26190: Incorrect rewrite of OPTIONAL with non-patterns on LHS

In some cases queries of the form: 

SELECT .. {  
  VALUES ..  
  ..  
  OPTIONAL { .. }  
}

could be optimized incorrectly, resulting in missing query results. This has been fixed.

Bug26188 - Executing query failed for query with VALUES

Certain queries of the form: 

SELECT .. {  
  VALUES ?x { <a> <b> <c> }  
  .  
}

could lead to an error: 

Error: the value of ag.sbqe::index is nil,  
       which is not of type ag.sbqe::plan-index-type.

this has been fixed.

Bug26181 - Incorrect :time-total addenda value for query on remote store

When executing a SPARQL query in AGWebView the Information tab shows query execution statistics, including the time it took to plan, execute, and output query results.

It could happen that the total time would be incorrect (too low). This has been fixed.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Distributed AllegroGraph

Rfe16457 - improve shard-local kb support

Previously, when defining a distributed store in the cluster config file, there was limited support for shard-local kb's by including localhost in the KB declaration. That is the following declaration would define a kb on localhost and would inherit whatever scheme/port were the default at the point where the declaration was made: 

KB localhost/catalogs/tests/repositories/kb4

If these defaults match the scheme/port used by each AG shard, then the KB will be shard-local. Now, any triple-store-spec that does not include a server part can be used to define a shard-local KB. Thus 

KB tests:kb4

no longer will inherit defaults from the environment at the point of the declaration, and will inherit the scheme/port of each shard, even if scheme/port differ across all shards.

See the Distributed Repositories Using Shards and Federation Setup document.

Bug26187 - Do not track servers in KB directives

In Distributed AllegroGraph, the servers that comprise a cluster are derived from the directives that refer to servers in the agcluster.cfg file. This includes server, repo, replica, and --incorrectly-- kb directives.

The servers referred to only in KB directives are no longer considered as part of the cluster. This allows remote KBs which do not reside on a cluster host.

AllegroGraph 7.0.0

AllegroGraph Server: General Changes

agtool load changes

The (undocumented) --blank-node-cache-size argument has been removed. (Even though this argument was undocumented, some users may have been told of its existence.) Blank nodes are no longer cached during data loading.

There is a new -w/--workspace option to agtool load. The value should be an existing directory which will be used for temporary files during data loading, particularly for the temporary storage of a table of blank node information. Absent this option, temporary data is stored in /tmp and if that directory is too small, the load will fail.

agtool now supports a triple-count tool

The triple-count tool returns the number of triples in a repository. See here in the agtool document.

agraph-control now takes a --cluster-config control option

The option allows the path of the cluster configuration file to be specified. This file defines distributed repositories (see Distributed Repositories Setup). The agraph-control program is described in the agraph-control of the Server Configuration and Control document.

Adding a triple with a predicate mapping and a datetype mapping might add two triples

Before this change, when adding a triple, if both a predicate mapping and a datatype mapping applied to the object of the triple, two triples would be added. The new defined behavior is that predicate mappings take precedence over datatype mappings.

index-literals argument passed when creating/modifying a freetext index

When creating a freetext index, the index-literals argument determines if and/or under what circumstances a literal will be indexed. The value passed can be either a boolean or a list containing UPIs, future-parts, or strings.

With this change, when passing a string to index-literals, it is expected to be equivalent to the value returned when calling part->value. That is, the string should not contain any embedded angle-brackets (< or >) or double-quotes (") used to denote the string represents a resource or literal, that are not part of the actual value represented by the string.

Rfe16417 - Deprecate style 2 indices

There are two triple index styles available, "style 1" and "style 2", with different storage structures and thus performance characteristics. In practice "style 1" was generally recommended, and "style 2" has been deprecated. The latter has been removed from the documentation and might be removed in a future release.

Rfe16386 - New BriefBacktrace configuration directive.

See the [administrator note](#admin-notes-700} for more information. See Server Configuration and Control for information on configuration directives.

Rfe16364 - New LicenseWarnInAdvance configuration directive.

See the administrator note for more information. See Server Configuration and Control for information on configuration directives.

Rfe16283 - New configuration directive to control the stale DNS retain time

There is a new configuration directive StaleDNSEntryRetainTime which specifies how long AllegroGraph will retain a stale DNS entry. If the DNS mapping from name to IP address may change while the application is running you'll want to specify a very small value (number of seconds) for this directive. See StaleDNSEntryRetainTime in Server Configuration and Control.

Rfe16244 - Update PARSE-NTRIPLES to support N-Triples 1.1 blank node syntax

When importing N-Triple data, certain blank node formats with dots, dashes or underscores were unsupported. For example importing this line: 

 <http://s> <http://p> _:a-b_c.d .

would fail with: 

 Error: N-Triples parser error while parsing string at line 1  
       (last character was #\_): Illegal character

Now AllegroGraph supports the blank node syntax as defined in the N-Triples 1.1 technical recommendation.

Rfe16239 - agtool load/export/archive from/to s3

agtool can access files in Amazon S3. To do so, AWS authentication must be specified on the agtool command line. See the section Accessing and operating on files on Amazon S3 in the agtool document. Files in S3 must be prefaced by s3://, like the following: 

 s3://bucketname/a/b/c/filename

Rfe14667 - Log THP warning only (don't show in AGWV)

The THP (Transparent Hugepages) warning is now logged to agraph.log only and not the console or AGWebView.

Bug26040 - agtool archive restore to a different repo.

The documentation printed by 'agtool archive restore --help' claimed that if the archive directory contained only one repo backup then you needn't name that backup if you restored to a different repo name. This claim has been removed from the documentation because the behavior isn't supported.

Bug26017 - Literal language in rdf/xml was not properly loaded

Previously, instead of setting the language of the literal object, an additional triple with the object being the language value and the predicate http://www.w3.org/XML/1998/namespacelang was incorrectly added to the triple store.

For example, <rdfs:label xml:lang="fr">Autriche</rdfs:label> loaded as: 

<subj> <pred> "Autriche"  
<subj> <http://www.w3.org/XML/1998/namespacelang> "fr"

Bug26011 - Allegrograph licensed cores check prints wrong core count in error message

Previously if the number of CPU cores exceeded the licensed limit, AllegroGraph would generate an error message like the following:

This machine exceeds the CPU core count specified in the license. Maximum licensed cores: X. This machine has: Y.

The value of X would be the license limit but the value of Y would potentially be incorrect (it would report the number of available threads, rather than the number of cores in the machine, which will be different values if hyperthreading is enabled). This has been fixed.

Bug26000 - Fti recovery crashes if stray fti files exist from a prior instance crash

In some circumstances instance recovery could fail with an error message like the following: 

opening of database myrepo has failed (The database system tried to  
allocate a new file with the logical name fti-1-chunk-1 which already  
exists in database myrepo)

This has been fixed.

Bug25999 - Crashed instance process can leave service daemon in confused state

Under some circumstances a crashed instance process would not be automatically restarted by the service daemon. This has been fixed.

Bug25986/Bug25997 - Fix agraph-backup tool initialization issues

When the repo spec was used to specify the repository, the credentials and host values from the spec were ignored and the tool always attempted to connect to 127.0.0.1. This commend thus 

$ agtool archive backup test:xyzzy@remotehost:10035/test test.bak

could fail with a Connection refused (errno 111) message because the host is not localhost. Specifying the credentials when restoring data also could cause the tool to fail (bug25997): 

 # Fails with the message 'Invalid arguments (:params,  
 # :restore-mode-p) found while creating triple-store of class  
 # db.agraph:remote-triple-store. ...'  
 $ agtool archive restore-all test.bak test:xyzzy@127.0.0.1:10035

Both these issues have been fixed. Note that using deprecated arguments like --host and --port could cause problems if the values specified did not match the values in the REPO-SPEC, even default (and thus not-visible) values in some cases. See Older repository specification arguments in the agtool General Command Utility document for examples.

Bug25959 - Fix ACID isolation violation in some ordered cursor operations.

Fixes a bug introduced in v6.5.0 where some ordered triple cursors could produce triples that were not visible in the current transaction. The scope of the problem was limited to triples committed in concurrent transactions that had yet to be written to disk.

Bug25907 - Document how to pass args to agraph using agraph-control

The ability to use agraph-control to pass arguments to the agraph program (which is called by agraph-control) when starting or restarting AllegroGraph existed prior to this change, but was not clearly documented. How to do so is now documented in the agraph-control --help text and in the description of agraph-control in the Server Configuration and Control document.

Bug25901 - Importing triples bypassed security-filters

It was possible for users with security-filters to import triples into a db that those filters were intended to prevent.

This problem is now fixed.

Bug25880 - Remove whitespace trimming in TRIX literals parsing

To restore the behavior specified by the standard, removed whitespace trimming in the TRIX parser while parsing literals.

HTTP API

No significant changes.

SPARQL

Rfe16433 - Remove full-scan-warning

The query warning type "warn-fullscan" and the query option "fullScanWarningSize" have been removed.

In practice these warnings could occur without there being a problem with the query. It would fill agraph.log with warning lines that would be ignored, defeating the purpose.

Usage of the franzOption_fullScanWarningSize query prefix will now lead to a warn-obsolete-query-option entry in the query result metadata: 

(run-sparql "PREFIX franzOption_fullScanWarningSize: <franz:210000000>  
             select ...")

has fourth return value: 

#<done  
  [warn] obsolete-query-option: fullScanWarningSize  
  [time] ...  
  ...>

Rfe16324 - Warn on ASK modifiers

An ASK query does not support LIMIT, OFFSET, or ORDER BY. Now a query warning is signaled if any of these is provided, with the exception of LIMIT 1.

Rfe16279 - Improve checking for unknown constants in the query

The check for unknown constants in the query at plan time is now more advanced, resulting in faster query execution.

For example if a resource <s> only occurs in the subject position of triples, a pattern like ?s <p> <s> will now be marked impossible at plan time.

Another example are references to standard IRIs that are present in every repository automatically, like <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>. Previously a pattern like ?s a ?foo would never be marked impossible, but now if there are no triples with such predicate, the pattern will be marked impossible.

This also has effect on magic properties like <http://franz.com/ns/allegrograph/3.0/temporal/intervalBefore> that under the hood expand into patterns with properties like <t:starttime>, <t:startpoint> and <t:time> (with http://franz.com/ns/allegrograph/3.0/temporal/ as t). If a repository only uses starttime, and never the startpoint/time combination, the plan will be optimized accordingly.

Bug26144 - CAAT ignores :pattern-filters of query patterns

Queries executed with the Chunk-at-at-Time (CAAT) strategy (so with a LIMIT, without DISTINCT or aggregators) could give incorrect results, if a specific query pattern was planned to act as filter for another pattern. This has been fixed.

Bug26143 - Graph <g> {} gives error

A query with an empty named graph clause, like: 

select ?s ?o {  
  graph <ex:g> {}  
  ?s ?p ?o .  
}

would fail with an error: Attempt to access the name field of {g} which is not a symbol. This is now fixed.

The meaning of graph <ex:g> {} as a pattern, is that some triple must exist in the given fixed graph <ex:g>. The meaning of graph ?g {} is that some triple must exist in any named graph.

Bug26137 - Spurious unknown-variables warning for VALUES in inner SELECT

A VALUES clause inside an inner SELECT, like in this query: 

SELECT ?region {  
  SELECT ?region WHERE {  
     VALUES ?type { <ex:Region> }  
     ?region a ?type .  
   }  
 }

would give a spurious warning about the VALUES variable: 

unknown-variables: Unknown variable used in VALUES clause: ?type_1_1

This is now fixed, so that there will be no more warnings.

Bug26116 - FILTER CONTAINS LCASE gives UPI-NOT-IN-STRING-TABLE-ERROR

Certain SPARQL expressions that create intermediate strings, like: 

select ?s {  
  ?s <ex:p> ?o .  
  filter ( contains(lcase(?o), 'something'))  
}

could signal the error: The UPI #(...)is not present in the repository's string table.

This has been fixed.

Bug26093 - GROUP_CONCAT interprets separator arg as format string

If a SEPARATOR argument containing a tilde was supplied to the GROUP_CONCAT aggregator, the resulting string could be incorrect. This has been fixed.

Bug26090 - Boolean value serialized as sparql-json lacks datatype

A boolean value serialized in sparql-json format would look like (here we use the Lisp function sparql:run-sparql but the results would be the same if the query was executed in AGWebView and exported in sparql-json format): 

(sparql:run-sparql "select * { bind(true as ?x) }"  
   :results-format :sparql-json)  
 
-> {"head": {"vars": ["x"]},  
    "results": {"bindings": [{"x": {"type": "literal", "value": "true"}}]}}

But the correct way to serialize a boolean is to include the datatype. Thus this: 

"x": {"type": "literal", "value": "true"}`

is incorrect and should instead be: 

"x": {"type": "literal",  
      "datatype": "http://www.w3.org/2001/XMLSchema#boolean",  
      "value": "true"}

This has been corrected.

Bug26078 - SPARQL FILTER on numeric range excludes UPI types that partially overlap

If a SPARQL query FILTER expression restricts a triple part to a numeric range, and if there is a triple whose part value falls in that range but is of a data type such that the filter range exceeds the data type range, then the triple would not pass the filter, which was incorrect.

For example: 

(add-triple "<ex:s>" "<ex:p>" (value->upi 127 :byte))

and query: 

select * {  
  ?s ?p ?o .  
  filter ((?o > 120) && (?o < 130)  
}

The query FILTER restricts the triple object to (120, 130). The ?o value 127 should pass this filter. The value 127 is of type :byte, which has range [-128, 127].

The incorrect behaviour is that the FILTER range (up to 130) exceeds the data type range (up to 127) and therefore the value 127 was incorrectly filtered out, meaning the triple would not be returned by this query.

This has been fixed.

Bug26071 - COUNT aggregate on unknown variable should return 0.

If a query aggregates a variable that is not otherwise used, like: 

SELECT (COUNT(?bogus) AS ?count) { ?s ?p ?o }

this would be evaluated as if the unknown variable ?bogus was replaced by true. The value of ?count would be the total triple count.

This has been corrected, so that the returned ?count is now zero.

Bug26020 - Invalid OPTIONAL reordering

If a query has an OPTIONAL pattern that binds a variable, followed by a non-OPTIONAL pattern that references that variable, like: 

SELECT * {  
  ?s <ex:p1> ?o .    # (1)  
  OPTIONAL {  
    ?t <ex:p2> ?o2 . # (2) possibly binds ?o2  
  }  
  ?s <ex:p3> ?o2     # (3) references (if bound already) or binds ?o2  
}

the query engine could create an incorrect query plan with evaluation order (1) (3) (2). The evaluation order of (2) and (3) would be reversed: variable ?o2 would be bound based on pattern (3). This is now corrected to have evaluation order (1) (2) (3).

Bug26005 - Incorrect SUM and GROUP_CONCAT default values for empty group

If a SPARQL query has no result rows (or if GROUP BY is used, and a specific group has no result rows) a SUM aggregation and a GROUP_CONCAT aggregation would both return an unbound value.

This was incorrect according to the SPARQL 1.1 spec. Now SUM will return 0, and GROUP_CONCAT will return the empty string.

Bug26004 - Queries that have a constantly true HAVING could fail.

A constantly true HAVING expression in a query, like: 

SELECT ?s (max(?o) as ?maxO) {  
  ?s ?p ?o  
} GROUP BY ?s  
HAVING (3 < 4)

could err with Received signal number 7 (Bus error). This has been fixed.

Bug25999 - Crashed instance process can leave service daemon in confused state

Under some circumstances a crashed instance process would not be automatically restarted by the service daemon. This has been fixed.

Bug25988 - SPARQL function REGEX should err on non-string

Previously AllegroGraph was liberal in accepting a constant resource as REGEX pattern. For example: 

SELECT (regex('ex:foo', <ex:foo>) as ?match) {}

would return "true".

This has been fixed in order to adhere to the SPARQL specification: if the target or pattern is not a string, a type-error is signaled. The above SELECT now returns an unbound value.

Bug25984 - SPARQL function LANGMATCHES should err on non-string

Previously AllegroGraph was liberal in accepting non-strings to function LANGMATCHES and converting them using STR.

For example: 

BASE <http://foo/>  
SELECT (langMatches(<ex>, <ex>) as ?a)  
   (langMatches(123, 123) as ?b) {}

was interpreted as: 

BASE <http://foo/>  
SELECT (langMatches( STR(<ex>), STR(<ex>)) as ?a)  
   (langMatches( STR(123), STR(123)) as ?b) {}

and would return "true" twice.

This has been fixed in order to adhere to the SPARQL specification: if either argument to langMatches() is not a string, the result is a type-error. The above SELECT now returns two unbound values.

Bug25982: Fix HAVING constraint on implicit single group without aggregates

If a query uses HAVING without using GROUP BY, and no aggregates (like MIN, COUNT), as for example in: 

SELECT * { ?s ?p ?o } HAVING (?s = <ex:s>)

then the HAVING clause acts like FILTER: 

SELECT ?s ?p ?o { ?s ?p ?o . FILTER (?s = <ex:s>) }

However in the first query AllegroGraph would incorrectly ignore the HAVING clause, meaning results that should have been filtered out were returned. This is now corrected.

Bug25941 - Blank node patterns not supported in subject position of BGP triples

A query pattern with a blank node pattern as subject, and a property path, like: 

select * {  
  [ <ex:p2> ?o2 ]  ( <ex:p1> | <ex:p2> )  ?o  
}

would give a parse error at the "(" character. This has been fixed.

Bug25936 - Inefficient join with UNION

The query execution planning for certain queries containing UNION has been improved, leading to faster execution. This impacts also magic properties that are implemented using UNION, like <t:intervalBefore>.

Bug25934 - Problem with BIND (EXISTS { pattern1, pattern2 } as ?x)

If a query binds a variable to an EXISTS expression, where the EXISTS contains multiple patterns, like: 

SELECT ?person ?known where {  
  ?person a <ex:Person> .  
  BIND ( EXISTS {  
           ?employee a <ex:Employee> ;  
                     <ex:knows> ?person  
         } as ?known )  
}

then the outcome could be incorrect: the second and later patterns could be ignored, or the query could result in an error like: 

Error: Bad variables for set: (?person).

This is now fixed.

Bug25933 - Duplicate patterns in blank node + property path expansion

For query triple patterns with a property path, and a blank node as object, like: 

?s  ^<ex:p>  [ <ex:p1> <ex:foo> ;  
               <ex:p2> <ex:bar> ]

the expansion into simple triple patterns could contain duplicates: 

1. ?bnode_1 ex:p ?s  
2. ?bnode_1 ex:p ?s  (duplicate)  
3. ?bnode_1 ex:p2 ex:bar  
4. ?bnode_1 ex:p1 ex:foo

The duplicated pattern resulted in superfluous work during query execution. This is now fixed.

Bug25931 - Sending SPARQL query with typed booleans to remote-triple-stores fails

Making SPARQL queries with typed booleans (e.g. true"^^xsd:boolean) would fail with an error indicating that AllegroGraph is unable to convert a boolean UPI into a future part.

This is now fixed

Bug25925 - SPARQL MAX does not handle unbound values correctly

The SPARQL MAX aggregator would return an unbound value as result, if any of the values to aggregate was unbound. This has been fixed.

Bug25920 - SPARQL SERVICE using https without chunking errs

When a SPARQL query was sent to remote SPARQL endpoint that uses HTTPS, and the endpoint sent results not using chunked encoding, then an error would be signaled: 

Error: the value of stream is #<excl::ssl-client-stream ..>,  
       which is not of type  
 (or net.aserve::unchunking-stream util.zip:inflate-stream  
           excl::hiper-socket-stream).  
  [condition type: type-error]

This could happen in a SERVICE <https://...> clause, or by proving a :HOST "https://.." argument to function run-sparql.

This has been fixed.

Bug25919 - SPARQL function STRLEN should err on non-string

Previously AllegroGraph was liberal in accepting non-strings to function STRLEN and converting them using STR.

For example: 

SELECT (STRLEN (<http://franz.com>) as ?s)  
       (STRLEN (123.45) as ?t) {}

was interpreted as: 

SELECT (STRLEN (STR (<http://franz.com>)) as ?s)  
       (STRLEN (STR(123.45)) as ?t) {}

and would return values 16 and 6.

This has been fixed in order to adhere to the SPARQL specification: taking the length of a non-string now gives a type-error. The above SELECT now returns unbound values.

Bug25916 - COUNT DISTINCT of expressions gives error

A query for COUNT DISTINCT where the value is not a variable (like ?o) but an expression (like "strlen(?o)"), for example: 

select (count(distinct(strlen(?o))) as ?olen) {  
  ?s ?p ?o  
}

used to signal an error: 

Received signal number 11 (Segmentation fault)

These queries are now supported.

Bug25898 - SPARQL error for property multimatch with filtered object range

Queries with a property path with alternatives, that have a filter on the object variable, where the object values are of a type that is hashed to a UPI, like: 

select * {  
  ?s ( <ex:p1> | <ex:p2> ) ?o .  
  filter (lang(?o) = 'en')  
}

could result in an error like: 

Error: The UPI #(0 0 0 0 0 0 0 0 0 0 0 3) (which is a literal-language)  
       is not present in the triple-store's string dictionary.

This has been fixed.

Bug25821 - SPARQL function SUBSTR should clip LENGTH to string length

If in SUBSTR the length argument is larger than then the string's length, this would result in an error: 

SELECT * { bind (substr('abc', 1, 100) as ?s) }  
 
Executing query failed:  
In subseq, `end' (100) is beyond the end of the sequence (3).

This is now fixed, so that the substring up to the end of the string is returned. In this case: abc.

Bug25806 - Fix ORDER BY on non-projected variable

For certain SPARQL SELECT queries that use ORDER BY on a non-projected variable, the results could be ordered incorrectly. This has been fixed.

Bug25694 - Fix simplifying SPARQL= into UPI=

It was possible for a SPARQL equality filter: 

FILTER (?a = ?b)

to return false if the values are of different types. For example, if ?a is the integer 2 and ?b the float 2.0. This has been fixed.

Bug25636 - Triple cursor FILTER containing [NOT] EXISTS applied incorrectly

If a query contains a FILTER clause that uses EXISTS or NOT EXISTS, there were cases where that FILTER was evaluated incorrectly and as a result the query would return incorrect results.

Concretely, if in the query plan the filter was integrated in the first triple pattern, it was possible for EXISTS to return true in certain cases where false was the correct answer; and for NOT EXISTS to return false where true was the correct answer.

This has been fixed.

Bug25585 - Support ORDER BY on an aggregate expression

Queries in which the ORDER BY expression refer to an aggregation, for example: 

SELECT ?s (MIN(?o) as ?minO) {  
  ?s ?p ?o  
}  
GROUP BY ?s  
ORDER BY (min(?o))

would fail with the error: 

:aggregate fell through a etypecase form

This is now fixed.

Bug24815 - Fix MINUS with an empty row on the RHS

A query that uses MINUS, where the second BGP contains only unbound values, like: 

SELECT ?x {  
  { VALUES (?x) { (1) (2) (3) (4) } }  
  MINUS  
  { VALUES (?x) { (UNDEF) }  
  }  
}

should have 4 results, namely ?x bound to each integer.

Previously AllegroGraph would return no bindings. This is now corrected.

AGWebView

Bug26010 - Fixed Delete statement confirmation dialog

The Delete statement confirmation dialog in AGWebView is now fixed. Instead of asking for the confirmation, if showed the message: function () { [native code] } and did not allow to confirm the operation. Now, the confirmation dialog is displayed properly and, if the operation is confirmed, it will complete correctly.

Bug25947 - AGWebView doesn't allow all session spec characters

In 6.6.0 AGWebView was too restrictive on the characters allowed in the Session Specification edit box on the main catalog page. This has been fixed.

Bug25856 - Back-up this repository on WebView repo page does not work

Fixed the inner API for remote backup request with target on the server side file system. Fixed the dialog to correctly describe what is expected from user.

Changes to the Lisp API

Improve Materializer support for remote-triple-stores

The keyword arguments :delete-first? and :delete-duplicates-first? are now supported when calling materialize-entailed-triples on a remote triple store via the AllegroGraph Lisp client, and can also be specified via the REST API as well.

Rfe16211 - add IF-DOES-NOT-EXIST parameter to OPEN-TRIPLE-STORE

This change extends the open-triple-store function with an :if-does-not-exist argument. The following values are supported:

Rfe12356 - HTTP proxy support for AllegroGraph Lisp client

New :proxy parameter for create-triple-store, open-triple-store and triple-store-exists-p functions of the AllegroGraph Lisp client API allow to specify an HTTP proxy to be used for all HTTP requests issued by client. Proxy settings can be specified in the form 

[<user>[:<password>]@]<host>[<port>]

Alternatively, components of the proxy specification can be provided explicitly by :proxy-user, :proxy-password :proxy-host :proxy-port parameters. Additionally, the list of domain name suffix strings, for which the proxy should be bypassed, can be specified via :proxy-bypass-list parameter.

Bug25974 - load-ntriples doesn't work with namestring source on remote-triple-store

Previously calling load-ntriples (or any of the other load-* functions) would fail for remote-triple-stores if the source argument was a string naming a file instead of a pathname. This has been fixed.

Prolog

No significant changes.

Documentation

rfe16275 - Storage and memory document for AllegroGraph

There is a new Memory Usage document which discusses memory and disk space requirements when using AllegroGraph.

nD Geospatial

No significant changes.

AllegroGraph 6.6.0

AllegroGraph Server: General Changes

Rfe16127 - Make AllegroGraph's scripting model permissions more strict

Superuser permission is now required to upload, list, view, or delete both sitescripts and reposcripts.

Additionally, reposcripts can no longer be loaded into shared backends.

Rfe16096 - Simplify AllegroGraph script naming.

Scripts no longer support hierarchical naming, in favor of a simple flat namespace. For example, when uploading a script: 

Legal:   http://example.com:10035/scripts/sc.cl  
Illegal: http://example.com:10035/scripts/com/app/module/sc.cl

Rfe16093 - Incoming IP restrictions mechanism for AllegroGraph

New AllowIP configuration directive provides a way to restrict access to AG instance to certain ranges of IP addresses.

Rfe15992 - Add MaximumBackupAge parameter to the agraph.cfg file

One can specify the maximum age in seconds that a backup of the multi-master controlling instance is considered acceptable to use to grow the cluster. See the Instance Settings section of the Multi-master Replication document and also the Top-level directives for multi-master replication clusters section of the Server Configuration and Control document).

Rfe15886, Rfe15470 - UI Improvements to the triple attribute pages.

Merges the 'Define Attribute' and 'Define Static Filter' pages into a single 'Triple Attributes' page that is more streamlined and user friendly. See the Attributes example in the AGWebView documentation.

Bug25833 - Using CorsAllowRegex would break session/backend creation

Setting the CorsAllowRegex directive (see Server Configuration and Control) in your agraph.cfg would cause the creation of dedicated sessions and shared backends to break with a confusing error message in the agraph.log file.

Bug25832 - Only compare CorsAllowOrigin directive against Origin host

Previously, each value specified by a CorsAllowOrigin directive was matched against the full value of the Origin header: scheme, host, and port. Now, each CorsAllowOrigin directive is matched only against the host, per the documentation.

Bug25812 - agtool query --standard-namespaces

If agtool query is given the argument '--standard-namespaces no' then the AllegroGraph set of standard namespace PREFIX definitions will not be prepended to the text of query. Furthermore if the query is going to an AllegroGraph repository that agtool will open itself then the set of namespaces known to AllegroGraph is cleared before the query is run.

Bug25805 - Fix bug in SNA path-finding functions used with remote-triple-stores.

SNA path-finding functions like depth-first-search and all-bidirectional-search-paths would fail when called on a remote-triple-store if either the START or END arguments represented literals. This is now fixed.

Bug25801 - CSV import hangs on surrogate pairs

The CSV importer has been fixed to work with UTF8 surrogate pairs.

Bug25784 - Agtool archive failed when using an https spec.

Trying to use agtool archive with an https spec would fail due to the tool attempting to make regular http requests to the ssl port. This problem is now fixed.

Bug25736 - improved handling of GRAPH in JSON-LD

A JSON-LD error is now signaled when a string which is neither a resource nor URL is passed to as the --graph argument to agtool load. :root is supported as a special value of GRAPH, resulting in using the toplevel subject as a default graph for all triples added for the JSON-LD object with that subject.

Bug25382 - Fix security flaw in AllegroGraph scripting.

Fixes a bug, introduced in v6.3.0, that allowed any file on a filesystem accessible to AllegroGraph to be loaded as a script. Scripts are now only searched for and loaded from their expected locations relative to the SettingsDir directive.

HTTP Client

Bug25820 - CORS does not work via HTTPS

Previously CORS configuration options were ignored for accesses to AllegroGraph made via HTTPS. This has been corrected.

SPARQL

Rfe16157 - Implement logging of slow queries

A new configuration directive SlowQueryLogThreshold, if set, causes SPARQL queries that have total run time in milliseconds greater or equal to the value, to be logged to agraph.log or to a log file specified by SlowQueryLogFile configuration directive.

Bug25831 - Fix optimization of GRAPH with a non-existant value in SPARQL

The query: 

select ?s ?p ?o  
{  
  VALUES ?s { <http://s> }  
  VALUES ?p { <http://p> }  
  GRAPH <http://non-existing> {  
OPTIONAL {  
  ?s ?p ?o  
}  
  }  
}

should return 1 result row with ?s, ?p bound and ?o unbound.

However due to an incorrect optimization of unreachable GRAPH clauses there would be no results. This has been fixed.

Bug25803 - Fix augmenting pattern with two IRI equivalence constraints

SPARQL queries that bind a variable multiple times, like: 

SELECT * {  
  BIND (<http://ex/p2> as ?p)  
  {  
BIND (<http://ex/p1> as ?p)  
?s ?p ?o  
  }  
}

could be optimized incorrectly, leading to the error message: 

Executing query failed:  
 nil is not of the expected type `number'

This has been fixed.

Bug25802 - Join query rewriters lose filters

Certain SPARQL queries where a join happens, and one side contains BIND and FILTER, like: 

SELECT ... {  
  ?s ?p1 ?o1  
  {  
    BIND (<http://p2> as ?p2)  
    ?s ?p2 ?o2  
    FILTER (.. ?o2 ..)  
  }  
}

could be optimized wrongly in that the FILTER would be dropped. This has been fixed.

Bug25794 - BIND variables disappear in join

Certain SPARQL queries where a variable was used in a filter and also projected could return results in which that variable was unbound due to an incorrect query optimization. For example: 

SELECT * {  
  {  
    bind (4 as ?four)  
    ?s ?p ?o .  
    filter (?o != ?four)  
  }  
  {  
    bind (3 as ?three)  
    ?s ?p ?o .  
    filter (?o != ?three)  
  }  
}

could return rows with ?three and ?four unbound. This has been fixed.

Bug25761 - Whitespace symbols are not escaped properly when writing in TSV format

According to the TSV SPARQL standard, when serializing RDF terms some whitespace symbols like tabs, newlines and carriage returns are encoded in strings as \t, \n and \r respectively. This has not been the case, as these symbols were serialized as-is, leading to some parsing problems later, specifically a tab character being treated as separator for TSV format. This has been corrected.

Bug25756 - Unused variable in complex property path gives errors

Queries of the form: 

select ?o {  
  ?s ( ^<http://foo> | <http://bar> ) ?o  
}

in which the property path was not very simple, and also one variable is not otherwise used (here ?s), could lead to an error during execution. This has been fixed.

Bug25752 - Fix missing results for DISTINCT + LIMIT

There could be missing results for queries that uses both DISTINCT and LIMIT due to the DISTINCT condition being applied incorrectly. This has been fixed.

Bug25749 - Filter (str(?var) = '..') marked as impossible

If a SPARQL query had filter pattern comparing a STR() value to a literal string, like: 

filter ('http://g' = str(?g))

this would result in no results and the warning: 

 Warning: `?g` cannot be a literal so the FILTER ?g is one of 1 thing will always fail

This has been fixed.

Bug25714 - SPARQL IRI() and URI() functions must return valid, absolute results

Previously, the SPARQL IRI() and URI() functions could return IRIs and URIs that were syntactically invalid and/or relative. This has been corrected. AllegroGraph will now signal SPARQL type errors which will be treated as UNBOUND for assignment and as FALSE for FILTER evaluation.

Bug22350 - Fix missing results when using CAAT with DISTINCT

When a SPARQL query containing DISTINCT or REDUCED was executed using the chunk-at-a-time strategy, there could be results missing due to incorrect handling of intermediate query results. This has been fixed.

AGWebView

Rfe16012 - Don't show AllegroGraph Utilities menu unless logged in.

When not logged in, AGWebview displayed the Utilities menu even though none of its menu items work without being logged in. Now, the menu is not presented until a user is logged in.

Changes to the Lisp API

Rfe16071 - Implement breadth-first-search-if functionality

The four breadth-first search SNA functions:

have all been enhanced such that the end argument can be a function of one parameter. If end is such a function, then the search code will call it on each node found during the search and will record a successful search if the function returns true.

Bug25759 - Change load-trix :default-graph argument to :graph

To ensure consistency with other load- functions, :default-graph argument to load-trix has been deprecated and will be removed in the subsequent release. :graph should be used instead.

Bug25738 - AGWebView users are notified when query results may be stale

When reloading queries via the 'Saved' or 'Recent' menus--but also when executing new queries--AGWV will now report when there's a possibility the results displayed are out of date.

Prolog

No changes

Documentation

No changes

AllegroGraph 6.5.0

AllegroGraph Server: General Changes

Rfe15938 - Deprecate --tlog arg to agtool view-tlog

The --tlog argument to agtool view-tlog is deprecated. One need only specify the tlog filename to view.

Rfe15836 - Improve efficiency of dynamic reasoner on rdf:type queries

Increased the speed of the dynamic reasoner when looking for all <SUBJECT rdf:type OBJECT> triples.

Rfe15774 - Make agtools use repo and server specs

All agtool commands are in the process of being modified to use a Repo Spec when referencing a repository and a Server Spec when referencing a server but not a repository. (Repo Specs also identify servers. Server Specs are used when a repository is not relevant to the command, such as with agtool user.) In this realease, the following commands have been modified to accept Repo Specs or Server Specs:

These changes are backward compatible. Previous options are supported but deprecated and may cause warnings to be signaled.

Rfe15748 - Add support for backing up a single remote repository

agtool archive now supports remote repository backup. This operation requires a user with AllegroGraph superuser privileges.

Rfe15745 - Check for no-commit mode before creating a new text index

Creating a new text index requires indexing all existing triples and preparing the new index configuration and datafiles for commit.

Previously creating a new text index on a no-commit repository (such as a warm standby replica) would perform the initial indexing and then fail to commit due to no-commit mode. Now the check for no-commit mode is made early to avoid doing work that will end up being discarded.

Rfe15695 - agtool load now reports the filename with buffer-size exceeded errors.

When the length of a line in an input file exceeds the buffer size an error is reported. Previously this error did not specify which file the error resulted from. This has been changed to report the filename.

Rfe15595 - Add AG directives to choose suppored SSL methods and SSL ciphers

It is now possible to specify SSL protocols and ciphers that will be accepted by AllegroGraph when using HTTPS. This can be achieved by using two new config directives - SSLProtocol and SSLCipherSuites. Configuration directives are described in the Server Configuration and Control document.

Rfe15517 - Change agtool load to use repository specifications

agtool load now accepts a repository specification (REPO_SPEC) argument when specifying a repository. A REPO_SPEC names the repository and also its catalog, and the server scheme, host, port, so all associated arguments which specified those values (--scheme, --host, --port, and --scheme) are deprecated.

Rfe15471 - Support For loading JSON and CSV data in agtool load.

agtool load can now load JSON and CSV files. See the section Loading Raw Data in the Data Import document to learn more about the supported workflows and parameters.

Rfe14214 - Add remote store support to agload

Remote store support has been added to agtool load. It is no longer necessary to run agtool load on the same machine as the AllegroGraph server and as the same user. A username and password must be specified when loading into a remote repository, and that user must have write permission in the repository into which data will be loaded.

Rfe13797 - Reconsider triggering checkpoints on tlog rollover

Automatic checkpoints on tlog rollovers are now disabled.

Rfe10713 - Two phase commit (2PC) support

AllegroGraph can now participate in two phase commit (2PC) operations. See the Two-phase Commit document for details.

Bug25668 - Turtle parser can fail to use same blank node context for all triples in a parse

The turtle parser could fail to unify blank nodes with the same label that appears in RDF lists and outside of those lists. For example, the following data would load with two different nodes instead of one: 

prefix : <ex://#>  
:a :p ( _:x1 _:x1 _:x1 ) .  
_:x1 :p 'object' .

This has been corrected.

Bug25653 - Opening of a repl that should be running

A multi-master replication instance repo will remain open until it is explicitly stopped or for the period of time when the AllegroGraph server is not running. If an instance dies for some reason and a request comes in to open its repo then the repo now will open and rejoin the cluster. Previously an explicit start was required to start the instance and open the repo.

Bug25641 - Unhelpful error message from inability to translate a character

The error message that occurred when a data import was unable to translate a character was unhelpful. This has been corrected.

Bug25596 - RDFS++ reasoner does not follow sameAs for domain or range assertions

The dynamic RDFS++ reasoner was not inferring triples that could be derived from rdfs:domain and rdfs:range statements in addition to owl:sameAs statements. This has been corrected.

Bug25589 - Improve domain and range RDFS++ reasoning when subject not supplied

AllegroGraph's RDFS++ dynamic reasoner was failing to find all rdf:type inferences that used rdfs:range or rdfs:domain when the subject portion was not supplied. This has been corrected.

Bug25588 - Rdfs++ reasoner did not infer range/domain via subProperties

AllegroGraph's dynamic RDFS++ reasoner was not making rdf:type inferences when these required rdfs:domain and rdfs:range reasoning through chains of rdfs:subPropertyOf or owl:inverseOf linkages. This has been corrected.

Bug25586 - Error during recovery of partially-created text index

Previously it was possible for a crash to occur when restarting a repository instance if a partially-created text index existed during at the time of the last checkpoint. This has been fixed.

Bug25578 - xsd:integer casting operator does not work on xsd:longs

The xsd:integer() casting operator was not working on values of type xsd:long. This has been corrected.

Bug25574 - Bidirectional search was inefficient when there is no path

AllegroGraph bidirectional search functions did more work than needed when there was no path to be found between two different nodes. This has been corrected.

Bug25511 - Dynamic restriction reasoner requires subclassing unnecessarily

The dynamic RDFS++ restriction reasoning now recognizes owl:Restrictions that are simple rdfs:subClassOf owl:Restriction and also those that are of rdf:type owl:Restriction.

Bug25496 - agtool upgrade will detect a running repo and exit

agtool upgrade will check if a repo is running and if so print a message that an upgrade is not necessary and then exit with a no-error status of zero. (If a repo can be opened successfully, it does not require upgrading. Once a repo is opened in a version of AllegroGraph later than the version in which is was created, it should not be opened in the earlier version again.)

Bug25390 - Anyone can delete roles

A bug that allowed a user to delete roles without logging in has been fixed.

Bug25353 - Fix various issues when using `agtool export --parallel ...'

Fixes a few issues when exporting in parallel (using --parallel or a --workers argument greater-than 1), and using the default --blank-node-handling strategy of `together'

NQX export could be incomplete due to worker threads incorrectly detecting and skipping triples containing blank-nodes. Now, blank-part detection when exporting NQX works correctly.

Only N-Quads export would check for blank nodes in the graph slot, meaning for other quad formats, some blank nodes would not be properly separated into their own file. Now, NQX, TriX, and TriG formats also check the graph part when saving statements containing blank nodes to a separate file.

Bug25201 - agexport output depends on the value of LANG

It was possible for the value of the LANG environment variable to alter the output of agtool export. This has been corrected.

Bug25114 - Fix printing nquad default graphs

When exporting triples in NQUAD format from a federated or distributed triple store, there could be an unintended "default graph" marker like "default-graph1748246237" after the subject, predicate and object. This has been fixed.

Bug24954 - Fix exporting in NQX format of remote stores

Exporting a remote triple store using "agtool export" in "nqx" format: 

agtool export -o nqx --user test:xyzzy ...

(with both "-o nqx" and a value for "--user")

would result in the error: 

http-triple-cursors cannot change whether or not they return  
extended-triples after they have been created.

This has been fixed.

Bug23424 - Rdfs++ reasoner does not infer subProperties of transitive properties

The dynamic RDFS++ reasoner was failing to infer some triples when they relied on following rdfs:subProperties of owl:transitive properites. This has been corrected.

Bug19476, Bug18483 - restriction reasoning improvements

AllegroGraph's dynamic restriction reasoning was missing some inferences when either the predicate portion of a query was wild or when they required additional rdfs:subClassOf inference on inferred triples. These have both been corrected.

HTTP API

The %D HTTP log directive change

The %D HTTP log directive (meaning response time in microseconds) will now use the high precision clock provided by the operating system rather than the one second resolution clock.

Rfe15876 - Allow specification of Accept-Type in the query parameters

In addition to supporting the normal HTTP Accept-type header, AllegroGraph now also allows an accept parameter to be included in the query portion of an HTTP SPARQL query request. This allows a client to specify the desired content type even when it does not control the headers that it sends. For example, if QUERY is the percent-encoded SPARQL query string, then this sample curl command would return the number of rows that the query produces: 

curl "http://REPOSITORY_URL?query=QUERY&accept=text%2finteger"

Rfe15737 - Add config directive to set HTTP keep-alive timeout

Added a new config directive HTTPKeepAliveTimeout to control the HTTP keep alive timeout. See Server Configuration and Control for information on config directives.

Rfe15731 - Add user-agent header to outgoing HTTP requests

To avoid problems with network proxies that refuse HTTP requests without a User-Agent header, this header is now included in outgoing HTTP requests to remote SPARQL endpoints.

SPARQL

Rfe16032 - Add some list and set handling functions to SPARQL

AllegroGraph now supports three new SPARQL functions:

The first two take a list of arguments and associate then with a variable (gensym). The third looks up an RDF list in the current SPARQL dataset and associates the contents with a variable. In each case, this variable can then be used with other SPARQL magic properties and functions to retrieve and manipulate the list or set.

For example: 

select ?member {  
  bind(<http://franz.com/ns/allegrograph/6.5.0/fn#makeSPARQLSet>(1, 2, 4, 1, 2) as ?set)  
  ?member <http://franz.com/ns/allegrograph/4.11/sna/members> ?set .  
}

Would return 1, 2, and 4.

Rfe16007 - Evaluate the WHERE clause of SPARQL UPDATE queries in REDUCED mode

If a repository contains duplicate triples, then evaluating the WHERE clause of SPARQL update queries could become very memory intensive. Evaluating these in REDUCED mode ameiliorates this problem.

Rfe15967 - Eager filter evaluation could lead to missing query results

If a SPARQL query was supplied parameter values (e.g., via the :with-variables argument to the Lisp API's run-sparql command) and these values produced type-errors, then the FILTER analysis machinery could become confused and produce an incorrect filter. This has been corrected. As an example, the following query and parameters should produce one result but before this bug was fixed, it would produce no results: 

select ?target {  
  optional {}  
  FILTER( ! bound(?target) || ! ( isLiteral(?target) && (lang(?target) = 'en')))  
}

parameters: ((?target . !<ex://test>)). The type-error occurs when the SPARQL lang() function tries to determine that language of the resource !<ex://test>.

Rfe15891 - Provide URI function in SPARQL to parse freetext queries

AllegroGraph now includes a URI function that can be used to see how freetext indexing queries will be parsed. For example: 

prefix test: <http://franz.com/ns/allegrograph/6.5.0/>  
select ?query ?expression {  
  values ?query {  
    '"soccer game"'  
    'soccer | fútbol'  
    'socc?r ping* '  
  }  
  bind(test:parseFreetextQuery( ?query ) as ?expression)  
}

would output: 

---------------------------------------------------------------  
| query           | expression                                |  
===============================================================  
| "soccer game"   | (:phrase "soccer game")                   |  
| soccer | fútbol | (:or "soccer " "fútbol")                  |  
| socc?r ping*    | (:and (:match "socc?r") (:match "ping*")) |  
---------------------------------------------------------------

which shows how each of the queries would be parsed for use by the fti:match magic property.

Rfe15890 - Enhance fti:match and fti:matchExpression to also bind predicates

AllegroGraph's built in Freetext Index magic properties can now also bind the predicate of any matching triples. For example, a query like: 

(?subject ?text ?predicate) fti:match 'sample query' .

Will search all text indices for matches on "sample" AND "query" and then bind ?subject, ?text, and ?predicate to the columns of each matching triple.

Rfe15717 - Query timeouts should only happen during execution not results output

Previously, AllegroGraph would check for query timeouts both during query execution and during result generation. This meant that a timeout could cause partial results to be sent over HTTP. AllegroGraph now only checks for timeouts while the query is executing which means that either no response will be sent or the entire result set will be sent.

Rfe15693 - Optimize some str(?a) = str(?b) filters

AllegroGraph will now execute queries involving str/str equality and inequality FILTERs where the variables in the calls to str are resources more efficiently. For example, if ?a and ?b are resources or plain literals, then this query will be able to execute without needing to look up the string values in the string dictionary: 

SELECT * {  
  :entry :has ?a .  
  :reference :contains ?b .  
  FILTER( str(?a) = str(?b) )  
}

Rfe15692 - Speed up query planning for queries with only simple BGPs

AllegroGraph now avoids computing estimates for queries that contain only single pattern BGPs. E.g., 

prefix f: <ex:foo#>  
select * {  
  graph ?g { ?s2 f:record ?o }  
  { select ?s2 { ?s2 f:record ?o2 } }  
}

Rfe15686 - Handle query + filter patterns more efficiently

This change allows the query + filter handling improvements from rfe15628 (implemented in release 6.4.2) to be used more often. In particular, the filter pattern can now be the last pattern in a BGP and the filter pattern can also introduce new bindings.

Rfe15529 - Let run-sparql intern query variables in one package

SPARQL query variables (the third return value from run-sparql) were sometimes located in the query.variables package, and sometimes in the db.agraph.sbqe package. Now they are always in query.variables.

Rfe13652 - Improve performance of some DISTINCT SPARQL sub-queries

AllegroGraph has long evaluated single variable DISTINCT queries using a specialized and more efficient querying technique. This technique did not, however, apply to sub-queries. This meant that these two equivalent queries would perform significantly differently: 

# Faster  
SELECT DISTINCT ?p { ?s ?p ?o }

and 

# Slower  
SELECT ?p { { SELECT DISTINCT ?p { ?s ?p ?o } } }

The query engine logic has been modified so that the two queries will perform similarly.

Rfe10047 - Basic support for date arithmetic in SPARQL

AllegroGraph now supports adding and subtracting xsd:durations from xsd:dates, xsd:times, and xsd:dateTimes. For example: 

## Find subjects modified since yesterday  
SELECT * {  
  bind( now() - 'P1D'^^xsd:duration as ?yesterday )  
  ?s :hasDate ?date .  
  FILTER( ?date > ?yesterday )  
}

AllegroGraph supports adding durations to date/times, adding date/times to durations and subtracting durations from date/times.

Bug26088 - FILTER(STR(?o) = '..') incorrect for literals with language tags

SPARQL queries like: 

SELECT * {  
  ?s ?p ?o .  
  FILTER(STR(?o) = 'Liechtenstein')  
}

are optimized by the query engine, so that the FILTER clause would be very efficient. However there was an error, in that if ?o were a literal with a language tag, like: 

'Liechtenstein'@de

it would not pass the filter. This has been fixed.

Bug25709 - Predicate statistics invalidation is incomplete

The query engine could incorrectly optimize away SPARQL type casting operations. This has been fixed.

Bug25669 - Logical ORs of inequality filters could produce too many solutions

SPARQL FILTERs that logically OR'd multiple inequality constraints together could be incorrectly processed leading to too many solutions. This has been corrected.

Bug25634 - Impossible OPTIONALs can lose solutions in some cases

If a query began with an OPTIONAL clause and that clause found no bindings and the query then required subsequent processing (e.g., via a BIND() clause), then solutions could be lost. This has been corrected.

Bug25564 - SPARQL queries that used constant expressions could fail over HTTP

AllegroGraph v6.4.2 introduced the eager evaluation of constant expressions (like bind(iri('string') as ?value)). This improvement sometimes caused failures when a query plan with these constants was reused because the value of the expression was not being saved by the plan which would lead to UPI is not in string table errors. This has been corrected.

Bug25545 - UUID() and STRUUID() should not be evaluated eagerly

The SPARQL UUID() and STRUUID() functions were being evaluated at plan time rather than query time. This has been corrected.

Bug25501 - Fix bug in t:intervalCotemporal using start and end times

The SPARQL magic property for t:intervalCotemporal worked correctly when using points on the time line (t:startpoint and t:endpoint) but not when using actual times (t:starttime and t:endtime). This has been corrected.

Bug25465 - Constraint analysis could cause queries with SERVICE clauses to fail

AllegroGraph's planning step could fail to process queries with SERVICE clauses that included FILTERs if those FILTERs attempted to query the current repository. This has been corrected.

The problem was introduced in AllegroGraph v6.4.1.

Bug25454 - Problem with SPARQL division by zero

If the operands to a SPARQL divide arithmatic expression were of type xsd:double, then AllegroGraph could incorrectly signal a SPARQL type error if the divisor was 0 instead of returning positive or negative infinity.

Bug25396 - SPARQL parser bug with unnamed aggregate expressions

AllegroGraph v6.4.1 introduced a bug such that the SPARQL parser would fail to parse a GROUP BY expression that did not provide a new variable. For example: 

SELECT (count(?s) as ?count) {  
  ?s ?p ?o .  
} GROUP BY str(?o)

Queries that did introduce a variable like: 

SELECT ?foo {  
  ?s ?p ?o .  
} GROUP BY (str(?o) AS ?foo)

would parse correctly. This has been corrected.

Bug25391 - SPARQL (count(distinct *) ...) can get the wrong answer

It was possible for SPARQL queries using COUNT(DISTINCT *) to produce the wrong answer. This has been corrected.

Bug25352 - Blank nodes from planning and parsing could incorrectly overlap

If a query used both anonymous blank nodes and property paths, then it was possible for the blank node bookkeeping to incorrectly assign identifiers. For example, a query like: 

select * {  
  [ a ?Class ] .  
  ?x :a/:b ?c .  
}

could be parsed as 

select * {  
  _:b0 a ?Class .  
  ?x :a _:b0 .  
  _:b0 :b ?c .  
}

instead of 

select * {  
  _:b0 a ?Class .  
  ?x :a _:b1 .  
  _:b1 :b ?c .  
}

This problem has been corrected.

Bug25342 - Alternation in Property Paths could return incorrect bindings

If a SPARQL query used alternation in a property path, it was possible that AllegroGraph would bind the subject side of the path to the variable on the object side. This has been corrected.

Bug25334 - SPARQL parser's variable classification was too strict

AllegroGraph was signaling an error if a GROUP BY variable was renamed via a SELECT expression. For example, the following valid query would fail to parse 

prefix : <http://example.org/>  
SELECT  (?entry AS ?other) (MIN(xsd:date(?date)) AS ?minDate) {  
  ?item a  :InternalProcedure .  
  ?entry (:logicalSuccessor)+ ?entry .  
  ?entry :eventDate ?date  
} GROUP BY ?event

This has been corrected.

Bug25198 - Variable mismatch for (run-sparql "ask ..") as :alists

There was a mismatch in the return values for: 

(run-sparql "ASK .." :results-format :alists)

as the alist (first value) could be: 

((?result . t))  
where the variable names (third value) would be: 
(?found)

Now both use the db.agraph.query.variables::?result symbol.

Bug24650 - Improve SPARQL parser's validation of VALUES clauses

AllegroGraph is now more strict when evaluating VALUES clauses where the number of variables in the clause does not match the size of each row of values.

Bug23066 - XSD casting could fail when used with expressions

If an XSD casting operator was used on the result of an expression, the result could be returned as nil. This has been corrected. For example, the following query used to fail with SPARQL type errors: 

select ?value {  
  values (?a ?b) { (1 2) }  
  bind( xsd:double( ?a + ?b ) as ?value )  
}

Bug22548 - Add page rank centrality to Prolog and SPARQL

AllegroGraph now supports computing page rank centrality via the page-rank-centrality Prolog function and also via a new magic property: (link http://franz.com/ns/allegrograph/4.11/sna/pageRankCentrality).

Bug19805 - Dynamic restriction reasoner could fail to infer some triples

The dynamic restriction reasoner was only inferring rdf:type triples when either the subject or the object (or both) was supplied. It would not make inferences when both sides were wild. This has been corrected. Note that inferring all type triples in a query like: 

SELECT ?o ?c { ?o a ?c }

can be expensive due to the amount of computation required.

AGWebView

Bug25510 - Bug in reposcript handling in AGWebVeiw

Previously if you saved or loaded a reposcript from the Utilities | Scripts page, the page would change to one which does not show reposcripts at all. This has been fixed.

Bug25336 - 'Cancel' button does not work in 'Import RDF from a text area input' form

The 'Cancel' button on the 'Import RDF from a text area' dialog used not to work unless an upload had been started. This has been corrected.

Bug25164 - Show metadata in WebView for Parallel SPARQL queries

Parallel SPARQL queries run in WebView against a federated triple store would not show query metadata in the "Information" tab. Now the tab shows a list of the query metadata for each shard of the federated store.

Changes to the Lisp API

Rfe15933 - Remove variable get-triples-list-limit

The variable *get-triples-list-limit*, if not nil, used to limit the number of triples returned by function get-triples-list. As get-triples-list already takes an explicit limit keyword argument, the variable has been removed.

Rfe15879 - Remove RUN-SPARQL :ENGINE argument, VALID-QUERY-ENGINES documentation

Since there is only one SPARQL engine in the current release, the :engine keyword argument to run-sparql and the operators get-sparql-query-engines and valid-query-engines have been removed.

Rfe15870 - Error when a query returns a cursor and there is no query environment

If run-sparql returns a cursor, then it is imperative that the code using that cursor be wrapped with the with-query-environment macro. The lack of wrapping can lead to hard to diagnose problems.

Rfe15828 - Remove preserve-strings and return-[non-]encoded-triples arguments

The Lisp API has long included a preserve-strings argument to the various functions that add triples to a repository (such as add-triple, load-ntriples, and others). When true, both a string version and an encoded version of triples whose objects were typed literals were added. Similarly, functions that retrieved triples (such as get-triples) included arguments to request either only the encoded version or the non-encoded version of said triples (return-encoded-triples and return-non-encoded-triples respectively). These three arguments have been removed from the API.

Rfe15603 - Improve behavior of db when closing an encapsulated triple store

The behavior of variable db when closing or deleting a triple store was not always intuitive. The new behaviour is that (close-triple-store :db db) only updates the binding of the variable db if it had db as its value before the call.

The new value of db depends on the type of db:

Rfe15282 - Allow warmup of individual index flavors

warmup-triple-store now accepts an indices keyword argument to allow specifying which indices to warm up. When the include-triples is true, indices specifies which indices to warm up. :indices nil or unspecified means warm up all indices.

Bug25553 - optimize-indices on a remote-triple-store ignored the wait-p argument.

In the lisp client, if optimize-indices was called on a remote-triple-store with :wait-p t, a non-True boolean value was passed in the HTTP request generated by the call. As a result, the call would return before the optimization operation had completed.

This is now fixed.

Bug25401 - Fix hanging (get-triples :return-extended-triples t) on remote store

Retrieving triples from a remote-triple-store could indefinitely hang if the number of triples was large (typically tens of thousands). This has been resolved.

Prolog

Bug24993 - Default graph in query results

Prolog queries that return the default graph object in a binding caused an error if CSV, TSV or TTL was used as the output format. Such queries also failed to display results in AGWebView. This has been corrected. Any variable bound to the default graph will be treated as unbound.

Bug22548 - Add page rank centrality to Prolog and SPARQL

AllegroGraph now supports computing page rank centrality via the page-rank-centrality Prolog function and also via a new magic property: (link http://franz.com/ns/allegrograph/4.11/sna/pageRankCentrality).

Documentation

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.4.6

AllegroGraph Server

Bug25699: Restoring upi-string got illegal value during recovery/replication

AllegroGraph 6.4.0 introduced a bug that could result in the following error message during crash recovery or when using replication: 

restoring upi-string got illegal value [xxx]

This could prevent a repository from starting up after a crash. This has been fixed.

Bug25653 - A multi-master repl repo that died will restart the next time it is opened

A multi-master repl instance repo (see Multi-master Replication) will remain open until it is explicitly stopped or for the period of time when the agraph server is not running. If an instance dies for some reason and a request comes in to open its repo then the repo now will open and rejoin the cluster. Previously an explicit start was required to start the instance and open the repo.

AGWebView

Rfe16002 - Move query from URL to body

Previously in AGWebView when running a query or downloading query a result, the query was in placed the request URL. This caused problems with long queries with HTTP proxies that limit the length of URLs. Now the query text is placed in the request body.

AllegroGraph 6.4.5

AllegroGraph Server

Bug25586 - Error during recovery of partially-created text index

Previously it was possible for a crash to occur when restarting a repository instance if a partially-created text index existed during at the time of the last checkpoint. This has been fixed.

SPARQL

Bug25564 - SPARQL queries that used constant expressions could fail over HTTP

AllegroGraph v6.4.2 introduced the eager evaluation of constant expressions (like bind(iri('string') as ?value)). This improvement sometimes caused failures when a query plan with these constants was reused because the value of the expression was not being saved by the plan which would lead to UPI is not in string table errors. This has been corrected.

Bug25545 - Uuid() and STRUUID() should not be evaluated eagerly

The SPARQL UUID() and STRUUID() functions were being evaluated at plan time rather than query time. This has been corrected.

AllegroGraph 6.4.4

AllegroGraph Server: Multi-master replication changes

Rfe15786 - Changes while adding a multi-master replication instance

In earlier releases, when a new MMR instance was added, it could not be removed until the addition completed. This caused problems when the addition took excessive time. Now additions can be removed at any time after they are added. Also, the Replication Cluster Management page in AGWebView would not always display correctly while a MMR instance was being added. That has been fixed.

AllegroGraph 6.4.3

AllegroGraph Server: Multi-master replication changes

Rfe15762 - In repl mgr do timeout for repl/stop http calls

Sending an HTTP request to non-existent machine can sometimes take a long time to timeout. The repl/remove and repl/stop commands for Multi-master Replication now accept a timeout parameter to control how long they wait for the affected host to respond before giving up or moving on.

Rfe15737 - Add config directive to set HTTP keep-alive timeout

The new config directive HTTPKeepAliveTimeout specifies the HTTP keep alive timeout. Configuration directives are described in the Server Configuration and Control document. HTTPKeepAliveTimeout is a top-level directive.

Rfe15712 - Improve replication temporary commit error

When it's not possible to commit at the moment due to replication parameters not being satisfied, an error will be signaled that will not backtrace to agraph.log and that will return a 409 "conflict" error from HTTP.

Bug25397 - Replication restart of two instances at the same time

Previously when two replication instances start up at nearly the same time they can get into a state where they fail to link up. This has been fixed.

Bug25364 - Return x-durability header in commit via HTTP

When an HTTP command results in a commit, "x-durability: missing=N" (where N is the number of of instances that are left to commit to satisfy durability) is supposed to be HTTP response headers but it was not. This has been fixed.

AllegroGraph 6.4.2

AllegroGraph Server: General Changes

Rfe15599 - Disallow creation of repos within the system catalog

Attempts to create repositories in the system catalog will now be met with the following error message: 

Repositories cannot be created within the 'system' catalog

Rfe15528 - Increase speed of finding distinct predicates or graphs in a reasoning triple-store

It is now much more efficient to find distinct predicates or graphs when RDFS++ or restriction reasoning is enabled.

Rfe15492: Use triple-store-spec for certain agtools

The following agtools (see agtool) have had their syntax changed to use a triple-store specification instead of the options --port, --user, --password, --host, and --catalog. The affected tools are: cancel-purge-deleted-triples, define-attribute, delete-static-attribute-filter, get-metadata, lookup-attribute-definition, memory-lock, memory-unlock, purge-deleted-triples, purge-rate-limit, set-purge-rate-limit, set-static-attribute-filter. The triple-store specification is 

http(s):[//[USER:PASSWORD@]HOST[:PORT]][/catalogs/CATALOG]/repositories/REPO

Items in brackets ([]) may be left out in some circumstances depending on what is required for the tool.

Rfe15482 - Change if-exists default for agtool repl create-cluster

In agtool repl create-cluster --if-exists argument used to default to supersede and now defaults to error. Existing scripts using agtool repl create-cluster may now fail to work until --if-exists supersede is added to the argument list.

Rfe15481 - Improve turtle serializer output for numbers

AllegroGraph's Turtle and TriG serializers now emit integers, decimals and double floats using the more concise numeric representation rather than the full XSD typed literal one.

Rfe14740 - agtool can now manipulate users and roles

Added agtool user and agtool role commands to manipulate user accounts and roles respectively. For more infomation, execute: 

agtool user --help  
agtool role --help

See the Managing Users document for further information.

Rfe11183 - Add --optimize option to agtool load

agtool load now accepts the --optimize option to request full (level 2) index optimization after loading completes. See the Data Import document. Index optimization is discussed in the Optimizing indices section of the Triple Indices document.

Bug25289 - Dropping a text index deletes files too early

Dropping a freetext index could result in instance startup errors if the instance crashed before a checkpoint occurred. This has been fixed.

Bug25288: Free Text Index state updates not recorded in tlog during commit

Starting in AllegroGraph v6.3.0, changes in text index configuration (such as creation and removal) were not stored in the transaction log during commit. This could result in disappearance of an added text index, or reappearance of a deleted text index after a repository crash restart. This problem is now fixed.

Bug25266 - sna:egoGroup function does not find its generator in all cases

The functional form of the sna:egoGroup magic property did not look up and register its SNA generator function if the generator was stored using triples. This has been corrected.

Bug25236 - Turtle serializer could emit invalid qnames

It was possible for the Turtle serializer to produce invalid output if resources contained characters that needed to be escaped. This has been corrected.

Bug25207 - Using Japanese tokenizer can result in errors

Using the Japanese text index tokenizer on a system that does not have the mecab package installed would result in an error like the following: 

tagger.cpp(153) [load_dictionary_resource(param)] param.cpp(71)  
[ifs] no such file or directory: /etc/mecabrc

This has been fixed.

Bug25201 - agtool export output depends on the value of LANG

It was possible for the value of the LANG environment variable to alter the output of agtool export. This has been corrected.

Bug25138 - agtool CMD will fail if run from a directory it cannot write to.

If the current working directory is not writable agtool query and agtool materialize would stop running with an error the first time they try to write to their respective log files. Now this behavior has been changed to warn the user that it was unable to write to the log file and then continues processing. It continues logging to the terminal as it would normally do. See agtool General Command Utility for a description of agtool.

Bug25115 - 'agtool export' errors when using --parallel and exporting NQX

Attempting to use agtool export with --output nqx and --parallel mode would result in an error like the following: 

error `#<consumer-cursor @ #x1000991a312> does not support extended-triples`  
occurred in worker 11  
interrupting...

This has been fixed.

HTTP API

Bug25246 - Improve HTTP socket reuse for remote repositories

Queries on remote triple stores now reuse the same underlying HTTP socket connection for efficiency.

SPARQL

Rfe15644 - Improve probing decision

AllegroGraph decides at query time whether it is more efficient to scan or probe to evaluate a triple pattern. However, if a pattern had certain constraints on some of its variables, then AllegroGraph was unable to probe. Now AllegroGraph is able to probe in more cases which can greatly improve performance for particular queries.

Rfe15628 - Handle query + filter patterns more efficiently

AllegroGraph now handles SPARQL queries where a pattern in a BGP can act as a filter on the existing set of solutions. As an example, consider this query: 

:target :select ?s .  
?s :has ?property .  
?property rdf:type :animal .

Prior to this change, AllegroGraph would first find all <?s :has ?property> triples and only then filter the results using the <?property rdf:type :animal> pattern. Depending on the shape of the dataset, this could generate a large number of intermediate results. AllegroGraph is now able to treat the <?property rdf:type :animal> pattern as a filter while evaluating the <?s :has ?property> pattern which can help speed the query.

Rfe15607 - Sort language tagged literals sensibly

The SPARQL 1.1 standard does not define an ordering on language tagged literals. AllegroGraph now orders language tagged literals by first comparing their languages and then their string values. Plain and xsd:string typed literals come before any language tagged literal.

Rfe15600 - Remove and warn about unused BINDings

AllegroGraph now issues a query warning if variables introduced by a BIND clause are not used elsewhere in the query. For example: 

select distinct ?o {  
  bind(23 as ?number)  
  ?s ?p ?o .  
}

Rfe15541 - Warn when a BIND form is extending an empty set of solutions

The SPARQL BIND construct operates by extending an existing set of bindings. If there are no bindings, then there is nothing to extend and so the assignment will uniformly fail. AllegroGraph now warns when it encounters this situation. For example, 

SELECT ?ABC {  
  BIND(?s AS ?ABC)  
  ?s ?p ?o .  
}

Previously, AllegroGraph was incorrectly rearranging the above query so that it would return bindings for ?ABC. Now the above query will leave ?ABC unbound and emit a :warn-empty-bind-clause warning.

Rfe15524 - Improve SPARQL property path efficiency

The data structures used to compute SPARQL property paths have been improved to provide better performance.

Rfe15523 - Improve SPARQL dataset efficiency

SPARQL datasets are now slightly more efficient when processing queries.

Rfe15522 - Use existing constant bindings when evaluating subsequent ones

AllegroGraph now uses existing constant BINDings when evaluating subsequent ones during query planning rather than waiting until query execution.

Rfe15515 - Support variable attributes in SPARQL attributes block

AllegroGraph SPARQL now supports variable attributes for SPARQL update operations. For example, the query: 

prefix attr: <http://franz.com/ns/allegrograph/6.2.0/>  
insert {  
  attributes ?attr { ?s ex:newPredicate ?o . }  
} where {  
  ?attr attr:attributes (?s ex:oldPredicate ?o }  
}

would replace each triple whose predicate was ex:oldPredicate with one having the predicate ex:newPredicate and the same attributes.

Rfe15501 - Make better use of FILTER constraints in property path queries

AllegroGraph now uses constraints derived from SPARQL property path queries in more cases. This can help to improve the performance of some queries.

Rfe15413 - Improve handling of constant BIND expressions

AllegroGraph now analyzes constant expressions in BIND forms and can use the results to improve the efficency of some queries.

For example, AllegroGraph previously would execute only the first of the following two logically equivalent queries efficiently: 

# BIND with a constant  
select * {  
  bind( <http://example.com#Mammal> as ?Class )  
  ?a rdf:type ?Class .  
}  
 
# BIND with a constant _expression_  
select * {  
  bind( iri('http://example.com#Mammal') as ?Class )  
  ?a rdf:type ?Class .  
}

Now both of the queries will execute efficiently.

Rfe15381 - Allow federated geospatial query if one store has no geospatial mappings at all

When executing SPARQL geospatial query on federated-triple-stores an error was thrown if federated stores had different geospatial type mappings. This was an overly strict limitation, so that error is now only thrown when regular type mappings differ.

Bug25271: Fix "agtool query" on SPARQL endpoint

Running a query on a SPARQL endpoint using "agtool query", e.g. 

./agtool query http://dbpedia.org/sparql

would, after printing results, signal the error: 

Error "http://dbpedia.org/sparql" is not a triple store occurred during setup

This has been resolved.

Bug25268 - Correct handling of query log line length

It was previously impossible to set the query log line length using the logQuery query option. The log length can now be set with the new logLineLength query parameter.

Bug25247 - Some SPARQL sub-queries with external filters could fail.

AllegroGraph could fail to correctly execute SPARQL queries when they involved a sub-query and FILTERs that applied to triple-patterns within the sub-query. For example: 

select ?s ?timestamp {  
  {  
    select distinct ?s ?timestamp {  
      ?s allegro:hasCreationDate ?timestamp .  
    }  
  }  
  FILTER (?timestamp >= '2018-04-12T20:32:53.213Z'^^xsd:dateTime>)  
  FILTER (?timestamp <= '2018-04-15T20:32:53.213Z'^^xsd:dateTime>)  
}

This has been corrected.

Bug25170 - Property path analysis of missing constants is too stringent

AllegroGraph was marking some property path queries as if they were impossible when elements of the path were not found in the repository. This led to some queries failing to return results. The problem has been corrected.

Bug25197 - Fix handling 'select *' results from remote endpoints

Queries of the form select * { .. } sent to a non-AllegroGraph endpoint could fail with a "result variables are in wrong order" error even though the query results sent by the endpoint were valid. This has been fixed.

Bug25169 - Impossible NOT EXISTS clauses should not make query impossible

The logic that notices when parts of a SPARQL query cannot succeed and marks them as impossible was incorrectly treating NOT EXISTS filters as if they were EXISTS filters which led to some queries failing to return results. This has been corrected.

Bug25151 - Problem with complex NOT EXISTS filters

Queries that used [NOT] EXISTS in a FILTER could lead to an error if the query inside the [NOT] EXISTS contained language filters.

Bug22482 - Improve property path estimates with fixed subject/object

AllegroGraph now does a much better job of estimating the cost of transitive closure (i.e., zero-or-more and one-or-more) property path queries. These improved estimates can lead to greatly improved plans.

AGWebView

Rfe15410: Add repository name and group columns to the AGWebView cluster table

On the replication manager page in Webview columns were added to display the repository name and catalog as well as the group of each instance. See the Managing Clusters Using AGWebView section for information on the WebView cluster table.

Bug25146: Ok button never reenabled after failed text area load

In AllegroGraph WebView when importing statements from a text area input the OK button didn't reenable after the loading process had failed. This has been corrected.

Changes to the Lisp API

Rfe15547 - Add type checking to the upi function

The upi function now includes type checking so that it will signal a clearer error message if passed an argument of the wrong type. The function is now also defined on UPIs in addition to future-parts.

Rfe15499 - delete-triple improvements

delete-triple now accepts a triple object as argument, as alternative to a triple id.

In either case the return value now indicates whether the deletion was successful.

Rfe15483 - Add part-to-namespace to the Lisp API

The Lisp API now includes part-to-namespace. This function takes a UPI, a future part, or a string representing a URI (with or without angle brackets) and returns the Prefixed Name and Local Part (as multiple values) if a registered namespace can be found to abbreviate the resource. For example: 

(part-to-namespace "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>")  
==>  
"rdf"  
"type"

Rfe14068 - Let run-sparql return query metadata for remote triple stores

Calling the run-sparql function on a remote triple store now returns query metadata like timing information as fourth value, just as for local triple stores.

Rfe8199 - Add a with-open-triple-store convenience macro

The Lisp API now includes with-open-triple-store macro which opens a triple-store, executes the body, and then closes it.

Bug25305 - Export cursor-peek from db.agraph package

The cursor-peek function was introduced in AllegroGraph v6.4.1 but was not exported from the db.agraph package. The symbol is now exported and also available in the db.agraph.user package.

Bug25192 - materialize-entailed-triples was not supported in the remote Lisp client

Both materialize-entailed-triples and delete-materialized-triples are now available in the remote Lisp client.

Bug25168 - Fix leaked http-cursor finalizer clash

Cursors can be leaked, for example by using return-from inside iterate-cursor. These cursors are eventually garbage-collected. There was a finalizer race condition in the collection of HTTP connection based cursors, that would lead to stream-closed-error being signaled to the user unexpectedly. This has been resolved.

Bug25168 - Improve cursor leak message for get-triples on remote-triple-store

If get-triples is called on a remote-triple-store outside with-query-environment, the cursor will open an HTTP socket to the triple store. If the cursor is leaked it will be garbage-collected eventually, but until that moment the socket will be kept open. This can negatively impact performance, therefore a warning will now be logged: 

[2018-03-12T09:06:27.250 p13791 W cursor] Cursor created by  
(get-triples) on remote-triple-store "db" was leaked,  
will be closed.

Bug25166 - remote-triple-stores reason on the client rather than the server

AllegroGraph now ensures that the dynamic reasoner will execute on the server rather than the client when using a remote-triple-store. This prevents a large number of get-triples from executing via HTTP when using the RDFS++ reasoner.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.4.1

AllegroGraph Server: General Changes

New agtool commands and some changed arguments

The *agtool command-line utility has new commands available. Further some commands have new arguments and some arguments have changed. See the agtool General Command Utility document for details. Most commands now have a --scheme argument which can be http or https.

agtool repl create-cluster adds --if-exists and --force arguments

agtool repl create-cluster previously would only create new, empty repositories and if an existing repository were specified, it would be deleted and a new, empty repository with the same name would be created. (This behavior differed from the behavior when creating clusters with AGWebView, where a repository had to exist and need not be empty when it was converted to an initial replication instance.) Now agtool repl create-cluster accepts an --if-exists argument with allowable values supersede (the default, meaning delete the existing repository and create a new one with the same name), overwrite (same as supersede), use (convert an existing, possibly non-empty repository), and error (signal an error if the repository specified already exists).

If use is specified, then any encoded id definitions will be examined and if any do not allow for 2^60 distinct strings and so will not work with next-encoded-upi-for-prefix, the conversion will fail and a warning will be printed. If the --force argument with value true is supplied, then the conversion will be done but none of the offending encoded id prefixes must be used again.

The Cluster Replication Manager page now displays statistics

These help users to monitor the state of the cluster. See the Managing clusters in AGWebView once created section in the Multi-master Replication document.

When adding a cluster instance in AGWebView, a progress log is displayed

When adding an instance in Webview you'll see that the log of actions is updated with progress reports describing what is being done. Also the AGWebView replication manager page has new columns and some columns have been renamed. See here for information on using AGWebView to manage clusters.

Rfe15465 - Disallow setting bulk-load-mode in replication instance

In a replication instance one cannot set bulk load mode since replication depends on the transaction logs being an accurate reflection of the changes to the repository and logging is disabled in bulk loading mode. Specifying the --bulk option to agtool load or clicking the Use bulk load mode checkbox when adding triples from an uploaded file in AGWebView will cause the load to fail.

Rfe15451 - Don't fail if unable to write to agload.log

If the current working directory is not writable agtool load would stop running with an error the first time it tries to write to the log file agload.log. Now this behavior has been changed to warn the user that it was unable to write to the log file and then continues processing. It continues logging to the terminal as it would normally do.

Rfe15426 - Expose and document the --parameter option for agtool load

agtool load now accepts the --parameter option for controlling the configuration of newly created repositories. See the --parameter option in the Data Import document for details.

Rfe15418 - Change StringTableCompression default to 'none'

AllegroGraph now in the default creates new repositories with string compression disabled. This results in faster out-of-the-box operation. String compression can be enabled by using the StringTableCompression directive in agraph.cfg. See the description of StringTableCompress in the Server Configuration and Control document for information.

Note that the string compression type can only been set when creating a new repository. The StringTableCompression parameter is further discussed here in the Performance Tuning document.

Rfe15190 - Remove --repl argument from agtool archive backup and agtool recover

The --repl argument for agtool archive backup and agtool recover is no longer present because it is no longer needed (whether or not a repository is a replication repository is now automatically determined).

Rfe14770 - Error message from agload loses most of the detail

When using agtool load the error messages signaled while loading triples were truncated. This is now corrected.

Rfe14389: AllegroGraph no longer supports Callimachus

AllegroGraph no longer supports Callimachus. Stores that were previously set up to integrate with Callimachus will open normally but they will no longer execute the Callimachus integration code.

Bug25087 - TempDir is not being used for placing temporary query files

Temporary query files were being placed in the repository's directory rather than in the location specified by the TempDir configuration parameter. This has been corrected.

Bug25075 - Bidirectional search can return bad paths

If there were many possible overlapping bidirectional paths between two nodes, AllegroGraph could return incorrect paths. This has been corrected.

Bug25102 - Subscript UPIs did not print correctly.

AllegroGraph was printing the prefix of a subscript UPIs twice. This has been corrected. Previously, they would print as 

<http://franz.com/ns/allegrograph/4.14/index#_http://franz.com/ns/allegrograph/4.14/index#_4>

Now they print as 

<http://franz.com/ns/allegrograph/4.14/index#_4>

Bug25062 - Fix uninformative error when running agraph-control as root.

When running agraph-control as root when there was no RunAs directive in agraph.cfg would result in the following uninformative error: 

Got error while executing plan: `nil' is not of the expected type  
`(array character (*))' for foreign type `(* :char)'

Now the error messages generated for no RunAs user and non-existent RunAs user, respectively, are: 

# bin/agraph-control --config lib/agraph.cfg start  
Running AllegroGraph as superuser is not recommended for security reasons.  
It is recommended that another less privileged user be used, specified  
via the `RunAs' directive in agraph.cfg.  
 
# bin/agraph-control --config lib/agraph.cfg start  
RunAs user `UserThatDoesNotExist' does not exist in the passwd database.  
 
Please run the AllegroGraph configuration script or adjust the `RunAs'  
directive in lib/agraph.cfg

HTTP API

Rfe15418 - Allow specifying string table compression method

Added a stringTableCompression parameter to the repository creation HTTP service.

SPARQL

Rfe15421 - Add control over duplicate triple reduction in SPARQL queries

AllegroGraph now has a new query option: enableDuplicateReduction which controls whether or not the SPARQL engine reduces the number of duplicate triples it processes. See SPARQL Query Options in the SPARQL Reference document.

If the store has no duplicate triples (based on subject, predicate, object, and graph) then this option has no effect.

SPARQL is defined on sets of triples (i.e., no duplicates) but AllegroGraph can store duplicates which can lead to duplicate rows of bindings returned by a query.

If this option is set to true, then AllegroGraph will ignore consecutive duplicate triples returned from the storage layer. Since the storage layer makes no guarentees on the order of triples it returns, this can lead to different results from the same query.

Note: Previous versions of AllegroGraph used duplicate reduction by default. This version turns if off by default. If your triple-store has duplicate triples, then this may change the results of your queries.

Rfe15419 - Include query modifiers when printing a SPARQL query plan

AllegroGraph now includes query modifiers like ORDER BY or LIMIT when it prints a query plan.

Rfe15391 - Warn when the variable HOST of a SERVICE clause is not bound

AllegroGraph now warns when the HOST variable of a SERVICE clause is unbound in the query.

For example, both of these queries will produce query warnings: 

# Query 1  
select * {  
 {  
   :myService :isAtURL ?url  
 } union {  
   service ?url { ?s ?p ?o }  
 }  
}  
 
# Query 2  
select * { service ?unknown { ?s ?p ?o } }

Rfe15354 - Improve handling of queries containing GRAPH and JOIN clauses

AllegroGraph will now handle queries involving GRAPH and JOIN clauses more efficiently.

Rfe15336 - Sanity check on JSON/XML data returned by SPARQL endpoint

If a SPARQL query includes a SERVICE call, the XML and JSON result data is now subject to more syntactic and semantic error checking. Nevertheless invalid result data from third parties might still lead to unexpected outcomes.

Rfe15273 - Add Javascript API for defining SPARQL extensions

AllegroGraph's server side Javascript engine now supports the following:

For details, see the Javascript documentation.

Bug25145 - Illegal keyword error when executing some queries

AllegroGraph could signal an Illegal Keyword error when executing some complex queries. This has been corrected.

Bug25113: SPARQL planner stopped optimizing certain queries

If a query has a FILTER like 

FILTER( ?var1 = ?var2 )

and it can be determined that both ?var1 and ?var2 are resources, then AllegroGraph can perform certain query optimizations. This feature was inadvertently broken in AllegroGraph 6.4 and has now been reinstated.

Bug25097 - Graph not in dataset warning signaled inappropriately

AllegroGraph v6.3.0 and following signal a query warning if the graph in a GRAPH clause is not in the dataset as in, for example, this query: 

select *  
from named <http://example.com#graph> {  
  graph <http://example.org#otherGraph> { ?s ?p ?o }  
}

AllegroGraph was signaling this warning more often than necessary. This has been corrected.

Bug25064 - GROUP BY expressions in a SUB-QUERY causes an error

AllegroGraph was signaling an error if a SPARQL query used GROUP BY expressions in a SUB-QUERY. This has been corrected. An example query that used to fail would be 

select * {  
  { select ?strWho (sum(?amount) as ?sum) {  
      ?who ?p ?amount .  
    } group by (str(?who) as ?strWho)  
  }  
}

Bug25057 - Values and OPTIONAL could lead to too many results

If a SPARQL query used a VALUES form inside the query and the VALUES form had only a single variable and a single binding for that variable and the query also used an OPTIONAL form that made use of the same variable, then AllegroGraph could return results with the variable unbound. This has been corrected.

As an example, AllegroGraph previously could have returned results with ?o2 unbound for the following query: 

SELECT ?s ?o1 ?o2 {  
  ?s ?p1 ?o1  
  OPTIONAL { ?s foaf:knows ?o2 }  
  VALUES (?o2) { (:b) }  
}

Bug25055 - Improve SPARQL parser error message for bad aggregation expression

AllegroGraph was signaling an error for bad aggregation expressions rather than indicating that the query failed to parse. For example, this query is invalid because it is illegal to GROUP BY an aggreagate: 

select (sum(?o) as ?sum) (sample(?s) as ?ss) {  
  ?s ?p ?o .  
} group by (sum(?o) as ?xxx) having (?xxx > 2)

AllegroGraph now signals a SPARQL parser error for queries like this.

Bug25034 - Catch additional SPARQL parse errors

SPARQL is defined such that repeated variables in the projection are ignored. Previously, AllegroGraph would include each repeated variable. Now, it will include only the first instance and will warn if the list includes repetition.

For example, this query: 

SELECT ?s ?g ?s { graph ?g { ?s ?p ?o } }

Will project only ?s and ?g and will warn that "The following variable is projected multiple times: ?s. Only the first projection will be included".

The variables in expressions and aggregation specifications may not overlap with other projected variables. AllegroGraph now catches errors like: 

SELECT ?o (str(?o) as ?o) {  
  ?s ?p ?o .  
}

or 

SELECT ?s (count(1) as ?s) {  
  ?s ?p ?o .  
} GROUP BY ?s

Previously, the above queries would have passed the parser.

AGWebView

Rfe15395 - Highlight all instances of the token under cursor in the query editor

AGWebView's SPARQL query editor now highlights any tokens that match the one currently under the cursor. This makes it easier to ensure that variable names are not mistyped.

Changes to the Lisp API

rfe15430 Use of next-encoded-upi-for-prefix in multi-master cluster instances

In order to use next-encoded-upi-for-prefix when the repository is a multi-master cluster instance, the template associated with the encoded id prefix must generate exactly 2^60 distinct strings. If it does not, the call to next-encoded-upi-for-prefix will signal an error. As noted just below, the "plain" will work as a template. See register-encoded-id-prefix.

Rfe15466 - Plain format encoded ids

register-encoded-id-prefix will treat the format argument of "plain" (a string) in this special way: The encoded id (the string after the @@) is a variable length integer in the range from 0 to (2^60 -1), much like blank nodes. This is a template suitable for use in multi-master cluster instances. See the Encoded IDs document.

rfe15430 Use of next-encoded-upi-for-prefix in multi-master cluster instances

In order to use next-encoded-upi-for-prefix when the repository is a multi-master cluster instance, the template associated with the encoded id prefix must generate exactly 2^60 distinct strings. If it does not, the call to next-encoded-upi-for-prefix will signal an error. As noted just above, the "plain" will work as a template. See register-encoded-id-prefix.

Rfe15379 - Remove in-line-data argument from the run-sparql

The in-line-data argument to run-sparql is no longer supported. If you need to execute the same basic query with different sets of BINDINGS, then use the VALUES form in your SPARQL query string.

Rfe15373 - Export peekable-cursors

The Lisp API now includes these new functions:

A peekable-cursor is like a regular cursor except cursor-peek can be used to see what cursor-next-row will return without modifying the state of the cursor. See the documentation for details.

Bug25119 - load-turtle not accepting a :base-uri arg in remote lisp client

The remote lisp client did not support the base-uri argument to load-turtle. This has been corrected.

Bug25070 - upi->value uses db when decoded encoded IDs

The upi->value function was using db to decode encoded IDs even when a specific value for db was passed into the function. This has been corrected.

Bug25043 - get-triples-nd-* functions were incompletely documented

The various nD get-triples functions (like get-triples-nd-within-bounding-box) return :part-not-found as a second return value if it can be determined that one of the search parameters is definitely not in the repository. This is now documented.

Bug23635: Make Lisp client :default-graph-uris keyword work with remote-triple-stores

SPARQL queries made against remote-triple-stores via run-sparql, that passed the :default-graph-uris keyword argument, would not return expected results. This has now been fixed.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

Bug25129: Bus error on geospatial search with a non-existing predicate

The bus error on the server happened when trying to obtain triples from polygon using predicate that was not previously added to the store.

Behaviour for obtaining triples from polygons was modified to properly handle this case.

AllegroGraph 6.4.0

AllegroGraph Server: General Changes

Rfe15302 - Remove support for using rapper as an external parser

The use-rapper argument to the AllegroGraph data import commands has been removed because AllegroGraph now supports all of the standard formats.

The rapper command line argument to agtool load has similarly been removed.

Rfe15219 - Add timeout and limit arguments to the SNA path-finding functions

The SNA path finding functions now have a timeout argument and a limit argument. The first specifies the number of seconds to execute the search whereas the second indicates that searching should stop once that many paths have been found.

The SNA path functions will return (as a second value) the status of the search. This will be either nil if the search was not interrupted, :timeout if the search reached the timeout, or :limit-reached if the limit on the number of paths was reached.

Note that the return format for the REST API to the SNA path functions is now versioned. The desired version if specified via a version parameter in the query string. If the version is 1, then the previous result format of a list of lists representing paths will be returned. Otherwise, the new format will be used. The new format consists of a JSON object with two keys: "paths" and "status". "paths" contains the list of paths and "status" contains the reason that the search completed. It will be one of "complete", "timeout", or "limit-reached".

Bug24980 - Turtle serializer escapes the PNAME_NS incorrectly

The Turtle serializer was escaping both the namespace portion and the local portion of a qualified name using the same logic. This was incorrect as the escaping for the two differ. The problem has been corrected.

HTTP API

No significant changes.

SPARQL

Rfe15337 - Have the NOW() SPARQL function include a timezone

The SPARQL NOW() function now returns dateTimes in GMT. Previously, the dateTimes had no explicit timezone.

Rfe15259 - Add a query option to limit the size of query log entries

AllegroGraph's SPARQL engine now truncates query log message to 512 characters by default. This limit can be changed by adding ":LINE_LENGTH" to the logQuery query option either in the AllegroGraph configuration file or on a per-query basis. For example, this PREFIX line will limit log lines to 100-characters: 

PREFIX franzOption_logQuery: <franz:yes:100>

Rfe15258 - Add maximumValuesCountForService query option

AllegroGraph uses the SPARQL VALUES clause to pass partial results to SPARQL endpoints. In general this can help the endpoint compute its results more efficiently but when there are many values the cost of transmitting the data can outweigh these benefits.

The maximumValuesCountForService provides a mechanism to limit the total number of VALUES sent.

Rfe15250 - Improve treatment of very large VALUES clauses

The query planner now handles large VALUES clauses significantly faster. In addition, the query executor now decides whether to probe or scan based on the number of VALUES and the expected number of rows that a cursor will need to traverse. These two changes can make some queries significantly faster.

Rfe11137: Support the TIMEZONE() and TZ() functions

AllegroGraph now supports the SPARQL 1.1. TIMEZONE() and TZ() functions.

Bug25003 - FILTERing on an expression like ( ! ?a = ?b ) caused an error

The above expression evaluates as (!?a) = ?b and AllegroGraph was not correctly handling the expression of the inner negation operator. This has been corrected.

Bug24985 - FILTER EXISTS and SUB-QUERY can lose solutions

If a FILTER EXISTS clause started with a sub-query then AllegroGraph could fail to find some solutions. This has been corrected.

Bug24657 - GRAPH filter can be applied to the wrong clause

If a graph variable appeared in the subject, predicate, or object position of a triple-pattern, it was possible for AllegroGraph to apply a GRAPH dataset constraint to it incorrectly. For example, AllegroGraph could have incorrectly determined that the ?g1 :predicate1 ?object pattern in the query below should be evaluated as if it was inside a GRAPH clause which could lead to incorrect results. 

select * {  
  GRAPH ?g1 { ?s1 ?p1 ?o1 } .  
  GRAPH ?g2 { ?s2 ?p2 ?o1 } .  
  ?g1 :predicate1 ?object .  
}

This has been corrected.

Bug24151 - Correct a SPARQL FILTER analysis issue

AllegroGraph's SPARQL engine would signal an error if a FILTER contained an expression like 

str( ?varOne ) = ?varTwo

This has been corrected.

Bug20593 - Support the SPARQL 1.1 BNODE( identifier ) function

AllegroGraph has long supported the BNODE() function with no arguments. It now also supports the version that takes a single plain literal or XSD string argument.

Bug24076 - Improve handling of DBPedia.org and other Virtuoso servers as SPARQL endpoints.

AllegroGraph works around an apparent problem with Virtuoso.

AGWebView

Rfe11480 - Added Shutdown instance operation to Repository Control section in AGWebView

See the desription of the repository page in the WebView document.

Bug24949 - Overview page broken for remote sessions

Opening a session to a single, remote store in WebView used to produce the following error message:

Loading failed: Service not supported in a session that does not wrap a single local store.

instead of the overview page. This has been corrected.

Bug24929 - Help messages appear out of view.

Help messages were often shown at the top of the Webview page, out of sight of the current page location. They will now be displayed in tooltips.

Changes to the Lisp API

Rfe15325: Make *db* a required thread binding

To help avoid accidental sharing of a repository connection between threads, each new thread will start with *db* bound to nil. Previously, *db* would retain its global value in new threads.

Given this sample code: 

(open-triple-store "test") ;; sets global value of *db*  
 
(mp:process-run-function "new thread"  
   (lambda ()  
     ;; The new thread wants its own connection to the repository  
     (open-triple-store "test")))

Prior to v6.4.0, the call to open-triple-store in "new thread" would overwrite the global value of *db*. In v6.4.0, "new thread" will have its own binding of *db* and the global value will not be affected.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

User-visible changes in earlier releases

See Change History for user-visible changes to earlier releases.