Introduction

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 6.4.6.

Release 6.4.6

Release 6.4.6 is a maintenance release with code fixes and enhancements.

Database internal format unchanged from version 6.4.5 (last change 6.4.0)

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.4.6 is unchanged from version 6.4.5. (The format was previously changed in release 6.4.0.) Therefore it is not necessary to upgrade 6.4.5 databases for them to be used with 6.4.6. Note that backing up repositories before before using them to a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether or not the database format is the same.

Release 6.4.5

Release 6.4.5 is a maintenance release with code fixes and enhancements.

Release 6.4.4

Release 6.4.4 is a maintenance release with code fixes and enhancements. Most changes relate to Multi-master Replication.

Release 6.4.3

Release 6.4.3 is a maintenance release with code fixes and enhancements. Most changes relate to Multi-master Replication.

Release 6.4.2

Release 6.4.2 is a maintenance release with code fixes and enhancements.

Release 6.4.1

Release 6.4.1 is a maintenance release with code fixes and enhancements.

Release 6.4.0

Release 6.4.0 is a major release with significant new features.

The most important and far-reaching change is support for multi-master replication. AllegroGraph has long supported single-master replication, where several AllegroGraph instances share data in a repository, but only one of them can make changes (adding or deleting triples). The rest then catch up. In multi-master replication, even though one instance is identified as the controlling instance, any instance can add or delete triples, with the remainder catching up with those changes while perhaps making other changes of their own. Single-master replication is still supported and is described in the Replication document. The new multi-master replication facility is described in Multi-master Replication.

Database internal format changed from version 6.3.0

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.4.0 is changed from version 6.3.0. (The format was previously changed in release 6.2.0.) Therefore it is necessary to upgrade 6.3.0 and earlier databases for them to be used with 6.4.0. Note that backing up repositories before before upgrading them to a new version is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

Programmer notes

These sections note changes which may affect how certain operations or code work compared to earlier releases. These are emphasized here because the changes may be unexpected. The complete list of user-visible changes below also mentions these changes in a shorter format.

6.4.6 Programmer notes

Several recent patches to Allegro CL modified the way URIs are handled in Allegro CL. These changes are incompatible with AllegroGraph 6.4.x and should not be applied to a Lisp client intended to be used with those versions of AllegroGraph. The patches are uri/uri.001 (posted in September, 2018) through uri/uri.006 (posted January 16, 2019). If you downloaded these patches, remove them from the uri/ subdirectory of the Allegro CL distribution directory and update your Lisp by executing update.sh/update.exe.

6.4.5 Programmer notes

See the note about URI patches ubder 6.4.6 Programmer notes above. That note applies to AllegroGraph 6.4.5.

6.4.4 Programmer notes

See the note about URI patches ubder 6.4.6 Programmer notes above. That note applies to AllegroGraph 6.4.4.

6.4.3 Programmer notes

6.4.2 Programmer notes

6.4.1 Programmer notes

6.4.0 Programmer notes

## User visible changes

These sections list changes in the current release and in some earlier releases. See the Change History for a list of changes in all earlier releases.

AllegroGraph 6.4.6

AllegroGraph Server

Bug25653 - A multi-master repl repo that died will restart the next time it is opened

A multi-master repl instance repo (see Multi-master Replication) will remain open until it is explicitly stopped or for the period of time when the agraph server is not running. If an instance dies for some reason and a request comes in to open its repo then the repo now will open and rejoin the cluster. Previously an explicit start was required to start the instance and open the repo.

AGWebView

Rfe16002 - Move query from URL to body

Previously in AGWebView when running a query or downloading query a result, the query was in placed the request URL. This caused problems with long queries with HTTP proxies that limit the length of URLs. Now the query text is placed in the request body.

AllegroGraph 6.4.5

AllegroGraph Server

Bug25586 - Error during recovery of partially-created text index

Previously it was possible for a crash to occur when restarting a repository instance if a partially-created text index existed during at the time of the last checkpoint. This has been fixed.

SPARQL

Bug25564 - SPARQL queries that used constant expressions could fail over HTTP

AllegroGraph v6.4.2 introduced the eager evaluation of constant expressions (like bind(iri('string') as ?value)). This improvement sometimes caused failures when a query plan with these constants was reused because the value of the expression was not being saved by the plan which would lead to UPI is not in string table errors. This has been corrected.

Bug25545 - Uuid() and STRUUID() should not be evaluated eagerly

The SPARQL UUID() and STRUUID() functions were being evaluated at plan time rather than query time. This has been corrected.

AllegroGraph 6.4.4

AllegroGraph Server: Multi-master replication changes

Rfe15786 - Changes while adding a multi-master replication instance

In earlier releases, when a new MMR instance was added, it could not be removed until the addition completed. This caused problems when the addition took excessive time. Now additions can be removed at any time after they are added. Also, the Replication Cluster Management page in AGWebView would not always display correctly while a MMR instance was being added. That has been fixed.

AllegroGraph 6.4.3

AllegroGraph Server: Multi-master replication changes

Rfe15762 - In repl mgr do timeout for repl/stop http calls

Sending an HTTP request to non-existent machine can sometimes take a long time to timeout. The repl/remove and repl/stop commands for Multi-master Replication now accept a timeout parameter to control how long they wait for the affected host to respond before giving up or moving on.

Rfe15737 - Add config directive to set HTTP keep-alive timeout

The new config directive HTTPKeepAliveTimeout specifies the HTTP keep alive timeout. Configuration directives are described in the Server Configuration and Control document. HTTPKeepAliveTimeout is a top-level directive.

Rfe15712 - Improve replication temporary commit error

When it's not possible to commit at the moment due to replication parameters not being satisfied, an error will be signaled that will not backtrace to agraph.log and that will return a 409 "conflict" error from HTTP.

Bug25397 - Replication restart of two instances at the same time

Previously when two replication instances start up at nearly the same time they can get into a state where they fail to link up. This has been fixed.

Bug25364 - Return x-durability header in commit via HTTP

When an HTTP command results in a commit, "x-durability: missing=N" (where N is the number of of instances that are left to commit to satisfy durability) is supposed to be HTTP response headers but it was not. This has been fixed.

AllegroGraph 6.4.2

AllegroGraph Server: General Changes

Rfe15599 - Disallow creation of repos within the system catalog

Attempts to create repositories in the system catalog will now be met with the following error message: 

Repositories cannot be created within the 'system' catalog

Rfe15528 - Increase speed of finding distinct predicates or graphs in a reasoning triple-store

It is now much more efficient to find distinct predicates or graphs when RDFS++ or restriction reasoning is enabled.

Rfe15492: Use triple-store-spec for certain agtools

The following agtools (see agtool) have had their syntax changed to use a triple-store specification instead of the options --port, --user, --password, --host, and --catalog. The affected tools are: cancel-purge-deleted-triples, define-attribute, delete-static-attribute-filter, get-metadata, lookup-attribute-definition, memory-lock, memory-unlock, purge-deleted-triples, purge-rate-limit, set-purge-rate-limit, set-static-attribute-filter. The triple-store specification is 

http(s):[//[USER:PASSWORD@]HOST[:PORT]][/catalogs/CATALOG]/repositories/REPO

Items in brackets ([]) may be left out in some circumstances depending on what is required for the tool.

Rfe15482 - Change if-exists default for agtool repl create-cluster

In agtool repl create-cluster --if-exists argument used to default to supersede and now defaults to error. Existing scripts using agtool repl create-cluster may now fail to work until --if-exists supersede is added to the argument list.

Rfe15481 - Improve turtle serializer output for numbers

AllegroGraph's Turtle and TriG serializers now emit integers, decimals and double floats using the more concise numeric representation rather than the full XSD typed literal one.

Rfe14740 - agtool can now manipulate users and roles

Added agtool user and agtool role commands to manipulate user accounts and roles respectively. For more infomation, execute: 

agtool user --help  
agtool role --help

See the Managing Users document for further information.

Rfe11183 - Add --optimize option to agtool load

agtool load now accepts the --optimize option to request full (level 2) index optimization after loading completes. See the AllegroGraph Loader document. Index optimization is discussed in the Optimizing indices section of the Triple Indices document.

Bug25289 - Dropping a text index deletes files too early

Dropping a freetext index could result in instance startup errors if the instance crashed before a checkpoint occurred. This has been fixed.

Bug25288: Free Text Index state updates not recorded in tlog during commit

Starting in AllegroGraph v6.3.0, changes in text index configuration (such as creation and removal) were not stored in the transaction log during commit. This could result in disappearance of an added text index, or reappearance of a deleted text index after a repository crash restart. This problem is now fixed.

Bug25266 - sna:egoGroup function does not find its generator in all cases

The functional form of the sna:egoGroup magic property did not look up and register its SNA generator function if the generator was stored using triples. This has been corrected.

Bug25236 - Turtle serializer could emit invalid qnames

It was possible for the Turtle serializer to produce invalid output if resources contained characters that needed to be escaped. This has been corrected.

Bug25207 - Using Japanese tokenizer can result in errors

Using the Japanese text index tokenizer on a system that does not have the mecab package installed would result in an error like the following: 

tagger.cpp(153) [load_dictionary_resource(param)] param.cpp(71)  
[ifs] no such file or directory: /etc/mecabrc

This has been fixed.

Bug25201 - agtool export output depends on the value of LANG

It was possible for the value of the LANG environment variable to alter the output of agtool export. This has been corrected.

Bug25138 - agtool CMD will fail if run from a directory it cannot write to.

If the current working directory is not writable agtool query and agtool materialize would stop running with an error the first time they try to write to their respective log files. Now this behavior has been changed to warn the user that it was unable to write to the log file and then continues processing. It continues logging to the terminal as it would normally do. See agtool General Command Utility for a description of agtool.

Bug25115 - 'agtool export' errors when using --parallel and exporting NQX

Attempting to use agtool export with --output nqx and --parallel mode would result in an error like the following: 

error `#<consumer-cursor @ #x1000991a312> does not support extended-triples`  
occurred in worker 11  
interrupting...

This has been fixed.

HTTP API

Bug25246 - Improve HTTP socket reuse for remote repositories

Queries on remote triple stores now reuse the same underlying HTTP socket connection for efficiency.

SPARQL

Rfe15644 - Improve probing decision

AllegroGraph decides at query time whether it is more efficient to scan or probe to evaluate a triple pattern. However, if a pattern had certain constraints on some of its variables, then AllegroGraph was unable to probe. Now AllegroGraph is able to probe in more cases which can greatly improve performance for particular queries.

Rfe15628 - Handle query + filter patterns more efficiently

AllegroGraph now handles SPARQL queries where a pattern in a BGP can act as a filter on the existing set of solutions. As an example, consider this query: 

:target :select ?s .  
?s :has ?property .  
?property rdf:type :animal .

Prior to this change, AllegroGraph would first find all <?s :has ?property> triples and only then filter the results using the <?property rdf:type :animal> pattern. Depending on the shape of the dataset, this could generate a large number of intermediate results. AllegroGraph is now able to treat the <?property rdf:type :animal> pattern as a filter while evaluating the <?s :has ?property> pattern which can help speed the query.

Rfe15607 - Sort language tagged literals sensibly

The SPARQL 1.1 standard does not define an ordering on language tagged literals. AllegroGraph now orders language tagged literals by first comparing their languages and then their string values. Plain and xsd:string typed literals come before any language tagged literal.

Rfe15600 - Remove and warn about unused BINDings

AllegroGraph now issues a query warning if variables introduced by a BIND clause are not used elsewhere in the query. For example: 

select distinct ?o {  
  bind(23 as ?number)  
  ?s ?p ?o .  
}

Rfe15541 - Warn when a BIND form is extending an empty set of solutions

The SPARQL BIND construct operates by extending an existing set of bindings. If there are no bindings, then there is nothing to extend and so the assignment will uniformly fail. AllegroGraph now warns when it encounters this situation. For example, 

SELECT ?ABC {  
  BIND(?s AS ?ABC)  
  ?s ?p ?o .  
}

Previously, AllegroGraph was incorrectly rearranging the above query so that it would return bindings for ?ABC. Now the above query will leave ?ABC unbound and emit a :warn-empty-bind-clause warning.

Rfe15524 - Improve SPARQL property path efficiency

The data structures used to compute SPARQL property paths have been improved to provide better performance.

Rfe15523 - Improve SPARQL dataset efficiency

SPARQL datasets are now slightly more efficient when processing queries.

Rfe15522 - Use existing constant bindings when evaluating subsequent ones

AllegroGraph now uses existing constant BINDings when evaluating subsequent ones during query planning rather than waiting until query execution.

Rfe15515 - Support variable attributes in SPARQL attributes block

AllegroGraph SPARQL now supports variable attributes for SPARQL update operations. For example, the query: 

prefix attr: <http://franz.com/ns/allegrograph/6.2.0/>  
insert {  
  attributes ?attr { ?s ex:newPredicate ?o . }  
} where {  
  ?attr attr:attributes (?s ex:oldPredicate ?o }  
}

would replace each triple whose predicate was ex:oldPredicate with one having the predicate ex:newPredicate and the same attributes.

Rfe15501 - Make better use of FILTER constraints in property path queries

AllegroGraph now uses constraints derived from SPARQL property path queries in more cases. This can help to improve the performance of some queries.

Rfe15413 - Improve handling of constant BIND expressions

AllegroGraph now analyzes constant expressions in BIND forms and can use the results to improve the efficency of some queries.

For example, AllegroGraph previously would execute only the first of the following two logically equivalent queries efficiently: 

# BIND with a constant  
select * {  
  bind( <http://example.com#Mammal> as ?Class )  
  ?a rdf:type ?Class .  
}  
 
# BIND with a constant _expression_  
select * {  
  bind( iri('http://example.com#Mammal') as ?Class )  
  ?a rdf:type ?Class .  
}

Now both of the queries will execute efficiently.

Rfe15381 - Allow federated geospatial query if one store has no geospatial mappings at all

When executing SPARQL geospatial query on federated-triple-stores an error was thrown if federated stores had different geospatial type mappings. This was an overly strict limitation, so that error is now only thrown when regular type mappings differ.

Bug25271: Fix "agtool query" on SPARQL endpoint

Running a query on a SPARQL endpoint using "agtool query", e.g. 

./agtool query http://dbpedia.org/sparql

would, after printing results, signal the error: 

Error "http://dbpedia.org/sparql" is not a triple store occurred during setup

This has been resolved.

Bug25268 - Correct handling of query log line length

It was previously impossible to set the query log line length using the logQuery query option. The log length can now be set with the new logLineLength query parameter.

Bug25247 - Some SPARQL sub-queries with external filters could fail.

AllegroGraph could fail to correctly execute SPARQL queries when they involved a sub-query and FILTERs that applied to triple-patterns within the sub-query. For example: 

select ?s ?timestamp {  
  {  
    select distinct ?s ?timestamp {  
      ?s allegro:hasCreationDate ?timestamp .  
    }  
  }  
  FILTER (?timestamp >= '2018-04-12T20:32:53.213Z'^^xsd:dateTime>)  
  FILTER (?timestamp <= '2018-04-15T20:32:53.213Z'^^xsd:dateTime>)  
}

This has been corrected.

Bug25170 - Property path analysis of missing constants is too stringent

AllegroGraph was marking some property path queries as if they were impossible when elements of the path were not found in the repository. This led to some queries failing to return results. The problem has been corrected.

Bug25197 - Fix handling 'select *' results from remote endpoints

Queries of the form select * { .. } sent to a non-AllegroGraph endpoint could fail with a "result variables are in wrong order" error even though the query results sent by the endpoint were valid. This has been fixed.

Bug25169 - Impossible NOT EXISTS clauses should not make query impossible

The logic that notices when parts of a SPARQL query cannot succeed and marks them as impossible was incorrectly treating NOT EXISTS filters as if they were EXISTS filters which led to some queries failing to return results. This has been corrected.

Bug25151 - Problem with complex NOT EXISTS filters

Queries that used [NOT] EXISTS in a FILTER could lead to an error if the query inside the [NOT] EXISTS contained language filters.

Bug22482 - Improve property path estimates with fixed subject/object

AllegroGraph now does a much better job of estimating the cost of transitive closure (i.e., zero-or-more and one-or-more) property path queries. These improved estimates can lead to greatly improved plans.

AGWebView

Rfe15410: Add repository name and group columns to the AGWebView cluster table

On the replication manager page in Webview columns were added to display the repository name and catalog as well as the group of each instance. See the Managing Clusters Using AGWebView section for information on the WebView cluster table.

Bug25146: Ok button never reenabled after failed text area load

In AllegroGraph WebView when importing statements from a text area input the OK button didn't reenable after the loading process had failed. This has been corrected.

Changes to the Lisp API

Rfe15547 - Add type checking to the upi function

The upi function now includes type checking so that it will signal a clearer error message if passed an argument of the wrong type. The function is now also defined on UPIs in addition to future-parts.

Rfe15499 - delete-triple improvements

delete-triple now accepts a triple object as argument, as alternative to a triple id.

In either case the return value now indicates whether the deletion was successful.

Rfe15483 - Add part-to-namespace to the Lisp API

The Lisp API now includes part-to-namespace. This function takes a UPI, a future part, or a string representing a URI (with or without angle brackets) and returns the Prefixed Name and Local Part (as multiple values) if a registered namespace can be found to abbreviate the resource. For example: 

(part-to-namespace "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>")  
==>  
"rdf"  
"type"

Rfe14068 - Let run-sparql return query metadata for remote triple stores

Calling the run-sparql function on a remote triple store now returns query metadata like timing information as fourth value, just as for local triple stores.

Rfe8199 - Add a with-open-triple-store convenience macro

The Lisp API now includes with-open-triple-store macro which opens a triple-store, executes the body, and then closes it.

Bug25305 - Export cursor-peek from db.agraph package

The cursor-peek function was introduced in AllegroGraph v6.4.1 but was not exported from the db.agraph package. The symbol is now exported and also available in the db.agraph.user package.

Bug25192 - materialize-entailed-triples was not supported in the remote Lisp client

Both materialize-entailed-triples and delete-materialized-triples are now available in the remote Lisp client.

Bug25168 - Fix leaked http-cursor finalizer clash

Cursors can be leaked, for example by using return-from inside iterate-cursor. These cursors are eventually garbage-collected. There was a finalizer race condition in the collection of HTTP connection based cursors, that would lead to stream-closed-error being signaled to the user unexpectedly. This has been resolved.

Bug25168 - Improve cursor leak message for get-triples on remote-triple-store

If get-triples is called on a remote-triple-store outside with-query-environment, the cursor will open an HTTP socket to the triple store. If the cursor is leaked it will be garbage-collected eventually, but until that moment the socket will be kept open. This can negatively impact performance, therefore a warning will now be logged: 

[2018-03-12T09:06:27.250 p13791 W cursor] Cursor created by  
(get-triples) on remote-triple-store "db" was leaked,  
will be closed.

Bug25166 - remote-triple-stores reason on the client rather than the server

AllegroGraph now ensures that the dynamic reasoner will execute on the server rather than the client when using a remote-triple-store. This prevents a large number of get-triples from executing via HTTP when using the RDFS++ reasoner.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.4.1

AllegroGraph Server: General Changes

New agtool commands and some changed arguments

The *agtool command-line utility has new commands available. Further some commands have new arguments and some arguments have changed. See the agtool General Command Utility document for details. Most commands now have a --scheme argument which can be http or https.

agtool repl create-cluster adds --if-exists and --force arguments

agtool repl create-cluster previously would only create new, empty repositories and if an existing repository were specified, it would be deleted and a new, empty repository with the same name would be created. (This behavior differed from the behavior when creating clusters with AGWebView, where a repository had to exist and need not be empty when it was converted to an initial replication instance.) Now agtool repl create-cluster accepts an --if-exists argument with allowable values supersede (the default, meaning delete the existing repository and create a new one with the same name), overwrite (same as supersede), use (convert an existing, possibly non-empty repository), and error (signal an error if the repository specified already exists).

If use is specified, then any encoded id definitions will be examined and if any do not allow for 2^60 distinct strings and so will not work with next-encoded-upi-for-prefix, the conversion will fail and a warning will be printed. If the --force argument with value true is supplied, then the conversion will be done but none of the offending encoded id prefixes must be used again.

The Cluster Replication Manager page now displays statistics

These help users to monitor the state of the cluster. See the Managing clusters in AGWebView once created section in the Multi-master Replication document.

When adding a cluster instance in AGWebView, a progress log is displayed

When adding an instance in Webview you'll see that the log of actions is updated with progress reports describing what is being done. Also the AGWebView replication manager page has new columns and some columns have been renamed. See here for information on using AGWebView to manage clusters.

Rfe15465 - Disallow setting bulk-load-mode in replication instance

In a replication instance one cannot set bulk load mode since replication depends on the transaction logs being an accurate reflection of the changes to the repository and logging is disabled in bulk loading mode. Specifying the --bulk option to agtool load or clicking the Use bulk load mode checkbox when adding triples from an uploaded file in AGWebView will cause the load to fail.

Rfe15451 - Don't fail if unable to write to agload.log

If the current working directory is not writable agload would stop running with an error the first time it tries to write to the log file agload.log. Now this behavior has been changed to warn the user that it was unable to write to the log file and then continues processing. It continues logging to the terminal as it would normally do.

Rfe15426 - Expose and document the --parameter option for agtool load

agtool load now accepts the --parameter option for controlling the configuration of newly created repositories. See the --parameter option in the agtool load document for details.

Rfe15418 - Change StringTableCompression default to 'none'

AllegroGraph now in the default creates new repositories with string compression disabled. This results in faster out-of-the-box operation. String compression can be enabled by using the StringTableCompression directive in agraph.cfg. See the description of StringTableCompress in the Server Configuration and Control document for information.

Note that the string compression type can only been set when creating a new repository. The StringTableCompression parameter is further discussed here in the Performance Tuning document.

Rfe15190 - Remove --repl argument from agtool archive backup and agtool recover

The --repl argument for agtool archive backup and agtool recover is no longer present because it is no longer needed (whether or not a repository is a replication repository is now automatically determined).

Rfe14770 - Error message from agload loses most of the detail

When using agtool load the error messages signaled while loading triples were truncated. This is now corrected.

Rfe14389: AllegroGraph no longer supports Callimachus

AllegroGraph no longer supports Callimachus. Stores that were previously set up to integrate with Callimachus will open normally but they will no longer execute the Callimachus integration code.

Bug25087 - TempDir is not being used for placing temporary query files

Temporary query files were being placed in the repository's directory rather than in the location specified by the TempDir configuration parameter. This has been corrected.

Bug25075 - Bidirectional search can return bad paths

If there were many possible overlapping bidirectional paths between two nodes, AllegroGraph could return incorrect paths. This has been corrected.

Bug25102 - Subscript UPIs did not print correctly.

AllegroGraph was printing the prefix of a subscript UPIs twice. This has been corrected. Previously, they would print as 

<http://franz.com/ns/allegrograph/4.14/index#_http://franz.com/ns/allegrograph/4.14/index#_4>

Now they print as 

<http://franz.com/ns/allegrograph/4.14/index#_4>

Bug25062 - Fix uninformative error when running agraph-control as root.

When running agraph-control as root when there was no RunAs directive in agraph.cfg would result in the following uninformative error: 

Got error while executing plan: `nil' is not of the expected type  
`(array character (*))' for foreign type `(* :char)'

Now the error messages generated for no RunAs user and non-existent RunAs user, respectively, are: 

# bin/agraph-control --config lib/agraph.cfg start  
Running AllegroGraph as superuser is not recommended for security reasons.  
It is recommended that another less privileged user be used, specified  
via the `RunAs' directive in agraph.cfg.  
 
# bin/agraph-control --config lib/agraph.cfg start  
RunAs user `UserThatDoesNotExist' does not exist in the passwd database.  
 
Please run the AllegroGraph configuration script or adjust the `RunAs'  
directive in lib/agraph.cfg

HTTP API

Rfe15418 - Allow specifying string table compression method

Added a stringTableCompression parameter to the repository creation HTTP service.

SPARQL

Rfe15421 - Add control over duplicate triple reduction in SPARQL queries

AllegroGraph now has a new query option: enableDuplicateReduction which controls whether or not the SPARQL engine reduces the number of duplicate triples it processes. See SPARQL Query Options in the SPARQL Reference document.

If the store has no duplicate triples (based on subject, predicate, object, and graph) then this option has no effect.

SPARQL is defined on sets of triples (i.e., no duplicates) but AllegroGraph can store duplicates which can lead to duplicate rows of bindings returned by a query.

If this option is set to true, then AllegroGraph will ignore consecutive duplicate triples returned from the storage layer. Since the storage layer makes no guarentees on the order of triples it returns, this can lead to different results from the same query.

Note: Previous versions of AllegroGraph used duplicate reduction by default. This version turns if off by default. If your triple-store has duplicate triples, then this may change the results of your queries.

Rfe15419 - Include query modifiers when printing a SPARQL query plan

AllegroGraph now includes query modifiers like ORDER BY or LIMIT when it prints a query plan.

Rfe15391 - Warn when the variable HOST of a SERVICE clause is not bound

AllegroGraph now warns when the HOST variable of a SERVICE clause is unbound in the query.

For example, both of these queries will produce query warnings: 

# Query 1  
select * {  
 {  
   :myService :isAtURL ?url  
 } union {  
   service ?url { ?s ?p ?o }  
 }  
}  
 
# Query 2  
select * { service ?unknown { ?s ?p ?o } }

Rfe15354 - Improve handling of queries containing GRAPH and JOIN clauses

AllegroGraph will now handle queries involving GRAPH and JOIN clauses more efficiently.

Rfe15336 - Sanity check on JSON/XML data returned by SPARQL endpoint

If a SPARQL query includes a SERVICE call, the XML and JSON result data is now subject to more syntactic and semantic error checking. Nevertheless invalid result data from third parties might still lead to unexpected outcomes.

Rfe15273 - Add Javascript API for defining SPARQL extensions

AllegroGraph's server side Javascript engine now supports the following:

For details, see the Javascript documentation.

Bug25145 - Illegal keyword error when executing some queries

AllegroGraph could signal an Illegal Keyword error when executing some complex queries. This has been corrected.

Bug25113: SPARQL planner stopped optimizing certain queries

If a query has a FILTER like 

FILTER( ?var1 = ?var2 )

and it can be determined that both ?var1 and ?var2 are resources, then AllegroGraph can perform certain query optimizations. This feature was inadvertently broken in AllegroGraph 6.4 and has now been reinstated.

Bug25097 - Graph not in dataset warning signaled inappropriately

AllegroGraph v6.3.0 and following signal a query warning if the graph in a GRAPH clause is not in the dataset as in, for example, this query: 

select *  
from named <http://example.com#graph> {  
  graph <http://example.org#otherGraph> { ?s ?p ?o }  
}

AllegroGraph was signaling this warning more often than necessary. This has been corrected.

Bug25064 - GROUP BY expressions in a SUB-QUERY causes an error

AllegroGraph was signaling an error if a SPARQL query used GROUP BY expressions in a SUB-QUERY. This has been corrected. An example query that used to fail would be 

select * {  
  { select ?strWho (sum(?amount) as ?sum) {  
      ?who ?p ?amount .  
    } group by (str(?who) as ?strWho)  
  }  
}

Bug25057 - Values and OPTIONAL could lead to too many results

If a SPARQL query used a VALUES form inside the query and the VALUES form had only a single variable and a single binding for that variable and the query also used an OPTIONAL form that made use of the same variable, then AllegroGraph could return results with the variable unbound. This has been corrected.

As an example, AllegroGraph previously could have returned results with ?o2 unbound for the following query: 

SELECT ?s ?o1 ?o2 {  
  ?s ?p1 ?o1  
  OPTIONAL { ?s foaf:knows ?o2 }  
  VALUES (?o2) { (:b) }  
}

Bug25055 - Improve SPARQL parser error message for bad aggregation expression

AllegroGraph was signaling an error for bad aggregation expressions rather than indicating that the query failed to parse. For example, this query is invalid because it is illegal to GROUP BY an aggreagate: 

select (sum(?o) as ?sum) (sample(?s) as ?ss) {  
  ?s ?p ?o .  
} group by (sum(?o) as ?xxx) having (?xxx > 2)

AllegroGraph now signals a SPARQL parser error for queries like this.

Bug25034 - Catch additional SPARQL parse errors

SPARQL is defined such that repeated variables in the projection are ignored. Previously, AllegroGraph would include each repeated variable. Now, it will include only the first instance and will warn if the list includes repetition.

For example, this query: 

SELECT ?s ?g ?s { graph ?g { ?s ?p ?o } }

Will project only ?s and ?g and will warn that "The following variable is projected multiple times: ?s. Only the first projection will be included".

The variables in expressions and aggregation specifications may not overlap with other projected variables. AllegroGraph now catches errors like: 

SELECT ?o (str(?o) as ?o) {  
  ?s ?p ?o .  
}

or 

SELECT ?s (count(1) as ?s) {  
  ?s ?p ?o .  
} GROUP BY ?s

Previously, the above queries would have passed the parser.

AGWebView

Rfe15395 - Highlight all instances of the token under cursor in the query editor

AGWebView's SPARQL query editor now highlights any tokens that match the one currently under the cursor. This makes it easier to ensure that variable names are not mistyped.

Changes to the Lisp API

rfe15430 Use of next-encoded-upi-for-prefix in multi-master cluster instances

In order to use next-encoded-upi-for-prefix when the repository is a multi-master cluster instance, the template associated with the encoded id prefix must generate exactly 2^60 distinct strings. If it does not, the call to next-encoded-upi-for-prefix will signal an error. As noted just below, the "plain" will work as a template. See register-encoded-id-prefix.

Rfe15466 - Plain format encoded ids

register-encoded-id-prefix will treat the format argument of "plain" (a string) in this special way: The encoded id (the string after the @@) is a variable length integer in the range from 0 to (2^60 -1), much like blank nodes. This is a template suitable for use in multi-master cluster instances. See the Encoded IDs document.

rfe15430 Use of next-encoded-upi-for-prefix in multi-master cluster instances

In order to use next-encoded-upi-for-prefix when the repository is a multi-master cluster instance, the template associated with the encoded id prefix must generate exactly 2^60 distinct strings. If it does not, the call to next-encoded-upi-for-prefix will signal an error. As noted just above, the "plain" will work as a template. See register-encoded-id-prefix.

Rfe15379 - Remove in-line-data argument from the run-sparql

The in-line-data argument to run-sparql is no longer supported. If you need to execute the same basic query with different sets of BINDINGS, then use the VALUES form in your SPARQL query string.

Rfe15373 - Export peekable-cursors

The Lisp API now includes these new functions:

A peekable-cursor is like a regular cursor except cursor-peek can be used to see what cursor-next-row will return without modifying the state of the cursor. See the documentation for details.

Bug25119 - load-turtle not accepting a :base-uri arg in remote lisp client

The remote lisp client did not support the base-uri argument to load-turtle. This has been corrected.

Bug25070 - upi->value uses db when decoded encoded IDs

The upi->value function was using db to decode encoded IDs even when a specific value for db was passed into the function. This has been corrected.

Bug25043 - get-triples-nd-* functions were incompletely documented

The various nD get-triples functions (like get-triples-nd-within-bounding-box) return :part-not-found as a second return value if it can be determined that one of the search parameters is definitely not in the repository. This is now documented.

Bug23635: Make Lisp client :default-graph-uris keyword work with remote-triple-stores

SPARQL queries made against remote-triple-stores via run-sparql, that passed the :default-graph-uris keyword argument, would not return expected results. This has now been fixed.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

Rfe13347 - Automatic nD datatype mapping is now on for new repositories

AllegroGraph now creates new repositories with automatic N-dimensional (nD) datatype mapping enabled. This means that adding triples with nD objects will add the necessary type mapping with no further intervention. See nD Geospatial Automation for details and a linke to further discussion in the geospatial examples document.

Bug25129: Bus error on geospatial search with a non-existing predicate

The bus error on the server happened when trying to obtain triples from polygon using predicate that was not previously added to the store.

Behaviour for obtaining triples from polygons was modified to properly handle this case.

AllegroGraph 6.4.0

AllegroGraph Server: General Changes

Rfe15302 - Remove support for using rapper as an external parser

The use-rapper argument to the AllegroGraph data import commands has been removed because AllegroGraph now supports all of the standard formats.

The rapper command line argument to agtool load has similarly been removed.

Rfe15219 - Add timeout and limit arguments to the SNA path-finding functions

The SNA path finding functions now have a timeout argument and a limit argument. The first specifies the number of seconds to execute the search whereas the second indicates that searching should stop once that many paths have been found.

The SNA path functions will return (as a second value) the status of the search. This will be either nil if the search was not interrupted, :timeout if the search reached the timeout, or :limit-reached if the limit on the number of paths was reached.

Note that the return format for the REST API to the SNA path functions is now versioned. The desired version if specified via a version parameter in the query string. If the version is 1, then the previous result format of a list of lists representing paths will be returned. Otherwise, the new format will be used. The new format consists of a JSON object with two keys: "paths" and "status". "paths" contains the list of paths and "status" contains the reason that the search completed. It will be one of "complete", "timeout", or "limit-reached".

Bug24980 - Turtle serializer escapes the PNAME_NS incorrectly

The Turtle serializer was escaping both the namespace portion and the local portion of a qualified name using the same logic. This was incorrect as the escaping for the two differ. The problem has been corrected.

HTTP API

No significant changes.

SPARQL

Rfe15337 - Have the NOW() SPARQL function include a timezone

The SPARQL NOW() function now returns dateTimes in GMT. Previously, the dateTimes had no explicit timezone.

Rfe15259 - Add a query option to limit the size of query log entries

AllegroGraph's SPARQL engine now truncates query log message to 512 characters by default. This limit can be changed by adding ":LINE_LENGTH" to the logQuery query option either in the AllegroGraph configuration file or on a per-query basis. For example, this PREFIX line will limit log lines to 100-characters: 

PREFIX franzOption_logQuery: <franz:yes:100>

Rfe15258 - Add maximumValuesCountForService query option

AllegroGraph uses the SPARQL VALUES clause to pass partial results to SPARQL endpoints. In general this can help the endpoint compute its results more efficiently but when there are many values the cost of transmitting the data can outweigh these benefits.

The maximumValuesCountForService provides a mechanism to limit the total number of VALUES sent.

Rfe15250 - Improve treatment of very large VALUES clauses

The query planner now handles large VALUES clauses significantly faster. In addition, the query executor now decides whether to probe or scan based on the number of VALUES and the expected number of rows that a cursor will need to traverse. These two changes can make some queries significantly faster.

Rfe11137: Support the TIMEZONE() and TZ() functions

AllegroGraph now supports the SPARQL 1.1. TIMEZONE() and TZ() functions.

Bug25003 - FILTERing on an expression like ( ! ?a = ?b ) caused an error

The above expression evaluates as (!?a) = ?b and AllegroGraph was not correctly handling the expression of the inner negation operator. This has been corrected.

Bug24985 - FILTER EXISTS and SUB-QUERY can lose solutions

If a FILTER EXISTS clause started with a sub-query then AllegroGraph could fail to find some solutions. This has been corrected.

Bug24657 - GRAPH filter can be applied to the wrong clause

If a graph variable appeared in the subject, predicate, or object position of a triple-pattern, it was possible for AllegroGraph to apply a GRAPH dataset constraint to it incorrectly. For example, AllegroGraph could have incorrectly determined that the ?g1 :predicate1 ?object pattern in the query below should be evaluated as if it was inside a GRAPH clause which could lead to incorrect results. 

select * {  
  GRAPH ?g1 { ?s1 ?p1 ?o1 } .  
  GRAPH ?g2 { ?s2 ?p2 ?o1 } .  
  ?g1 :predicate1 ?object .  
}

This has been corrected.

Bug24151 - Correct a SPARQL FILTER analysis issue

AllegroGraph's SPARQL engine would signal an error if a FILTER contained an expression like 

str( ?varOne ) = ?varTwo

This has been corrected.

Bug20593 - Support the SPARQL 1.1 BNODE( identifier ) function

AllegroGraph has long supported the BNODE() function with no arguments. It now also supports the version that takes a single plain literal or XSD string argument.

Bug24076 - Improve handling of DBPedia.org and other Virtuoso servers as SPARQL endpoints.

AllegroGraph works around an apparent problem with Virtuoso.

AGWebView

Rfe11480 - Added Shutdown instance operation to Repository Control section in AGWebView

See the desription of the repository page in the WebView document.

Bug24949 - Overview page broken for remote sessions

Opening a session to a single, remote store in WebView used to produce the following error message:

Loading failed: Service not supported in a session that does not wrap a single local store.

instead of the overview page. This has been corrected.

Bug24929 - Help messages appear out of view.

Help messages were often shown at the top of the Webview page, out of sight of the current page location. They will now be displayed in tooltips.

Changes to the Lisp API

Rfe15325: Make *db* a required thread binding

To help avoid accidental sharing of a repository connection between threads, each new thread will start with *db* bound to nil. Previously, *db* would retain its global value in new threads.

Given this sample code: 

(open-triple-store "test") ;; sets global value of *db*  
 
(mp:process-run-function "new thread"  
   (lambda ()  
     ;; The new thread wants its own connection to the repository  
     (open-triple-store "test")))

Prior to v6.4.0, the call to open-triple-store in "new thread" would overwrite the global value of *db*. In v6.4.0, "new thread" will have its own binding of *db* and the global value will not be affected.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

User-visible changes in earlier releases

See Change History for user-visible changes to earlier releases.