Introduction

This document lists all changes for the most recent major release (releases with number X.Y) and subsequent minor releases (with numbers X.Y.Z and perhaps further identifiers), including major and important changes and a list of all user-visible modifications. Change History repeats the list of user-visible modifications for this release and includes similar lists for earlier releases.

The release described in the documentation set is 6.3.0.

Release 6.3.0

Release 6.3.0 is a major release with new features in addition to adding general improvements to and fixing bugs in version 6.2.3. The changes are described below. The most notable new features include:

Database internal format unchanged from version 6.2.3

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.3.0 is unchanged from version 6.2.3. (The format was previously changed in release 6.2.0.) Therefore it is not necessary to upgrade 6.2.3 databases for them to be used with 6.3.0. Note that backing up repositories before moving them to a new version (whether or not upgrading is required) is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

Release 6.2.3

Release 6.2.3 is a maintenance release which fixes bugs in version 6.2.2. There are also other improvements and optimizations. The changes are described below.

Database internal format unchanged from version 6.2.2

The database format is changed from time to time. When the format is changed, databases must be upgraded to the new format before they can be used. Upgrading is discussed in the Database Upgrading document. The format in version 6.2.3 is unchanged from version 6.2.2. (The format was previously changed in release 6.2.0.) Therefore it is not necessary to upgrade 6.2.2 databases for them to be used with 6.2.3. Note that backing up repositories before moving them to a new version (whether or not upgrading is required) is always recommended. Once a database has been used in a later release, it cannot be used in an earlier release regardless of whether the database format is the same.

Release 6.2.2

Release 6.2.2 is a maintenance release which fixes bugs in version 6.2.1. There are also other improvements and optimizations. The changes are described below.

Release 6.2.1

Release 6.2.1 is a maintenance release which fixes bugs in version 6.2.0. There are also other improvements and optimizations. The changes are described below.

Release 6.2.0

Release 6.2.0 is a major release with significant changes to various parts of AllegroGraph. The attributes facility has been improved and a new command-line tool, *agtool combines most previous command-line programs into one. The changes are described below. We also list some important changes here.

The Server Load and Performance Charts

6.3.0 Programmer notes

6.2.3 Programmer notes

There are no programmer notes for version 6.2.3.

6.2.2 Programmer notes

6.2.1 Programmer notes

There are no programmer notes for version 6.2.1.

6.2.0 Programmer notes

It is now: 

    agtool command arguments-and-options

although the command name is shortened in most cases (so, for example, agload becomes load).

User visible changes in AllegroGraph 6.3.0

These sections list changes in the current release and in some earlier releases. See the Change History for a list of changes in all earlier releases.

AllegroGraph Server: General Changes

Catalogs may no longer be named root or system.

The names root and system are reserved as catalog names. root interferes with the default catalog (which is referred to as the root catalog) and the system catalog is used for auditing and other system data. If you have catalogs named root or system, contact [email protected] if you have difficulty upgrading to 6.3.0.

'agtool archive' no longer accepts --catalog, catalog now part of repo name

The agtool archive program (see Backup and Restore) now specifies catalogs and repositories together using the catalog:repository format. The --catalog option to agtool archive is no longer accepted.

Rfe15210 - Log which user starts a session

When a session is started, AllegroGraph now logs the name of the user that started the session. (Although not documented, the AllegroGraph log has always said when a session was started and stopped.) A sample log message: 

[2017-09-21T15:59:48.028 p21266 back] Session on port 43704 started by user "test".

Rfe15194 - Support XQuery and XPath Math Functions

AllegroGraph now supports the following XPATH math functions

See http://www.w3.org/2005/xpath-functions/math/ for details.

Rfe15132 - agtool archive list can operate on backup directories

agtool archive list now can operate not only on single archive file but can be given backup directory as source. In that case it displays details about every repository inside backup.

Additionally it can be given the optional DB-IN-ARCHIVE argument to display details about single repository in ARCHIVE even if other repositories are present.

New option --summary allows agtool archive list to only list repositories inside ARCHIVE, without printing details about every one.

agtool archive is described in the Backup and Restore document.

Rfe12362 - CORS support

CORS (Cross-Origin Resource Sharing), if enabled, allows scripts run on a web page from one server to make HTTP requests to the (different) server where AllegroGraph is running. CORS is not enabled by default. You enable it with various donfiguration directives described in the CORS directives section of the Server Configuration and Control document.

Rfe12201 - Switching a repository between commit and no-commit modes

This now requires superuser privileges.

Bug24927 - Turtle serializer tried to escape capital A

The turtle serializer could try to escape the letter capital A which could lead to invalid output. This has been corrected.

Bug24910 - Unhelpful Turtle parser error message

Improved the error message the Turtle parser produced when the object of a triple was missing.

Bug24905 - Audit log does not display in AG 6.2.3

The Audit log display feature in AGWebView was inadvertently broken in release v6.2.3. This has been corrected.

Bug24842 - Custom services should not use user namespaces

Custom services were being executed in the context of the current user rather than in the context of the code that defined the service. This has been corrected.

Bug23757 - If a script file generates errors (warnings) while loading, don't start the session.

Previously, any errors or warnings that occurred when compiling a Lisp script were written to the AllegroGraph log file and the session would still start. Now, any problems are still written to the log file but the session will not start and the text of the problem(s) will be returned in the HTTP response.

HTTP API

Bug20845 - /version/info web service removed

The /version/info web service caused the server to report detailed internal information about the server and the machine on which the server was running. It was decided that the information was not in fact useful and could expose security vulnerabilities. The URL now returns 404.

SPARQL

Defining your own magic properties

AllegroGraph lets you define your own Magic Properties using the Lisp API. The Defining Magic Properties Tutorial describes how to do this and provides numerous simple examples.

Rfe15182 - All of AllegroGraph's query warnings are now documented.

See the Query Warnings section of the SPARQL Reference document.

Rfe15152 - Propagate impossible query conditions outward in the plan

When a query contains a triple pattern that cannot possibly match, it will mark the pattern as impossible. The impossible marking will propagate outward (e.g., from the pattern to the BGP to the enclosing JOIN and so on). This allows queries with typos or other errors to fail quickly.

Rfe15135 - Warn if any SPARQL type errors occur during query execution

SPARQL will now log a warning if any type errors are encountered during query execution. For example, it is an error to try and cast the string '1.23e4' to an integer because the lexical representation of an XSD integer consists of "a finite-length sequence of decimal digits (#x30-#x39) with an optional leading sign." (cf https://www.w3.org/TR/xmlschema-2/#integer). Therefore, a query like: 

select * { bind(xsd:integer('1.23e4') as ?int) }

will leave ?int unbound. Previously, this type error would have been surpressed. Now, AllegroGraph will issue a warn-sparql-type-errors warning.

Rfe15112 - Improve cancelQueryOnWarnings option

The cancelQueryOnWarnings query option now cancels queries immediately for most plan or query warnings. This option defaults to off but can be turned on by placing 

QueryOption cancelQueryOnWarnings=yes

in the AllegroGraph configuration or by prefixing a SPARQL query with the PREFIX 

PREFIX franzOption_cancelQueryOnWarnings: <franz:yes>

Rfe12249 - Improve magic predicates for Solr and MongoDB

The magic predicates that interface AllegroGraph's SPARQL engine with Solr and MongoDB have been improved. The three magic predicates:

can now take an optional second parameter in their list of subject arguments. This second parameter will be bound to the matching objects of any triples found by the search in the case of solr:matchID and to the associated linking identifier in the cases of solr:match and mongo:find. For example, the following query will bind ?s to the subjects found and ?text to the object: 

prefix solr: <http://franz.com/ns/allegrograph/4.5/solr/>  
select ?s ?text {  
  (?s ?text) solr:matchId 'boatswain' .  
}

Rfe11193 - Improvement performance of SPARQL expression compilation

The SPARQL query expression compiler is now approximately 45 times faster than before. This means that queries that use many expressions (e.g., in FILTERs, BIND, ORDER BY, etc.) will perform more efficiently.

Note that aggregation expressions still use the older and slower compiler.

Rfe11678 - Full-scan warning should take dataset into account

AllegroGraph signals a warning when a SPARQL query needs to perform a full scan and the store contains more than one million triples. AllegroGraph was incorrectly signaling this warning when the query was executed against a SPARQL dataset. This has been corrected.

Bug24898 - Fix problem with magic SNA centrality predicates and neighbor caches

Some queries that used AllegroGraph's SNA centrality magic predicates and neighbor caches could cause an error. This has been corrected.

Bug24861 - Sparql parser confused by "?o<='foo'^^<...> "

The SPARQL parser was incorrectly treating the inequality operator as the start of a URI. This has been corrected.

Bug24681 - Parsing should fail if a blank node label is used in more than one BGP

AllegroGraph did not signal a parsing error if a SPARQL query used the same blank node in more than one basic graph pattern (BGP). This has been corrected.

AGWebView

Rfe15198 - Index management page

An index management page has been added to AGWebView. The new page can be accessed from the Repository Overview page by following a link labeled 'Manage triple indices'. The list of indices has been removed from the overview page and the 'purge deleted triples' link has been moved to the index management page. See The Manage triple indices page for more information.

Rfe14563 - Repository reports

A set of reports describing disk usage and other repository properties has been added to WebView. The reports can be accessed through links on the repository overview page. See the Reports section in the Repository Overview page description for more information.

Rfe13419 - Add warmup triple-store to AGWebView

Add new option Warmup store in Repository Control section in repository overview page. See the description of the Repository Overview Page in the WebView document.

Rfe11915 - Creating freetext index prefills stop words with default ones.

The AGWebView dialog for creating a freetext index now starts with a default selection of stop words (rather than starting with no stop words and the option to add a default selection). See the Free-Text Indices page in the WebView document for more information.

Rfe11256: Import RDF statements directly.

It is now possible to import RDF statements directly by typing or pasting them into a text box in AG WebView. See the Load and Delete Data section of the Repository Overview page description.

Bug24824 - Spurious warning for ASK queries in AGWebView

AllegroGraph WebView was showing a query warning for ASK queries when the "Limit to 1000 results" checkbox was checked. This has been corrected.

Bug24480: "Ignore errors" option was ignored when loading triples from server file

The "Ignore errors" error handling option was not being honored when loading a server-side file using AG WebView. This has been fixed.

Bug23607 - Improve accuracy of query timing results in AGWebView

Previously, the timing results in AGWebView represented the time it took to find the first result. Now, it represents the time it takes to find all results. The total time to display may still be longer as it will also include the time it takes the web browser to render the returned data.

Changes to the Lisp API

Remove :plain datatype mapping

The :plain datatype mapping converted typed string literals into the equivalent plain literal. E.g., it would convert "bear"^^xsd:string into "bear". :plain mappings are no longer necessary because AllegroGraph handles typed strings more efficiently automatically. Mappings are specified with setf and datatype-mapping and predicate-mapping.

Note that any existing datatype or predicate mappings of type :plain will be automatically removed from the store.

Bug24857 - path-finding over HTTP fails when paths include literals

If an SNA path included typed literals, then a remote-triple-store would convert them into the wrong UPI type leading to UPI not in string-table errors. This has been corrected.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.2.3

AllegroGraph Server: General Changes

Rfe15101: Restore tlmgr to tlmgr.restored

The tlmgr file contains information about replication jobs. Since the existence of replication job information can result in transaction log files accumulating, the tlmgr file is no longer restored as is. Now it is renamed to tlmgr.restored during a restore. After a restore, replication jobs must be reestablished by the user.

Rfe12408 - Add graph filtering support to agmaterialize

agtool materialize can now limit materialization to a specified list of graphs via the --graph (short form -f -- note that -g specifies the default graph for inferred triples) command line argument. The inferred graph (specified with --inferred-graph or -g) will always be included in the list. See the Materializer document.

Bug24729 - Spaces in filenames prevent server-side file import.

Importing a server-side file with spaces in its path would fail with an error message about illegal characters in a URI. This issue affected both WebView and the HTTP API, but not agtool load and has been corrected.

HTTP API

Bug20772 - Loading JSON format may fail to commit all triples

Using the 'commit=number' parameter when importing statements through the HTTP API could result in some statements not being committed. The issue only affected JSON imports and has been corrected.

SPARQL

Rfe15077 - Signal a query warning when CONSTRUCT tries to generate invalid triples

AllegroGraph will now signal a query warning if any triples produced by the CONSTRUCT template are invalid. Formerly, these triples were silently discarded.

Rfe15075 - Improve parsing efficiency of large USING [NAMED] datasets

The SPARQL parser now parses UPDATE commands with very large dataset specifications using the USING and USING NAMED clauses more efficiently.

Rfe15074 - SPARQL parser has non-linear behavior for large VALUES clauses

Improved the SPARQL parser performance for queries with 1000s of elements inside of VALUES clauses.

Rfe15061 - Improve efficiency of SPARQL queries with large datasets

AllegroGraph now determines at query time whether it would be more efficient to issue getStatements calls for each graph in a dataset or to issue a single getStatements call and filter the results to include only graphs in the dataset. Depending on the triple-store and the dataset, this can be significantly more efficient.

Rfe15006 - Improve GRAPH and PROPERTY PATH query planning and execution

AllegroGraph now handles property path queries inside of GRAPH clauses more efficiently when the graph is a variable and the clause contains no property path queries that use any of the ZeroOrMorePath, OneOrMorePath, or ZeroOrOnePath operators.

Rfe14450 - Improving cross product warning

The warning issued when AllegroGraph detects a cross product in a SPARQL query is now more descriptive.

Bug24779: Constructed triples should not allow blank nodes in the predicate postion

AllegroGraph was incorrectly allowing CONSTRUCT to produce blank nodes in the predicate position of triples. This has been corrected.

Bug24762 - SPARQL DELETE/INSERT/WHERE and Chunk processing could fail

If the chunk size was too small, a SPARQL DELETE/INSERT/WHERE command could fail to correctly process triples.

Bug24755 - Problems in intervalAfterDatetime and intervalBeforeDatetime

The intervalAfterDatetime magic property could fail to find solutions when using a constant datetime rather than a variable. Additionally, both intervalAfterDatetime and intervalBeforeDatetime could fail to find solutions when the intervals were defined on explicit times rather than using named time points.

Bug24709 - FILTER EXISTS and unbound variables on the RHS

AllegroGraph used to return incorrect results for FILTER EXISTS expressions when the RHS contained any unbound variables. This has been corrected.

Bug24704 - Improve strategy used to find unique values in a store

AllegroGraph uses several techniques to determine all of the unique values stored for subject, predicate, object, or graph. Depending on the quads in the store and the shape of the query, one of these techniques can be much more efficient than the others. It was possible, however, for AllegroGraph to choose the wrong technique in some cases. This has been corrected.

Bug24665 - SPARQL queries with :dateTime inequality filters could lose results

Query results could be lost if a SPARQL query used a non-inclusive inequality filter on xsd:dateTimes or xsd:times. This has been corrected.

Bug24455 - Incorrect results when join variables might be unbound

AllegroGraph would sometimes return incorrect results from joins where a part of the join key was unbound.

This has been corrected.

Bug24215 - Unbound values in join keys

When merging solutions during a join, AllegroGraph would sometimes not produce bindings for variables that are only bound on the RHS of the join and are a part of the join key.

For instance when running this query: 

select * {  
  { values(?x ?y) {(1 undef)} }  
  { values(?x ?y) {(undef 2)} }  
}

AllegroGraph would return (?x = 1, ?y = undef) instead of the expected (?x = 1, ?y = 2).

This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug24796 - intern-resource error on future-part with unregistered namespace abbreviation

A call to intern-resource on a future-part whose namespace abbreviation was not defined would signal a bus error. This will now signal a condition of type cannot-resolve-namespace-for-future-part-error.

Prolog

No significant changes.

Documentation

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.2.2

AllegroGraph Server: General Changes

Rfe14926: Reserve parentheses characters

Parentheses are no longer allowed in triple store or catalog names. These characters are reserved for use in triple store specifications.

Bug24700 - Unfriendly error message when session ports are unavailable

Previously, if a request to create a session cannot be completed due to lack of an available session port, an unfriendly error message was reported. This has been fixed and now a clear error message is returned.

Bug24687 - Deleting a tlog when free list is full throws error.

Fixes a bug introduced in v6.1.4 where a tlog that was deleted due to the free list being full would be repeatedly deleted by AllegroGraph despite it no longer existing. This bug could prevent the freeing/deletion of old tlogs, causing them to accumulate.

Log entries that indicate this bug, would be similar to the following: 

[2017-05-22T13:49:21.800 p17318 tlarchiver] Store "bug24687", free list is full, will just delete "tlog-2477c96f-0579-ba54-b60b-7083f2293fa8-27"  
 
... later ...  
 
[2017-05-22T13:51:34.996 p17318 tlarchiver] Store "bug24687", free list is full, will just delete "tlog-2477c96f-0579-ba54-b60b-7083f2293fa8-27"  
[2017-05-22T13:51:34.999 p17318 E tlarchiver] Store "bug24687", Attempt to delete tlog-27 gave error There is no file named /dbs/bug24687/tlog-2477c96f-0579-ba54-b60b-7083f2293fa8-27: No such file or directory [errno=2].: No such file or directory [errno=2].

Bug24673 - FTI merger can create empty data file

The FTI merger process sometimes fails and logs an error message similar to the following: 

Process 'repo FTI Merger' raised: opening  
#<mapped-file-simple-stream  
#P"/agdata/data/rootcatalog/repo/fti-2-chunk-114-data" mapped  
closed @ #x100054959b2> resulted in error (code 22): Invalid  
argument.

this has been corrected.

Bug24663 - TriG parser uses the wrong graph on the first blank node

In TriG nested blank nodes can be specified in a graph other than the default graph. When the TriG parser encountered the first nested blank node inside a non-default graph the graph for the triples derived from the parsing of that nested blank node would incorrectly be set to the default graph. This has been fixed to use the correct graph.

Bug24642: Incorrect text index query warning

AllegroGraph v6.1.4 added a check and warning for text index queries which could not possibly match. Under certain circumstances this check would warn even though the query could match. This has been corrected.

Bug24623 - SNA path finding in remote-triple-stores did not suppport blank nodes

If a path contained blank nodes, then the SNA path finding functions would signal an error if used on a remote triple-store. This has been corrected.

Bug24615 - Static filter function not re-esablished when attaching to a store

Previously when opening an existing triple store, the static attribute filter was not reestablished. This has been fixed.

Bug24588 - load-nqx does not properly track line numbers

When an error occurs loading extended triples via load-nqx and agload the incorrect line number is reported. This has been corrected.

Bug24548: TransactionLogArchive directive doesn't create catalog and repository directories when tlarchiver is initiated

AllegroGraph did not previously create the TransactionLogArchive directory when starting a database instance. Instead a "no such file or directory" error was logged when the tlarchiver ran. This has been fixed. The directory (and any subdirectories) are now created when a database instance starts, if needed.

Bug24545 - Store specification containing a remote URL could fail

A "missing value for argument db.agraph::user" error would occur when starting a session using a store specification containing a remote URL without authentication information. This has been fixed.

Bug24342 - load-turtle loses default-attributes for RDF collections

Previously the Turtle parser would fail to include attributes when parsing an RDF collection such as: 

prefix : <ex://foo#>  
:a :p (1 2 3) .

This has been fixed.

Bug21595 - agload bad error message when given a filespec that does not exist

The agload program reported misleading error messages when attempting to load files that did not exist or contained wildcards. This has been improved by reporting an accurate and more useful error message. See Data Loading for more information on agload.

HTTP API

Rfe14967 - Ensure that the statements REST API uses namespace abbreviations

The statements REST API was not using user defined namespaces to abbreviate RDF resources in formats that supported them like turtle or TriG. This has been corrected.

Rfe14930 - Improve error message when a file cannot be accessed

The agload process for loading a local file via the HTTP interface reported misleading and inaccurate error messages when the local file was inaccessible. This has been improved by returning an accurate and useful error message in the HTTP response.

SPARQL

Rfe14978 - Improve efficiency for SPARQL datasets with many graphs

AllegroGraph now operates on SPARQL datasets with many graphs more efficiently.

Rfe14975 - SPARQL dataset validator is inefficient for large dataset specifications

The SPARQL engine validates any dataset specification before it begins execution. This change improves the performance of the validation which helps query speed especially when the dataset has more than a few thousand graphs.

Rfe14974 - SPARQL parser is inefficient for large dataset specifications

SPARQL queries with thousands of FROM and FROM NAMED clauses in their dataset description now parse much more efficiently.

Rfe14931 - Improve how property path queries implement filtering

SPARQL Property Path queries now handle FILTERs more efficiently.

Rfe14899 - Warn if query LIMIT/OFFSET and externally imposed LIMIT/OFFSET conflict

Pursuant with the changes made for Rfe14597 (described below), AllegroGraph will now warn if a query's internal and external LIMITs differ or if a both an internal and external OFFSET are supplied.

Rfe14597 - Reduce confusion between internal and external query limit and offsets

Previously, any externally imposed LIMIT or OFFSET would override the LIMIT or OFFSET in the query string. This was most evident in AGWebView because the web interface uses a dropdown selector to pick the LIMIT and this LIMIT overrode any specified in the query.

Now, AllegroGraph will:

Bug24684 - Property Paths and MINUS could interact badly

It was possible for queries that used MINUS and Property Paths to generate incorrect answers. This has been corrected.

Bug24610 - Trivial equivalence constraints cause an error.

Queries containing trivial equivalence constraints, such as filter(?x = ?x), used to cause the following error: 

missing value for argument db.agraph.sbqe::var2

This has been corrected.

Bug24609 - User attributes set via the SPARQL prefix could be ignored

Depending on how a query was executed, any user attributes set via the userAttributes SPARQL query prefix option could be ignored. This has been corrected.

Bug24608 - Spurious join results.

When performing joins where the left hand side contained unbound values (resulting from the use of OPTIONAL or UNDEF keywords) AllegroGraph could sometimes return the same solution row multiple times. For instance, the following query: 

SELECT * {  
  { values (?a ?x ?y) {(0 1 2)} }  
  optional { values (?a ?y ?z) {  
		(0 2 3)  
		(0 undef 4)  
		(undef 2 5) }  
  }  
}

Would return 4 results, including two copies of the (0 1 2 5) row. This has been corrected.

Bug24599 - concatenated-cursors don't support extended triples

When a SPARQL query which uses attributes-related magic properties is issued against a graph-filtered triple store, an error like the following can occur: 

Executing query failed: #<concatenated-cursor @ #x1000732dc52>  
does not support fat-triples

This has been fixed.

Bug24538 - Invalid effective boolean value of constant strings in filters

A query like: 

SELECT * {  
 ?s ?p ?o  
 FILTER ("bruce")  
}

was returning no results when it should have returned every row in the store. Non-blank strings are supposed to have an effective boolean value of "true". Blank strings have an effective boolean value of "false". This has been fixed.

Bug24530 - SPARQL-JSON result format does not handle encoded boolean properly

Previously, boolean values in SPARQL-JSON results were represented as the strings "0" and "1". Now boolean values are correctly represented as "false" and "true".

AGWebView

Rfe14968 - Remove the deprecated planner option from AGWebView

Remove the long deprecated planner option from AGWebView's query page. This option has had no effect for some time.

Rfe13239 - Provide "no limit" option for query results in AGWebView

Subject to the resources of the web browser, AGWebView can now return an unlimited number of results from a query. By default, AGWebView will display 1000 results or use the limit specified by the query (if it is smaller than 1000). Because of this, the More Results button has also been removed. See the WebView document for information on AGWebView.

Bug24616 - N-Triples parser errors mention NQX

When trying to load a .nt or .nq file through AGWebView and a parsing error occurs the error message states that a "NQX" parsing error occurred when it should have stated it was an N-Triples (for .nt) or N-Quads (for .nq) parsing error. This has been fixed to report the correct format type.

Changes to the Lisp API

Bug24669 - serialize-nqx does not work on remote-triple-stores

Serialization of attributed triples was not working correctly for remote-triple-stores. This has been corrected.

Bug24658 - load-nqx fails on remote-triple-stores

An error would occur if no default attributes were passed to load-nqx on a remote-triple-store. This has been corrected.

In order to reduce future confusion, the :attributes argument to the various load functions in the Lisp API has been changed to :default-attributes. Note this is an incompatible change. Code using :attributes will signal an error. See the 6.2.2 Programmer Notes for more information.

Bug24549: open-store-from-specification does not work

Previously, attempting to open a triple store using a URL could fail like so: 

(open-triple-store "http://localhost:10443/repositories/kennedy")  
Error: Invalid character in triple-store name  
       "//localhost:10443/repositories/kennedy".  
The name may not contain these characters /, \, ~, :,  
Space, $, {, }, <, >, *, +, |, [, ].

This has been fixed.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

Bug24680 - Tonativestring is broken on Python 2

The Python client sometimes failed while processing values with non-ascii characters, showing the following error message: 

UnicodeEncodeError: 'ascii' codec can't encode characters in position ??: ordinal not in range(128)

This has been corrected.

Bug24443 - getStatements sometimes breaks for raw Python values

The getStatements function can be called with a raw Python value, instead of an RDF term, as its third argument (object filter). For instance, the following call: 

conn.getStatements(None, somePredicate, True)

would retrieve all triples with the predicate of somePredicate and the object equal to "true"^^xsd:boolean.

This mechanism did not work correctly if the second argument (predicate filter) was None. In that case an AttributeError exception would be raised.

This has been corrected.

Java Client

Rfe14990 - Provide methods to download query results

The Java client now contains methods to download and save query results to a file. The new methods are:

Both methods have multiple overloads, allowing for the output path and desired output format to be passed in a variety of ways.

Rfe14970 - Speed up AllegroGraph java client connection pools.

Depending on the connection pool configuration, many unnecessary requests could be made to the AllegroGraph server when borrowing/returning connections to/from a connection pool.

These calls have been cleaned up, resulting in improved connection pool performance.

Bug24648 - inferredGraph parameter missing from AGMaterializer

When materializing triples through the Java API it is now possible to specify the graph into which the generated triples will be added.

Bug24644 - Fix TriG import via the Java client.

The AllegroGraph Java client would fail when attempting to import TriG data due to the content-type "application/x-trig" being unsupported in the server.

AllegroGraph now supports TriG data imported with a content-type of either "application/trig" or "application/x-trig".

nD Geospatial

No significant changes.

AllegroGraph 6.2.1

AllegroGraph Server: General Changes

Rfe14881 - Cross attribute comparisons

AllegroGraph 6.2.0 added a restriction against using an attribute filter operator with two different types. An attempt to perform such an operation would result in a "Comparisons between attributes of different types is not supported" error message.

This restriction has been removed. Note, however, that ordered comparisons between different attribute types is still not allowed since there is no defined ordering between two ordered attributes. Triple attributes are discussed in Triple Attributes.

Bug24567 - Initial superuser account file owned by root if configure-agraph run as root

Previously, if AllegroGraph was installed, configured, and started as root, the directories and account file for the initial superuser remained owned by root. This resulted in the following error the first time the superuser made access to AllegroGraph: 

mkstemp failed: Permission denied [errno=13]

This has been corrected.

Bug24557 - Fix bug restoring repositories with a non-default stringTableSize.

When restoring repositories with a non-default stringTableSize directive to v6.2.0, a bug in an upgrade step would cause the repo's string table to be set to the default size of 16M. If the stringTableSize was larger than this default, the string table would be truncated, causing data loss and possible corruption.

This bug has been fixed.

Bug24548 - TransactionLogArchive directive doesn't create catalog and repository directories when tlarchiver is initiated

AllegroGraph did not previously create the TransactionLogArchive directory when starting a database instance. Instead a "no such file or directory" error was logged when the tlarchiver ran. This has been fixed. The directory (and any subdirectories) are now created when a database instance starts, if needed.

Bug24342 - load-turtle loses default-attributes for RDF collections

Previously the Turtle parser would fail to include attributes when parsing an RDF collection such as: 

prefix : <ex://foo#>  
:a :p (1 2 3) .

This has been fixed.

HTTP API

No significant changes.

SPARQL

Bug24538 - Invalid effective boolean value of constant strings in filters

A query like:  
SELECT * {  
 ?s ?p ?o  
 FILTER ("bruce")  
}

was returning no results when it should have returned every row in the store. Non-blank strings are supposed to have an effective boolean value of "true". Blank strings have an effective boolean value of "false". This has been fixed.

Bug24530 - SPARQL-JSON result format does not handle encoded boolean properly

Previously, boolean values in SPARQL-JSON results were represented as the strings "0" and "1". Now boolean values are correctly represented as "false" and "true".

Bug24529 - Some queries using VALUES produce wrong results.

Some queries using the VALUES keyword and joins or unions produced incorrect results. More specifically some rows were missing from the result.

Two examples of such queries are: 

select ?x { { values(?x) {(1)} } union { values(?x) {(2)} } }

and 

select ?x ?y ?z { { values (?x ?y ?z) {(1 2 3)} }  
                  { values () {()} }}

The first one would produce only a single result (2), while the second one would not produce any results. This has been corrected.

Bug24528 - Wrong join results when using the CAAT strategy

When using the chunk-at-a-time execution strategy some queries that performed left joins with the identity set would produce spurious results. One such query is: 

select ?a ?b ?c {  
  { ?a ?b ?c } union { ?c ?b ?a }  
  optional { bind(42 as ?x) }  
}

This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug24545 - parse-remote-store-specification returns inconsistent results

A "missing value for argument db.agraph::user" error would occur when starting a session using a store specification containing a remote URL without authentication information. This has been fixed.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

Rfe14695 - Replace pycurl with requests.

The HTTP library used by the AllegroGraph Python client has been changed from pycurl to requests. This change means that the Python client no longer depends on any libraries that require native extensions. This results in a simpler installation process, as it is no longer necessary to have a working C development environment on the target machine.

Note that pycurl will still be used if it is installed, since it might offer better performance in some scenarios.

Bug24582 - AGRAPH_PROXY variable not used

Proxy settings specified through the AGRAPH_PROXY environment variable used to be ignored.

This has been corrected.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.2.0

AllegroGraph Server: General Changes

Rfe14846 - Revised triple attribute filter operators

There are extensive changes to attribute filter operators, allowing better control when comparing user attributes with triple attributes. New operators include 

or, not, empty, overlap, attributes-overlap, subset, superset,  
equal, attribute-set\<=, attribute-set<, attribute-set=,  
attribute-set>, attribute-set>=

The existing operator attribute>= has been renamed attribute-set>= (the old name is still accepted but users are advised to switch to the new name). See Triple Attributes for information on attribute filter operators.

Rfe14806 - Audit attribute and static filter definitions

If auditing is enabled an audit record is made for defining an attribute and for defining the static attribute filter. See the Auditing document for more information on auditing.

The rdf:type of the new records are: 

<http://franz.com/allegrograph/4.11/audit-log#defineAttribute>  
<http://franz.com/allegrograph/4.11/audit-log#deleteAttribute>  
<http://franz.com/allegrograph/4.11/audit-log#defineStaticAttributeFilter>

Rfe14773 - Changes to agraph-control

The agraph-control program that starts and stops AllegroGraph has a new command restart, which requests that the server shut itself down, if running, and then start back up again. The reload command, which reloaded the configuration file, is no longer supported. See the agraph-control section of the Server Configuration and Control document.

Rfe14797 - Add permissions for user-attribute specification

Added user permissions user-attributes-header (if true, user can send attributes via HTTP as the value of the x-user-attributes header), and user-attributes-prefix (if true, user can specify the user attributes via the SPARQL prefix franzOption_userAttributes. See the Managing Users section of the WebView document for information on user permissions.

Rfe14733 - String comparison operator for attribute filter

Added the attribute= operator which can be used to compare attributes to other attributes, attributes to strings, or strings to strings. See Triple Attributes for information on attribute filter operators.

Rfe14671 - New configuration option StringTableCompression

The StringTableCompression configuration option allows specifying the compression type (including no compression) for the string table. The default is lzo999 compression. See Server Configuration and Control for information on configuration options.

Rfe14661 - agtool combines many command-line programs

The *agtool program replaces the programs agraph-backup, agmaterialize, agquery, agexport, agload, agraph-recover, agraph-replicate, and convertdb. agtool takes as its first argument the task to perform (usually indicated by a shortened version of the old program name but sometimes quite different -- so agraph-backup becomes archive and convertdb becomes upgrade) and then the various argument and options for that task. For example, in order to load the Ntriples file mytriples.nt into the mystore database in the server listening on port 10035, in earlier releases you ran this command: 

% agload --port 10035 mytriples.nt mystore

now you use agtool as follows: 

% agtool load --port 10035 mytriples.nt mystore

(Other argument/option combinations will do the same as those shown.) For most tasks, the arguments and options are the same as with the earlier tools, with the following differrences:

agraph-backup (archive), agmaterialize (materialize), The --ext-backup-command is no longer supported. Otherwise arguments and options are the same.

agraph-recover (recover), convertdb (upgrade), these changes: -c accepted for --catalog, -p accepted for --port.

agload (load), this change: -u not accepted as synonym for --base-uri.

agquery (query) and agexport (export), this change: --username is now --user and -u is a synonym of --user.

agraph-replicate (replicate), this change: -c accepted for --catalog, -u accepted for --user.

See the agtool General Command Utility document for more information and links to all the documentation for the individual tasks.

Rfe13254 - Tools for saving and loading metadata

Metadata, that is attribute and static filter definitions, can be saved from a database and loaded into a database. The methods for doing so include using the new --save-metadata option to agtool export (see Data Export) and the new --metadata option to agtool load (see Data Loading), along with new Lisp functions and REST API, discussed under Lisp API and HTTP API below.

Bug24425 - install-agraph no longer creates a log/ directory

install-agraph would incorrectly create a log/ directory alongside the lib/ and bin/ directories it creates. It no longer does so, leaving the creation of the log directory to tools like configure-agraph, agraph-control, or the agraph server itself.

HTTP API

Rfe13254 - load/save attribute metadata

GET and POST metadata commands allow obtaining metadata (attribute and static filter definitions) from a repo and loading them into another repo. See the Attributes section of the REST/HTTP interface document for details.

SPARQL

Rfe14812 - Support attributes in INSERT DATA and INSERT/WHERE

The syntax of SPARQL INSERT DATA and DELETE/INSERT/WHERE commands has been extended to support the fine-grained specification of attributes on a per-triple basis. See Triple Attributes for more information on attributes.

Rfe14726 - New prefixes for attributes

The defaultAttributes SPARQL query option specifies the attributes to be associated with each triple added via the INSERT DATA, DELETE/INSERT/WHERE, or LOAD commands. The userAttributes query option specifies the user attributes to be used when processing the query. See Triple Attributes for more information on attributes.

Rfe13305 - Old-style SPARQL GEO queries are no longer supported

Because AllegroGraph's magic properties support a richer range of geospatial features than the old-style SPARQL GEO syntax, the old syntax was deprecated in 2014 and has now been removed from the code. Any queries that use the old-style syntax will need to be rewritten.

Magic properties support both the deprecated 2D geospatial encodings and the newer nD ones.

Bug24481 - Wrong results for VALUES with a single variable

When processing a VALUES clause in a SPARQL query AllegroGraph would return incorrect results if there was exactly one variable bound by that clause and the list of values contained either:

This has been corrected.

Bug24470 - Spurious results for SELECT DISTINCT on a bound variable

When executing a query which tries to find unique values of an already bound variable like in the example below: 

select distinct ?s WHERE {  
  bind(ex:A as ?s) .  
  ?s ?p ?o .  
}

AllegroGraph would ignore the binding and return all results that match the pattern. This error could be observed only if the single-set execution strategy was used.

This has been corrected.

AGWebView

Rfe14576 - Make deleting namespaces easier on AGWebView Query page

You can now delete namespaces on the AGWebView Query page.

Rfe14569 - New vmstat-like server performance charts added to WebView

A new menu item in the WebView Utilities menu brings up a page of performance charts on the server machine. The several charts show approximately the same machine load data printed by the vmstat command, charted for the past five minutes. AGWebView is documented in the WebView document. The new Server Load page is documented here in that document.

Changes to the Lisp API

Rfe14797 - New user-attributes-prefix-permission-p keyword argument to run-sparql

The function run-sparql now have a user-attributes-prefix-permission-p keyword argument. It must be given a true value in order to run a SPARQL query including "PREFIX franzOption_userAttributes".

Rfe14754 - Allow #\: to be used as the catalog and repository delimiter

In addition to using the slash character (#\/), the colon (#\:) can now be used to designate a triple-store in a different catalog. I.e., the following are equivalent: 

(open-triple-store "test" :catalog "production")  
(open-triple-store "production/test")  
(open-triple-store "production:test")

Note that the format AllegroGraph uses to display triple-stores will now be catalog:name rather than catalog/name.

Rfe14705 - The triple-store-user (db.agraph.user) package uses the sparql package

The triple-store-user package, also named the db.agraph.user package, now uses the sparql package which means that symbols in the sparql package like run-sparql can be used more easily as they do not need to be package qualified.

Rfe14703 - :return-fat-triples keyword argument is now :return-extended-triples

The return-fat-triples keyword argument to get-triples, get-triples-list, and freetext-get-triples has been renamed return-extended-triples. The new functions extended-triple-triple and extended-triple-attributes return the triple and the attributes (respectively) of an extended triple. The new function copy-extended-triple makes a copy of an extended triple.

Rfe13254 - New functions to load/save attribute metadata

The functions get-metadata. set-metadata, load-metadata, and save-metadata allow retrieving metadata (attribute and static filter definitions) from a store and adding metadata to a store. The metadata can be transmitted as a string or written/read from a file.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.1.6

AllegroGraph Server: General Changes

Rfe14785 - Allow default session idle timeout to be set in agraph.cfg

Two new configuration parameters have been defined for controlling the default and the maximum allowed session idle timeout. DefaultSessionTimeout specifies the default timeout and MaximumSessionTimeout specifies the maximum timeout. See the description of those directives in the list of top-level directives in Server Configuration and Control for details.

Bug24519 - Path-finding broken on remote-reasoning-triple-store

Path-finding operations (such as depth-first-search) were broken for remote-reasoning-triple-stores since v6.1.4. This has been fixed.

Bug24389 - Bus error when adding triples with unspecified graph to security-filtered-triple-store

Previously, adding a triple to a security-filtered-triple-store could result in a bus error. This has been fixed.

Bug24221 - String table configuration mismatch during restore.

An error would occur when trying to restore a database that was backed up in AllegroGraph prior to version 6 if the catalog it was restored to had a string table size different than the default 16M entries.

The error was: 

> String table configuration mismatch: Configured number of slots X  
 does not match recovered number of slots (16777216)

This has been corrected.

HTTP API

No significant changes.

SPARQL

Bug24463 - Binding produced index out of range error during query

Some queries using BIND used to fail with the above error message. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

Bug24427 - Javadocs missing from agraph server distribution

Previously the Javadocs for the AllegroGraph Java client were missing from the installed documentation set. This has been fixed.

Python Client

Rfe14803 - Support for decimal literals

Literal objects constructed from Python's Decimal values will now have the default datatype of xsd:decimal.

Rfe14694 - Drop support for cjson

The Python client no longer uses the cjson library. Python 2.6 users should install the simplejson library instead to avoid a significant degradation in performance. Other versions of Python are unaffected.

Bug24428 - Creating literals from time objects with timezone

RepositoryConnection.createLiteral used to raise an error when passed a datetime.time instance that contains time zone information. This has been corrected - such calls will now return literals of type xsd:time with time value converted to UTC.

Bug24405 - Non-ASCII characters in addData()

The Python client used to raise an exception (specifically a UnicodeDecodeError) if the data passed to addData was a Unicode string containing non-ASCII characters.

This has been corrected.

Bug24322 - Python clients reads imported file into memory

When uploading data to the server the Python client used to read the whole file into memory before sending it. This could cause out of memory errors for larger files. To avoid this issue, the client now sends files in chunks. This method requires only a limited amount of memory.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.1.5

AllegroGraph Server: General Changes

Bug24442 - Fix intermittent query failures that report a "broken pipe" error

A bug was introduced in v6.1.3 where requests could fail inappropriately under certain conditions, with a "broken pipe" error message appearing in agraph.log or reported in AG WebView. This has been fixed.

Bug24412: agload incorrect line number reporting on invalid character

The agload program (see Data Loading (agload)) could report incorrect line numbers if errors were encountered when loading large N-Triples and N-Quads files in parallel. This problem could only occur if the files were larger 4MB. This has been corrected.

HTTP API

No significant changes.

SPARQL

Bug24352 - In FILTERs can be handled very inefficiently

SPARQL queries that constrain a particular binding to a set of values could operate very inefficiently in some circumstances. Such queries include ones that use the IN FILTER or that use the VALUES clause to supply a list of possible matches. The inefficiency has been corrected.

AGWebView

Bug24423 - WebView hangs if HTTP tracing is enabled.

Previously AGWebView would hang on page load if HTTP tracing was enabled. This has been fixed.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.1.4

AllegroGraph Server: General Changes

Rfe14558 - If possible increase the file descriptor limit at startup

If the agraph server is started as root it will set the number of file descriptors agraph can use to 65536 thus not being limited by the default operating system setting (which is usually quite low).

If the file descriptor limit is raised you'll see a line like the following in agraph.log just after the server starts: 

[2016-11-08T17:07:23.956 p13419 daemon] Set fd limits soft: 65536, hard: 65536

Rfe11277 - Warn when a freetext query tries to match on things that cannot possibly match

AllegroGraph now generates a query warning when a freetext search expression cannot possibly find matches. For example, the default stop words in an AllegroGraph freetext index include the and an so it is impossible to search for matches on the expression (or 'an' 'the'). This means that a query like 

select * { ?s fti:match 'the || an' . }

will return no results. Previously, AllegroGraph would have returned no results without generating a warning. Now, AllegroGraph will warn that The freetext expression the || an cannot succeed.

Rfe11046 - Release fds for free text index disk tries

Making use of a free text index will require far fewer file descriptors than before.

Bug24395 - TriG parser errors could print that they were from the Turtle parser

Some TriG parser errors incorrectly indicated that the current parser was using Turtle syntax rather than TriG. This has been corrected.

Bug24349: Transaction log archiving to same filesystem broken

Transaction log archiving was broken in v6.1.2 when archiving to the same filesystem. This problem is now corrected.

Bug23354 - Check SSL cert when frontend starts

If the SSL certificate information you put in your configuration file is not correct AllegroGraph will log an error and exit during server startup. Previously you didn't find out about SSL certificate errors until the first https request came to the server.

Bug24332 - AllegroGraph doesn't fail gracefully if it cannot claim desired ports

Previously, if AllegroGraph was unable to bind the TCP port specified by the Port configuration directive, it would leave some lingering processes after printing the error message. This has been fixed.

Bug24297 - <http://www.w3.org/2005/xpath-functions#:translate> was broken

The XPath translate function could not be used between AllegroGraph 6.0.2 and 6.1. This has been corrected.

If a query attempted to use the translate function, it would get a parse time error.

HTTP API

No significant changes.

SPARQL

Rfe14722 - Improve planning speed for SPARQL queries that use IN filters

The code generated for evaluating SPARQL IN queries has been optimized for some common cases. This will result in faster plan times for such queries.

Bug24383 - Sparql Update can fail if the WHERE part uses string literals

If the WHERE portion of a SPARQL DELETE/INSERT/WHERE command used string literals that were not already interned in the triple-store in FILTER expressions, then it was possible for the UPDATE to fail. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Rfe14736 - Add graph parameter to SNA path finding functions

The various SNA path finding functions like all-depth-first-search-paths and bidirectional-distance all take a new graph/s parameter. This can specify either a single graph or a list of graphs that will be used to restrict path search to only triples from these graphs.

Rfe14721 - Path finding functions accept lists of predicates as generators

The Lisp API to AllegroGraph's Social Network Analysis path finding algorithms has been extended. Previously, the functions took a generator that must have been previously defined. Now, you can pass a list of predicates in the place of a generator and these will be used to create a generator on the fly. Note that this added flexibility does come at the cost of between 5 and 10 percent slower performance so predefined generators are to be preferred.

The syntax of the predicate list is as follows. Each element in the list must be one of

For example, we can find the paths between ex:A and ex:B using undirected links on either ex:knows or ex:likes via the following: 

(all-breadth-first-search-paths  
  !ex:A !ex:B  
  '((:undirected !ex:knows) (:undirected !ex:likes)))

This change also adds a new REST API to the path finding algorithms via sna/paths. See the HTTP reference for details.

Bug24293 - Some plain literals could be printed incorrectly

Changes in AllegroGraph 6.1 made it possible for some plain literals to print incorrectly in some situations when using remote-triple-stores. For example, the literal a/b/c could print as a/b"/c.

This has been corrected.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

Rfe14693 - Allow data import from strings

A new method named addData has been added to the connection class. This method makes it possible to add RDF data stored in a string to the store. All RDF data formats supported by addFile can be used.

Rfe14692 - Add more RDFFormats

The only formats supported during data import (addFile) used to be N-Triples and RDF/XML. It is now possible to use other formats, namely Turtle, TriG, N-Quads, NQX and TriX. The addFile method is now capable of determining the correct format based on the filename. When using this capability it is also possible to import gzip compressed files.

bug24379: HTTPS in Python does not work with Basic auth

The Python client used to only allow HTTPS connections if a client certificate was used for authentication.

It is now possible to connect to HTTPS servers by passing either a host value starting with 'https://' or a protocol value of 'https' when creating the server object.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.1.3

AllegroGraph Server: General Changes

Rfe14363: Add binary-only, non-interactive modes for install-agraph

Two command-line options have been added to install-agraph to support non-interactive installation:

Installation and configuration are described in the Server Installation document.

Bug24253 - Triple-store statistics could become stale when new datatypes were added

It was possible for the statistics AllegroGraph keeps on each predicate to become stale if triples were added with heretofore unseen XSD datatypes. This has been corrected.

Bug24245 - Text index word filters and wildcard search do not work together

A freetext index that used word filters like drop-accents or the stemmer could fail to find matches when using wildcard searches. This has been corrected.

HTTP API

No significant changes.

SPARQL

Bug24273 - The SPARQL RAND() function was being evaluated at plan time

AllegroGraph was mistakenly evaluating the SPARQL RAND() function at plan time rather than query execution time. This could lead to incorrect results and has been corrected.

Bug24214 - Querying using the direct reification magic property returns at most one match

AllegroGraph's direct reification magic property only returned the first matching triple. This has been corrected so that all matches are returned.

AGWebView

Rfe14205: Maintain and provide information on live HTTP requests

A new page, Admin -> Requests, was added in AGWebView. It shows all the currently running and recently finished HTTP requests handled by AllegroGraph and its subprocesses.

Bug24256 - Regular users cannot see catalogs

The catalogs list displayed on the main WebView page used to contain only the catalogs for which the current user has 'read' access to all repositories therein.

The list now contains catalogs for which the user has 'read' access to at least one repository therein.

Changes to the Lisp API

No significant changes.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

Bug24300: Python shutdown hang

Previously if a session was created but not closed before Python was ready to exit, the shutdown of Python could be delayed by a very long time. This problem has been fixed. However, it is always best to call closeSession() on a session when it is no longer needed.

Bug24290 - Changes to Python tutorial

The Python tutorial is now configurable using environment varaibles which provide information about the catalog, username, password, etc. See the Python Tutorial for more information.

Java Client

Bug24238 - Java API Unicode issues

The Java client used to incorrectly process non-ascii characters in results of CONSTRUCT queries. Such characters were replaced by question marks. A similar problem could be observed with results of some other calls (e.g. evalInServer) on platforms where the default encoding used by Java is not UTF-8. A notable example is Windows, where the encoding tends to be CP-1252 (although that depends on the language).

Both issues have been corrected.

nD Geospatial

No significant changes.

AllegroGraph 6.1.2

AllegroGraph Server: General Changes

Rfe14561 - Remove the deprecated --encoding option from agload

Previously, the -C/--encoding option was used to allow agload to process N-Triples and N-Quads files encoded in a character encoding other than UTF-8.

This option has been removed. All N-Triples and N-Quads files with non-standard encoding must be converted with a separate tool, such as iconv, prior to being processed by agload. agload is described in the Data Loading (agload) document.

Bug24233 - Install-agraph.sh errors on tutorial directory when overwriting

When installing AllegroGraph to an existing installation directory, an error would occur when attempting to install the tutorial directory. This is now fixed.

HTTP Client

No significant changes.

SPARQL

No significant changes.

AGWebView

Rfe14407 - Use RDF syntax for displaying namespaces in AGWebView

The display style of namespace abbreviations in AGWebView was changed so that they can now be copied and pasted into a SPARQL query.

Bug24194 - Free text search URI incorrectly parsed slashes in WebView

In WebView, URLs of full-text search and script pages were incorrectly parsed if their arguments contained slashes. This has been corrected.

Bug24172 - Namespace sorting and duplicate inconsistencies in WebView

In AGWebView, namespace abbreviations on the Namespaces and Query pages are now sorted alphabetically. Also, adding a namespace with a duplicate prefix is now forbidden. To modify a namespace, delete it and add it, modified, again.

Bug22807 - WebView error when asking for the plan of a SPARQL update command

AllegroGraph signaled an error when asked to display the plan of a SPARQL Update command. This has been corrected.

Changes to the Lisp API

Bug23492 - with-query-environment is not documented

Documentation has now been added for with-query-environment. That macro ensures all cursors created in its body are discarded when the body completes.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

Javascript Client

Bug24207 - Javascript client library could leak cursors

It was possible for scripts using the Javascript client library to leave cursors in an active state. Although these cursors are eventually freed, they use resources and can cause lock contention. The problem is now corrected: Javascript scripts will always close all cursors automatically.

nD Geospatial

No significant changes.

AllegroGraph 6.1.1

AllegroGraph Server: General Changes

Rfe14493 - Print progress messages during long restores.

During a database restore by agraph-backup a progress report is periodically printed to stdout, showing the fraction of the data file which has been processed, the recent processing rate, and an estimate of the completion time. See Backup and Restore.

Bug24168 - Wrong triple count after a delete with duplicate suppression

Deleting an uncommited triple could cause AllegroGraph to report an incorrect triple count (possibly negative) if duplicate suppression was active at commit time.

This has been corrected.

Bug24153 - owl:sameAs interferes with owl:transitiveProperty during materialization

The presence of owl:sameAs assertions could lead to AllegroGraph failing to infer all owl:transitivePropery triples during materialization. This has been corrected. (Note that this problem only occurred using the materializer and not the dynamic RDFS++ reasoner).

Bug24147 - Support "null" as a synonym for the default graph

The remove-graph-uri, insert-graph-uri and using-graph-uri parameters of the SPARQL and Prolog HTTP endpoints now allow the default graph to be referred to by using the string "null" as the URI.

This resolves a problem with the Java client in which SPARQL Update queries issued through the Jena API did not work as expected. Specifically, a WHERE clause would fail to match any triples unless a graph was specified explictly in the query.

HTTP Client

No significant changes.

SPARQL

No significant changes.

AGWebView

Bug24062 - AGWebView uses "Repository" and "Store" interchangeably

In WebView, both "repository" and "store" (and also "triple store) were used to describe repositories. Now, "repository" is used in all places. The documentation has been updated to reflect the change,

Bug24173 - TypeError when going back to query view with saved query result in AGWebView

In AGWebView, going back to a recent query view with cached result caused a harmless error to appear on the browser console. This has been corrected.

Changes to the Lisp API

Bug24179 - Unicode issues in the remote Lisp client.

The remote Lisp client in AllegroGraph 6.1 used to fail in an unpredicatable manner if non-ascii characters were present in query results.

This has been corrected.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

nD Geospatial

No significant changes.

AllegroGraph 6.1

Server Changes

Rfe14282 TBC 5.1.4 plugin

AllegroGraph plugin for TopBraid Composer has been updated to be compatible with TBC 5.1.4. See TopBraid Composer Plugin for more information.

Rfe14279 - agraph-backup should shut down instances

Previously, agraph-backup would leave instances open and lingering after performing a backup or restore operation. This could result in hangs due to memory exhaustion when running backup-all or restore-all operations across many repositories. The likelihood of hang would depend on a combination of repository size and the value of the InstanceTimeout directive.

Now, agraph-backup will attempt to shut down the instance process of all repositories it operates on, as soon as the operation has completed. agraph-backup is described in the Backup and Restore document.

Rfe14273 - Reduce repeated query log messages

AllegroGraph was repeating some messages unnecessarily when logging query execution. This repetition has been reduced.

Bug24130 - Access to the system initfile should be protected

It was possible for any user to access the contents of the AllegroGraph init file. This has been corrected. Now only users with superuser privileges will be allowed to view the file.

Bug24128 - Blank nodes in RDF/XML collections

When parsing an RDF/XML file containing a collection with a blank node as one of its elements, AllegroGraph used to generate an invalid triple (using "li" as the predicate, instead of one of 1, 2, _3, ...).

This has been corrected.

Bug24126 - RDF/XML parser incorrectly handles empty elements.

The RDF/XML parser used to incorrectly handle empty literals encoded as property elements with empty bodies. The parser would generate a blank node instead of an empty literal.

This has been corrected.

Bug24079 - agload with '--error-strategy ignore' and unrecognized file types

agload will now print a warning before ignoring a file of an unrecognized type regardless of the value of the --error-strategy argument. agload is described in the document Data Loading. Example: 

agload --error-strategy ignore repo wrong-file  
Cannot guess the format of wrong-file. Use --input to specify it.  
Load finished 0 sources in 00:00:01 (1.00 seconds).  No triples added.  
Terminating agload processes, please wait...

Compare with previous behavior: 

agload --error-strategy ignore repo wrong-file  
Load finished 0 sources in 00:00:01 (1.00 seconds).  No triples added.  
Terminating agload processes, please wait...

Bug24066 - Sessions did not close their databases after idle timeout expires

When a session expired after a given timeout, it would not close its database until another session was started. This has been corrected.

Bug23964 - Incorrect escaping of IRIs in Turtle output

AllegroGraph used to incorrectly escape some characters when using the Turtle format. Furthermore, spaces and a few other characters that are not legal in IRIs were not escaped at all. Other characters, such as double quotes, were escaped using an invalid syntax (\" instead of \u0022).

Qualified names were affected by a similar problem.

This has been corrected - all characters are now properly escaped in both IRIs and qualified names.

Bug23960 - gz file import is broken

Importing .gz or .bz2 files using single-threaded import used to cause an error as the file was not correctly uncompressed.

This has been corrected.

Bug23828 - CSRF vulnerability in file upload

The import dialog in AGWebView was not adequately protected against Cross-Site Request Forgery (CSRF) attacks. When a logged-in user visited a malicious web site, an attacker could import arbitrary triples into any store the user can access.

This has been corrected - a CSRF attempt will now result in a password prompt.

Bug23820: Prevent repository access when a restore is in progress

Previously, an attempt to access a repository while it was in the process of being restored could result in random errors and problems, including being unable to close a repository. This change improves the way in which AllegroGraph notes that a repository is being restored, and improves the check for restore-mode so that these issues are prevented.

Bug23705 - A triple-store must be open to define SPIN magic properties

Changes in AllegroGraph 6.0 required that a triple-store be open before any SPIN magic properties could be defined. This has been corrected.

Bug23339 - Canonical escaping in N-Triples

AllegroGraph used to produce invalid escape sequences in URIs when printing triples in the N-Triples format. Some reserved characters, such as double quotes, were escaped with a backslash. Others were not escaped at all.

This has been corrected. All reserved characters are now properly escaped using the \u notation.

In addition, escaping of strings in N-Triples has been modified to make the output match the canonical form described in the standard. The most notable change is that non-ascii characters are no longer escaped.

Bug23181 - Consistency bug when backing up deleted bitmap

If a triple was deleted while backup was in progress, that triple would also be marked as deleted in the restored store. This has been fixed, backup now represents a consistent state of the store at the time it was started.

Bug22115: Fix configure-agraph handling of passwords.

configure-agraph could crash when entering passwords that contain certain non-alphanumeric characters in them. This bug has now been fixed.

Bug21655 - serialize-rdf/xml can emit illegal xml characters

In XML, control characters (e.g. NUL, BEL, STX, ...) cannot be represented in any form. The RDF/XML serializer (with the Lisp function serialize-rdf/xml, used by agexport when passed --output rdfxml) used to silently output such characters, producing files that could not be parsed.

This has been corrected - if the serialized data contains any invalid characters an error will be reported, as in the following example: 

bash-4.2$ agexport -o rdfxml store output.rdf  
open the triple-store store on port 10035 and export to ...  
Warming up the store  
Error Unable to escape character U+0008 (Backspace). occurred during processing  
logging error to /.../agexport.log

Bug20220 - Fix bug when validating the --loaders argument to agload

The --loaders argument to agload accepts as a value a positive integer between 1 and 32. On machines with a large number of cores, agload would default to a number of loaders greater than 32 and then error due to the value being too large. This validation bug is now fixed.

HTTP Client

Rfe14445 - Add REST API for [catalogs/CATNAME]/respositories/REPONAME/exists

AllegroGraph's REST API now includes 

[catalogs/CATNAME]/respositories/REPONAME/exists

which can be used to determine whether or not the named repository exists. It will return a 404 code if the repository does not exist and a 200 response with the body true if it does.

Bug23942 - Handle HTTPS scheme for SPARQL UPDATE LOAD command

The SPARQL UPDATE LOAD command would fail when using the HTTPS URL scheme. This has been corrected.

Bug23817 - Running a query that used MINUS repeatedly via REST could error

Under certain circumstances, if a SPARQL query that used MINUS was evaluated more than once using the HTTP REST API, AllegroGraph could signal an error. This has been corrected.

SPARQL

Rfe14278 - Improve SPARQL SERVICE efficiency for AllegroGraph endpoints

AllegroGraph now uses a more efficient binary data format when communicating with AllegroGraph-based SPARQL endpoints. This can lead to large performance improvements for SPARQL SERVICE calls.

Rfe14257 - DISTINCT|REDUCED mode should be passed on to SERVICE calls

When the main query uses DISTINCT or REDUCED, any use of SPARQL federation will pass on the modifier to endpoints. This can greatly reduce overall IO.

Rfe14254 - SERVICE should remove duplicates from any VALUES it passes along

If the main query is using the DISTINCT or REDUCED modifier, then the SERVICE requests should remove duplicate rows from the list of VALUES that is passed to the endpoint. This can make the query much more efficient.

Rfe14086 - New SPARQL functions for performing haversine calculations.

These SPARQL functions accept various combinations of latitude/longitude coordinates and nD encoded literals. They compute distances on the Earth's surface.

The individual names indicate the required argument types: a Lat or Lon must be a number interpreted as degrees. A Loc must be an nD literal that encodes a latitude and a longitude. Its nD encoding must be registered in the open database, but is not needed in any of the function calls themselves.

Each of these functions accepts an optional additional argument specifying the units of the computed result. It may be :km (default) or :miles expressed as a resource in the keyword namespace, i.e., <http://franz.com/ns/keyword#km> or <http://franz.com/ns/keyword#miles>.

See the SPARQL functions list for a list of SPARQL function with links to further documentation.

Rfe11390 - Modify SPARQL 1.1 lexer/parser to match changes to standard

The SPARQL parser used to fail when the input contained any:

All mentioned cases are now parsed correctly, in accordance with the SPARQL grammar.

Bug24161 - Correct possible segmentation faults during SPARQL DELETE operations

Under some circumstances, a SPARQL DELETE command could cause a segmentation fault. This has been corrected.

Bug24103 - Correct problem in nD magic property SPARQL queries

It was possible to get an error when executing a query that combined nD magic properties and other patterns. This has been corrected.

Bug24030 - An empty IN filter is ignored if there are any other IN filters

If a SPARQL query contained two or more IN filters that were associated with the same triple pattern and one of the IN filters was empty, then both IN filters were ignored. This has been corrected.

Bug24026 - Fix query failure when using DISTINCT, SERVICE and magic predicates

It was possible for a query to fail during planning if it used multiple SERVICE clauses, the service clauses used magic predicates, and the query asked for DISTINCT results. This has been corrected.

Bug24020 - Property path queries in saved plans with Chunk-at-a-Time engine can lose solutions

It was possible for some solutions to be lost on subsequent runs of a saved query plan using the Chunk-at-a-Time execution engine if the query used SPARQL OneOrMorePath or ZeroOrMorePath property path expressions. This bug sometimes appeared when running queries in AGWebView because it directs AllegroGraph to use saved plans. This has been corrected.

Bug23988 - Fix query failure in federations of reasoning and non-reasoning stores

Some queries could fail with an error in a federated triple-store composed of a mix of reasoning and non-reasoning leaf stores. This has been corrected.

Bug23949 - Filtering on multiple language tags could fail

The FILTERing machinery in AllegroGraph's SPARQL engine could fail to correctly process collections of language tag checks. For example, a query with a filter like: 

FILTER( lang(?x) = 'fr' || lang(?x) = 'en' )

could fail. This has been corrected.

Bug23897 - Query warnings are incorrectly cached

Query warnings saved in SPARQL plans were not being updated to take account of changes in the underlying store.

Bug23893 - Federated-triple-stores and statistics caching could lead to poor plans

The caching of store statistics could lead to incorrect plans when querying a federated-triple-store. This has been corrected.

Bug23889 - Correct breakage in solR magic property queries

The SPARQL magic properties for solR queries were broken in AllegroGraph 6.0. This has been corrected.

Bug23884 - Constant pattern only CONSTRUCT could fail when executed via a saved plan

If a CONSTRUCT query template contained only constant patterns (i.e., no variables), and there was a limit on the query, and the query was being executed via a saved plan, then only the first execution would return results and subsequent executions would return nothing. This has been corrected.

Bug23882 - Algebraic query combines some BGPs too aggressively

The query planner was sometimes combining BGPs and inadvertently producing cross-products which could lead to slower query performance. This has been corrected.

Bug23881 - FILTER clause could be lost in complex SPARQL queries

It was possible for a FILTER clause to be lost during query planning. This has been corrected.

Bug23873 - Neither limit nor planner should require eval permission

Users without 'eval' permissions were unable to choose a planner or limit for a query. This has been corrected.

Bug23870 - Restriction reasoning could cause error during SPARQL query

Depending on the way it was invoked, a SPAQRL query could signal a error when restriction reasoning was used. This has been corrected.

Bug23841 - Do not allow triples in query results to be deleted

It was possible to remove triples from the database by clicking the X icon next to CONSTRUCT or DESCRIBE query results. This possibility has been disabled. CONSTRUCT and DESCRIBE queries do not return enough information to uniquely identify a triple in the triple-store. Offering to delete them is error prone and ambiguous. To delete triples, either use a SPARQL DELETE statement or the node view (click any part of a triple in the results).

Bug23755 - Problem with alternating property path in sub-queries

A sub-query that used a property path pattern with alternation could incorrectly bind the predicate of matched triples to output variables. An example of the problem would have been: 

select ?s ?o {  
  { select ?s ?o {  
     ?s :hasChild | :hasParent ?o .  
    } }  
}

This query would bind the predicate of matched triples to ?o. This has been corrected.

AGWebView

Rfe14371 - Allow the number of loaders to be selected in AGWebView

It is now possible to specify the number of loaders when invoking agload through AGWebView.

Rfe13428 - Display query information for CONSTRUCT, DESCRIBE and ASK queries

The query information tab is now displayed alongside results of CONSTRUCT, DESCRIBE and ASK queries.

Bug24047 - WebView: download duplicates ignores mode

The Export duplicate statement command in Store Control ignored the choice of SPO/SPOG. This has been corrected.

Bug23948: Fixed paged behavior after using the browser's back button

Previously, the Download and Load more buttons would function incorrectly if used after clicking the browser's back button. This has been corrected.

Bug23888 - Upload progress indicator not showing

AGWebView will now display progress information when uploading a file. This was previously not visible due to a bug. Progress is reported as a text line similar to this: 

Uploading file triples.nt(1475 Kb uploaded)

Bug23887 - Automatic scrolling in the AGWebView view server log pane broken

The server log in AGWebView would not auto-scroll when new log data appeared. This has been corrected.

Bug23857 - Load more does not work in Free Text search

Loading the next page of Free Text search results did not work. This has been corrected.

Bug23856 - Repeated text search breaks navigation

Page navigation would sometimes break when a user entered the same query twice in the Free Text search field. This has been corrected.

Bug23840 - Wrong even/odd row coloring when odd limit chosen

Row coloring could appear incorrect when using an odd limit and paging in the query view. This has been corrected.

Bug23839 - Planner selector grayed-out after language detection

It was possible to disable the planner selector in the query view if language autodetection was used. This has been corrected.

Bug23827 - Process list in WebView crashes on Firefox

The process and job list views of AllegroGraph WebView did not display any processes or jobs on Firefox. This has been corrected.

Bug23811 - AGWebView import fails for RDF/XML

Multi-core file import in AGWebView would fail if the chosen format was RDF/XML. This has been corrected.

Bug23810 - Importing via file upload no longer works in AGWebView

The multi-core file import feature in AGWebView was not working unless the file type was set directly. This has been corrected. In addition, AllegroGraph now recognizes application/n-triples and application/n-quads as media-types for the import of N-Triples and N-Quads (respectively).

Bug23800 - Query progress spinner only appears on every other query in Agwebview

The "Loading..." indicator was not displaying the first time that a new SPARQL query was executed via AGWebView's query editor. Now the loading indicator is shown every time.

Bug23774 - CONSTRUCT queries not displaying results in AGWebView

Results from CONSTRUCT queries were not being displayed in Safari under OS X. This has been corrected.

Bug23752 - Agwebview Lisp editor fails when digits are at the end of a line

The Lisp and Prolog script editors would become unresponsive if the last token on a line was a number. The JavaScript console showed the following error: 

TypeError: undefined is not an object (evaluating 'stream.peek().toUpperCase')

This has been fixed.

Bug23141 - AGWebView's CONSTRUCT doesn't show the MORE button

Previously, if the number of results of a CONSTRUCT query exceeded the limit set by the user, there was no way to show more results. Now, there is a MORE button at the bottom of the results which functions similarly to the one used for SELECT queries.

Bug22314 - Federated-triples-stores, encoded IDs and AGWebview don't mix

Attempting to use AGWebView to query a federated triple-store whose leaf nodes used encoded IDs led to an error. This has been corrected. Note: Correct behavior of a federation of stores that use encoded IDs assumes that each leaf node has the same definitions for encoded ID prefixes.

Changes to the Lisp API

Rfe11567 - Support store specifications in the Lisp client

In addition to using the catalog keyword argument, AllegroGraph now accepts store names in the form CATALOG-NAME/STORE-NAME. E.g., 

(open-triple-store "general/test-data")

would open the store test-data in the catalog general.

You can also supply a triple-store specification in calls to open-triple-store. This will build a store instance from the specification, bind it to *db* and return it. In this case, the store instance will be given a name like store12341.

Finally, you can query store specifications directly using any of the HTTP clients by using a URL encoded version of the specification in place of a simple store name. E.g., 

curl -X GET -v "http://localhost:11040/repositories/%3Ca%3E%2B%3Cb%3E?query=select%20*%20%7B%20%3Fs%20%3Fp%20%3Fo%20%7D%20limit%205"

would send the query 

select * { ?s ?p ?o } limit 5

to the federation of triples stores a and b (as specified by <a>+<b>).

Note that in this case, AllegroGraph will not start a new session so Prolog functors or other things defined in the servers initialization file will not necessarily be present.

Bug24105 - Deprecate serialize-rdf-n3 in favor of serialize-turtle

AllegroGraph can serialize data in turtle format but not in N3 so it is better to use the correct name. The now deprecated function serialize-n3-rdf will be removed in a future release and should no longer be used. serialize-turtle should be used instead.

Bug24006 - pcache Lisp module conflicts with AllegroGraph

A package conflict made it impossible to load AllegroCache and pcache in the same Lisp execution as AllegroGraph (without tiresome package manipulation). The problem has been fixed in this release.

Bug23846 - warmup-triple-store does not work in remote-lisp-client

Calling warmup-triple-store on a remote store used to have no effect. The call did not fail, but no operation was performed. The remote client will now correctly pass the request to the server it is connected to, preparing it for subsequent operations.

Prolog

Bug23877 - Prolog select exposes a security hole

There was a remote code execution vulnerability in Prolog query handling code where users without eval permissions could execute arbitrary code on the server. Executing Prolog queries now requires eval permissions. (This bug was fixed in various maintenance releases including version 6.0.2. We repeat it here because of its importance.)

Documentation

Bug23874 - Documentation search in local AllegroGraph docs doesn't work

The documentation search field in the upper left corner of AllegroGraph documentation pages now uses Google to search for the given phrase within: 

franz.com/agraph/support/documentation/<version>/

where <version> is vX.Y[.Z] (v6.1 in the case of the current release). This means that even if you have a local copy of the documentation the search is done in the web copy on the Franz Inc. website.

Python Client

RepositoryConnection.createCoordinate arguments renamed

The lat and long arguments of RepositoryConnection.createCoordinate have been renamed latitude and longitude.

Bug23901 - RepositoryResult.asList() always returns None

The asList method in the RepositoryResult class was buggy and always returned None. It now correctly returns all statements from the result object.

Java Client

No significant changes.

Geospatial

Rfe12345 - Unnecessary geospatial functions removed

The functions geospatial-subtype-index-in-this-image and geospatial-subtype-index-in-db are neither useful nor necessary in the geospatial API. They have been unexported and removed from the documentation.

Bug23933 - Geospatial subtypes could be mistranslated on remote stores

It was possible for encoded geospatial data to be mistranslated when connecting to AllegroGraph via a remote-triple-store in the Lisp environment. This has been corrected.

Bug23848 - nd:inCircle :units incorrectly defaults to :miles, not :km

The :units argument to get-triples-nd-within-radius in Lisp and the similarly named functor in Prolog, and nd:inCircle in SPARQL, incorrectly defaulted to :miles instead of :km. This has been fixed so the behavior now agrees with the documentation. Since a kilometer is less than a mile, the change may cause previous queries that did not specify :units now to return fewer results. The extra results before the change were bogus.

Bug23767 - automate-nd-datatype-mappings could fail under parallel load

Spurious metadata conflict errors could be signaled if a store using automatic nD-datatype mappings was being accessed from multiple clients (for example, via agload). This has been corrected.

Bug23584 - nD query support for ISO6709 time

In an nD query, the min and max limits for a :time type ordinate may be expressed as a real number, as a dateTime UPI, or as a string literal in ISO6709 syntax (either as an xsd:dateTime typed string or an untyped string). These are supported in both SPARQL and Lisp/Prolog queries. Previously ISO6709 was not accepted in all contexts.

AllegroGraph 6.0.2

Prolog

Bug23877 - Prolog select exposes a security hole

There was a remote code execution vulnerability in Prolog query handling code where users without eval permissions could execute arbitrary code on the server. Executing Prolog queries now requires eval permission. User permissions are discussed in the section Managing Users in the WebView document.

User-visible changes in earlier releases

See Change History for user-visible changes to earlier releases.