AllegroGraph 4.7 user-visible changes

AllegroGraph Server: General Changes

Rfe11587 - Change InstanceTimeout defaults

The configure-agraph script now supports instance timeouts in interactive mode and via the --instance-timeout command line option. It defaults to one week.

Also, the default value the InstanceTimeout configuration directive (effective when there is no global or catalog level InstanceTimeout) has been changed from 0 (= disabled) to one hour.

Rfe11536 - Transitive inference queries could be slow

The RDFS++ reasoning for transitive queries could be slower than necessary for some datasets. The code has been reimplemented to be significantly faster.

Rfe11285 - Improve query constraint analysis

AllegroGraph now does a better job of applying filter constraints throughout a query. For example, previous versions of AllegroGraph would not have transformed either of the two BGPs (on either side of UNION) into range queries (which are more efficient).

SELECT * {  
  ?a ex:d ?c .  
  { ?a ex:b ?c . } UNION { ?a ex:c ?c . }  
  FILTER (?c > 2)  
} 

Now it will.

Bug21114 - Instances lingering too long

Instances with non-zero InstanceTimeout only timed out when there was activity involving the service daemon. This has been corrected. Instances with an InstanceTimeout of zero (the default) are not affected.

Bug21094 - Queries against federations could signal UPI-not-in-string-table errors

It was possible for queries against federated triple-stores to lead to UPI-not-in-string-table errors. This has been corrected.

Bug21089 - agload division by 0

Progress reporting failed with division by 0 if the operation completed within 1ms.

Bug21077 - Expressions can be analyzed too eagerly

It was possible for constant expressions to be analyzed to their final values too quickly which could lead to an error if the expressions were used in a BIND. For example, this query would fail:

select * { bind(str('hello') as ?x } 

rather than producing a single binding of ?x to the string hello. This has been corrected.

Bug21069 - A warm-standby client now exits non-zero if error occurs during commit replay

Replication secondary (warm-standby client) now exits non-zero if error occurs during commit replay. This is to ensure transactional behavior in the replica.

Bug21059 - Improve efficiency of all predicates cache when RDFS++ is enabled

The query engine keeps track of the predicates in the triple-store to optimize queries which contain clauses where the predicate position is a variable. If RDFS++ was enabled and the cache was not already fresh, then AllegroGraph would try to find all of the store's predicates with reasoning enabled. Since new predicates cannot be inferred, this could be hugely inefficient. This has been corrected.

Bug21057 - Fix log rotation causing crashes

When the SIGHUP handler closes a log stream, attempts to log could fail with a 'stream closed' error potentially crashing the server. This bug has been fixed.

Bug21040 - Turtle data import should use UTF-8 encoding

The Turtle specification mandates that Turtle files are encoded using UTF-8. AllegroGraph was not enforcing this in all cases which could mangle Unicode data on import. This has been corrected.

Bug21022 - Group expressions with unbound elements could be dropped

AllegroGraph could lose rows in a aggregation query if the GROUP BY expression resulted in unbound columns. This has been corrected.

Bug21008 - Correct XSD datatype for certain division operations

When performing division of two xsd:integers or two xsd:decimals, AllegroGraph was emitting the result as an xsd:float rather than an xsd:decimal. This has been corrected.

Bug21007 - Error creating a session with non-default graph for inferred triples

The syntax for starting a reasoning session that uses a different graph for its inferred triples is

<store>[rdfs++#<graph>] 

AllegroGraph was failing to parse this specification correctly and signaled an error instead. This has now been corrected.

HTTP Client

Rfe11601 - HTTP and WebView: MongoDB interface config

Added resources to HTTP protocol for MongoDB parameters:

GET /repositories/[name]/mongoParameters  
POST /repositories/[name]/mongoParameters 

See Mongo Interface, The HTTP Protocol, GET mongoParameters, and POST mongoParameters.

SPARQL

Rfe11558 - Add query logging support

To add in query debugging and tuning, SPARQL queries using the 1.1 query engine can now be logged to the AllegroGraph log file. To log a query, use the prefix option notation by pre-pending this prefix to your query:

prefix franzOption_logQuery: <franz:yes> 

The log output will go directly to the AllegroGraph log. In this release, the output is relatively cryptic. Please consult with Franz ([email protected]) to interpret the data.

Rfe11492 - Add support for SPARQL CSV and TSV

Added CSV (comma separated values) and TSV (tab separated values) as native SPARQL output formats. Also corrected problems with using CSV output on result sets with unbound values.

Rfe11486, Rfe11538 - improve SPARQL 1.1 engine geospatial integration

The SPARQL 1.1 query engine did not implement the POINT extension for SPARQL FILTER functions so that queries like:

SELECT * {  
   ...  
} ORDER BY geo:haversine-km(?loc,POINT(105.842328, 21.0184040)) 

would fail. This has been corrected. Additionally, the geospatial query analyzer has been improved so that more computations are carried out at query analysis time rather than at query execution time. This can speed up some geospatial queries tremendously.

Bug21117 - Some SPARQL 1.1 path queries could parse incorrectly

The SPARQL 1.1 parser could combine some combinations of alternation and sequences incorrectly. For example, this query

SELECT ?xxx ?yyy {  
    ?xxx (dc:aaa|dc:bbb/dc:ccc) ?yyy .  
} 

would have been parsed as if it read (dc:aaa|dc:bbb|dc:ccc). This has been corrected.

Bug21093 - SPARQL UPDATEs could fail to save newly created strings

If run with chunkProcessingAllowed, a SPARQL UPDATE query that generated new strings could fail to save some of these new strings in the underlying triple-store. This has been corrected.

Bug21082 - Optional cross-product with no bindings could fail

If the final step of a query was a LEFT-JOIN and that LEFT-JOIN was also a cross-product (i.e., the right hand side shared no variables with the left hand side) and the right hand side produced no bindings of its own (e.g., it was all constant or its variables could be elided because they were not used anywhere else in the query), then it was possible for AllegroGraph to incorrectly remove answers. This has been corrected.

Bug21078, bug21083 - Handle sub-query SELECT expresions and filters correctly

SELECT expressions in sub-queries were not being handled correctly by AllegroGraph. This has been corrected so that queries like

SELECT ?a (?a * 2 as ?b) ?c ?d {  
  BIND(1 as ?a)  
  { SELECT ?c (2* ?c as ?d) {  
    BIND(20 as ?c)  
  } }  
} 

now perform properly.

It was also possible for AllegroGraph to incorrectly optimize FILTER expressions in sub-queries that returned all variables (*). This too has been corrected.

Bug21072 - unused variable check could remove variables in EXISTS filters

The SPARQL 1.1 query planner removes variables that are not used in the query from the set of bindings maintained during query execution. Unfortunately, the check to see whether or not a variable was used could fail to properly descend into EXISTS (and NOT EXISTS) filters. For example, the following query would have failed:

PREFIX :    <http://example/>  
PREFIX  rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  
SELECT DISTINCT (?s1 AS ?subset) (?s2 AS ?superset)  
WHERE  
{  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        ?s1 :member ?x .  
        FILTER ( ?s1 = ?s2 || NOT EXISTS { ?s2 :member ?x . } )  
} 

This has been corrected.

Bug21064 - Unneeded graph variables with DISTINCT query could error

If the variable in a GRAPH clause was not needed by the query (because, e.g., the query asks for DISTINCT results and the graph variable is not used anywhere else in the query), then the query planner will remove the variable from the plan. Unfortunately, the query executor could still try to assign a value to the graph variable which would lead to an error at query execution time. This has been corrected.

Bug21046 - Multiple MINUS clauses could lead to query execution error

A SPARQL query with multiple MINUS clauses could cause the SPARQL 1.1 query engine to incorrectly determine the variables on which to join the clauses. For example, this query would fail:

SELECT (?s1 AS ?subset) (?s2 AS ?superset) {  
    # All pairs of sets  
    ?s2 rdf:type :Set .  
    ?s1 rdf:type :Set .  
 
    MINUS {  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        # Assumes ?s1 has at least one member  
        ?s1 :member ?x .  
        # If we want to exclude A as a subset of A.  
        # This is not perfect as "?s1 = ?s2" is not a  
        # contents based comparison.  
        FILTER ( ?s1 = ?s2 || NOT EXISTS { ?s2 :member ?x . } )  
    }  
    MINUS {  
        # If we don't want the empty set being a subset of itself.  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        # Choose the pair (empty set, empty set)  
        FILTER ( NOT EXISTS { ?s1 :member ?y . } )  
        FILTER ( NOT EXISTS { ?s2 :member ?y . } )  
    }  
} 

This has been corrected.

Bug21037 - Construct queries with Turtle results format can cause string lookup error

If a SPARQL CONSTRUCT query used expressions to create strings that were not already in the underlying triple-store and if the RDF format requested with Turtle, then a UPI not in string table error could be generated. This has been corrected.

Note that the other RDF formats do not exhibit this behavior.

Bug21031 - Remove unstable optimization

The SPARQL 1.1 query engine attempted to reduce the amount of work needed to evaluate certain joins. Unfortunately, it was possible to create queries that exposed a conflict between this optimization and other optimizations which would lead to an error at query evaluation time. This optimization has been temporarily removed. It will be added again once the conflicts it exposed have been resolved.

Bug21023 - Correct datatype of the seconds() function

The SPARQL 1.1 seconds() function should return a value of type xsd:decimal but AllegroGraph was returning xsd:float. This has been corrected.

The date and time functions (like month() and seconds()) were also not behaving correctly on literals of type xsd:time. This has been corrected.

Bug21019 - Filter EXISTS with fixed triple patterns could fail

If a SPARQL query used FILTER (NOT) EXISTS and the EXISTS clause contained fixed triple-patterns, then AllegroGraph could signal an error. For example, this query would fail:

SELECT * {  
  ?a a ex:Fruit .  
  FILTER EXISTS { ex:apple a ex:Plant . }  
} 

This has been corrected.

Bug21016 - UPDATE with blank nodes could lead to error

A SPARQL UPDATE query that included blank nodes in the INSERT template could lead to an error. This has been corrected.

Bug20994 - Correct (in)equality test between plain and typed literals

Some comparisons between plain literals and typed literals could return false rather than signaling a SPARQL type error. This meant that a FILTER that tested for (in)equality could succeed where it should have failed. This has been corrected.

Bug20840 - Improve comparisons between xsd:dates and xsd:dateTimes

The SPARQL operator mapping table is quite strict in the sort of comparisons that are allowed. AllegroGraph was allowing inequality comparisons between xsd:dates and xsd:dateTimes even though these datatypes are not strictly comparable. These sort of comparisons will now signal a type error leading FILTER conditions to fail. I.e., a query like

... FILTER ( ?aDate > ?aDateTime ) ... 

will now fail (correctly) even though it may have resulted in results previously.

Bug20736 - Federated-triple-stores can fail to find equivalent literals

In SPARQL filters, typed-literal equality is based on the value of the literal, not its lexical form. I.e., these literals are all the same:

• '4'^^xsd:byte
• '4'^^xsd:long
• '4'^^xsd:unsignedByte

To reduce the amount of work during query execution, the SPARQL engine checks to see whether any triples exist with literals of these (encoded) types. This check was being handled incorrectly for federated triple-stores which could lead to answers being left out of federated result sets. This has now been corrected.

Bug20770, Bug21020 - improve SPARQL 1.1 aggregation support

SPARQL 1.1. allows assignment in a GROUP BY clause. For example:

... GROUP BY (str(?name) as  ?theName) ... 

AllegroGraph did not allow for this syntax and required the work-around of:

... BIND(str(?name) as ?theName) ... } GROUP BY ?theName ... 

This omission has been filled.

Secondly, AllegroGraph did not allow aggregate expressions in the HAVING clause or SELECT expression involving more than one aggregate. For example, these queries would fail:

SELECT (avg(?value) as ?avg) ((min(?value) + max(?value)) / 2 as ?half {  
  ...  
} 

and

SELECT * {  
  ...  
}  
group by ?value  
having(sum(?value) > 20) 

Both of these omissions have also been corrected.

Bug20165 - Unnecessary variable in a BIND clause could cause error

If the variable in a BIND clause was not needed by the query (because, e.g., the query asks for DISTINCT results and the variable is not used anywhere else), then the query planner will remove the variable from the plan. Unfortunately, the query executor could still try to assign a value to the variable which would lead to an error at query execution time. This has been corrected.

AGWebView

Rfe11601 - MongoDB interface config

Added support in WebView for setting MongoDB parameters, in the Repository Overview page. See Mongo Interface.

Rfe11557 - WebView: serve doc files

AllegroGraph documentation, a collection of HTML files installed with the server, is now served by WebView, and the Documentation menu in WebView links to these local files in addition to those on www.franz.com.

Rfe11474 - Execute query without changing text

In the Query editor, a query may now be executed without editing the text.

Rfe11398 - Update query editor to CodeMirror v2.24

The query editor for SPARQL and Prolog, implemented by CodeMirror, has been upgraded from v2.15 to v2.24, including javascript and css.

Bug21085 - Webview: Recent queries should not add items redundantly

WebView Recent Queries now keeps the most recently used queries. If a previous query (including the same planner and limit) is repeated, that query is moved to the top of Recent Queries, with the number remaining the same. Limit and planner are displayed on the Recent Queries page.

Bug21062 - Recent queries in multiple browser tabs

Using WebView in multiple tabs of a browser caused a javascript error leading to a blank screen below the menu. This has been fixed by consistently using localStorage.

Note for using multiple browser tabs/windows: the last tab to execute query will overwrite localStorage of "recent queries" that will be seen if a new browser tab is opened.

Bug21050 - WebView: CodeMirror Query editor steals arrow keys

When the cursor is in Query Editor, Alt-LeftArrow and Alt-RightArrow will now perform the default browser behavior instead of navigating within the editor page.

Bug21036 - Editor empty after 40 recent queries

WebView holds up to 40 recent queries, but incorrectly restored the text when more than 40 were made. This is fixed.

Note that logout and login will clear the list of recent queries.

Bug20998 - Webview: failures in IE7

Many links to navigate between WebView pages did not work in Internet Explorer 7 (IE7) due to usage of browser history. In the Query editor page, limit and planner options did not take effect in IE7. These problems have been fixed.

Bug20982 - Query limit and planner selections ignored

In the Query editor screen, the limit and planner selections were ignored. Now, they are both used for Execute and Download. Also, they will remain set after Execute, and following a link from the Recent Queries page will remember these settings for that query (but they are not stored for a Saved Query).

Changes to the Lisp API

Bug21042, Bug21043 - standardize remote-triple-store SPARQL cursors

The cursors returned by run-sparql for remote-triple-stores behaved differently from ones returned for other triple-store classes. In particular, SELECT queries could not request a results-format of :cursor and the rows returned by cursors were lists of bindings rather than arrays of bindings.

The cursors for remote-triple-stores are now standardized to behave in the same way as cursors in the rest of AllegroGraph. This item is repeated above under AllegroGraph Server: General Changes, where it is also pointed out this applies to Lisp clients only.

Bug20067 - Make session ping threads more resilient

Remote clients ping their sessions to keep them from timing out on the server. Intermittent network errors could cause the ping thread to fail. This fix makes pinging attempts continue until the session timeout passes without successful pings.

Documentation

Bug21080 - Doc layout bad in IE intranet-mode

AllegroGraph docs now use html5 doctype and style. For Internet Explorer (IE), the highest mode available will be used because IE7 layout is not optimal.

Python Client

Rfe11244: Python Support for client X.509 certificate authentication

To support X.509 certificate authentication with SSL, AllegroGraphServer objects now have cainfo and sslcert keyword arguments for indicating the client security information. verifyhost and verifypeer can also be used.

These arguments correspond to the pycurl.CAINFO, pycurl.SSLCERT, pycurl.SSLVERIFYHOST, and pycurl.SSLVERIFYPEER settings on the Curl object if you want to read about them in the pycurl documentation for more information.

user and password keyword arguments can be omitted when using certificate authentication.

See src2/franz/openrdf/tests/tests.py's test_ssl for example use.

Java Client

Rfe11523 - Upgrade to Sesame 2.6.5 jars

With this change, the Java client has been updated to use the Sesame 2.6.5 jars.

Rfe11358 - Upgrade Clojure to 1.4.0

The Clojure client API now uses Clojure 1.4.0 and Leiningen 1.7.1.

Bug21099: java pool holds sockets open

Problem: the AGConnPool used one MultiThreadedHttpConnectionManager per AGRepositoryConnection, but only closed it with the pool rather than with the repo. This used multiple connections to the main AGraph port and left http socket connections open for them and a session port after the AGRepositoryConnection was closed. Session ports are not recycled by the AGraph server if SessionPorts is not specified.

Fix: use a SimpleHttpConnectionManager for each AGRepositoryConnection and close both at the same time. This minimizes socket connections and avoids the secondary pool within MultiThreadedHttpConnectionManager.

Bug21092 - Clojure: errors in tutorial

Enhanced repo-federation to work without rcon-args as was used in the tutorial. This fixes tutorial example16.

Enhanced value-factory to work with either Repository or RepositoryConnection. This reduces reflection warnings in the tutorial.

AllegroGraph 4.6 user-visible changes

AllegroGraph Server: General Changes

Rfe11449 - Timeout for database instances

Support has been added for making database instances linger for some amount of time after their last close. This allows for reducing the likelihood that a subsequent access to an otherwise idle database will require a long startup time. The time to linger is determined by the global and per catalog InstanceTimeout directive whose default is 0s.

InstanceTimeout is from the instant the last session or backend that has the store open closes it.

New function ensure-not-lingering of the same argument as open-triple-store.

close-triple-store has a new keyword arg :ensure-not-lingering.

Rfe11445 - Queries with large intermediate result sets could generate segfaults

Queries with very large intermediate result sets could generate segmentation faults. This has been corrected.

Rfe11440 - Remove trailing slash from server URL

With this change, AGServer now removes any trailing slash from the specified server URL. Previously, an errant trailing slash on the server URL resulted in an obscure error message about "No suitable response format available" that was difficult to diagnose.

Rfe11418 - improve efficiency of AllegroGraph's turtle parser

Improved the memory usage and speed of AllegroGraph's turtle parser making it more than twice as fast as previous versions.

Rfe11404 - Optimized opening stores

By reducing inter-process communication latency, opening triple stores has been made significantly faster.

Rfe11177 - Agload --relax-syntax flag

agload now uses --relax-syntax flag instead of --relax-for-btc flag. This flag now allows for underscores and dashes to be used in anonymous node strings for N-Triples and N-Quads. agload is documented in Data Loading.

Rfe11126 - SSL client certificate and CRL support

New https-verification keyword argument to open-triple-store and create-triple-store.

Rfe8881 - Optionally suppress duplicates at commit time

AllegroGraph now has a switch duplicate-suppression-strategy, which will prevent new duplicate triples (both duplicates already in the store, and duplicates within the same commit) from being committed to the store.

Bug20997 - Solve latency issue while starting replication.

Fixed a bug where transaction logs (tlogs) were repeatedly (and unnecessarily) examined when performing replication. Now each tlog is examined once. This results in the time required to be O(n) instead of O(n*n) (that is, time is linear with the number of tlogs rather than increasing by the square of the number of tlogs).

Bug20984: Turtle serializer emitted invalid blank node identifiers

The rdf-n3 (turtle) serializer was emitting blank node identifiers without prefixing them with ":_". This has been corrected.

Bug20981 - Exhausted cursor recycling happens too early

Exhausted cursors were being recycled too early which could sometimes result in program crashes. This has been fixed.

Bug20968 - Improve string dictionary cache performance

When heavily used, AllegroGraph's string dictionary cache could become bogged down in bookkeeping resulting in very poor performance. This change prevents this excessive bookkeeping and provides the expected near-constant cache performance.

Bug20943 - Free text index bug under replication

When a new free-text index was created while replication was enabled, it caused an error in the standby process. This is now fixed.

Bug20913: Xsd date and time coercion functions could signal error

If given invalid input, the type coercion functions for dates, times and dateTimes could signal an error other than a SPARQL type error. This would lead to a query error rather than a filter failure. For example, this query:

prefix xsd: <http://www.w3.org/2001/XMLSchema#>  
select ?ds ?d {  
  bind( '2012-12-04 is my favorite day' as ?ds )  
  bind( xsd:date(?ds) as ?d )  
} 

would signal an error rather than returning a single result for which the binding for ?d is empty.

This has been corrected.

Bug20910 - Queries with variable predicates could lose results

If AllegroGraph finds a query pattern with variable predicates and distinct results (and certain other conditions that are data dependent are also true), then it will optimize the pattern by converting it into an iteration over all of the distinct predicates in the store. A typical query of this form is

select distinct ?p {  
  ?s a ex:Animal .  
  ?s ?p ?o  
} 

Because computing all of the distinct predicates is expensive, AllegroGraph caches this information. Previously, AllegroGraph could fail to update the cache correctly when new triples were added. This could lead to queries returning too few results.

This has been corrected.

Bug20896 - Hang when max chunks for an index reached

As part of their operation, the automatic index optimization processes will sometimes merge some number of input chunks into a single output chunk in order to keep the total chunk count below a certain threshold. However, under some circumstances, some indexes would never undergo the optimization process so this chunk reduction would not occur. Eventually the chunk limit would be reached and the database would hang. This problem has been corrected.

Bug20887 - AllegroGraph XML parsers should not follow external references

AllegroGraph's RDF/XML and TriX parsers could try to follow external references in input files and fail. This has been corrected.

Bug20877 - xs:date could fail on some short plain literals

The xs:date constructor function could fail to correctly create a date if the argument passed to it was a small plain literal. E.g., constructs like

BIND (xsd:date( concat(str(?date), '-01-01')) as ?begin_date) 

could pass a string whose length was less than 10 to the xs:date function and this could lead to the failure.

This has been corrected.

Bug20833 - openFederated should handle remote stores

AllegroGraphServer.openFederated now works on remote stores. Previously it always attempted to open supplied arguments as local stores.

The openFederated API still has some limitations. To open a federated session with all federation minilanguage features such as reasoning and graph filtering, pass a properly formatted spec string to AllegroGraphServer.openSession directly.

Bug20822 - Cascading error can occur if database create/open fails

Previously, if there was an error during database open/create, another error could occur during the cleanup process, obscuring the actual problem. This has been fixed.

Bug20819 - problem with some combinations of multiple OPTIONAL clauses

Some query patterns with OPTIONALs whose left-hand side was empty could lead to incorrect results. An example of a query that could have failed was:

select distinct ?value ?type ?name {  
  OPTIONAL {  
    ?value <http://example.com#isa> ?type .  
  }  
  OPTIONAL {  
    ?value <http://example.com#name> ?name .  
  }  
  FILTER( REGEX(STR(?name),'Sa') && BOUND(?type) )  
} 

This has been corrected.

Bug20814 - Make session urls unique

In order to avoid accidental sharing of sessions, include a unique session id in the session-url. Behind the scenes, what previously looked like http://localhost:55555 is now something like: http://localhost:55555/sessions/7e8df8cd-26b8-26e4-4e83-0015588336ea.

Bug20806 - fix loading of initfile and sitescripts

Initfiles and sitescripts longer than 4096 characters were presented with 4096 character long blocks in reversed order. This bug is now fixed.

Bug20799 - turtle-parser was case sensitive when parsing numbers

AllegroGraph's turtle parser was failing to parse floating point numbers that used an upper-case E before the exponent. This is now corrected.

Bug20793 - RDFS++ queries with free predicate could infer invalid triples

If a query involving RDFS++ reasoning used an unbound predicate, the reasoner could use inverse property reasoning to infer invalid triples. This has been corrected.

Bug20757 - Problems when starting agraph while another agraph is running

Previously if you tried to start AllegroGraph when it was already running, the second start would fail but it would cause the already-running AllegroGraph to stop operating correctly. This has been fixed.

It has always been acceptable to run two or more AllegroGraph servers simultaneously as long as they, at the very least, used different settings for the following parameters in their configuration files:

• Port

• SettingsDirectory

• GlobalShmServerDirectory (defaults to the value of SettingsDirectory)

The operating system already enforced a unique Port setting. This change adds enforcement of unique SettingsDirectory and GlobalShmServerDirectory settings. Therefore some (dangerous) side-by-side AllegroGraph configurations that used to work may no longer start up. An instructive error message will be displayed in this case.

Bug20628 - Changes to warmstandby API

PUT /repositories/[name]/warmstandby

If one attempts to start a replication task that already exists, an http 409 Conflict response will be returned.

Bug20533 - Agload --rapper flag documentation

The --rapper flag is now specified in the group "Other options". Previously it was not given a group in the command line help. agload is documented in Data Loading.

Bug20508 - Remote triple store can't use the RDFS++ reasoner

Added support for apply-rdfs++-reasoner on a remote client triple-store.

HTTP Client

Rfe10812 - Improve http tracing

HttpTrace is now a global configuration directive of the same meaning as the --http-trace command line option. The global directive HttpTraceOptions and the command line option --http-trace-options were added, the value of which is a comma separated list of options. Options starting with the character + turn on the corresponding log category. Those starting with - turn them off. Use the max-message-size= option to truncate overly long messages. The default is +log,max-message-size=1000.

Bug20798 - Improved http error handling

Previously, if the client that was performing an http request died and the backend couldn't transmit the response, then the backend was killed and the request was retried on another backend that also failed because the client socket was closed which wasted resources. The new behavior is to let the backend live on (without retrying), if the error is really from the client side.

SPARQL

Rfe11469: add line number to SPARQL parser error messages

The SPARQL parser will now indicate the line at which an error is encountered. The error message has also been reformatted to make it easier to understand the parsing problem.

Rfe11357 - enhance SPARQL 1.1 engine for changes to standard

The SPARQL 1.1 standards committee recently made several small changes to the standard. To accommodate these, AllegroGraph's SPARQL engine has:

• added support for the functions STRBEFORE, STRAFTER and REPLACE
• removed support for the function SHA224
• relaxed the type validation for REGEX so that language tagged literals and typed literals of type xsd:string can be searched without resorting to the STR function.
• returns rdf:langString as the datatype of a language tagged literal
• accepts solution modifiers for ASK queries

There are several other changes to the standard which AllegroGraph will support in a future release. These include:

• Grammar: Backslash character escapes in prefixed names
• Grammar: Fix: Allow ASK to take solution modifiers
• Grammar: Fix: AdditiveExpression: '?' => '*'
• Allow aggregates in ORDER BY clause

For additional information, please see the SPARQL release notes.

Bug20951 - SPARQL could fail to match plain and typed strings

SPARQL specifies that a plain literal (like "hello") is equivalent to a literal with type xsd:string (like "hello"^^xsd:string). The SPARQL 1.1 query engine was failing to provide this entailment in some cases. This has been corrected.

Bug20939 - CLEAR only recognized if it's the first verb

The SPARQL 1.1 UPDATE verb CLEAR was only recognized correctly if it was the first verb in a SPARQL statement. This has been corrected.

Bug20936 - OFFSET 0 and LIMIT did not mix

If a SPARQL query used an explicit OFFSET of zero, the LIMIT would be ignored. I.e., a query like:

SELECT * { ?a a ex:Animal . }  
LIMIT 10 OFFSET 0 

would return all ex:Animals rather than just 10. Leaving the OFFSET off entirely would produce the correct behavior. This has been corrected so that OFFSET 0 is treated as expected.

Bug20912 - Inequality filters could signal an error

Inequality filters in SPARQL queries can be transformed into fast range queries. If a query used a filter like

FILTER( ?a < 0.0 ) 

then the SPARQL 1.1 query planner could attempt to decrement an unsigned number below zero which would lead to a segmentation fault.

This has been corrected.

Bug20911 - SPARQL could lose precision on very small numbers

SPARQL numeric conversion routines could treat double-floats and decimals as single-floats which would lose precision. For example, a query with a filter like

FILTER( ?x > 4.5e-45 ) 

would have been interpreted as

FILTER( ?x > 0 ) 

This has been corrected.

Bug20898 - Complex queries could fail on federated triple-stores

Because of the way statistics were calculated on federated triple-stores, SPARQL queries with many clauses in a basic graph pattern (BGP) could fail when the query engine attempted to re-order them. This has been corrected.

Bug20891 - Improve SPARQL comparator selection

SPARQL specifies difference comparison operations depending on the type of the things being compared. The SPARQL 1.1 query engine could select the wrong operation if both the left and right hand sides were variables. This has been corrected.

Bug20884: improve Lisp client remote-triple-store SPARQL interface

The remote-triple-store implementation of the Lisp client was unable to execute SPARQL queries with non-trivial filter clauses. This has been fixed. (Repeated in the Lisp API section below.)

Bug20867: REPLACE function in SPARQL 1.1 used incorrect syntax for matches

The SPARQL 1.1 REPLACE function was implemented using a different syntax for pattern replacement than that specified by the standard. Replacements are now correctly specified using the $<number> syntax.

Bug20863 - Some SPARQL XSD numeric computations could fail

Depending on the datatypes of the numbers involved, it was possible for a SPARQL numeric computation to signal a type error rather than correctly returning a value. This has been corrected.

Bug20859 - Some filter combinations could cancel each other out

If a SPARQL query used IN (or several FILTERS like ?uri = constant) and if there were other filters on variables that overlapped with the variables in the IN construct, then it was possible for AllegroGraph to lose track of the additional filters in some circumstances. This happened due to two separate query optimizations handling the filters inconsistently. This problem has been corrected.

Bug20852 - remote-triple-store and SPARQL UPDATE don't mix

The Lisp remote-triple-store client could signal an error after evaluating a SPARQL UPDATE statement. The UPDATE would occur but the result would be mis-interpreted and signal an error. This has been corrected.

Bug20851 - SPARQL FILTERs could be too strict

Certain combinations of triple patterns in a BGP and FILTERs could cause AllegroGraph to misapply the FILTERs. For example, depending on the underlying triple-store, this query

SELECT ?first_film ?actor ?second_film {  
  ?first_film dbpp:starring dbpr:Jack_Black , ?actor .  
  ?second_film dbpp:starring dbpr:Joan_Cusack , ?actor .  
  FILTER ( ?actor != dbpr:Joan_Cusack )  
  FILTER ( ?second_film != ?first_film )  
  FILTER ( ?actor != dbpr:Jack_Black )  
} 

could have misapplied the ?actor filter and produced fewer results than expected.

This has been corrected.

Bug20838 - SPARQL CONSTRUCT queries could fail for large result sets

Due to limitations in the internal mechanisms used to keep track of intermediate results, SPARQL CONSTRUCT queries that returned more than 100,000 rows would cause a segmentation fault. This has been corrected.

Bug20830 - Some inequalities could use the wrong comparisons

Some SPARQL 1.1 queries could select the wrong comparison code when comparing two variable clauses with either >= or <=. This could lead to incorrect results. This has been fixed.

Bug20821 - EXISTS or NOT EXISTS and additional filters did not mix

The new SPARQL 1.1 EXISTS and NOT EXISTS filters failed to parse correctly when combined with other FILTER clauses. This has been fixed.

Bug20805 - encodeforuri could produce incorrect results

The SPARQL 1.1 encode_for_uri could produce incorrect results for some UNICODE characters. This has been corrected.

AGWebView

Rfe10885 - Setting duplicate suppression from AGWebView

Added "Suppress duplicate statements" feature to the repository overview page. This sets the duplicate-suppression-strategy switch, which can be set to prevent new duplicate triples (both duplicates already in the store, and duplicates within the same commit) from being committed to the store.

Bug20996 - Webview: js error creating a new repo

This fixes a javascript error when creating a repository from the Catalog page.

Bug20993 - Webview: Internet Explorer (IE) compatibility mode

AGWebView now includes an html meta header for IE9 so that IE9 will use its documentMode instead of IE7 compatibility. Also tested with IE8.

Also using html5 doctype to get standards mode in all browsers.

Bug20963 - Fix session url in AGWebView

Urls for sessions are changed in Bug20814 (described under AllegroGraph Server: General Changes above). This fixes AGWebView and changes the url for its use of sessions to include the uuid.

Bug20894 - AGWebView not working with ie9 on intranet

When connecting to AGWebView on an intranet site IE may automatically use Compatibility Mode. A syntax error causes it to only display the "Loading" message and not continue. Work-around is to use a full url, or otherwise not use compatibility mode. The syntax error is now fixed so an intranet does not cause the error.

Bug20843 - AGWebView sometimes clears the editor

The execute button would clear the query editor (and the results area). This happened when the same query had been executed sometime before in the same browser followed by logout/login. The browser history mechanism had not been fully cleared. Recent queries are cleared after logout. This is now fixed.

Changes to the Lisp API

Bug20884: improve Lisp client remote-triple-store SPARQL interface

The remote-triple-store implementation of the Lisp client was unable to execute SPARQL queries with non-trivial filter clauses. This has been fixed. (Repeated in the SPARQL section above.)

Bug20883: improve the Lisp client's remote-triple-store caching interface

The Lisp client's remote-triple-store implementation did not support the spogi-cache-enabled-p or spogi-cache-size methods. This has been corrected.

Bug20185 - Remote lisp client used wrong value for restriction reasoning

For a remote reasoning store, this method used the wrong string internally, leading to an error when a query was made:

(setf (restriction-reasoning-enabled-p *db*) t) 

For a remote reasoning store, this method enabled restriction reasoning if the key :restriction-reasoning-enabled was present regardless of the key's value. Now respects t or nil.

(apply-rdfs++-reasoner :restriction-reasoning-enabled nil)  

Documentation

No significant changes.

Python Client

Rfe11313 - Remove export*, use response capture

RepositoryConnection.export and exportStatements have been deprecated. They used client side serialization of statements that was buggy and not up to spec. The functionality has been replaced with a way to capture server responses of any supported mime type.

A new context manager function on RepositoryConnection is called saveResponse, and can be used as follows:

with open('out', 'w') as response:  
    with conn.saveResponse(response, 'application/rdf+xml'):  
        # The server response is written to response  
        conn.getStatements(None, None, None) 

You can reference the function testsaveresponse in src2/franz/openrdf/tests/tests.py for examples.

Java Client

Rfe11493 - Support writing N-Quads from an AGModel

With this change, the Jena adapter now supports writing to N-Quads via AGModel.write(out, "N-QUADS").

Rfe11438 - Provide N-Quads parser/writer META-INF/services in agraph.jar

With this change, the agraph.jar now provides N-Quads implementations for org.openrdf.rio.RDFParserFactory and org.openrdf.rio.RDFWriterFactory in META-INF/services (Sesame doesn't yet provide N-Quads implementations).

Rfe11436 - Add Sesame 2.6.2 src jar

With this change, the openrdf-sesame-2.6.2-onejar-src.jar has been added to eclipse project classpaths (repo and distribution) for convenience.

Rfe11397 - Support RepositoryFactory and Repository configuration

With this change, the Java client now supports Sesame's RepositoryFactory api and Repository configuration from RDF data. See javadoc for the new com.franz.agraph.repository.config package for details.

Rfe11240 - Support user/role management

With this change, AllegroGraph superusers can now dynamically manage users and roles. See the AGServer javadoc for details.

Rfe11230 - Support masquerading as another user

With this change, an AllegroGraph superuser can now masquerade as another user when issuing requests. See the javadoc for AGRepositoryConnection#setMasqueradeAsUser for details.

Bug20946 - IE javascript error in Mold: innerHTML is null

Work-around: the error dialog can be dismissed and the search results are still displayed. Fixed.

Bug20811 - Allow prolog queries from javascript

To run javascript the user must have the "execute arbitrary code" permission which should be enough to run prolog queries too. The bug that was fixed prevented the user from running prolog queries from javascript.

AllegroGraph 4.5 user-visible changes

AllegroGraph Server: General Changes

Rfe11338 - CJK bigram tokenizer

A new tokenizer :simple-cjk for Chinese/Japanese/Korean (CJK) text has been added. It uses bigrams for consecutive CJK characters as words. To use this tokenizer, specify :simple-cjk as the tokenizer option when creating or modifying a freetext index.

The bigram tokenizer can be used for CJK mixed text, but its simplicity may result in false positives. It also tends to index a much larger number of words compared to other tokenizers. See Free-text Indices and create-freetext-index.

Rfe11298 - Spread access to the same triple store across shared backends

Previously, when using shared backends, operations for a particular triple store tended to be routed to a single shared backend, even if it was busy. Now concurrent operations for a particular triple store are routed to a shared backend that is not already processing a request for that same store. Additional backends (up to the value of the "Backends" configuration parameter) will be started if necessary to achieve this goal. This is expected to increase concurrency in such situations.

Dedicated sessions are unaffected by this change.

Since this change is most beneficial when there are many shared backends available, the default setting of the "Backends" configuration parameter has been changed from 3 to 10.

Rfe11257 - Backup/restore user namespace settings

agraph-backup is enhanced in two ways:

The user namespace settings are now handled by the backup-all/backup-settings/restore-all subcommands.

The backup-settings subcommand can take a path to the executable of an older AllegroGraph's version of agraph-backup, and call it to archive the store data. With this feature, archiving an entire repository from an older AllegroGraph repository becomes a single command.

See Backup and Restore.

Rfe11099 - Add Apache Solr support to AllegroGraph

Now you can connect an external Solr text search engine from AllegroGraph server. With new prolog functions you can issue a query to the Solr engine to retrieve associated document IDs. See Solr Text Indices.

Bug20750 - Unused store closer holds lock too long

The thread which closes idle triple stores in backends held an important lock while trying to acquire locks on individual stores. If a lock on a store was being held by another thread (e.g., one running a long query), this could cause the whole backend to become unresponsive to other unrelated requests. This has been corrected.

Bug20736 - Federated-triple-store could mistreat xsd:strings in queries

In some cases a federated-triple-store could fail to account for the equality of plain literals and literals with a datatype of xsd:string. This has been corrected.

Bug20717 - Fix the UPI not in string table query issue

Depending on the result format and the client, queries that generated new strings (e.g., using SPARQL 1.1's BIND clause) could lead to a UPI not in string table error. This has been corrected.

Bug20712 - Implement xs:date and xs:time XPath constructor functions

The constructor functions for xs:date and xs:time are now implemented.

Bug20697 - Zero length property path with future parts

Zero length property path patterns now correctly match future parts that are not yet in the db:

?x :p{0} :a . 

Note that the same pattern with a literal worked even without this fix:

?x :p{0} "o" . 

Bug20690 - Property path results cardinality

In the presence of cycles, property path expressions did not always return results with the right cardinality. This has been fixed.

Bug20616 - Fix race condition in triple store termination

AllegroGraph now waits for a triple-store's subprocesses to exit when the last connection is closed instead of killing them while they could potentially be executing a clean shutdown.

Avoid Logging a backtrace during unclean shutdown

Under certain circumstances (such as when 'killall agraph' was used), there was a race condition during process termination that could result in an error and backtrace being logged to agraph.log. This has been corrected.

HTTP Client

Rfe11308 - Support UPDATE via POST /repository/ID/statements

The Sesame 2 protocol allows statements to be added to a repository by sending a POST request to /repository/ID/statements. AllegroGraph previously only supported update via POST requests to /repository/ID.

Now SPARQL update statements may be directed to either location. You can also POST an RDF document or data in the form of an application/x-rdftransaction to /repository/ID/statements.

Rfe11213 - Allow superusers to run queries with another user's security filters

AllegroGraph sessions belonging to superusers now allow their owners to use the X-Masquerade-As-User header in HTTP requests to activate another user's security filters.

Bug20671 - List the Hub process in the HTTP API

AllegroGraph now lists the Hub process in the HTTP API's process list and in WebView. The Hub process can not be killed or asked to perform a backtrace.

SPARQL

Rfe11320 - Trying to use SERVICE should signal an error

AllegroGraph's SPARQL engine does not support federated query (http://www.w3.org/TR/sparql11-federated-query/). Previously queries using the SERVICE keyword would fail silently. Now these queries will signal an error.

Rfe11302 - Support clauses with no variables

The SPARQL 1.1 engine now supports basic graph patterns where one or more of the clauses have no free variables. Previously, a query like

select * {  
   :a :p1 :c .  
      :p2 ?value .  
} 

could produce incorrect results. This has been corrected.

Rfe11262 - Improve SPARQL 1.1 filter transformation

AllegroGraph's SPARQL 1.1 engine is able to move some filters out of the query processor and into the storage layer. For example, a query like

select * {  
  ?a rdf:type ?class .  
  filter( str(?class) > 'z' )  
} 

will move the inequality into a filter involving only the first clause. In some cases, however, the query processor could leave behind a stub filter which would unnecessarily add to total query time (without altering the results). This is now corrected.

Several other minor efficiency improvements have also been added to the query engine.

Rfe11254 - Improve SPARQL 1.1 sub-query implemenation

A problem in the sub-query implementation led to two failures in the DAWG test-suite. These have been corrected and additional tests have been added to the regression tests

Rfe11245 - Optimize Queries where the right-hand side of a join is a BGP

The SPARQL 1.1 engine now handles queries where the right-hand side of the join is a BGP more efficiently. Previously, it would compute the RHS separately and then join the two sets. Now, it merges the RHS BGP into the existing LHS set directly. This can provide some speed and memory improvements for some queries.

Rfe11135 - Implement SPARQL 1.1 DISTINCT aggregation

AllegroGraph's SPARQL 1.1 engine now implements DISTINCT in its aggregates. This means that a query like

select ?gender (count(distinct ?profession) as ?pCount) {  
  ?person k:profession ?profession .  
  ?person k:sex ?gender .  
} group by ?gender order by ?pCount 

will now correctly return a count of the distinct professions for each gender rather than a count of all professions.

Rfe11098 - Integrate Apache Solr with SPARQL 1.1 query engine

Assuming that you have setup Apache Solr with an AllegroGraph triple-store, you can now query it using the SPARQL 1.1 query engine. See the description of using Solr in Solr Text Index. When used with SPARQL 1.1, there are two storage strategies: addinng a triple that associates the text with a Solr document ID and telling Solr to associate the text with the triple-id directly. Both are described in the referenced document.

Rfe11070 - Improve performance of variable predicate queries

AllegroGraph's SPARQL engine now recognizes queries that have variables in the predicate position and provides custom methods to optimize them. An example of such a query is:

SELECT ... {  
 ... clauses ...  
 ?s ?p ?o . 

The optimizations are only possible when AllegroGraph has already computed the complete list of predicates in the store. This can be accomplished using the ComputeStatistics query option. For example

PREFIX franzOption_ComputeStatistics: <franz:yes>  
SELECT ... 

Bug20778 - Several fixes for various SPARQL 1.1 issues

There were numerous issues in the initial release of the SPARQL 1.1 query engine. Some of the problems were caught by customers and others by internal review. This release of AllegroGraph fixes:

• issues with several functions including: encodeforuri, some new mathematical functions, isLiteral, etc.

• fixes several code paths that could lead to the UPI is not in string dictionary error,

• fixed some edge cases in the SPARQL parser,

• made some conversion functions more stringent so that, for examples, 10 dollars would raise a SPARQL type error rather than being treated as the number 10.

There remain some known issues with SPARQL; please see the SPARQL release notes for details.

Bug20760 - Several improvements in SPARQL aggregation.

• Do not include unbound variables in COUNT aggregation. Previously, AllegroGraph would include these in the count.

• Remove bindings for SUM, AVG, etc. if any of the bindings in the group are unbound. Previously, AllegroGraph would simply ignore the unbound values are return the SUM of the remaining elements.

• Allow non-numeric bindings for MIN and MAX. Previously, AllegroGraph assumed that MIN and MAX only applied to numeric bindings.

Bug20756 - Multiple OPTIONAL clauses for the same variable could fail

The SPARQL 1.1 query engine could fail to find some bindings when multiple OPTIONAL clauses referred to the same variable. For example, a query like

SELECT ?first ?last {  
  ?s ab:lastName ?last .  
  OPTIONAL { ?s ab:nick ?first . }  
  OPTIONAL { ?s ab:firstName ?first . }  
} 

could fail to find bindings for ?first that come from the second OPTIONAL. This is now corrected.

Bug20752 - Queries with both filters and property paths could fail

If a SPARQL 1.1 property path query used FILTERs and these FILTERs were moved into the triple patterns, then it was possible for the filters to be ignored. This has been corrected.

Bug20732 - SPARQL 1.1 property path improvements

In some cases, SPARQL 1.1 property path queries could do too much work and return too many results. This has been corrected.

Bug20723 - Improve handling of SPARQL filter predicates

Some of the SPARQL 1.1 filter predicates were failing to function properly when used in some contexts because their return value was not being correctly transformed from an internal representation to an external one. This has been corrected.

Bug20722 - SPARQL 1.1 parser treated whitespace differently

The SPARQL 1.1 parser required whitespace after a period and before an additional SPARQL keyword. This meant that a query like

select * { ?a ?b ?c.filter( ?c > 3) } 

would signal a parse error. This has been corrected.

Bug20721 - Improve handling of certain OPTIONAL edge cases

Depending on the data in the store, the SPARQL 1.1 engine could sometimes fail to process queries like:

select ?a ?b ?c ?d {  
  optional { ?a a ?b . }  
  { ?c <urn:a:b> ?d . }  
} 

where the OPTIONAL could lead to an empty group of bindings on the left-hand-side during query processing. This has been corrected.

Bug20711 - Improve some date and dateTime comparison issues

The SPARQL 1.0 and 1.1 engines behaved inconsistently with respect to one another for certain comparisons between xsd:dates and xsd:dateTimes. This could lead to differences in query behavior. These issues have been reconciled.

Bug20695 - Problem when computing range-queries on dateTimes

Both the SPARQL 1.0 and the SPARQL 1.1 engine could fail to correctly compare dateTime datatypes with fractional seconds. This has been corrected.

Bug20689 - Parsing of SPARQL property path expressions where a group is followed by a + as in

?x (ex:p)+ ?y 

no longer results in a error.

AGWebView

Bug20751 - Agwebview could submit queries twice in some browsers

On some browsers AGWebView could submit queries twice. This led to unnecessary work for SPARQL or Prolog SELECT queries and to duplicate data for SPARQL UPDATE queries. This has been corrected.

Bug20738 - AGWebView could not start remote sessions

A recent change in AllegroGraph broke AGWebView's ability to start sessions on remote stores. This is now corrected.

Changes to the Lisp API

No significant changes.

Documentation

Rfe11236 - Fix command line documentation of the agdebug tool

The agdebug tool now correctly indicates a non-optional argument.

Python Client

Rfe11229 - Add support for user masquerading to python client

Added support for user masquerading to Python client by adding RepositoryConnection.runAsUser(username).

Java Client

Rfe11329: AGConnPool, remove unnecessary dependency on Jena

If using AGConnPool with Sesame API, but not Jena, the Jena libraries are no longer required.

Rfe11310: AGConnPool should rollback when borrowing

Pool will now always rollback when borrowing a connection. This makes the transaction catch up to the most recent.

The 'testOnBorrow' property for the pool is now redundant, so generally not recommended.

Rfe11242: Upgrade to Sesame 2.6.2 jars

With this change, the Java client has been updated to use the Sesame 2.6.2 jars, including support for AGUpdate and AGRepositoryConnection#prepareUpdate methods. Java Sesame Tutorial example13 has also been updated to reflect use of prepareUpdate.

Bug20795: AGConnPool does not close connections

Closing the AGConnPool now closes sessions properly.

AllegroGraph 4.4.0.1 user-visible changes

AllegroGraph Server: General Changes

Bug20667 - XSD float exponent fix

Due to rounding issues when encoding, exponents of floats in the :sparql-xml result format were - very seldom - off by one. This has been fixed.

Bug20615 - TrxLogRetain replication now works on secondaries

The catalog directive: TransactionLogRetain replication now archives properly for replication secondaries.

HTTP Client

No significant changes.

SPARQL

Rfe11219 - Add query option support for all SPARQL constructs

AllegroGraph's SPARQL parser supports specifying query options as part of the query string. These take the form

prefix franzOption_OPTIONNAME: <franz:OPTIONVALUE> 

In particular, you can tell AllegroGraph which query engine to use with the queryEngine parameter. For example, to use the SPARQL 1.0 engine in a query, you would add the prefix

prefix franzOption_queryEngine: <franz:sparql-1.0> 

This extension is now supported for all SPARQL constructs: ASK, CONSTRUCT, DESCRIBE, SELECT and UPDATE.

Rfe11218 - Improve maximum solution size handling

Because the SPARQL 1.1 engine operates on the entire query solution simultaneously it can be more memory intensive than the SPARQL 1.0 engine for some queries. There was an internal check to warn if query solution sets were becoming too large. This check was causing some queries to fail rather than simply signaling a warning. This has been changed so that these queries will write a warning to the log rather than failing.

Rfe11202: SPARQL Update example with batch update

With this change, Java tutorial example 13 has been augmented to show how to evaluate SPARQL Update queries, including one that batches updates in a single request. [This item is repeated in the Java Client section.]

Bug20768 - SPARQL 1.1 property paths could return incorrect results

AllegroGraph was failing to filter non-matching rows for property path queries like

select ?a {  
  ?a owl:sameAs{1} ?a .  
} 

meaning that it would emit results when it should not. This has been fixed.

Bug20692 - Sparql 1.1 CONTAINS function can fail

Corrected a problem with some instances of the SPARQL 1.1 CONTAINS function.

Bug20676 - Geospatial datatype mappings could cause query errors

Changes to some query optimization code in the AllegroGraph SPARQL engines (both sparql-1.0 and sparql-1.1) were incompatible with geospatial datatype mappings (geospatial predicate mappings were not effected). This could lead to parse errors during query processing. This has been fixed.

Bug20665 - SPARQL-1.1 UNION doesn't see with-variable bindings on the RHS

The SPARQL 1.1 engine was losing track of the variable bindings on the right-hand side of a UNION. This could cause queries like

select ?s1 ?s2 {  
  { ?s1 a ?p . }  
  union  
  { ?s2 a ?p . }  
} 

(with a variable binding for ?p) to produce incorrect results. This has been corrected.

Bug20664 - SPARQL-1.1 CONCAT function signaling errors with typed literals

The SPARQL-1.1 CONCAT function could signal an error instead of generating the correct result when one or both of the arguments with a typed literal. This has been corrected.

AGWebView

Rfe11198 - Improve WebView's bulk loading UI

WebView no longer conflates bulk load mode with agload usage. It's now possible to use agload's multi-core optimized loading mechanism and to optionally turn bulk load mode on. Some aspects of the file loading UI in WebView have also been optimized: The format selection element is now a combo box, and is located closer to the file input field.

Bug20662 - Reload triple count in WebView if load fails

WebView will now display the correct triple count in a store after an import got stopped by an error (e.g. the triple license limit).

Bug20641 - Improve restoring backups from WebView

When restoring from a backup in the WebView catalog overview, AllegroGraph will now automatically upgrade the newly-created store to the newest version. Additionally, there is now an option to set the store up for use as a replication secondary.

Bug20637 - Fix problem where SPARQL Update did not work with AGWebView

Changes to SPARQL UPDATE caused some UPDATE queries to fail in AGWebView. This has been corrected.

Changes to the Lisp API

Bug20640 - Fix printing of encoded-ids in remote lisp client

AllegroGraph's remote lisp client now correctly translates encoded IDs' UPIs back into strings. It can also correctly translate variable-width encoded IDs into UPIs now.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe11202: SPARQL Update example with batch update

With this change, Java tutorial example 13 has been augmented to show how to evaluate SPARQL Update queries, including one that batches updates in a single request. [This item is repeated in the SPARQL section.]

AllegroGraph 4.4 user-visible changes

AllegroGraph Server: General Changes

Rfe11082: Transactional duplicate deletion

INCOMPATIBLE CHANGE: AllegroGraph no longer supports duplicate deletion at index optimization time. The database creation parameters indicating duplicate deletion strategy are no longer accepted.

Instead, AllegroGraph now allows deletion of duplicates using the delete-duplicate-triples API function, which is also accessible via the HTTP API call DELETE /statements/duplicates.

Rfe11039 - Include AllegroGraph version in ps output

AllegroGraph subprocesses modify their command line so that they can supply identifying information to programs like 'ps' and 'top'. Now, in addition to the name of the subprogram, the version of AllegroGraph is included. Also, for the AllegroGraph Service Daemon process, the port number that it is listening on is included.

Rfe11017 - Add an agdebug command line utility

AllegroGraph now ships with a command line utility called agdebug that will output a summary the contents of the transaction log files in a store. This is primarily useful for diagnosing problems with transactions and the database's storage system.

Rfe10984 - Fix serialization issues with encoded IDs

Both the turtle and the RDF/XML serializers failed to correctly handle encoded IDs. This has been corrected.

Rfe10981 - Improve upgrading/migration process

The agraph-backup tool has two new commands, backup-all and restore-all, to help migrating existing databases to the new version of AllegroGraph or copying all catalogs and repositories to a new location. See Backup and Restore and Database Upgrade.

Rfe10948 - Log secondary index creation

Adds log entries to the AllegroGraph log when a secondary index (also called a UPI map) is created.

Rfe10853 - Support fti charset in Python client

Keyword arguments innerChars, borderChars, and tokenizer have been added to createFreeTextIndex and modifyFreeTextIndex.

getFreeTextIndexConfiguration also returns these new configuration properties and values. See put-freetext-index.

Rfe10783 - Add per triple-store control of secondary indices

Modified the Lisp and HTTP APIs so that each triple-store can specify whether or not it should use existing secondary indices and/or create new secondardy indices during query. Since these indices must be maintained and updated when modifying the triple-store, automatic creation should be turned off until the store is quiescent. The default is to use secondary indices when they exist but not to create new ones. Use [create-secondary-indices-p[]/(setf create-secondary-indices-p), and use-secondary-indices-p/(setf use-secondary-indicies-p)* to control the behavior.

Rfe10568, Rfe11172 - Set the default query engine via agraph.cfg

AllegroGraph supports two SPARQL query engines. The algebra engine has been available for several years and is well tested, stable and robust. The newer, set-based, engine, named sparql-1.1 can be significantly faster than the algebra engine but may contain bugs that the algebra engine does not.

Out of the box, AllegroGraph uses the sparql-1.1 engine. This can be changed by using the QueryEngine parameter in the configuration file. If you include

QueryEngine algebra 

Then AllegroGraph will use the algebra engine. If you leave the setting off or use

QueryEngine sparql-1.1 

Then AllegroGraph will continue to use the set based query engine.

The value of this configuration is read each time a triple-store is created or opened.

Rfe10179, Rfe10540 - SNA optimizations

Improved the efficiency of SNA operations in several ways. All SNA generator functions are now slightly faster. SNA generators that use select now save the select plan and re-use it. This can make these generators hundreds of times faster than before. The SNA system will also use secondary indices (UPI-maps) when available.

Rfe10398 - Improved speed of low-level AllegroGraph operations

Optimized the code used to copy UPIs and parts of triples. This produces a query speedup in SPARQL queries and most Prolog select queries. Some count-only Prolog select queries can run slightly slower.

Rfe10090, Rfe10541 - optimizations to AllegroGraph secondary indices

Improves both the creation of secondary indices (UPI-maps) and their efficiency in use. Queries that make heavy use of secondary indices will operate more quickly.

Rfe9786 - Make shm server global

A new configuration directive GlobalShmServerDirectory is added to specify a directory where the global shared-memory server creates its socket. When omitted, the value of SettingsDirectory is used.

Bug20619 Optionally not inherit trx log arch params during restore

The configuration file catalog directive TransactionLogArchive is no longer inherited from a backup database during a restore. In addition, when the --replica flag is set during agraph-backup restore, TransactionLogRetain and TransactionLogReplicationJobname are no longer inherited from the backup and must be set manually in the config file catalog section prior to restore. This is so that a replication primary and replication secondary can have independent transaction log archiving behavior. The flag --replica also ensures --nocommit is set and --newuuid is not set, and should be used as a replacement for --nocommit when restoring backups for replication client purposes.

Bug20460 - Encoded IDs and reasoning triple-stores mix not

If reasoning was used during a query, then encoded IDs were not correctly recognized which led to incorrect results. This has been corrected.

Bug20443 - Encoded IDs did not work with the TriX serializer

The TriX serializer was incorrectly emitting encoded IDs as if they were plain literals instead of resources. This has been corrected.

Bug20355 - Durably save transactions on warm-standby client

The warm-standby secondary machine now saves transaction log entries in a way that ensures they've been flushed to disk durably as soon as the entire commit has been read.

Bug20332 - Trix parser can get confused and try to encode plain literals

Under some circumstances, the TriX parser could attempt to encode plain literals as if they were typed literals which could cause a data import to fail. This has been corrected.

Bug20230 - Fix deletion slowdown under replication

The replication server used a connection to the replicated triple store that ensured it would never expunge deleted triples, slowing down operations if many deletes had occurred. Now, the replication server correctly expunges these triples and does not slow down.

Bug19957 - Turtle parser should support type-mapping

The turtle parser was not respecting a triple-store's type-mapping settings. This meant that triples added via turtle would not have their objects converted into encoded UPIs. This has been corrected.

Spr38044 - Restrict possible values for TransactionLogRetain

AllegroGraph will now signal an error when invalid values for TransactionLogRetain are encountered in the config file.

HTTP Client

No significant changes.

SPARQL

Rfe10985 - Spin functions and magic-properties

SPIN functions and magic-properties have been implemented. The SPIN implementation is described in Spin document.

Rfe10737 - SPARQL 1.1 BIND support

Add support for SPARQL 1.1's BIND language construct. In particular, queries like

select ?title ?costPlusTax {  
?book <http://purl.org/dc/elements/1.1/title> ?title .  
?book <http://example.org/ns#price> ?cost .  
?book <http://example.org/ns#tax> ?tax .  
bind( ?cost * (1 + ?tax) as ?costPlusTax )  
} 

are now supported.

Rfe10736, Rfe10743 - SPARQL 1.1. exists, not exists, and minus

AllegroGraph's SPARQL engine now handles the MINUS operator of SPARQL 1.1. It also handles the new EXISTS and NOT EXISTS filter operators.

Rfe10700 - SPARQL 1.1 assignment

SPARQL 1.1 assignment is now implemented so that queries like

select ?title ?cost ?discount (?cost * ?discount as ?totalCost)  
where {  
?book :title ?title .  
?book :price ?cost .  
?book :discount ?discount .  
} 

are now supported.

Rfe10699 - SPARQL 1.1 UPDATE support

The SPARQL parser and implementation now support the UPDATE specification at http://www.w3.org/TR/2011/WD-sparql11-update-20110512/.

Note: This change is incompatible with some existing uses of SPARQL UPDATE.

AGWebView

Rfe11131: Allow triggering deletion in WebView

It is now possible to invoke duplicate deletion from WebView.

Rfe10631 - Allow loading files via agload in WebView

WebView now has a "Bulk Load" option to speed up loading large files. Since this option uses the agload utility internally, it also allows specifying an error handling strategy other than "Cancel".

Bug20492 - Improve code editor in WebView

The text editor in WebView (used for e.g., queries) has been improved for browser compatibility (mainly with Internet Explorer) and functionality. It now expands automatically to fit the text contained, displays line numbers and automatically indents code.

Changes to the Lisp API

Rfe11082: Transactional duplicate deletion (repeated from the General section)

INCOMPATIBLE CHANGE: AllegroGraph no longer supports duplicate deletion at index optimization time. The database creation parameters indicating duplicate deletion strategy are no longer accepted.

Instead, AllegroGraph now allows deletion of duplicates using the delete-duplicate-triples API function, which is also accessible via the HTTP API call DELETE /statements/duplicates.

Documentation

No significant changes other than new features documented.

Python Client

Rfe11122 - Add "delete duplicates" API function to python client

RepositoryConnection.deleteDuplicates:

Delete all duplicates in the store. Must commit.

mode - "spog" or "spo"

Java Client

Rfe11121 - Support transactional duplicate deletion

With this change, the Java client now provides the method AGRepositoryConnection#deleteDuplicates(comparisonMode) for deleting spog and spo duplicates in a transactional manner. See the javadoc for more details.

See Change History for user-visible changes to earlier releases.

Rfe11117: java api for SPIN (adds to rfe10988)

New methods added to AGRepositoryConnection: listSpinFunctions() listSpinMagicProperties()

New classes: AGSpinFunction, AGSpinMagicProperty

Rfe10622 - Http response handlers now have their own package

With this change, the Java client's HTTP response handlers have been refactored and moved into a com.franz.agraph.http.handler package.

AllegroGraph 4.3.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10984 - Fix serialization issues with encoded IDs

Both the turtle and the RDF/XML serializers failed to correctly handle encoded IDs. This has been corrected.

Rfe10967 - Nicer CSV output option

AllegroGraph now supports another CSV output format (MIME type application/processed-csv) that uses the UTF-8 encoding, prints literals with numeric types as numbers, and doesn't wrap string literals in quote marks.

Bug20460 - Encoded IDs and reasoning triple-stores failure

If reasoning was used during a query, then encoded IDs were not correctly recognized which led to incorrect results. This has been corrected.

Bug20451 - Fix encoded ID API on remote lisp client

The remote lisp client was missing a few functions in the encoded ID API: The functions that were missing are next-encoded-upi-for-prefix, encoded-id-prefix-registered-p and encoded-id-prefix-in-use-p. These are now implemented and available on the remote lisp client.

Bug20443 - Encoded IDs did not work with the TriX serializer

The TriX serializer was incorrectly emitting encoded IDs as if they were plain literals instead of resources. This has been corrected.

Bug20416 - Use proper n-triples syntax for text/csv

The CSV generator (when requesting MIME type text/csv) for query results will now always generate valid CSV such that the entries imported into a spreadsheet or a database are in correct n-triples part syntax.

HTTP Client

No significant changes.

SPARQL

No significant changes.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe10983 - Query execution with initial bindings

With this change, the Java client's Jena adapter now supports creating query executions with initial bindings using the method: AGQueryExecutionFactory#create(query,model,initialBinding);

Welcome to AllegroGraph 4.3.2

AllegroGraph 4.3.2 user-visible changes

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

Bug20424 - Disable 5-minute HTTP request timeout

AllegroGraph now allows HTTP API requests to run (and generate results) for longer than 5 minutes.

SPARQL

No significant changes.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug20272 - NPE when no response stream is available

Previously, the Java client's HTTP response handler could throw an opaque NullPointerExeption if no response stream is available (this might occur due to a server crash). With this change, the response handler checks for a null stream and throws an exception advising the user that no response stream is available and to inspect the server log file for possible explanations.

Welcome to AllegroGraph 4.3.1

AllegroGraph 4.3.1 user-visible changes

AllegroGraph Server: General Changes

Rfe10889 - Provide guaranteed duplicate removal behavior

As long as no triples are concurrently being added, completion of a level 2 database optimization will now ensure that all duplicate triples (as defined by the current duplicate triple removal mode) are deleted.

Rfe10850 - Ag3 plugins for TBC 3.5

TBC 3.5 no longer provides built-in support for AG3 databases. With this change, AG3 plugins are made available to work with TBC 3.5. The plugins are not compatible with previous versions of TBC due to conflicts with the built-in support for AG3.

AG3 plugins for TBC 3.5 will be available from this Eclipse update site:

http://www.franz.com/ftp/pub/agraph/eclipse/tbc/3.5

Bug20408 - Implement encoded-ids for lisp remote-triple-store

It is now possible to register, unregister and list encoded-id prefixes when using the remote lisp client.

Bug20397 - Load triples from files faster on remote lisp client

The remote lisp client now can load large files into the server, and will no longer run out of memory when doing so.

Bug20378 - Improve duplicate reduction in UPI-map based cursors

AllegroGraph reduces the number of triples examined during SPARQL queries by skipping subsequent duplicates when iterating over a cursor. This duplicate reduction was not occuring for cursors based on secondary indices (UPI-maps) which could lead to decreased query performance. This has been corrected.

Bug20372 - Fix part printing for literals and resources with same name

AllegroGraph would incorrectly print literals that had the same name as a resource. This is now fixed.

Bug20355 - Durably save transactions on warm-standby client

The warm-standby secondary machine now saves transaction log entries in a way that ensures they've been flushed to disk durably as soon as the entire commit has been read.

Bug20336 - Fix inconsistency on warm standby secondaries

Replication secondary instances for AllegroGraph would sometimes query and attempt to display inconsistent data while replication was copying transactions from the primary instance. This has now been fixed: AllegroGraph ensures that only data from transactions that have been fully been replayed is used.

Bug20230 - Fix deletion slowdown under replication

The replication server used a connection to the replicated triple store that ensured it would never expunge deleted triples, slowing down operations if many deletes had occurred. Now, the replication server correctly expunges these triples and does not slow down.

HTTP Client

Rfe10893 - Allow "unlimited" concurrent HttpClient connections

Previously, the Java client's AGServer class would default to using an HttpClient that limited the number of concurrent connections per host to 20 (up from the HttpClient's default of 2, per RFC 2616 sec 8.1.4), which could still result in unnecessary confusion and socket churn when interacting with application level connection pool management. With this change, the defaultMaxConnectionsPerHost and maxTotalConnections are both increased to Integer.MAX_VALUE to avoid conflict with application level connection pool management.

SPARQL

Bug20382 - Sparql queries that returned * could confuse remote-triple-stores

If a SPARQL query returned all of its variables (e.g.. SELECT * ...), then the remote-triple-store could fail to determine the actual variables returned by the query. This could in turn lead to incorrect results from clients that use this information (e.g., Gruff). This is now corrected.

AGWebView

Rfe10924 - Enable download of server log file from WebView

WebView now allows viewing and downloading the agraph.log server file for diagnostic purposes. To open the log, select the logfile entry in the Admin menu.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug19798 - Illegal resource in Jena reified statements

Previously, the Jena adapter's createReifiedStatement method could create reified statements containing illegal blank node ids that were based on generated UUIDs, manifesting in invalid part errors from the server. With this change, these reified statements are now created using legal resources. AGGraphMaker instances can now be created with a specified ReificationStyle (the default style is ReificationStyle.Minimal).

Welcome to AllegroGraph 4.3

AllegroGraph 4.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10790 - improve logic for finding the sort-upi-map binary

There were cases when the Lisp client could fail to locate the path to the sort-upi-map external program. This has been corrected.

Rfe10665 - AG4 plugins for TBC 3.5.0

With this change, the AG4 plugins have been modified to work with TBC 3.5.0. The plugins are not backwards-compatible with previous versions of TBC due to changes in its API.

AG4 plugins for TBC 3.5 will be available from the update site: http://www.franz.com/ftp/pub/agraph/eclipse/tbc/3.5

Rfe10636, Bug20160 - signal an error if a blank node ID is too big, support for more blank nodes per store

AllegroGraph previously supported only 2^32 blank-nodes per triple-store. This is now increased to 2^56. Furthermore, if AllegroGraph tries to create a blank-node with an ID larger than the maximum supported, it will now signal an error rather than silently reusing existing nodes.

Rfe10632 - Enable per-user security-filtering of stores.

AllegroGraph now has a new class, security-filtered-triple-store, which allows specifying criteria that triples must fulfill in order to be accessible in that store.

AllegroGraph also automatically filters stores for users of a remote-triple-store if these users (or any of their roles) have security-filters defined.

Rfe10090, Rfe10541 - Optimizations to AllegroGraph secondary indices

Improves both the creation of secondary indices (UPI-maps) and their efficiency in use. Queries that make heavy use of secondary indices will operate more quickly.

Rfe10508 - Japanese full text search

An alternative text indexer for Japanese is provided. From the Lisp client, it is available by supplying :japanese as the :tokenizer keyword argument to create-freetext-index. When you use the Japanese tokenizer, we recommend you set :minimum-word-size to 2 or even 1 (the default is 3, but there are many Japanese words that consist of 2 characters).

The following is an example configuration for indexing Japanese text in literals of the object part of triples whose predicate is ! .

(create-text-index "ja" :predicates '(!<description>)  
                        :stop-words nil  
                        :index-fields '(:object)  
                        :tokenizer :japanese  
                        :minimum-word-size 2)  

Rfe10020 - add support for owl:SymmetricProperty

AllegroGraph's RDFS++ reasoner now supports owl:SymmetricProperty so that it will make inferences like:

P rdf:type owl:SymmetricProperty  
a P b  
===> b P a 

In the process of implementing this addition, several other small efficiency improvements were made so that the reasoner is now usually faster even though it is doing more work. For example, the time to run all 14 LUBM-50 queries went from 10.04 seconds down to 9.44 seconds.

Rfe8278 - Add ability to customize the tokenizer used by freetext indexing

The Lisp API create-freetext-index takes two additional keyword arguments, border-chars and inner-chars, to fine-tune the word tokenization behavior. Each argument takes a freetext-charset object, specifying a set of characters that can appear at the borders of a word, or inside a word. A freetext-charset object can be created by a new API freetext-charset.

The following example creates a freetext index in which only alphanumeric characters and a hyphen #\- consists a word; that is, #. and #_ breaks words.

(create-freetext-index "custom" :predicates '(!<description>)  
                       :inner-chars (freetext-charset '(:alphanumeric #\-))) 

Bug20289 - Do not allow stored procedures to enter the debugger

If a stored-procedure written in lisp calls the function error with a non-error condition, the AllegroGraph backend process running the stored procedure would enter the debugger. This resulted in an unresponsive backend.

Now, the backend will catch invocations of the debugger, report the failure, and continue running.

Bug20273 - Suboptimal index queries broken

Under certain circumstances, suboptimal index queries could return triples that did not match the query pattern. This has been fixed.

Bug20160, Rfe10636 - signal an error if a blank node ID is too big, support for more blank nodes per store

AllegroGraph previously supported only 2^32 blank-nodes per triple-store. This is now increased to 2^56. Furthermore, if AllegroGraph tries to create a blank-node with an ID larger than the maximum supported, it will now signal an error rather than silently reusing existing nodes.

Bug20133 - Select queries should create filtered triple-stores only as needed

Prolog select queries were sometimes creating temporary graph-filtered triple-stores unnecessarily which wasted resources and time. This is now corrected: graph-filtered triple-stores are created only when necessary.

Bug19834 - Resource manager connection management bugs

Previously, global variables were used to maintain connection information between Lisp clients and one of the AllegroGraph subprocesses. This could cause problems if a single Lisp client made connections to two separate AllegroGraph servers or if an AllegroGraph server was stopped and restarted. This has been fixed.

Bug20037 - allow trailing slash on Server URL in AG4 dialogs

Previously, putting a trailing slash on the Server URL in AG4 dialogs would result in an error message that was hard to debug. With this change, dialogs are now robust to trailing slashes.

HTTP Client

Rfe10839 - Freetext index charsets and tokenizer in HTTP API

The HTTP API for freetext index management now supports the optional arguments "tokenizer" as well as "borderChars" and "innerChars", which specify the tokenizer and character sets for word boundaries.

Bug20250 - Fix federated remote stores in HTTP API

The /session resource didn't correctly create sessions when specified using the syntax. Now, POSTing to /session correctly opens a session on a remote triple store.

SPARQL

Bug20305 - Encoded IDs were incompatible with SPARQL CONSTRUCT queries

SPARQL CONSTRUCT queries will not produce triples whose subject or predicate are literals. The check for this was incorrectly excluding triples with encoded IDs in the subject (or predicate) because these IDs have a different UPI type-code that the one used for resources. This has been corrected.

Bug20288 - Encoded IDs don't work with SPARQL/XML and SPARQL/JSON output

The SPARQL/XML results format would crash when used with encoded IDs; The SPARQL/JSON results format generated output but emitted encoded IDs as typed literals rather than as resources. Both of these problems are corrected.

AGWebView

Rfe10728 - Improve WebView store overview user interface

The AllegroGraph WebView store overview page now is better structured and allows easier access to commonly-used functions.

Bug20304 - Fix codemirror for Internet Explorer 9

Internet Explorer 9 users can now use WebView's text editor with syntax highlighting.

Bug20302 - Fix loading more results in Internet Explorer

WebView can now load more than 100 results (using the [more] link) in Internet Explorer.

Bug20296 - Fix query result downloads in Firefox 3.

Fix a JavaScript incompatibility that caused downloading query results from WebView to fail in Firefox version 3.

Bug20203 - Fix WebView for newer Internet Explorer versions.

Internet Explorer 8 and 9 support a new history tracking mechanism, and deprecated features that WebView used to keep track of history before. Now, WebView makes use of this new mechanism when it's available. This allows Internet Explorer to correctly run AllegroGraph WebView again.

Bug20069 - Uploading file into a context from WebView is broken

The error " is not a valid part" would always appear. This is now fixed.

Changes to the Lisp API

Bug20146 - Export UPI hash-table functions from db.agraph.user

In the transition from AllegroGraph 3 to AllegroGraph 4, the UPI hash-table functions ceased being exported from db.agraph.user. This is now corrected.

Bug20145 - (Lisp API only) loading AllegroGraph should provide module :agraph4.

AllegroGraph previously only provided the module :agraph which meant that using

(require :agraph4) 

could load agraph4.fasl unnecessarily. This is fixed.

Bug20144 - Fix interference from remote-triple-store ping

To ensure that a remote-triple-store remains opened, AllegroGraph's lisp API sends a ping message in the background in regular intervals. Previously, it would use the same session object to record responses from the server as a user's requests. This would cause errors when a ping message interrupts a long-running request.

Now, AllegroGraph uses a separate session object on the client's side to send these ping messages.

Documentation

Bug20149 - Correct documentation of datatype and predicate mappings (Lisp)

The Lisp reference manual had several typos in its discussion of datatype and predicate mappings. These have been fixed.

Python Client

No significant changes.

Java Client

Rfe10135 - Improve javadocs

Added overview pages, more docs for classes and methods, and links to AllegroGraph documentation for introduction, catalog, federation, etc.

Rfe10745 - Add Jena text indexing code example

With this change, a Jena text indexing code example is now available in the distribution in JenaTutorialExamples.java as example 12; this will eventually also be written up in the Jena tutorial document, but the focus here is on making the code example available as it is useful on its own.

Rfe10624 - AGQuery limit and offset modifiers

With this change, the Sesame and Jena AGQuery classes now allow limit and offset to be specified by the query object, outside of any limit or offset specified in the query string, making it easier for apps to override these modifiers.

Rfe10623 - Jena AGQueryExecution count methods

With this change, the Jena api now supports counts queries via the AGQueryExecution#countSelect() and countConstruct() methods. Solutions are counted server-side and only the count is returned to the client.

Rfe10492 and Rfe10493 - javadoc for AGRepositoryConnection

Add javadoc that points to SessionPorts for relevant methods. Add javadoc for type mappings, linking to http and lisp docs.

Rfe10452 - support N-Quads responses in the Java client

With this change, the Java client now supports N-Quads responses from the server. The property -Dcom.franz.agraph.http.defaultRDFFormat=NQUADS can now be used to configure the client to request N-Quads format by default for RDF statement responses; when this property is not present, the default format is TriX. The method AGHttpRepoClient#setPreferredRDFFormat() can also be used to override the default RDFFormat on a per connection basis.

Rfe10450 - Jena Prolog and transaction code examples

With this change, Jena Prolog and transaction examples are now available in the distribution in JenaTutorialExamples.java as examples 17, 18, and 22; they will eventually also be written up in the Jena tutorial document, but the focus here is on making the code examples available as they are useful on their own.

Rfe10447 - add AGGraphQuery#count() method

With this change, the java client now supports counting for graph queries (e.g CONSTRUCT and DESCRIBE) with the AGGraphQuery#count() method. Also added javadoc for the AGTupleQuery#count() method.

Rfe10446 - add count() examples to the Java tutorial

With this change, the Java tutorial shows how to use the AGTupleQuery#count() and AGGraphQuery#count() methods to to obtain query result counts without returning results (counting is done server-side, only the count is returned).

Rfe10445 - add example/test getting triple ids via prolog

With this change, example17 in TutorialExamples.java shows how to get triple ids via Prolog. TripleIdTests has also been augmented to get triple ids via Prolog.

Rfe10411 - Streaming large query results

When AGTupleQuery.evaluate() is used for sparql and prolog queries, the results are returned by the server in sparql/xml format. This is parsed by the Sesame library using a SAX XML parser and the results are collected in an ArrayList before the TupleQueryResult is returned. For very large result sets, this will cause an OutOfMemoryError.

Alternatively, AGTupleQuery.evaluate(handler) can be used where the handler methods are called as results are parsed. This can handle any length results without a memory error, but the API is less convenient. A new example has been added to the tutorial to demonstrate this: TutorialExamples.example3a()

A new class AGStreamTupleQuery can be used to wrap an AGTupleQuery so the TupleQueryResult will be streamed back. This provides the familiar iteration API rather than a handler, but the http response is only parsed as the iterator is advanced. The usage of this API is explained further in the javadocs: AGTupleQuery query = conn.prepareTupleQuery(QueryLanguage.SPARQL, "SELECT ..."); query = new AGStreamTupleQuery(query); TupleQueryResult results = query.evaluate();

There is also a new method in AGRepositoryConnection that will cause the connection to automatically wrap all tuple queries to use streaming: AGRepositoryConnection.setStreamResults(boolean streamResults)

Instead of SAX, the AGStreamTupleQuery uses an XMLStreamReader for parsing. Initial performance results suggest that in addition to being resistant to OutOfMemoryError this uses less overall memory than the SAX parser for small and medium result sets (10-500).

This approach of streaming results has not yet been applied to getStatements().

Rfe9445 - Upgrade slf4j, funnel through jcl, and add source jars

With this change, the java client now uses slf4j-api-1.6.1, and uses slf4j-jcl-1.6.1 to funnel sesame and jena logging through commons-logging-1.1.1 (used by httpclient-3.1). Source jars are also included for javadoc and to faciliate debugging.

Welcome to AllegroGraph 4.2.1c

AllegroGraph 4.2.1c user-visible changes

AllegroGraph Server: General Changes

Clarify what retain for replication means

If the TransactionLogRetain is set to replication then all transaction logs will be saved until replication starts at which point those log files no longer needed will be archived. This behavior may be further modified in a future release.

Bug20257 - fix convertdb for some rare cases

The convertdb utility failed to upgrade some stores that had transaction logs with a certain size. Now it can correctly upgrade these stores.

Bug20236 - Fix startup when HTTPProxy is specified in config

AllegroGraph now starts up correctly when a HTTP proxy specification is present in the configuration file.

Bug20223 - Fix agload for Turtle files

A change in agload broke loading Turtle-formatted files with agload. Now, agload can correctly load the Turtle format again.

HTTP Client

No user-visible changes.

SPARQL

No user-visible changes.

AGWebView

No user-visible changes.

Bug20200 - Clarify format support in import forms in WebView.

AllegroGraph now supports loading .gz-compressed files (and autodetecting their file format) from the server's local file system. The file import dialogues in WebView also have a note about the file name format for autodetection.

Changes to the Lisp API

Rfe10790 - improve logic for finding the sort-upi-map binary

There were cases when the Lisp client could fail to locate the path to the sort-upi-map external program. This has been corrected.

Documentation

No user-visible changes.

Python Client

No user-visible changes.

Java Client

No user-visible changes.

Welcome to AllegroGraph 4.2.1a

AllegroGraph Server: General Changes

Rfe10689 - Increase unique string limit to ~4B

The limit on the number of unique strings in the string table was previously 1,431,655,765. This limit has been increased to 4,294,967,295 entries.

Rfe10562 - Change encoded-id marker to @@ (from @#)

The marker used to indicate that a URI represents an encoded-ID has been changed from @# to @@.

Rfe10532 - Return encoded ids in ascending order

The .../encodedIds?prefix=""&amount=# http request was returning the ids in reverse order. This has been fixed to return them in ascending sequential order.

Rfe10521 - Deleted hash table management based on generation

Deleted hash tables are now cleaned up as soon as they are no longer needed, preventing the store from accumulating unnecessary files.

Rfe10200 - Regulate speed of mergers

Triple index mergers now regulate their speed based on the number of triples to be merged and the rate at which triples are added after a merge has started.

Welcome to AllegroGraph 4.2.1.

AllegroGraph Server: General Changes

Rfe10483, Rfe10528, Bug20012 - Inadequate transaction log durability

When using the default TlogSyncMethod of "odirect", the durability of transaction log files was not correctly ensured. This release fixes that issue by preallocating transaction log files and fsyncing them before using them. Additional transaction log files are created as needed. In addition, transaction log files that are no longer needed are now recycled instead of deleted.

This change will have an impact on database creation time due to the initial preallocation. To compensate for this, the default TransactionLogSize parameter has been changed from 1 gigabyte to 256 megabytes.

There is a new configuration parameter DesiredTlogFiles which specifies the number of transaction log files which should be preallocated at database creation time. The default value is 2. Specifying a larger value helps lower the probability of additional transaction log files being created during commits. This parameter is documented in Server Configuration.

Rfe10475 - Improve I/O behavior under memory pressure

When under memory pressure, Linux can sometimes generate far more I/O than expected when minor page faults occur during accesses to memory mapped files. This can result in very poor performance, especially when the I/O device is already congested. This change instructs the operating system not to perform these extra I/Os.

Rfe10439 - RDSF++ reasoner now does not create duplicate inferred triples

In some cases, the RDFS++ reasoner could follow multiple owl:sameAs paths and generate many duplicate inferred triples. This increased AllegroGraph's memory footprint and decreased RdFS++ reasoning performance.

The reasoner now generates only a single inferred triple for each owl:sameAs path when searching for sameAs statements with a fixed subject and predicate or for ones with a fixed predicated and object.

Rfe10434 - Rename Lisp API function to duplicate-deletion-strategy

The Lisp API function duplicate-deletion-strategy alters the way AllegroGraph manages duplicate triples during index optimization operations. It can be one of:

• t - the same as :spog (see below).

• nil - do nothing to duplicates.

• :spo - ensure that no more than one triple with a given Subject, Predicate and Object exists after index optimization operations are complete. A triple may be deleted if it differs from another only in its graph.

• :spog - ensure that no more than one triple with a given Subject, Predicate, Object and Graph exists after index optimization operations are complete.

Previously, this function was called delete-duplicates-during-merge-p.

This setting is also available via the HTTP interface.

Rfe10413 - Add check for 64-bit Linux

Added a check to install-agraph and configure-agraph scripts to ensure that they're running on a 64-bit Linux. This should reduce confusion in the case where AllegroGraph Server was accidentally installed on a 32-bit Linux host.

Rfe10400 - Minor cursor implementation changes

Some changes were made to enhance AllegroGraph's cursor mechanisms. These do not alter existing cursor behavior but help pave the way for future improvements.

Rfe10399 - Verify lisp client and server version match

As the lisp client is very sensitive to changes in the backend API exposed by the AllegroGraph server, it now checks that it is connecting to the correct version of the server.

Rfe10363 - Improved error message when a file cannot be found

The error message generated when a database file is not found has been improved to make it more clear what is causing the problem.

Rfe10354 - Redesigned --quiesce feature of agtestclient

The quiesce functionality in agtestclient was unnecessarily complicated and inefficient. The feature has been redesigned to require running on the host of the secondary server in question and also require the expected count and a timeout. This is sufficient for the purposes for which the quiesce function is used. Expected count must either be queried from the primary or the triple-count must be known beforehand.

A bug was also fixed with the order of the arguments in the delete function. (They had been reversed.) Also, help documentation was created, which can be accessed by running agtestclient with no arguments. Finally, the --add argument was renamed to the more appropriate --add-random.

Rfe10345 - Don't log warnings when opening NFS triple-stores.

Opening a triple-store on NFS caused unnecessary warnings to be issued. These warnings are no longer logged.

Rfe10342 - Command line wrapper for the lubm-generator Lisp module

Note: lubmgen can easily contaminate or erase a production database if used without clear understanding of its purpose.

Rfe10337 - Improve cursor resource management

Improve the management of the cursors AllegroGraph uses internally for triple counting and triple deletion.

Rfe10335 - Improve logging of (adjusted) store parameters

AllegroGraph will now log human-readable configuration values when opening a store if a human-readable presentation is available. (The system determines whether a value has a human-readable presentation or not. That is not under user control. Values which do not have a human-readable presentation are also logged but not in a human-readable way.)

Rfe10308 - AllegroGraph can use UPI-maps to run queries in more cases

Increased the number of times when AllegroGraph can use UPI-maps (secondary indices) to run queries. This improves the enhancements made in Rfe10141 and Rfe10142.

Rfe10243 - opening database should honor existing merge mode as default

It may have been possible that the duplicate removal mode is rolled back accidentally by reopening the database. The problem is fixed.

Rfe10184 - Improve custom hash table efficiency

Improve the way AllegroGraph handles some internal hash table computations. This will provide some very small performance improvements to general AllegroGraph operations. Note that none of these operations occur in inner loops so the performance gains will not be markedly noticeable.

Rfe10114 - Change default for TransactionLogRetain to "recovery"

In prior releases, the default value for the TransactionLogRetain setting was "all", which meant that, unless configured otherwise, all transaction log files since the the database was created would be saved. This release changes the default to "recovery", which means that only the transaction log files required for instance crash recovery are retained. See Transaction Log Archiving.

Rfe9338 - Improved error message due to session port connection failure

When using methods that start a session (setAutoCommit, addRules, and registerSNAGenerator), if there is a connection error, the error message points to the documentation about SessionPorts.

Bug20122 - Handle disk full on transaction log writing

Previously, encountering a full filesystem while writing to a transaction log would result in a crash. Now, AllegroGraph will (if possible) log a message to the logfile and to stderr noting the problem and will periodically retry the write operation until it succeeds.

Bug20118 - graph type-mapping could lead to incorrect data

If a triple was added with a graph field that was mapped to an encoded UPI (for example, a geospatial point or a typed literal) and the object field was not mapped (for example, a plain literal or a resource), then AllegroGraph would fail to correctly store the triple's object. This has been fixed.

Bug20089 - encoded-ids not working in non-object fields for clients

AllegroGraph was failing to translate encoded-IDs in fields other than the object field for any non-direct Lisp client. This has been fixed.

Bug20050 - Improve speed of bulk load mode in agload

agload was incorrectly handling the --bulk parameter which caused it to perform significantly more slowly when bulk mode was turned on. With this change, bulk mode is 6 to 7 times faster than it was before.

Bug20031 - agraph-backup backs up the same file twice

Previously, agraph-backup was backing up one of the database files twice. Though this did not result in a corrupt backup, it was wasting time and space. This problem has been fixed.

Bug19996 - Repair durability guarantees for NFS stores.

The change to implement rfe10216 in AllegroGraph 4.2 accidentally broke durability guarantees on NFS stores: Transaction logs on NFS were opened in an asynchronous write mode, which could lead to commits getting lost in the case of a crash.

Bug19969 - Correct problem printing times with fractional seconds

If a fractional second had a value less than 0.001, then AllegroGraph would incorrectly print the time (or dateTime) using scientific notation. It now prints these values correctly. E.g., it now prints 16:46:40.000133 rather than 16:46:40.1.33e-4.

Bug19954 - Commit/rollback could sometimes result in crash

Fixed a programming error which could potentially cause a commit or rollback to result in a program crash.

Bug19950 - Handling of tranaction log files corrected

In this release the 'recovery' and 'archive' values for the TransactionLogRetain catalog parameter function correctly. Previously these two values were treated as if 'all' were specified.

Bug19933 - Made agload recognize "nq" file format (nquads).

Previously, agload would reject files with extension .nq. Now they are correctly identified as nquad files and loaded.

Bug19916 - Improve :terse output

Modified the heuristic for the :terse output so that it will not emit empty strings. Previously, a URI like 'http://www.foo.com/bar/' would have a terse representation of ''; now it will use 'bar/'.

Bug19836 - Date and datetime with negative year

XSD requires BCE dates, for example, these are now accepted:

(value->upi "-0002-04-12" :date)  
(value->upi "-0002-04-12T12:23:00Z" :date-time) 

Bug19682 - Federated-triple-stores error for type-mappings

Type mappings can now be used with federated stores where all component stores have the same type mapping, for example in methods such as collect-all-type-mappings and get-triples. Function collect-all-type-mappings returns the intersection of mappings for all component stores.

Mappings can still only be set on component stores, not a federated store.

Bug19116 - Datatype-mappings are not applied to graph

When adding a triple, if the graph is a literal with a datatype that can be mapped like object, it will be. Subjects and predicates are not mapped at all.

At most 3 triples can be added, which is the same as before:

1. with object and graph mapped by datatype
2. with object mapped by predicate-mapping and graph by datatype if this results in a different triple than #1
3. with no mappings if :preserve-strings is t

HTTP Client

Rfe10360 - Better HTTP error messages for encoded-id errors.

Stack traces are no longer logged for too-wide encoded-id formats or bad re-registration of such ids.

Bug20124 - Make number of initial HTTP workers configurable.

The number of initial HTTP workers started by the AllegroGraph server used to be exactly the number of backends. However, that doesn't take into account any frontend sessions like WebView. This would cause long-running requests on startup (like opening a triple store) to cause shorter-lived requests to wait until the longer-running request was finished.

AllegroGraph now has a new top-level configuration parameter HTTPWorkers (defaulting to 50), which lets administrators specify how many HTTP worker threads to launch on startup.

Bug20022 - Add location-rewriting to the AG reverse-proxy.

The HTTP reverse-proxy that dispatched requests to server back-ends did not handle redirect responses with Location headers in any special ways. This could sometimes lead to clients being redirected to a backend URL, which did not work. The proxy now properly rewrites the Location header to point at the front-end URL.

Bug20013 - Requests to back-ends with Expect: headers fail

Due to a bug in the HTTP system, sending a request with a body an Expect: header (which CURL does by default) would cause a deadlock in the intra-server communication, and thus a timeout. This has been fixed.

Bug19948 - Allow full HTTP delete-duplicates control

The HTTP API used to expose deleteDuplicates as a boolean value, but there are actually three states: off, delete by SPO, and delete by SPOG. The strings "spo" and "spog" (as well as the existing booleans) can now be used in the HTTP API when inspecting or setting duplicate deletion.

SPARQL

Rfe10486 - SPARQL query planning should not require the PSOGI index

When a query involves filtering on potentially encoded triples (e.g., numeric or date ranges), then the SPARQL algebra query engine checks to see if certain ranges of triples exist during its planning phase. The checks that SPARQL makes requires the POSGI index and are inordinately slow if that index is not available. Previously, AllegroGraph would make the checks regardless. Now, it will now skip these checks if the index is not there.

Rfe10374 - Improve efficiency of some SPARQL queries

SPARQL's notion of equality requires AllegroGraph to examine several variants of some IRIs. For example, the literal "4" should be equivalent to the literal "4"^^xsd:string.

This change improves the efficiency of AllegroGraph's SPARQL equality computations. The performance improvement will depend on the query.

Rfe10371 - Improve SPARQL's handling of DISTINCT LIMIT queries

Improve SPARQL's performance for queries with a LIMIT whose solutions must be DISTINCT.

Rfe10348 - Iri= for subject and predicates (generalize SPARQL constraints)

Extended IRI= constraints for subjects and predicates. This allows queries like:

select * where { ?s ?p ?o . filter( ?s = ex:foo ) } 

to operate as if they were written:

select * where { ex:foo ?p ?o . } 

If multiple constraints are simultaneously operative, then only one will be used for a given pattern. Currently, these will be selected in the order: subject, object, predicate.

Queries for which this optimization is relevant will see greatly increased speed.

Rfe10344 - Improve SPARQL statistical BGP reordering logic

The statistical BGP reordering function could produce sub-optimal results if a SPARQL query had multiple levels of nested UNIONs or OPTIONALs. This could cause queries to take much longer than necessary.

Rfe10312 - Optimize IRI= for object fields in SPARQL queries

This optimization converts IRI= constraints for object fields in SPARQL queries into the equivalent storage level get-triples calls. Previously, a query like

SELECT ?article  
WHERE { ?article rdf:type ?type .  
        ?article swrc:pages ?value  
        FILTER (?type = bench:Article ) } 

would fetch all of the rdf:type triples and then filter for those whose object field was bench:Article. Now the query will be executed as if it had been written as

SELECT ?article  
WHERE { ?article rdf:type bench:Article .  
        ?article swrc:pages ?value  
 }  

Rfe10118 - Added interface to SPARQL query analysis to Python client

After calling RepositoryConnection.prepareTupleQuery() on a SPARQL select query, one can call analyze() on the resulting query object to get a string returned with information about which indices would be used to satisfy the query.

Currently, the query object's method is:

def analyze(self, analysisTechnique=None, analysisTimeout=None):

Analysis is only available for SPARQL queries.  
 
analysisTechnique defaults to "executed", which executes  
the query to perform dynamic analysis. "static" analysis is  
the other option.  
 
For analysisTimeout, pass a float for the maximum number of  
seconds to run the query if executed. 

Rfe9507 - Add initial support for SPARQL subqueries

SPARQL subquery support includes ORDER BY, LIMIT, and DISTINCT. An example query would be:

PREFIX fn:    <http://www.w3.org/2005/xpath-functions#>  
PREFIX boat:  <http://example/boats/vocab/>  
PREFIX boats: <http://example/boats/>  
SELECT ?boat ?boatName  
FROM boats:boats1  
FROM boats:boats2  
WHERE {  
  # Select the 2 boats with the shortest names.  
  { SELECT ?boat WHERE {  
    ?boat boat:placed ?position ;  
          boat:name ?name .  
    } ORDER BY asc(fn:string-length(?name)) LIMIT 3 }  
 
  # Select their crew and the boat's name.  
  ?boat boat:name ?boatName .  
}  
ORDER BY desc(fn:string-length(?boatName)) desc(?boatName) 

Note that subqueries are a pre-release SPARQL 1.1 feature; documentation and conformance tests are not yet available from the W3C, semantics are subject to change, and the implementation might be incomplete in a manner that is visible to user code. Please contact Franz support if you suspect that you have found a discrepancy.

Bug20070 - fix problem with geospatial encoded UPIs and some output formats

If a SPARQL query returned geospatial encoded UPIs and used the SPARQL-XML result format, then an error would occur. The same error could occur when serializing geospatial UPIs in the turtle format.

There was also a problem correctly serializing geospatial output using the SPARQL-JSON format: incorrect latitude and longitude values were sometimes generated.

All of these problems are corrected.

Bug20047 - Can't download query results with missing values as CSV

Taking a SPARQL query that produced unbound variables (say, with OPTIONAL) and asking for a comma-separated value return format would return an error. This has been fixed.

Bug19989 - Sparql CONSTRUCT could generate invalid blank nodes

SPARQL CONSTRUCT using the N-Triples format could generate invalid blank nodes.

Bug19832 - Correct bug in low level SPARQL equality code

Correct a bug in some rarely executed SPARQL equality code. The bug would have caused some queries in some datasets to fail to run.

AGWebView

Rfe10370 - Use only relative URLs in WebView

This makes it possible to set up a reverse-proxy that proxies, for example, host:80/allegrograph/ to host:10035/, without breaking WebView.

Bug20051 - Downloading query results from WebView was broken

After recent patch (bug20022), trying to download query results from WebView always resulted in a 404 response. This fixes that.

Bug20016 - Fix horizontal scrolling in WebView for Firefox

A bad CSS rule was preventing Firefox from ever displaying a horizontal scrollbar for WebView. This made it hard to inspect, for example, query result tables that were wider than the window. This is now fixed.

Bug20009 - 4.2 WebView can't handle some data saved by 4.1

Saved queries saved in 4.1 would cause an error when used in 4.2, because of a change in the way the client-side code is handling JSON data. This change fixes that, by falling back to the old parsing method if the new one fails.

Bug19972 - Webview breaks when it doesn't find JSON support

On older browsers without a JavaScript JSON object, WebView's detection of that object was failing, causing the whole script to abort. It now properly detects, the functionality, replacing it when not present.

Changes to the Lisp API

Rfe10420 - Add lisp client function optimize-indices

AllegroGraph's Lisp client now has a function optimize-indices that will cause the server to rearrange the indices for optimal query performance. This optimization pass can take a long time and cause a lot of I/O operations to be performed on large stores.

Rfe10066 - Add option to use rapper for load-turtle

The lisp API load-turtle now has a :use-rapper-p keyword parameter. See load-rdf/xml documentation for argument usage details.

Rfe9715 - Lisp client package now includes Lisp tutorial examples

The subdirectory tutorial now holds the Lisp and data example files discussed in the documentation.

Bug19672 - Triple-store-exists-p leaks server information

The Lisp client's successful return value from triple-store-exists-p sometimes provided the store's path on the server's filesystem.

The API has been fixed to return only the values nil or t.

Bug19133 - get-triple now applies geospatial mappings

get-triple now correctly applies geospatial mappings to the triple returned. Also, the get-triple now returns a fresh triple (rather than one which might be reused by internal mechanisms).

Documentation

New JavaScript document for programming the server

See JavaScript.

New Security document describing data security

Rfe10341 - Improve documentation search

Added basic documentation searching to the AllegroGraph documentation.

Python Client

Rfe10505 - use session to get new blankNodeIds

AGValueFactory was using the main repository url to get blank node ids. Use connection session instead if available. Using the session can reduce load on the primary server.

If the ValueFactory was created by Repository instead of RepositoryConnection, the old behavior remains.

Rfe10418 - Call indices/optimize from Python

Added an optimizeIndices(level, wait) method to RepositoryConnection for exposing the http indices/optimize POST request to Python client.

Rfe10326 - Support deleteDuplicates setting from Python client

The Catalog.createRepository function now has a deleteDuplicates keyword argument for setting the store's deleteDuplicates behavior on creation. If unspecified, the behavior is the server's default setting.

The store's deleteDuplicates setting is also available from a Repository object instance either as a property variable (delete_duplicates) or as setDeleteDuplicates and getDeleteDuplicates methods. This is a per-store setting.

See the HTTP Protocol - SPARQL Endpoint documention for setting values and behaviors (text search for deleteDuplicates).

Rfe10325 - Added evalJavaScript to Python client

Added an evalJavaScript function to the RepositoryConnection class for evaluating server-side JavaScript code.

See the server's JavaScript documentation for details on programming the server in JavaScript.

Java Client

Rfe10389 - Upgrade to sesame 2.3.2

With this change, the java client distribution has been upgraded to use openrdf sesame 2.3.2, a minor upgrade from 2.3.1 (see Sesame).

Rfe10373 - Support deleteDuplicates mode in java

With this change, the Java Client supports setting a repository's policy for deleting duplicates using methods AGRepository#setDeleteDuplicatesMode and AGRepository#getDeleteDuplicatesMode.

Rfe10372 - Support periodic commit during add/load

With this change, the java client now supports the /statements commit parameter for periodic commits during add/load via the AGRepositoryConnection's setUploadCommitPeriod and getUploadCommitPeriod methods.

Rfe10257 - Support script loading in sessions

Added method to AGRepositoryConnection: addSessionLoadScript(String scriptName)

When a dedicated session is created, the named scripts on the AllegroGraph Server will be loaded.

Rfe10256 - Support loadInitFile switch in java

Added method to AGRepositoryConnection: setSessionLoadInitFile(boolean loadInitFile)

When a dedicated session is created, the initFile on the AllegroGraph Server will be loaded if this param is true.

Javadocs on AGRepositoryConnection reorganized to describe sessions.

Rfe10201 - Support adding N-Quads from java

With this change, the java client now supports adding N-Quads data from Sesame and Jena using the RDFFormat AGRDFFormat.NQUADS and the language "NQUADS", respectively.

Rfe10177 - Support getting statements by id

With this change, the java client now supports getting statements by triple id using the AGRepositoryConnection getStatements(String... ids) method. Note that there is currently no method for obtaining a statement's id in the java client, so this api is for experimental use by developers and subject to change.

Rfe10174 - Support adding Turtle from Sesame and Jena

With this change, Turtle formatted data can now be added to a repository using the Sesame and Jena APIs.

Rfe10059 - SPOGI triple cache management

With this change, the SPOGI-cache can now be managed from the java client using AGRepositoryConnection methods enableTripleCache(size), getTripleCacheSize(), and disableTripleCache().

Rfe9436 - Session lifetime management

With this change the lifetime (maximum idle time) of a session can be set using the AGRepositoryConnection#setSessionLifetime method. The default session lifetime (in effect when not set with the method above) can be set with the AGHttpRepoClient.setDefaultSessionLifetime method. When none of these are set, the default session lifetime is 1 hour.

Rfe8362 - Allow geospatial queries against graph/context field

A boolean useContext parameter can be given to /geo/box, /geo/circle, /geo/haversine, and /geo/polygon, to make them query the context field, rather than the object field of the triples.

Bug19906 - Writing a JavaScript NaN value as JSON loops forever

When running, for example, the JavaScript program 'NaN', and asking for a JSON response, the server would never respond, because the float-printing code didn't handle 'special' float values. We now convert NaN and Infinity to strings in JSON output.

Bug19342 - Java LUBM Prolog and SPARQL queries

With this change, Java LUBM Prolog and SPARQL queries are updated to use distinct in appropriate places and located in the test.lubm package as AGLubmProlog and AGLubmSparql; ant targets lubm-prolog and lubm-sparql have been added to facilitate running the queries from the command line.

Welcome to AllegroGraph 4.2

The new (in 4.2) document Release Notes has information about the 4.2 release. The document you are reading simply lists, for the current and for previous releases, user-visible changes (new features, bug fixes, and other modifications) in an abbreviated format. All the 4.2 changes listed here are also listed in the Release Notes but that document does not list changes in earlier releases. We suggest that users start with that document and refer to this one only if they need information about earlier releases.

Welcome to AllegroGraph 4.2. See Release Notes for general release information. The following, organized by category, are user-visible changes in AllegroGraph 4.2.

AllegroGraph Server: General Changes

Rfe10293 - Warn against using ReiserFS.

As ReiserFS causes long delays on checkpoints, AllegroGraph now warns (in the server log file) if a store resides on ReiserFS. To ease debugging, it also logs the file system types of every store directory now.

Rfe10267 - Do not request duplicate removal if there are no duplicates detected

Internal handling of duplicate removal process is slightly improved. This fix also eliminates excessive "Dupes deleted: 0" messages from the agraph.log file.

Rfe10255 - Allow minimum text-index word size of 1

It is now possible to create freetext indices with a minimum word size of 1 character (the minimum used to be 2).

Rfe10216 - Support alternative tlog synchronization methods.

AllegroGraph now supports a new per-catalog configuration option, TlogSyncMethod, which allows users to specify one of three synchronized writing methods for transaction logs: ODIRECT, SYNC, and fsync.

As before, ODIRECT is the default and the recommended choice on ext3 file systems. For catalogs residing on non-ext3 file systems, the other choices may yield performance benefits.

Rfe10191 - Have install-agraph pass its arguments to configure-agraph

You can now do './install-agraph /dir/ --non-interactive ...' to install and configure agraph in a single command.

Rfe10182 - Improve efficiency when waiting for an in-memory chunk

Previously, when waiting for an in-memory chunk to become available, a commit operation would poll once per second. The polling has been replaced with a wait on a semaphore which will result in immediate notification when an in-memory chunk becomes available. This results in lower average times for commits that must wait for in-memory chunks.

Rfe10173 - Upgrade plugins to support TBC 3.4.0-Beta1

Previously the AllegroGraph plugins were only supported on TBC 3.3.x; with this change, the plugins can also be used with TBC 3.4.0-Beta1.

Rfe10167 - Log euid and runas information at daemon startup.

AllegroGraph now logs which user started the AllegroGraph service daemon, which user it was meant to switch to once running, and the currently effective user ID.

Rfe10162 - Perform a clean shutdown on agraph-control stop

Previously, running the agraph script with the stop argument would not verify that open databases were shut down correctly, leading to a forced recovery from transaction logs upon next startup.

Now, running the agraph script with the stop argument will wait for up to 60 seconds for open triple-stores to shut down cleanly.

To get the quicker shutdown behavior, use force-stop argument or kill the Service Daemon process with the QUIT signal.

Rfe10156, Rfe10157 - Improve lock acquisition and store creation on NFS.

Previously, creating a triple-store left open a very narrow window in which another process could clobber the store parameters. Now, the triple-store is locked immediately on creation. Additionally, newly-created triple-stores can now be cleaned up correctly when created on NFS.

Rfe10149 - Support setting a graph for inferred triples in HTTP sessions

Using the syntax [rdfs++# ] it is now possible to specify which graph should be used for the inferred statements in a reasoning store created as an HTTP session.

Rfe10147 - Export 'custom-service' macro from the db.agraph package

This macro was crudely inserted directly into db.agraph.user before. It is now imported there from db.agraph, like all other AllegroGraph functionality.

Rfe10143 - support bulkMode in connections

With this change, connections can now be put into bulkMode. When in bulkMode, data can be added/loaded more quickly, but there is no guarantee of durability in the event of a crash. Added method AGRepositoryConnection#setBulkMode to set the connection's bulkMode (defaults to false), and added method AGRepositoryConnection#isBulkMode to check the mode.

Rfe10141, Rfe10142 - Use UPI-maps more often inProlog queries

Improved the Prolog query optimizer to use UPI-maps in more cases. Also added a new function, qm, that will use a UPI-map whenever possible. The optimizer changes will help existing queries run more quickly; the new functor will allow more hand-tuned query optimizations.

Rfe10117 - implement run-time (dynamic) query index analysis

Query index analysis now comes in two flavors: the existing :static flavor (from rfe10075) and the new :executed flavor. The later provides an accurate account of the index usage because it actually runs the query while monitoring which indices are touched. Because the query is run, this flavor of analysis can be much more time consuming.

Note that the default analysis technique is to use the new :executed flavor. You can use a different technique by using the queryAnalysisTechnique query parameter. You can also specify a timeout using the queryAnalysisTimeout parameter. See Query Analysis.

Rfe10069 - AllegroGraph v4.0.6 introduced a dependency on a patch to update message digest functions. Now, loading the AllegroGraph client will fail early if this patch is missing, and will report an actionable error.

In addition to the check for the SHA1 patch, AG will now report missing patch names to the user, and offer to install these patches specifically.

Rfe10021 - Support Soundex freetext indices

This adds a supported freetext wordfilter 'soundex', which will reduce words to their Soundex equivalent when indexing or searching. Soundex is described in this Wikipedia article.

Rfe9877 - Lazier collection of FTI results

The triple-ids matching a free-text query will no longer all be accumulated at the start of the query, but (as far as possible) lazily, while triples are being fetched. This should improve performance of limited queries with lots of results.

Rfe9861 - Improve method of sending an x-rdftransaction

Previously, sending an application/x-rdftransaction request required escaping to the client's http layer. With this change, the method AGRepositoryConnection#sendRDFTransaction can now be used; this method is useful for bundling add/remove operations into a single request and minimizing round trips to the server. Changes are committed iff the connection is in autoCommit mode. For increased throughput when sending multiple rdftransaction requests, consider using autoCommit false and committing less frequently.

Rfe9746 - Various improvements to agload

• Added the --rapper command line option to tell agload to use the external rapper program when parsing RDF/XML.
• Added :trix to the list of supported input formats
• Added :save as an error strategy to write errors to the file agload.log and continue.
• Added --recursive to process source directories recursively.

Rfe9657 - Unattended execution of configuration script

The configure-agraph script now has --non-interactive mode, where all its parameters are specified on the command line, so that it can be called from a script without human intervention.

Rfe9631, Rfe10028 - db-get-triples improvements, UPI map usage

Improved the performance of AllegroGraph's internal get-triples functors and extended them to use secondary indices (UPI maps) when they are available.

This greatly improves the performance of many standard queries for both SPARQL and Prolog select. For example, the SP2 benchmark for 10,000 authors goes from a total time of 112.58 seconds to 37.91 seconds.

Rfe9622 - support getting the server's version information

With this change, the client can now get the server's version, build date, and revision information using the AGServer methods getVersion(), getBuildDate(), and getRevision(), respectively.

Rfe9517 - Unify AServe and AGraph messages in the AGraph log

Our HTTP server used to output log messages in its own format. We now route these through the AGraph logging system, causing them to follow a unified format.

Rfe9515 - Drop deleted triples from text index during merge.

The freetext index used to never un-index anything it had indexed. It now leaves out triples that have been deleted when merging index chunks, which will prevent unbounded index growth in stores where lots of triples are being deleted.

Rfe9470 - Reduce madvise calls during compressed index lookup

Small performance improvement during load and query.

Rfe9296 - Transactionally safe duplicate triple elimination

There are improvements in the functionality to remove duplicate triples. Removal is done more eagerly and more robustly. When opening a database, you can pass one of the following values as the :delete-duplicates-during-merge argument:

• nil - do not remove duplicates (default)
• :spog - duplicate triples with the same s, p, o and g are deleted.
• :spo - duplicate triples with the same s, p, o are deleted
• t - same as :spog, for backward compatibility.

Rfe8414 - Add encoded-ids to AllegroGraph

Adds support to register and query encoded-ids and to use these to generate sequences of UPIs in a triple-store.

Bug19910: agraph-recover now returns 0 when successful, 1 otherwise

Agraph-recover now correctly returns 0 to shell when successful. (see Point-in-Time Recovery).

Bug19897 - SHM hangs if socket creation fails

Previously, if a socket creation failure occured in the SHM server (which is a critical subprocesses associated with every database instance), database startup would hang. Now the error is immediately reported, allowing AllegroGraph server to otherwise continue operating.

Bug19891 - Fix log level test for log-backtrace and when-debugging.

Error logging no longer ignores the preferred log level for certain messages.

Bug19888 - Prevent logging gratuitous errors on forced shutdown.

AllegroGraph no longer leaves error messages in the server log file when force-terminated with the killall command. (Trying to write such log messages could result in the service daemon trying to communicate with dead processes.

Bug19864 - Some cursors could fail to correctly free resources

There were situations in SPARQL querying where a cursor could fail to correctly release its resources even when it was correctly discarded. This could lead to resource starvation. This is now fixed.

Bug19848 -- Improve behavior of qm when UPI-maps are not available

The qm functor now gracefully drops back to the behavior of the q functor when UPI-maps are not available. Previously, it would signal an error.

Bug19842 - Error message server-pid-and-client-pid-must-match-error

Fix error message when server is running as agraph and client is running as root.

Bug19813 - Fix store parameter merging for configured values.

Opening a triple-store by passing configuration parameters to open-triple-store now correctly overrides those values that are already present in the store's parameters configuration file.

Bug19816 - FTI Merger sometimes produces corrupt chunks

In a database with deleted triples, it was possible for the text-index merger to output a chunk that could not be read, which would bring down the whole FTI subsystem.

Bug19800 - support empty enumerations in owl:oneOf

AllegroGraph's RDFS++ reasoner signaled an error when an owl:oneOf claimed that some class X was one of the empty set. Now it works correctly for this case too.

Bug19791 - Fix checkdb's --port argument.

Previously, the checkdb utility could check triple-stores only if they were served on the default port, 10035. Now, it uses the value from its --port argument to determine the port to connect to.

Bug19786 - Checkdb now exits with 0 on success.

The checkdb utility now follows the unix convention of returning 0 if no problems were detected.

Bug19785 - Convertdb now exits with 0 on success.

The convertdb program now follows the unix convention of returning the status code 0 if there were no errors.

Bug19777 - Replication to multiple secondaries improved

The speed of replication will no longer be determined by the slowest secondary. Each replication job will now proceed as fast as each secondary database can process the data

Bug19766 - Sub-optimal index selection

Previously, in certain circumstances (such as when a range query was being used and one or more of the standard index flavors were not available) the index flavor selection code would exclude some index flavors that it could have used to satisfy a query more efficiently. This has been fixed.

Bug19773 - Dropping an index is not crash recovery proof

Fixed a bug where dropping an index followed by uncleanly shutting down a database would result in an error during restart recovery.

Bug19765 - Detect more cases of string table corruption.

The checkdb program can now detect many more inconsistencies in string tables.

Bug19764 - N-Quad output was incorrect

The output of AllegroGraph's N-Quad serializer when printing single triples was incorrect.

This is now fixed.

Bug19697 - improve performance of part printing

Increased the speed and memory efficiency of part printing.

Bug19657 - Fix race-condition in text-index merger

The merger could sometimes pick the chunks it was to merge from an outdated set of chunks, resulting in chunks being merged multiple times. This would cause duplicate triples to show up in FTI search results.

Bug19600 - direct-upi wire format does not handle nil values

The wire format used to communicate query results to a remote-triple-store broke when given a nil value, though the result of SPARQL queries with an OPTIONAL part may return them. This is now fixed.

Bug19587 - Suboptimal encoding of some characters in shm-queue.

Some characters were being encoded with an extra unnecessary byte. This problem has been corrected.

Bug19578 - Clearnamespaces XMLTransaction tag is broken

There was a bug in the XMLTransaction parser that causes requests containing a tag to fail. This has been rectified.

Bug19550 - Fix reification in rdf/xml parser

Fixed a bug in which certain statements imported from RDF/XML files would not be reified correctly.

Bug19549 - RDF/XML parser loses the trailing /

When importing from RDF/XML sources, AllegroGraph used to rewrite URIs that came without a path, but with a trailing slash and import them without the trailing slash (e.g., "http://franz.com/" would be imported as "http://franz.com"). Now, it imports all URIs verbatim.

Bug19530 - Fix off-by-one error in text-index code.

Adding a 255-character (the maximum) word to a text index would cause searching and merging of index chunks to break, because a too-small buffer was allocated when traversing the index. This is now fixed.

Bug19522 - Prevent spurious warning in some Prolog select queries

If both :limit and :count-only were specified in a Prolog select query, AllegroGraph would signal a spurious warning. This is now corrected.

Bug19517 - improve Lisp API for remote-triple-stores

Previously, if a :url parameter was used, AllegroGraph would ignore other parameters to create-triple-store like :user and :port that may be inconsistent with :url value. AllegroGraph now ensures that the :url and any other parameters agree and merges parameters into the :url if necessary. This means that you can now use constructs like:

(create-triple-store "foo"  
                     :triple-store-class 'remote-triple-store  
                     :url "http://blade4.franz.com"  
                     :catalog "java-tutorial"  
                     :user "test" :password "xyzzy"  
                     :port 11101) 

Bug19515 - Correct precision loss in printing of RDF time and dateTimes

Certain RDF time and dateTime values with fractional seconds could fail to print all of the digits stored for the value. This patch corrects this problem.

Bug19508 - Prevent text-index from leaking files on crashes

When processes crashed 'hard' (hang, or kill -9), the text index would sometimes leave files around that it never cleaned up. It now takes care to (at some point) delete every file it creates.

HTTP Client

Rfe10278 - Alphabetize store and catalog lists in HTTP responses

The lists of repositories and catalogs returned through the HTTP API were ordered more or less randomly. This made finding a repository in a long list in WebView cumbersome. They are now sorted by name.

Rfe10234 - Inefficient limited Prolog queries over HTTP

The HTTP server used to fetch all results, then throw away the extra ones, when executing a Prolog query with a limit argument. It now only fetches needed results.

Rfe10230 - Accept application/x-turtle as a synonym for text/turtle

Some systems are still using the pre-standard MIME type for Turtle documents. The AllegroGraph HTTP server now accepts that as a content-type when uploading triples.

Rfe10220 - Provide an HTTP interface for controlling bulk mode

It used to be only possible to turn on bulk-load mode per load request. This was slow when using many small requests. In a session, it is now possible to persistently turn bulk mode on and off through requests to /session/bulkMode.

Rfe10210 - Don't backtrace on unsupported-db-version error

The HTTP server no longer logs a backtrace and returns a server-error code when trying to open a store with the wrong version. Instead, it returns a PRECONDITION FAILED error.

Rfe10165 - Expose turtle loading over HTTP.

The /statements HTTP service now accepts request bodies in Turtle format, using a Content-Type header of text/turtle. Such files can now also be loaded from WebView.

Rfe10078 - Make HTTP protocol no longer hide inaccessible repositories.

It used to be that HTTP services /repositories and /catalogs hid all repositories and catalogs not accessible by the current user. This was unnecessarily confusing. Now all present objects are returned, with readable and writable flags indicating whether the current user has access to them.

Rfe10009 - Implement Turtle loading

Loading of files in the Turtle (Terse RDF Triple Language) format is now supported.

Rfe9986 - Support Sesame 2.2-style XML transactions

The Sesame project, which is the origin of the XML transaction format our HTTP server supports, have slightly changed the format to be simpler and more flexible. We now support this update, while remaining compatible with the old-style transactions.

Rfe9912 - Remove SharedMemSize parameter from HTTP protocol and HTTP docs

The SharedMemSize parameter was still mentioned in the HTTP protocol docs, and understood by some HTTP services, even though it is no longer used.

Rfe9910 - Reuse sockets for backend-frontend communication

Use HTTP keep-alive to reuse the sockets that the front-end uses to talk to the back-end.

Rfe9838 - Use HTTP keep-alive in the remote Lisp client.

Lisp remote-triple-stores now re-use their sockets when making HTTP requests.

Bug19909 - Inter-server HTTP communication should use local addresses

When the server tried to address itself over HTTP, it would often use the URL provided in the HTTP request that trigerred this communication. If that came from outside the local network, the address in this URL might not be useable. It now uses a local address.

Bug19871 - HTTP server does not unescape \u in resources

In resources passed to the HTTP server either through HTTP parameters, or through JSON data, N-Triples backslash escape sequences were left in the literals, rather than being unescaped. The server now properly handles these.

Bug19846 - Ensure the HTTP client explicitly frees SPARQL cursors

SPARQL queries from the HTTP client did not explicitly discard the cursor they created. This could cause resource exhaustion in situations where many, many queries were operating simultaneously. This is now fixed.

Bug19801 - HTTP character encoding should not depend on system locale

The system's default character encoding was accidentally being used when writing HTTP responses. This worked fine on systems with a UTF-8 locale, but caused the server to return inconsistently encoded responses on other locales.

Bug19661 - HTTP JSON output for query results truncates row on first unbound result.

The code that wrote out SELECT query results in application/json format would stop writing a row at the first null it encountered, leaving that null and any values after it out of the result.

Bug19616 - Allow multiple contexts to be specified in HTTP /size

Contrary to the Sesame docs, we only accepted a single context argument to our /size HTTP service. This has now been fixed -- you can pass multiple context parameters, and the returned value will be the sum of their sizes.

SPARQL

Rfe10207 - Efficiency improvement for some SPARQL graph queries

Improve the way the SPARQL engine handles simple GRAPH clauses with a variable graph argument that does not appear within the rest of the clause. For example:

select ?someGraph ?o  
where {  
 ex:a ex:inGraph ?someGraph .  
 graph ?g { ex:a a ?o . }  
} 

SPARQL semantics dictate that the query act as if the graph clause is checked for every graph in the dataset. In this case, however, it is clear that the value of ?g cannot alter the behavior of the inner query and that the intent is to determine the graph slots of any of the selected triples.

Note that this optimization is currently very conservative and only occurs when the graph clause is a simple BGP. Future work will extend the behavior to more complex queries.

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing

Nested instances of the SPARQL negation-as-failure idiom were not being parsed correctly leading to incorrect query results. This fix corrects the problem.

Rfe10030 - Share statistical planner between Prolog and SPARQL

Use the foundations of the Prolog select statistical planner in SPARQL queries to reorder BGPs. This improves the speed of many SPARQL queries. Note that this planner requires the POSGI index. If this index is not available, then the coverage planner is used instead.

Because the planner is statistical in nature, it will not always choose the best plan. In general, performance is improved (for example, the 10,000-triple SP2 benchmark queries almost double in speed (from 56-seconds down to 33-seconds) but some queries go slightly slower using the new planner. We will continue to enhance the statistics used and the planner's capabilities in future releases.

If your queries are running more slowly than expected, you can tell SPARQL to use the older coverage-based heuristic planner or tne identity planner (i.e., the planner that does no clause rearranging. Planner selection varies by client. See the client SPARQL documentation for more details.

Bug19809 - fti:matchExpression is sometimes treated like fti:match

There was a bug in the SPARQL code that sometimes caused S-espression-style text queries (as per fti:matchExpression) to be parsed as textual queries (fti:match).

Bug19681 - fix SPARQL problem in queries using both FROM and FROM NAMED

A SPARQL query that used FROM and FROM NAMED together to create the dataset could incorrectly omit some of the the graphs mentioned in the FROM clause. This could cause the query to produce the wrong results.

This problem has been corrected.

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing (repeated from Rfe10061 listing)

Nested instances of the SPARQL negation-as-failure idiom were not being parsed correctly leading to incorrect query results. This fix corrects the problem.

AGWebView

Rfe10300 - Allow optimizing of stores from WebView

There is now an optimize store control in the store overview in WebView. This can be used to optimize a store for querying after a bunch of data has been added.

Rfe10280 - Expose scripting in WebView

There is now a 'Scripts' link in the navigation bar, where one can run JavaScript and (with the right permissions) Lisp scripts interactively, as well as store and edit server-side user script files.

Rfe10277 - Improve help text for WebView part input

The examples given now demonstrate that both backslash-escapes (\n, \uHHHH) and unicode characters are allowed.

Rfe10244 - Add JavaScript server-scripting support

It is now possible to send an /eval request with content-type "text/javascript", and have the body be evaluated as JavaScript. Custom services can also be written in JavaScript.

Rfe10236 - Disable query planner selector in WebView for Prolog queries

To make it clear that the query planner dropdown doesn't apply to Prolog queries, it is now greyed out when selecting a non-SPARQL language.

Rfe10185 - Allow the user to specify a limit for queries in WebView

For heavy queries, the default limit (200) imposed by WebView might be too big. There is now a dropdown above the query input that allows one to choose the amount of results that should be retrieved.

Rfe10181 - Leave WebView widgets in the default browser style

This removes the custom CSS styles for WebView inputboxes and buttons, making the the application look a little more native in modern browsers.

Rfe10155 - Allow a query planner to be selected in WebView

Adds a drop-down menu to the WebView query input page where one can select which planner to use.

Rfe10128 - Rework WebView namespace management

Both the store's default namespaces and the user's namespaces are now managed in the same place. This to reduce confusion---people assumed they could use the namespaces they defined for the store in their queries, without first selecting them.

Rfe10126 - Show PIDs in WebView's process list

Under #processes, the Unix PID of each of the processes is now listed after the process name. The /processes HTTP service now returns tuples instead of just names.

Rfe10116 - Show running sessions in WebView

Once you navigated away, the browser history used to be the only way to get back to a session through WebView. There is now a list of active sessions (if any) in the catalog overview page.

Rfe10039 - Support backup and restore through WebView.

There is now (for superusers) the option to create stores from backups, and to back up existing stores, through the WebView interface. (This is not recommended for large stores.)

Bug19880: Resizing the query input field in WebView is broken

You could drag the resize corner widget, but when you released it nothing happened. This is now fixed.

Bug19868 - Webview tries to write to the installation dir

The server would try to cache gzipped JavaScript files in its installation directory, which fails when it is running under a user that doesn't have write access to that directory. It now uses the settings directory (called the Directory to store data and settings by the configuration program, with default name .../data/) for the cache.

Bug19858 - AGWebView accidentally shows Turtle as export format

We don't currently support exporting to Turtle, so this shouldn't be selectable.

Bug19762 - Fix problem in process page in WebView

A bug had snuck into the WebView code that prevented the #processes page from coming up when in a catalog view.

Bug19741 - Add example to context input field help in AGWebView import

It used to say only that it accepted 'N-Triples' syntax.

Bug19523 - Fix several bugs in WebView's query result display.

Prolog queries returning the default graph now display properly. Prolog queries returning non-flat-lists (such as (?x (?y ?z))) now produce an error message rather than no feedback at all. Literals ending in a backslash are now parsed properly, rather than treated as invalid.

Changes to the Lisp API

Rfe10296 - Check minimum build id on client startup.

The AllegroGraph lisp client will now check if the lisp has all the required runtime patches installed, and will present an error and resolution options if that is not the case.

Rfe7148 - Optimize away keyword scanning in get-triples

db.agraph:get-triples now uses a compiler macro to optimize the calls so no runtime CPU time is spent keyword scanning.

In Lisp if you want to trace get-triples calls, you should declare the function notinline. Alternatively, you can trace db.agraph::get-triples-positional, the non-exported internal function, which the compiler macro uses.

Bug19890 - load-rdf/xml-from-string (Lisp API) could cause memory error

If called with a very long string and no explicit base-uri, load-rdf/xml-from-string tried to create a base-uri by querying the file system with the string. In some situations, this could cause a buffer overflow which would lead to abnormal client termination. This commit fixes the base-uri resolution algorithm so that it handles strings without querying the file system. The underlying buffer problem has also been corrected.

Bug19678 - serialize-rdf-n3 fails with encoded UPIs

The Lisp API function serialize-rdf-n3 would fail if it was asked to serialize triples with any encoded UPIs. This is now corrected.

Bug19069 - improve error handling of N-Triples parser

The continue-on-errorp function to load-ntriples can now specify how to behave when an error occurs more precisely. It can return one of

:continue (the default)

to ignore an error and continue

:stop-parsing

to ignore the erorr and stop parsing the rest of the current source

:error

to signal the error in any case

Documentation

Rfe9657 - server-installation.html document changed to describe non-interactive running of configure-allegrograph.

Python Client

Rfe10221 - BulkMode support added to Python client

Bulk Mode is a repository-wide setting that affects all connected clients.

Use the bulk_mode property on a repository object to set and get the bulk_mode state. Here are some examples.

repository.bulk_mode = True print "The bulk mode state is ", repository.bulk_mode

If you only have a connection object, you can access the property with conn.repository.bulk_mode.

Any commits while repository.bulk_mode is true are done so using BulkMode on the server.

Java Client

Rfe10303 - Support revised bulk-load mode interface

With this change, the Java client now supports the revised bulk-load mode interface. Bulk mode is now persistent (sticks even after closing and reopening the store), and is controlled with the AGRepository#setBulkMode and AGRepository#isBulkMode methods, rather than per session.

Bulk-load mode can now only be controlled through a store-wide toggle (/bulkMode over HTTP), since it is a store-wide setting, and turning it on and off a lot is very slow.

Rfe10197 - Support for encodable-ids and generated URIs

With this change, a namespace (viewed as a set of URIs) can now be registered as encodable. URIs in the namespace have local parts that follow a declared format, allowing for efficient encoding and in some cases generation of unique URIs in the namespace. Once registered, a conformant URI in the namespace is auto-encoded when added to the store, resulting in a smaller string table and faster query speed. (This is a Java client version of the encoded-ids feature, see Encoded IDs for more information.)

Rfe10189 - Java Interface for Stored Procedures

The stored proc feature and API are experimental, and subject to change in a future release. For this reason, the new methods are marked as deprecated.

The AllegroGraph server defines a new API for defining Stored Procedures and they are installed like Custom Services.

The primary API is:

AGRepositoryConnection.callStoredProc(functionName, moduleName, args) 

A low-level API is also exposed:

AGHttpRepoClient.callStoredProcEncoded()  
AGSerializer.serializeAndEncode()  
AGDeserializer.decodeAndDeserialize()  

Rfe10119 - Add analyze() method to provide query analysis

Sesame AGQuery subclasses can now call the analyze() method to obtain a query analysis for the query rather than evaluating it.

Rfe9971 - JAVA API for HTTP protocol checkVariables

Related to bug19157: SPARQL error if ORDER variable isn't in query

Add method setCheckVariables(boolean) to both query classes: com.franz.agraph.jena.AGQuery com.franz.agraph.repository.AGQuery

Rfe9858 - Support ResultSet nextSolution() method

Previously, the legacy ResultSet API method nextSolution() would throw an UnsupportedOperationException; with this change the method is now supported.

Rfe9846 - JENA graph unions and graph-scoped reasoning

With this change, the method AGGraphMaker#createUnion can be used to create a union of named graphs. AGInfModels can now be created over models backed by a union graph, and reasoning will be properly scoped to just the union graph; this change allows sound reasoning over a proper subset of data in the store.

Rfe9534 - Add source jar to the client download

With this change, an agraph-[version]-src.jar file is now included in agraph-[version]-client-java.tar.gz downloads. The download's Eclipse project's .classpath is modified to attach the source jar to the agraph-[version].jar to facilitate browsing and debugging.

Welcome to AllegroGraph 4.1.1

Welcome to AllegroGraph 4.1.1. This release of AllegroGraph provides overall product stability and efficiency improvements, with special attention to improvements in AGWebView and documentation.

AllegroGraph Server: General Changes

Rfe9796 - Make tlog-write-checkpoint-indicator use :allocate-temporary

The checkpoint indicator file, which records the location of the last checkpoint record that has been written to the transaction log file, is now written using the new internal API for creating temporary files. This ensures atomic updates on disk.

Rfe8994 - Change behavior when a filesystem is full

A change has been made to the AllegroGraph server that makes the database server block if one of the file systems is found to be full during a write operation. Operations automatically continue once space has been made available by the system administrator.

Rfe9319 - Freetext index configuration and search support

With this change a free text index can be created by specifying its configuration, and free text performed can be performed w.r.t particular freetext indices.

Rfe9534 - Add sources for jena, arq, and json jars

Previously the source code for these 3rd party libs had to be obtained separately; this change includes them as a debugging convenience in the git repository, and sets up the .classpath to reference them in Eclipse.

Rfe9969 - Update clojure api

AGraph Clojure API updated for agraph v4, clojure 1.2. Build support for leiningen 1.3.1, as well as ant.

Clojure and Java source are in the Git repository: http://github.com/franzinc/agraph-java-client Added README.md for use and development instructions.

Clojure and Java jars are available in the Clojars Maven repository: http://clojars.org/groups/com.franz agraph-java-client and openrdf-sesame-onejar When AGraph v4.1 is released, these will be updated and agraph-clj-4.1.jar added.

Changed license to EPL, same as the java code.

Removed agclj.sh in preference for 'lein repl' or 'lein swank'.

Rfe9999 - Auto-adjust MinMergeChunks

If MergeThreshold is configured to a value lower than MinMergeChunks, the value of latter is now automatically adjusted to the value of the former. This ensures that a merge will actually be done when the threshold is reached.

Rfe10119 - Add analyze() method to provide query analysis

Sesame AGQuery subclasses can now call the analyze() method to obtain a query analysis for the query rather than evaluating it.

Bug19658 - Agload needs bin directory symlink

The symbolic link for the agload program was not being created in the 'bin' directory by install-agraph and the RPM installation. This has been fixed.

Bug19674 - After-the-fact index creation should mind the amount of files it opens

Fixed a bug that caused "too many open files" error and terminating the server when large number of chunks had to be merged.

Bug19698 - Changes to prepareGraphQuery and prepareBooleanQuery

prepareGraphQuery and prepareBooleanQuery now also accept the string "SPARQL" or "PROLOG" as a query language parameter. Previously, only prepareTupleQuery allowed the string values as a convenience in place of the QueryLanguage.SPARQL or QueryLanguage.PROLOG objects.

Bug19720 - Fix "don't attach as root" check

The check which ensures that you don't attach to a triple store as "root" was checking the real uid, not the effective uid, preventing it from working properly under su/sudo. This has been fixed.

Bug19739 - Compressed index files not unmapped when closed

Triple index files which were mapped during queries were not being properly unmapped when they became obsolete. This problem has been fixed.

HTTP Client

Rfe9756 - Return 400, not 500, when trying to delete an open store over HTTP.

The HTTP server was returning (and logging) an internal server error when it tried to delete a store that was still being held open (through a DELETE or PUT request to the store's URL). It now returns a 400 (bad request) response.

Rfe9971 - Java api for http protocol checkVariables

Related to bug19157: SPARQL error if ORDER variable isn't in query

Added method setCheckVariables(boolean) to both query classes: com.franz.agraph.jena.AGQuery and com.franz.agraph.repository.AGQuery.

Rfe10120 - Make sure custom-service exists in agraph4.fasl

This adds the HTTP backend code to the agraph fasl, so that it is possible to compile custom services in an image that has this fasl loaded.

SPARQL

Bug19157 - SPARQL error if ORDER variable isn't in query

When the order by clause has a var not known from the query, ignore it (the default), or identify it in an error message (when sparql-check-variable-errors-p or checkVariables in the http protocol is true).

Bug19618 - Properly escape strings in SPARQL/JSON output.

Newlines and other special characters are now escaped according to JSON rules in SPARQL/JSON results returned by the SPARQL engine.

Bug19619 - Fix typed literal writing in SPARQL/JSON output.

The SPARQL engine was not writing out typed literals in SPARQL/JSON documents properly. Those in UPI form got a wrong type property ("literal" rather than "literal-typed"), those in future-part form were written out as XML.

AGWebView

Rfe9492 - Add export to AGWebview

Adds the possibility to download repositories and query result sets through AGWebView. Several output formats are supported, depending on whether triple data (n-triples, n-quads, rdf/xml, trix) or select results (sparql+xml, csv) is downloaded.

Rfe9587 - Show progress indication in WebView uploads

Rather than just seeing a spinner animation, users doing an upload in WebView now see how much data has been uploaded already.

Rfe9898 - Add drop-down menus to WebView's navigation bar.

To make it possible to link more functionality from the navigation bar without running out of space, groups of links are now put into drop-down menus.

Rfe9900 - Optimize loading of WebView scripts and style-sheets.

Serves the JavaScript and CSS files used by WebView in concatenated form, allowing gzipped responses. This causes the base webview page to go from 25 files totalling over a quarter megabyte to 2 files and 68 kilobyte.

Rfe9950 - Provide a way to start exploring a store in WebView

This adds a paragraph to a repository overview containing links to some simple queries that can help the user explore the store.

Rfe9951 - In WebView user management, show drop-downs for stores.

When granting permissions on a store, the user no longer has to type the names, but can select them from a drop-down menu.

Rfe9964 - Ask confirmation before letting user un-superuser himself in WebView

Once your user loses its superuser flag, it can no longer manage users, so one wrong click here could lock yourself out. There is now a confirmation dialog to ensure you intended to do this.

Rfe9990 - Do not display \uHHHH character escapes in WebView

For non-ASCII characters, WebView showed the N-triples escape sequence instead of the character itself, even when 'short' part display was turned on. This is now fixed.

Rfe10025 - More robust recent-query saving in WebView

On modern browsers, you'll now be able to list your recent queries even after navigating away from WebView and back. On top of that, the list of recent queries is no longer tied to a specific repository.

Rfe10026 - WebView now shows a spinning animation when loading a file

Before, there was no proper feedback indicating that something was happening during a dataset upload.

Rfe10027 - In WebView's graph view, fix nodes after dragging

Nodes can be rearranged by dragging in the graph view. A node now stays fixed after having been dragged, making it much easier to lay out the graph.

Rfe10050 - Make specifying a graph when adding from WebView easier.

The graph input field in the add-triple dialog is now always visible. WebView will try to retrieve a list of graphs that are used in this store, and allow you to choose them from a drop-down.

Rfe10051 - Allow a graph/context to be specified when importing in WebView.

This extends the import file and import URL dialogs with a context field, which is used to specify a graph to import into.

Rfe10052 - Add a general delete-triples UI to WebView

This adds a 'delete statments' control to the repository overview page, from which you can delete triples, by wildcard. Also includes the existing-graph select control from rfe10050, to make it easy to drop a graph.

Rfe10127 - Allow init files to be loaded in sessions created from WebView

There is now a checkbox 'load initfile' in the widget that creates new sessions from a catalog view in WebView.

Bug19581 - Newly registered users in WebView don't get enough access

When user-registration is turned on, new users were supposed to get the same access permissions as the anonymous user, but due to a bug they did not get any access. This is now fixed.

Bug19536 - In WebView's graph view, clicking on a node was broken.

Clicking on a node in the graph did not actually take you to that node's overview page, but raised a script error instead. This is now fixed.

Bug19552 - Can't start a session from catalog page in WebView

There was a bug that would prevent users from starting a session from a (non-root) catalog page in WebView. This has been fixed.

Documentation

Rfe9574 - Lisp client documentation

The AllegroGraph Lisp Client Quick-Start guide has been expanded to include explicit Lisp Client set-up instructions and an explanation of local vs. remote triple-store connections.

Rfe9750 - Improve document navigation

The navigation panel (to the right side of most documentation pages) and the document map page have been revised and extended to provide more organized and thorough access to the many documents and web pages that describe AllegroGraph.

Rfe9883 - Javadoc included in the distribution

Javadocs for com.franz.agraph are in the javadoc folder in the distribution file, and they link to external javadocs for java, jena, and sesame.

Rfe9929 - User manual for 4.1.1 AGWebView

The AllegroGraph documentation has been extended to include a User's Guide for WebView. This document presents task-oriented guidance on using the WebView interface for managing users and repositories.

Rfe9936 - Document TopBraid Composer plugin

In this release the TopBraid Composer (TBC) plugin installation notes got an upgrade with numerous screen-shot illustrations and further notes on how to integrate TBC with the AllegroGraph Virtual Machine.

Rfe10046 - AllegroGraph Text Indexes

There is a new, high-level document describing AllegroGraph's free-text indexing features.

Rfe10077 - Add user-selected triple indices to tutorials

The Python, Java, and Jena tutorials have an augmented example1() showing how to add and remove triple indices for greater efficiency.

Rfe10108 - AllegroGraph Triple Indexes

There is a new, high-level document describing AllegroGraph's system of triple indexes and how to manage them.

Rfe10139 - Emphasize "agraph" user vs. AG superuser

The server-installation document now emphasizes the differences between the "agraph" operating system account and the AllegroGraph superuser account. Users sometimes confused the two accounts, which caused connection issues.

Rfe10159 - Rapper documentation

There is a new document describing the use of the Rapper command-line utility for pre-processing incoming data files to make them load more quickly in AllegroGraph.

Bug19620 - Fix typos in documentation of create-freetext-index

The description of the index-fields argument was a jumble of a sentence.

Welcome to AllegroGraph 4.1

Welcome to AllegroGraph 4.1. This release of AllegroGraph provides overall product stability and efficiency improvements, with special attention to backup/restore, point-in-time recovery, and warm standby features.

Rfe9535 - Use managed files throughout the storage layer

The storage layer has been changed so that all files are allocated through the central resource manager mechanism. This makes all files eligible for placement through FilePlacementRule parameters in the database configuration. Previously, some files could not be placed using such rules.

Rfe9772 - More advanced server scripting

This introduces server-side script files, which can be loaded into sessions, and a convenient way to define custom HTTP services from such files. See the HTTP protocol documentation for details.

Rfe9784 - Keep track of which pages of deleted bitmap have been touched

In order to speed up backup operations, the AllegroGraph server now keeps track of which pages in the deleted bitmap file are used.

Rfe9785 - Revise backup and restore

AllegroGraph now uses a new, single-file archive format for storing backups. The new backup and restore functionality is described in the server documentation.

Rfe9789 - Support callback parameter when writing JSON output

Requests returning JSON data can now specify a callback=[x] HTTP parameter, which will cause the response to be wrapped in [x]([response]), so that the resulting document can be loaded as a JavaScript program (which is useful for working around cross-domain restrictions in browsers).

Rfe9798 - Allow relative directories in config files

In the AllegroGraph server configuration file, relative pathnames can now be specified for file and directory parameters. Such pathnames are considered relative to the directory where the server configuration file is located.

Rfe9805 - Point in time recovery

The point in time recovery facilty allows the a database backup to be restored to its state at a particular point in time.

Rfe9835 - Use stream API to allocate blocks in sparse deleted bitmap

The database server code has been updated to explicitly allocate blocks in the deleted bitmap file using file system calls. That way, full filesystem conditions will be handled in a graceful manner.

Rfe9840 - Improve "disk full" behavior in agraph-backup

When agraph-backup detects a "disk full" condition, it now performs an interactive recovery if it was started from a terminal. When started non-interatively (i.e. from cron), no recovery is performed and an error message is generated.

The exit code of agraph-backup is now always 0 if the operation succeeded, 1 if it failed.

Progress reporting in agraph-backup has been improved.

Rfe9842 - Agraph-backup: merge parameters.dat while restoring

When restoring a database from a backup, the parameters established in the parameters.dat file stored in the backup database are copied to the restored database. File placement related parameters are skipped in this process.

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

When a back-end crashed, this often caused two back-traces to be logged, one for the back-end and one for the front-end. The second one was just noise, so it is now suppressed.

Rfe9889 - Allow resource manager to forget a file

A new internal API function has been added to the resource manager library to support the archival of transaction log files.

Rfe10055 - Properly expose the no-commit flag

The no-commit mode, used primarily by the warm standby replication process, is now something that can be explicitly turned on and off.

Rfe10056 - Require authentication for warm-standby connections.

The socket on which a warm-standby server listens used to be accessible by anyone. It now requires a valid username and password when initiating a connection.

Bug19391 - Agraph-install no longer outputs paths with double slashes

The helper script agraph-install would output paths with double slashes when given a path with a trailing slash as argument. This was mostly harmless, and is now fixed.

Bug19420 - Chunk writers leak disk space when db crashes

A problem has been corrected that could lead to stray files being left on disk when a database was brought down uncleanly.

Bug19446 - Catalog parsing problem - confusing error message

The error message that the configuration file parser generates when overlapping directories are specified in the configuration file has been improved.

Bug19487 - Solve ':void external-format' errors in logs.

The AG log files occasionally showed unexpected errors of 'Attempt to translate characters using a :void external-format'. These were harmless, but they are now handled properly, and no longer logged.

Bug19526 - Back up tlog file only up to backup checkpoint

The backup code has been changed so that transactions which executed while the backup was running are not included in the backup.

Bug19684 - Include the triple-id index in the standard indices

If specific indices are not requested when a store is created, then it will have the standard set of seven: :ospgi, :posgi, :spogi, :gospi, :gposi, :gspoi, and :i.

Prior to this change, the :i index was not included in this standard set.

Lisp Client

Bug19442 - Make sure remote-triple-store accepts :server argument.

The :server argument to the remote-triple-store class was being incorrectly rejected, making it impossible to open stores on other machines.

SPARQL

Bug19474 - Improve SPARQL describe queries

• excess variables are no longer included in the output of DESCRIBE

• CBD (Concise Bounded Description) logic is now implemented so that we correctly handle blank nodes

• Many general efficiency improvements

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

AGWebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

This fixes a problem where Internet Explorer 7 would display "Internet Explorer cannot display the webpage" when opening WebView (or any other page served by AG). Internet Explorer fails to handle deflated content, so we now serve it gzipped responses instead.

Rfe9899 - Fix typo and wrong word choice in WebView.

When adding a notable node, 'Desription' was corrected to 'Description'. When looking at a text index that indexes all predicates, it used to say 'None' under predicates, now it says 'All'.

Bug19500 - Fix problem with displaying geospatial results in WebView

When a query result contained geospatial data, a bug in WebView was triggered that caused the result to fail to display. This bug has been fixed.

Bug19525 - Fix failure when opening a federated session in WebView.

Trying to create a federated session in WebView resulted in a PRECONDITION FAILED error. This was caused by a bug in the way session data was being returned by the server, causing WebView to believe it was not on a federated store. The problem has been fixed.

Documentation

Spr37148 - Question about connecting using java

Two lines of code were inconsistent between the Java tutorial HTML file and the tutorial examples code. The tutorial has been edited to correct this issue.

Rfe9783 - AllegroGraph's isolation model

The AllegroGraph Overview document has been augmented with a new section on AllegroGraph's ACID-compliant isolation model.

Rfe9806 - Enhance SPARQL examples in tutorials

In the Python, Java and Jena tutorials, example13() now uses the SPARQL "contruct" to create has-grandchild triples, and then "describes" a grandparent and a grandchild.

Rfe9821 - Make SPARQL examples more robust

Example10() of the Python/Java tutorials has been extended to show examples of SPARQL's FROM, FROM DEFAULT, and FROM NAMED operators, and an example of the UNION operator.

Rfe9831 - Python/Java tutorials, illustrate use of conn.size() with contexts.

Example10() of the Python and Java tutorials has been extended to show examples of the conn.size() method returning the number of triples found in specific subgraphs.

Rfe9915 - Java and Python tutorials demonstrate SPARQL and RDFS++

Example19() of the Java and Python tutorials has been augmented to demonstrate how to use a SPARQL query with RDFS++ reasoning.

Rfe10104 - Point-in-time recovery document

There is a new document describing AllegroGraph's point-in-time recovery feature.

Rfe10105 - Replication and warm-standby document

A new document on triple store replication and warm stand-by has been added to the document set.

Rfe10106 - Update backup/restore document

The backup/restore document has been updated to reflect the new features available in AllegroGraph 4.1.

Rfe10107 - Transaction Log Archiving document

Transaction logs increase without limit unless archived. A new document describes the transaction log archiving feature of AllegroGraph.

Bug19454 - Python tutorial explains how "int" becomes "long"

In example15() of the Python tutorial we show registerDatatypeMapping() requesting a nativetype "int" and AllegroGraph responding with XMLSchema.LONG. This is correct, and the text now explains why.

Document MinimumCheckpointInterval and MaxRecoveryTime parameters

The meaning of these parameters have been documented in the daemon configuration documentation.

Welcome to AllegroGraph 4.0.6a

Welcome to AllegroGraph 4.0.6a. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9837 - Avoid log warning when opening an existing repo

Added method AGCatalog.hasRepository(id) which tests for existance. Added AGCatalog.openRepository(id) which will open but not create. Added AGCatalog.createRepository(id,strict) where the strict option will throw an exception if the repository already exists.

Rfe9980 - getCatalog now works with the root catalog

Added AGCatalog.isRootId(id) to identify id's for the root catalog. Enhanced AGServer.getCatalog(id) to accept root catalog identifiers. Also added AGServer.getCatalog() to return the root catalog.

Rfe9983 - Support the AllegroGraph Lisp Client in ANSI Lisp

Add support to the AllegroGraph Lisp client so that it works in both mlisp and alisp.

Rfe10041 - Implement owl:oneOf

AllegroGraph's RDFS++ reasoner now includes the following inference:

if the class X is owl:oneOf A, B and C  
then each of A, B and C are of type X. 

Rfe10075 - Implement query index use analyzer, version 1

If analyzeIndicesUsed=true is added to a SPARQL HTTP query string, then AllegroGraph will analyze the query to determine which indices it will use. The results are returned as plain text and written to the AllegroGraph log. They will look like

(desired spogi optimal 1 actual spogi optimal 1)  
(desired posgi optimal 6 actual ospgi suboptimal 4) 

Where each line refers to one of the indices required by the query. The line indicates which index flavor the query really wants and which index the query used based on what is available to the store. The additional information indicates whether or not the index choice was optimal, suboptimal or required a full scan. The numbers indicate whether or not additional filtering was required and can be ignored for now.

Bug19650 - Fix problem federating repositories from the root catalog

Previously an exception was thrown when attempting to federate repos from the root catalog; this change addresses that.

Welcome to AllegroGraph 4.0.6

Welcome to AllegroGraph 4.0.6. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9535 - Use managed files throughout the storage layer

The storage layer has been changed so that all files are allocated through the central resource manager mechanism. This makes all files eligible for placement through FilePlacementRule parameters in the database configuration. Previously, some files could not be placed using such rules.

Rfe9620 - Increase limit on number of index chunks

Previously, the maximum number of index chunks was based on the limit on the number of file descriptors (RLIMITNOFILE) which, in a default Linux configuration, resulted in a limit of 100 chunks. Now the maximum is based on the limit on the number of memory mappings (determined by reading /proc/sys/vm/maxmap_count), which usually results in a much higher limit.

If the limit on the number of index chunks is reached, further commit activity will be blocked. Therefore this higher chunk limit will reduce the likelihood of blocked commits.

Rfe9731 - Change transaction log aligment calculation

An internal change was made to the way the alignment for transaction log records is calculated so that opening transaction log files can be defered until records are written.

Rfe9762 - Add someValuesFrom and allValuesFrom restriction reasoning

AllegroGraph's hasValue reasoning has been renamed restriction-reasoning and now covers owl:hasValue, owl:someValuesFrom and owl:allValuesFrom. In particular, AllegroGraph will now infer:

C owl:onProperty P  
C owl:hasValue v  
a rdf:type C  
==> a P v  
 
C owl:onProperty P  
C owl:hasValue c  
a P c  
==> a rdf:type C  
 
C owl:onProperty P  
C owl:someValuesFrom D  
a P b  
a P x  
b rdf:type D  
(and will not infer x rdf:type D)  
==> a rdf:type C  
 
C owl:onProperty P  
C owl:allValuesFrom D  
a type C  
b P x  
==> x type D 

Because the reasoner now implements more than owl:hasValue, it has been renamed the restriction-reasoner. The accessor function has-value-reasoning-enabled has been deprecated and the new accessor is restriction-reasoning-enabled.

The HTTP protocol changes to use restriction instead of hasvalue as the name of the reasoner that supports hasValue reasoning (and the other restrictions).

Rfe9769 - Log merge statistics

In addition to logging how long a merge took, the following additional information is logged: Number of triples read, number of triples written, number of deleted triples expunged, and number of duplicate triples dropped.

Rfe9772 - More advanced server scripting

This introduces server-side script files, which can be loaded into sessions, and a convenient way to define custom HTTP services from such files. See the HTTP protocol documentation for details.

Rfe9776 - Standardize blank node serialization

Use the same blank node syntax in all of AllegroGraph's serializers (N-Triples, RDF/XML, TriX, etc.).

Rfe9789 - Support callback parameter when writing JSON output

Requests returning JSON data can now specify a callback=[x] HTTP parameter, which will cause the response to be wrapped in [x]([response]), so that the resulting document can be loaded as a JavaScript program (which is useful for working around cross-domain restrictions in browsers).

Rfe9803 - Bulk loading support

AllegroGraph now supports a bulk loading mode that allows changing the triple-store without updating the transaction log. Both the HTTP statements service (used to load data into a store) and the agload bulk loader have been enhanced to use this new feature.

Use the bulkMode parameter to specify that you want bulk loading on in the HTTP service

Use the --bulk option to specify that you want bulk loading on when using agload.

Depending on hardware and the actual data, loading with the transaction log turned off is between 10 and 33% faster than loading with it on.

Rfe9812 - Add client API for triple-store index manipulation

Add the following new functions to the Lisp API:

• add-index - add a new index to a store
• drop-index - remove an index from a store
• triple-store-indices - return a list of current indices for a store
• valid-index-flavors - return a list of all valid index flavors

Note that changes made by 'add-index' and 'drop-index' are not made until 'commit-triple-store' is called.

Rfe9889 - Allow resource manager to forget a file

A new internal API function has been added to the resource manager library to support the archival of transaction log files.

Rfe9907 - Add with-indices support to agload

agload now supports custom index selection using the --with-indices command-line option.

Rfe9913 - Improve single-float printing

AllegroGraph could lose precision when printing single float values. For example, 0.0018333651 would print as 1.833365E-3 rather that 1.8333651E-3. This has been corrected.

Rfe9930 - Support adding, dropping, listing repository indices

Added methods addIndex, dropIndex, listIndices, and listValidIndices to AGRepositoryConnection to support managing repository indices.

Rfe9953 - Share transaction log buffer across attachees

Each process (backend) that attached to a store used to allocate its own in-memory buffer for writing to the transaction log.

A single buffer is now allocated in shared memory and used across all backends, saving valuable RAM.

Rfe9966 - Front-end server-socket backlog parameter is too low

The kernel will only save a certain amount of pending connection attempts. If the server doesn't accept them quickly enough, they get discarded. The default backlog size was too small. It is now set to use the maximum the operating system allows, making the chance of discarded connections much, much lower.

Rfe9970 - Upgrade path for configurable indices

Code has been added to convert pre-v4.0.6 databases to be useable with v4.0.6 in terms of configurable indices.

Rfe9972 - Updates to database upgrade code

The internal API for database upgrades has been improved to make more information about the upgraded database available to the code that performs (part of) the upgrade.

Rfe9992 - Backup/restore improvements

agraph-backup now accepts a --catalog command line parameter. Also, agraph-backup now only backs up the used portions of the string table files, resulting in faster backup and restore times.

Rfe10012 - Consistency checker for sstab

A utility has been added to the AllegroGraph distribution that validates the contents of the on-disk string table data structures.

To validate the integrity of a database, the new "checkdb" utility should be used.

Rfe10019 - Setup automatic datatype-mapping for triple-ids

There is now an automatic datatype mapping between triple-id UPIs and the datatype . This means that these UPIs will be be read and written in this format. For example, the triple

ex:test rdf:refersTo "18"^^<http://franz.com/ns/allegrograph/4.0/triple-id> . 

will be imported such that its object is an encoded UPI that refers to triple ID 18 using the AllegroGraph specific triple-ID reification.

Bug19171 - Importing RDF files with long text literals is much faster.

AllegroGraph now uses a string output stream to collect characters in text literals instead of string+'ing them together. This speeds up importing long literals with many XML escapes. Files that previously took 4 hours to import now take ~2 seconds.

Bug19181 - Correct newline parsing for n-triple imports

Previously, the AllegroGraph n-triple format parser would count newline and carriage return characters as white space, allowing incorrectly-formatted triples to be imported. Now AllegroGraph allows a triple to be specified on one line only.

Due to this change, the AllegroGraph n-triple parser now reports the correct line number when a parse error occurs. This line number can now be used in a text editor to jump to the location of the problem.

Bug19253 - Accept resources for :base-uri in load-rdf/xml.

Previously, the RDF parser disregarded resources used as :base-uri in the RDF import. Now, it accepts all resources and other future-parts and uses their value to determine the base URI for the import.

Bug19266 - Correct cursor re-use problem

AllegroGraph was aggressively re-using storage layer cursors which could lead clients to point at cursors that were no longer valid. This could lead to incorrect query results.

Bug19391 - Agraph-install no longer outputs paths with double slashes

The helper script agraph-install would output paths with double slashes when given a path with a trailing slash as argument. This was mostly harmless, and is now fixed.

Bug19407 - Allow the direct Lisp client to log to the server log file

The direct client (which is used internally in the HTTP server) can now write to the server log file directly.

Bug19420 - Chunk writers leak disk space when db crashes

A problem has been corrected that could lead to stray files being left on disk when a database was brought down uncleanly.

Bug19442 - Make sure remote-triple-store accepts :server argument.

The :server argument to the remote-triple-store class was being incorrectly rejected, making it impossible to open stores on other machines.

Bug19446 - Catalog parsing problem - confusing error message

The error message that the configuration file parser generates when overlapping directories are specified in the configuration file has been improved.

Bug19460 - Issue installing agraph 4.0.5 over 4.0.4

A bug has been fixed that resulted in errors when trying to open a database that was created with AllegroGraph v4.0.4. This problem has been resolved.

Bug19569 - Java client now helps delete orphaned saved queries

Preparing a Sesame AGQuery currently has the side effect of saving the prepared query on the server in memory for the duration of the session. A client app that prepares many queries during a session thus uses up server memory. If an AGQuery instance goes out of scope, the saved query is no longer needed on the server and is currently only deleted when the session expires. With this change, the client library no longer uses the saved query service during a prepare query, pending further cost-benefit analysis of the saved query service. Client code can explicitly request that a query be saved using an experimental AGQuery.setSaveName() method. The client library also tries to help delete orphaned saved queries sooner by tracking when AGQuery instances are garbage collected on the client and issuing delete requests for the orphaned saved queries in subsequent query requests.

Bug19474 - Improve SPARQL describe queries

• Excess variables are no longer included in the output of DESCRIBE

• CBD (Concise Bounded Description) logic is now implemented so that we correctly handle blank nodes

• Many general efficiency improvements

Bug19487 - Solve ':void external-format' errors in logs.

The AG log files occasionally showed unexpected errors of 'Attempt to translate characters using a :void external-format'. These were harmless, but they are now handled properly, and no longer logged.

Bug19489 - Directory where pidfile resides is chown'd to agraph user

In prior releases, when running as root, the agraph-control script would chown the directory where the pidfile resides to the agraph user. This was okay when the recommended pidfile path of /var/run/agraph/agraph.pid was used, but if another path that was shared with non-agraph files was used, the chown was undesirable. This problem has been fixed.

Bug19497 - Fix convertdb

As AllegroGraph v4.0.6 changes the on-disk database format, existing databases need to be converted. The conversion process is performed by the "convertdb" utility, which has been brought up to date with respect to recent storage layer changes.

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

Bug19509 - Agload could load inconsistently on virtual machines

Some virtual machines could reveal a fault in agload that caused inconsistent triple loading. This could result in agload reporting a negative triples-per-second load rate and in different amounts of triples being loaded on different runs. This patch corrects the problem.

Bug19539 - Support dynamic catalogs

Support for dynamic catalogs as documented in the AllegroGraph manual has been restored.

Bug19548 - Correctly import default namespaces in rdf/xml literals.

Previously, when importing XML literals that had a default namespace set, the RDF/XML import in AllegroGraph would silently drop the default namespace. Now, the correct xmlns clause is set for these xml literals.

Bug19551 - Fix handling of blank nodes in RDF/XML parser.

Previously, the RDF/XML parser failed to import empty xml elements like by creating references to empty string literals instead of blank nodes like the RDF/XML syntax specification mandates. Now, the RDF/XML parser correctly creates blank nodes.

Bug19560 - Provide sstab upsizing upgrade path

v4.0.5b increased the sizes of the string table files without changing the database version number. Databases created with v4.0.5a or earlier must be upgraded using the convertdb utility to have this problem corrected.

Bug19562 - Improve error message when AllegroGraph loaded on an unsupported platform

AllegroGraph is currently only supported on linux-64 but the error message produced when trying to load agraph4.fasl on a non-supported platform was nonsensical. This patch improves the error message to make it clear what has gone wrong and what remedies to take.

Bug19565 - Crash can occur when file descriptor limit is large

When running as a user with a large file descriptor limit (larger than 1024), it was possible for AllegroGraph to crash under certain circumstances. This problem has been corrected.

Bug19567 - Log handles remain open after SIGHUP

Due to a mistake in the way the daemon detached from the terminal, the initial log file would never be closed, even when a HUP signal was sent to the daemon. This is now fixed.

Bug19595 - Repair createGraph() for anonymous graphs

Previously createGraph() in the no-arg case was trying to use an illegal blank node id based on a Jena-internal AnonID; this fixes that problem.

Bug19602 - String table fails to recover short strings from transaction log

An error has been corrected that could have resulted in the loss of string table data if a database was uncleanly shut down.

Bug19614 - Bus error during prolog query

Some queries that returned AllegroGraph specific UPI types (e.g., for reified triple-ids or default-graphs) could cause a bus error rather than signaling the correct message that these types cannot be serialized using RDF/XML. Corrected.

Bug19632 - N-Quads serializer was emitting the default-graph incorrectly

The N-Quads serializer was needlessly including the graph even for triples in the default-graph of the triple-store. Furthermore, it was emitting these graphs using invalid syntax. Both of these issues are fixed.

HTTP Interface

Rfe9885 - Fix HTTP connection keep-alive for 204 responses.

HTTP requests that caused a 204 (no content) response incorrectly had their connection closed by the server, even when they requested keep-alive. This meant the client had to create a new socket for their next request, causing unnecessary overhead.

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

When a back-end crashed, this often caused two back-traces to be logged, one for the back-end and one for the front-end. The second one was just noise, so it is now suppressed.

Rfe9906 - Support index inspection and manipulation over HTTP.

Under /indices, a triple store's indices can now be inspected, created, and dropped over HTTP. Adds an index argument to store-creation requests, to set the indices at creation time.

Bug19506 - Make HTTP server more cautious about closing stores

It was possible for the HTTP server to close a triple store that it was still reading from. This would cause the reading request to fail. The server now waits until pending requests have finished when closing one of its stores.

Documentation

Rfe9936 - User document for Top Braid Composer and AG

Added an installation page for the AllegroGraph TopBraid Composer plugin.

Lisp Client

Bug19452 - Race condition in lisp client startup

While connecting to a database, the Lisp client could sometimes crash with the following error message:

The value of db.agraph.spawn::hub is nil,  
which is not of type db.agraph.spawn::hub 

A fix has been developed to prevent that problem from occuring.

Python Client

Rfe9906 - Add support for index changes to the Python client

A connection object in the Python client now has listIndices, addIndex, and dropIndex methods for inspecting and manipulating the indices on a triple store.

Rfe9965 - Python miniclient now supports retry on ECONNRESET

The python miniclient now retries on connection failures with errno ECONNRESET. Previously it would just fail the request.

Java Client

Rfe9845 - Support owl:Restriction reasoning in the Java client

The Java client now supports hasValue, someValuesFrom, and allValuesFrom reasoning. In the Sesame AGQuery API one can now specify which entailment regime to use when including inferences in query results; the new Restriction regime is now available as an option, while RDFS++ remains the default. Similarly, using the Jena AGReasoner API, one can set up AGInfModels with either a Restriction reasoner or the default RDFS++ reasoner.

SPARQL

Bug19474 - Improve SPARQL describe queries

• Excess variables are no longer included in the output of DESCRIBE

• CBD (Concise Bounded Description) logic is now implemented so that we correctly handle blank nodes

• Many general efficiency improvements

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

WebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

This fixes a problem where Internet Explorer 7 would display "Internet Explorer cannot display the webpage" when opening WebView (or any other page served by AG). Internet Explorer fails to handle deflated content, so we now serve it gzipped responses instead.

Bug19500 - Fix problem with displaying geospatial results in WebView

When a query result contained geospatial data, a bug in WebView was triggered that caused the result to fail to display. This bug has been fixed.

Bug19525 - Fix failure when opening a federated session in WebView.

Trying to create a federated session in WebView resulted in a PRECONDITION FAILED error. This was caused by a bug in the way session data was being returned by the server, causing WebView to believe it was not on a federated store. The problem has been fixed.

Welcome to AllegroGraph 4.0.5d

Welcome to AllegroGraph 4.0.5d. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9966 - Front-end server-socket backlog parameter is too low

The kernel will only save a certain amount of pending connection attempts. If the server doesn't accept them quickly enough, they get discarded. The default backlog size was too small. It is now set to use the maximum the operating system allows, making the chance of discarded connections much, much lower.

Welcome to AllegroGraph 4.0.5c

Welcome to AllegroGraph 4.0.5c. This release of AllegroGraph v4.0 provides efficiency improvements in the area of reasoning.

AllegroGraph Server: General Changes

Rfe9887 - Exclude inferences from Sesame queries by default

Sesame queries (Tuple, Graph, and Boolean) will now be set to exclude inferences by default (this applies when the setIncludeInferred method has not been called on the query instance).

Bug19484 - Jena always uses reasoning

Fixed so reasoning is off by default and is enabled when InfModel is used.

Bug19485 - setAutoCommit(true) causes exception

True is the default, but calling this would raise an exception, so this has been fixed.

Bug19491 - Jena reasoning not on full graph

In the Jena API, AGInfModel, reasoning was previously done on graphs (default or named), and is now done on the full repository. Reasoning on scoped graphs will be included in a future release.

Welcome to AllegroGraph 4.0.5b

Welcome to AllegroGraph 4.0.5b. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9866 - Increase limit on string file sizes

Previously, the files that held string data were limited to 32GB each. This release expands that limit to 512GB each.

Bug19503 - Reclaim cursor resources more aggressively

Certain SPARQL operations could acquire cursor resources and not release them immediately. Depending on garbage collection and server load, this could eventaully hang the server.

This patch ensures that SPARQL releases cursors immediately during triple-existence checks which prevents the resource issues from arising.

Welcome to AllegroGraph 4.0.5

Welcome to AllegroGraph 4.0.5. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9334 - License support for agraph with triple-limit and expiration

AG now requires a license code which includes a limit on the number of triples and an expiration date, both of which may be unlimited.

If the license code is missing, invalid, or expired, it defaults to Free version, 50 million triples with no expiration.

Rfe9348 - Added offset keyword argument to getStatements/evalFreeTextSearch

The getStatements and evalFreeTextSearch methods on RepositoryConnection objects now support an optional offset keyword argument for skipping a number of results before returning values from the result set.

Rfe9471 - Limit maximum checkpoint frequency by configuration variable

A new configuration variable, MinimumCheckpointInterval, has been added that can be set to modify the frequency at which checkpoints are written by AllegroGraph. It defaults to the the CheckpointInterval period or five minutes if interval based checkpointing is not used.

The SkipPostMergeCheckpoint configuration variable has been removed.

Rfe9564 - Alternate mode for checkpoint triggering

In addition to triggering checkpoints in regular time interval, AllegroGraph now offers an alternate mode for triggering checkpoints based on the expected recovery time. It is enabled by the MaxRecoveryTime catalog/database parameters and needs to be set to the maximum number of milliseconds that AllegroGraph should spend recovering transactions from the transaction logfile during recovery. When AllegroGraph detects that the established maximum recovery time would be exceeded, a checkpoint is written. Note that the maximum recovery time period does not include the flat startup fee for a database that is independent of the number of transaction to recover from the transaction log file.

Rfe9604 - Suppress unnecessary checkpoints

In previous releases, AllegroGraph automatically created checkpoints when a database was started, stopped or a certain time interval had elapsed. This has been changed so that checkpoints are only written in these situations if there had been any commits since the last checkpoint has been created.

Rfe9608 - Have HTTP /eval service handle multiple returned values

Extra values returned by evaluated forms used to be discarded. The server now attempts to return them as a list of values, if the Accept format supports such a thing.

Rfe9691 - Remove race condition in internal API

In order to support a new use case, a race condition has been removed from the internal resource manager API.

Rfe9709 - Improved automated test for database recovery

Automated test coverage for database recovery has been improved.

Rfe9713 - Document load-nquads

Added the documentation for load-nquads and load-nquads-from-string to the Lisp Reference Guide.

Rfe9726 - Minor agload improvements

• improve agload output so that it is easier to understand,
• correct agload failure to load the TriX format,
• expose support for N-Quad format, and
• other minor internal improvements

Rfe9729: Make it possible to disable interval based checkpointing

Interval based checkpointing can now be disabled by setting the CheckpointInterval database parameter to zero.

Rfe9732 - Disable interval based checkpointing when recovery time based checkpointing is enabled

When recovery time based checkpointing is enabled, AG now switches off interval based checkpointing by default.

Rfe9766 - Ensure that the AllegroGraph server and the Lisp client use the same UID

Because the Lisp client must have full read/write access to a database's files and shared memory data structures, it must run under the same UID as the AllegroGraph server. This change enforces this requirement to avoid potentially confusing errors.

Bug19101 - with-variables->bindings-array is too strict

AllegroGraph's SPARQL engine raises an error when imposed bindings (the :with-variables keyword argument to run-sparql) are not a subset of the variables mentioned in the body of the query.

This error checking failed to include variables mentioned in a CONSTRUCT template or DESCRIBE variable list, which meant that query invocations which imposed bindings for variables mentioned in the CONSTRUCT template alone would raise an error.

This is now corrected.

Bug19168, Bug19332 - Ensure that user functions are compiled correctly

This patch improves the way that functions generated at run-time are compiled. This includes SNA generators and select query plans. These should all perform slightly better with this change.

Bug19209 - Fix bug with reasoner and rollback

In certain situations, the RDFS reasoner could become confused and fail to update its data caches after the triple-store was rolled-back. This is corrected.

Bug19339 - Update UPI serialization

There was a printer escape problem in the N-Triple serializer such that only one backslash was output when two should have been. I.e., \a instead of \a. This prevented serialized triples from being re-imported into AllegroGraph. This problem has been corrected.

Bug19429 - Problems with catalog option FilePlacementRule in agraph.cfg

A defect has been fixed that resulted in an error when a database created from a catalog that specified a FilePlacementRule was opened.

Bug19431 - Wrong parameters logged during database startup

A defect has been fixed that made the AllegroGraph server potentially log wrong parameters to the server log file during database startup.

Changes to the Lisp API

Rfe9643 - Remove synchronize-automatically from the Lisp API

In AllegroGraph 3, synchronize-automatically controlled whether or not a triple-store was immediately synchronized after the addition of new triples. Because this level of control is unnecessary in AllegroGraph 4, the variable is being removed.

Bug19188 - [Lisp API] create-triple-store :if-exists :error problem

The Lisp API function create-triple-store behaved incorrectly if the :if-exists keyword argument was :error. An error would occur whether or not the triple-store already existed. Fixed.

Documentation

Rfe9475 - Document catalog parameters vs database parameters

The AllegroGraph server configuration documentation has been updated to include information on how existing databases can be reconfigured.

Rfe9721 - Restore Lisp freetext tutorial

Adds a tutorial that describes the way freetext indexing works in version 4 of AllegroGraph.

Bug19411 - Configuration and performance documentation updated

The server documentation has been updated to reflect the removal of the 'Aux' catalog directive and the addition of the StringTableDir and FilePlacementRule directives.

Java Client

Rfe9743 - Upgrade to Sesame 2.3.1

Previously the Java client was using Sesame 2.2.4; this upgrades the client to the latest version of Sesame 2.

Jena Client

Rfe9744 - Upgrade to Jena 2.6.2

Previously the Java client was using Jena 2.6.0; this change upgrades the client to the latest version of Jena.

Welcome to AllegroGraph 4.0.4

Welcome to AllegroGraph 4.0.4. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe8792 - Clean up partially created triple stores

When creating a triple store fails, the AllegroGraph server now deletes any files and directories that it may have created up to the point where the error occured. This makes it easier to retry creating the triple store once the condition that made creation fail has been removed.

Rfe9123 - Extend test functions to start and stop db server

The AllegroGraph server can now be stopped by providing the --stop-server switch to the agraph executable. The --config or --pid-file parameters must be present on the command line.

Rfe9292 - AllegroGraph Loader

Added the AllegroGraph Loader tool for parallel loading of large files.

Rfe9367 - AllegroGraph log should include the config file

A change has been made to AllegroGraph so that configuration file directives are written to the server log file at "info" log level during configuration file parsing.

Rfe9451 - Pause text-index merges during checkpoints

We make sure the text-index merger is paused when a checkpoint is taking place, to prevent resource contention from making the checkpoint take long, which would make other subsystems wait.

Rfe9497 - Ensure that select queries release temporary indices

Previously, select queries that used temporary indices (UPI maps) would not free them when the query completed. This could result in poor memory management and eventual poor system responsiveness. This is now corrected because queries release their temporary indices when they complete.

Rfe9508 - Add configuration options for manual control of file placement

Two new configuration options, StringTableDir and FilePlacementRule, have been added to the catalog and database configuration. They can be used to control the placement of individual database files.

Rfe9518 - SHM server handles client failures

Increased robustness of the background shared memory server so that it won't crash when a client shuts down prematurely while communicating with the server.

Rfe9579 - Include catalog name in conditions signaled by the storage layer

Error messages generated by the storage layer now include the catalog name in addition to the name of the database that was being acted upon when the error occured.

Rfe9580 - Use less strict permissions on UPI map and transaction log files

In prior releases, UPI map and transaction log files were created with mode 0600, which prevented them from being read by other users. This made is difficult to share databases. Now, UPI map and transaction log files are created with mode 0666 and the umask is used to determine the final file permissions. This keeps the permissions on these two types of files in line with the other database files.

Rfe9602 - Optimize various UPI coercion functions

Optimized some of the code paths that AllegroGraph uses to convert strings and future-parts into UPIs. This will provide a small performance benefit (1-2%) in some aspects of triple-store loading and querying.

Rfe9642 - Transaction logging adjustments

The recovery code has been adjusted so that automatic recovery after a machine crash is possible if the crash happened while a record was written to the transaction log.

Rfe9642 - Transaction logging adjustments for warm standby

The transaction log now includes marker records for the beginning of a commit that is used by the warm standby mechanism. Existing databases must be upgraded to accomodate for the new transaction log format.

Rfe9654 - Remove Aux configuration parameter

The Aux configuration parameter that was used in previous versions of Allegrograph to make a database use multiple disks for its files has been removed. In order to make use of multiple disks, the TransactionLogDir, StringTableDir and FilePlacementRule should now be used.

Rfe9710 - Make warning about uploading big files through WebView clearer.

Since browsers aren't very good at uploading multi-gigabyte files, WebView makes it clear that there are better alternatives for doing that.

Bug19199 - Lock database directories

When an Allegrograph server opens a database, it will now lock the database directory so that if a second server is configured to use the same directory, it cannot open the database. This protects the database from corruption which would be the result of two servers opening it at the same time.

Bug19203 - Restore the :use-planner option of the select query planner

The :use-planner option of the select query planner was being ignored. It now functions correctly.

Bug19221 - Durability issue related to file deletion

A problem has been identified that could result in database files to be wrongly deleted during checkpoint processing, resulting in database corruption. The problem has been fixed.

Bug19251 - Spurious warning message about nonexistent file ctl0

During database creation, the server logged a spurious warning message about a file "ctl0" that could not be deleted. This has been fixed.

Bug19256 - Database recovery fails with partially written checkpoint

A problem was identified that could cause database recovery to fail when the transaction log contained an incomplete checkpoint record. This problem has been corrected.

Bug19263 - Server log contains cut-off lines

Due to a programming error, it could happen that lines in the server log file were garbled or cut off. This program has been corrected.

Bug19264 - Opening databases is slower than it needs to be

A bug has been found that caused opening databases to be much slower than needed. The reason was that the "df" command was invoked by the database server for every file instead of once per directory. This has been corrected.

Bug19279 - Deleted triple data structure maintenance only happens once

After each merge (but only if 15 minutes has elapsed), a task runs which performs maintenance on the data structures which track deleted triples. Due to a logic bug, this operation was only happening once, even if subsequent merges occurred. This has been corrected.

Bug19282 - Improve parsing of XSD single and double floats

AllegroGraph was parsing XSD doubles using a single float parser which led to some values being incorrectly converted to infinity. AllegroGraph was also treating numbers outside the range of IEEE floating point representations inconsistently. It now always converts numbers smaller than the minimum representable value for a float to zero and numbers larger than the representable value for a float to positive or negative infinity (as appropriate).

Bug19303 - Improve algorithm used to find sort-upi-map program

In some cases, AllegroGraph was unable to find an external program needed for UPI map creation. This patch fixes the problem.

Bug19308 - Make namespaces available to code evaluated server-side.

Code run through the eval-in-server service is now given access to the namespaces that the user defined. It used to run in an environment without namespaces.

Bug19328 - Make remote stores raise an error when created while open

It is now an error to call create-triple-store on a store that is still being held open as a remote-triple-store (in Lisp).

Changes to the Lisp API

Bug19298 - Improve the printing of future-parts

(Lisp only) Future-parts were not printing properly when print-circle was bound to true. In addition, future-parts like ! were not printing correctly (the angle brackets were left off). This patch corrects both of these problems.

Documentation

Rfe9339 - Improve 3-to-4 code conversion document

The Java and Jena sections of the 3.x to 4.x code conversion document have been greatly expanded.

Rfe9549 - Many improvements in the Lisp documentation

Improved and clarified many minor issues in the Lisp reference guide.

Rfe9690 - Documentation for AllegroGraph Loader

A documentation page for the AllegroGraph Loader has been added to AllegroGraph 4.0.4.

Rfe9711 - More detail in Java client installation instructions

The instructions for importing the AllegroGraph Java client into Eclipse have been augmented with more detail to make the process easier for new users.

WWW495 - Add LUBM Benchmarks to documentation navigation list

A link to the web page on AllegroGraph's performance on the Lehigh University benchmark (LUBM) has been added to the navigation bar of the Franz documentation pages.

Text Indexing

Rfe8932 - Allow sorting of freetext-searches by relevance.

freetext-get-triples and the HTTP /freetext service now support an argument that will cause the results to come back in sorted order of relevance, using a simple weighting scheme that takes into account word frequencies in the document and commonness of words across documents.

Rfe9356 - Add fuzzy text-index searching.

Freetext search expressions can now contain fuzzy terms, written as (fuzzy "term" [max-distance]) in s-expression queries, and term~ or term~2 in textual queries, which will match anything with a Levenshtein distance less than max-distance (or 1/5 word length when no explicit distance is given).

WebView

Rfe9504 - Improve handling of WebView file uploads to use less memory.

File uploads through WebView went through a few too many memory buffers, causing them to fail on systems without a lot of free memory. They are now directly streamed, and never stored in memory in their entirety.

Welcome to AllegroGraph 4.0.3

Welcome to AllegroGraph 4.0.3. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe8999 - Use full blank-node IDs in RDF/XML output.

The nodeID properties found in RDF/XML documents created by AllegroGraph can now be sent back to the server to identify the blank node they refer to. They used to be shorter IDs, which were meaningless outside of the document.

Rfe9363 - Revise use of fsync and fdatasync and move to checkpoint time

In previous versions, AllegroGraph used the fsync() system call in various places to synchronize the on-disk state of database files. This could result in unexpected I/O activity when databases resided on ext3 file systems, as fsync() on ext3 always synchronizes all files, not just the file that has been requested to be synchronized. To improve this, AllegroGraph now syncs files only at checkpoint time.

Rfe9490 - increase application's initial heap size

Increase AllegroGraph's initial heap size to reduce garbage collection frequency and thereby improve application performance.

Rfe9505 - Improve efficiency of string dictionary lookup

Increased the efficiency of string lookup by roughly 20%.

Rfe9541 - Server installation script avoids writing double-slashes to agraph.cfg

Previously in configure-agraph, if the user gave a directory name with a trailing slash, the cfg file would be written with double-slashes in various dir names.

Now, the trailing slash is removed to avoid double-slashes. A double-slash is legal in Unix, but is visually unappealing.

Rfe9560 - Truncated cursors free resources more aggressively

Truncated cursors now clean up their environment more quickly which improves general AllegroGraph resource management.

Rfe9567 - Don't print a backtrace on socket timeout errors.

The server no longer logs a backtrace when a socket timeout is raised.

Rfe9586 - Prevent unneeded file reads in metadata-rollback

This provides an increase in efficiency when rolling back a triple store. The metadata system was re-reading its status file on rollback, even when it didn't have to.

Rfe9593 - Optimize string-dictionary

Optimized and stream-lined AllegroGraph's string-dictionary code. This provides a 5 to 6-percent speed increase in many string related operations.

Rfe9598 - Improve efficiency of string table intern operations.

Previously when strings were interned, more work than necessary was being performed if an entry already existed for a string. This extra work is now avoided.

Rfe9665 - Adjusted merge invocation algorithm

Changes have been made to the algorithm which determines when index merges should occur. These changes should make merge behavior more predictable and efficient.

Rfe9673 - Make sort-upi-map operation more verbose

In order to aid in tracking down a hard-to-reproduce bug, the sort-upi-map program now logs information about which files it is manipulating.

Bug19041 - Delete-triple-store only deletes files from the main directory (not the aux)

Previously, if a database used Main and Aux directories and was deleted using delete-triple-store, the files in the Aux directories were not deleted. This has been corrected.

Bug19085 - Fixed race condition in delete-list.dat handling

A file named "delete-list.dat" is used to maintain a list of which files should deleted at the next checkpoint. There was a race condition in the handling of this file which could result in it being deleted at an inopportune time. This problem has been corrected.

Bug19178 - Fix race condition in metadata system

In some circumstances, opening a store or rolling back could cause the system to try to read a non-existent file. This is now solved.

Bug19182 - Performance regression with concurrent writes.

A performance problem has been identified in the mechanism that Allegrograph uses to translate from internally-used logical file names to real on-disk path names. Previously, such lookups were made by a centralized process using a remote procedure call. Now, lookups are solely made within the requesting process without requiring an inter-process exchange.

Bug19192 - Merger may not notice that the database is shutting down

Previously, under certain circumstances, the merger may fail to shut down in an orderly manner during an otherwise normal database shutdown. This problem has been fixed.

Bug19189 - Prevent HTTP back-ends from holding their lock too long

This bug was most visible when using cursors in remote-triple-stores -- the client had to read out the whole cursor before it could make other requests. This is now fixed.

Bug19201 - Catch and log errors raised in the process-services thread.

Errors raised within the per-process thread that managed, for example, back-trace requests were not properly logged, but caused the debugger to be entered. This is now fixed.

Bug19206 - Unrecoverable database due to checkpoint file id reuse

A problem has been identified that could result in an unrecoverable database in situations where there have been unclean shutdowns during recovery. This problem has been fixed.

Bug19222 - Do not fsync temporary files during checkpoint processing

Previously, AllegroGraph called fsync for all files, including temporary files, that make up a database. This has been changed so that temporary files are not fsynced.

Bug19223 - Recovery hangs

A problem has been identified that could cause AllegroGraph to hang during database recovery afer an unclean shutdown. This problem has been fixed by starting the merger processes earlier so that they are available during the recovery process.

Bug19236, Bug19237 - improve geospatial subtype mapping support

The geospatial sub-type mapping code was occasionally being called incorrectly which could lead to spurious errors during query. In addition, the error message reported when a geospaital subtype was not found was being displayed incorrectly. Both of these problems are now corrected.

Bug19244 - Delete old deleted-shmem-files

It has been discovered that the deleted-shmem-files, which are used by the triple deletion functionality, have never been deleted when they became obsolete. This wasted disk space with stale data. The problem has been fixed.

Bug19257 - Deleted hash file deleted at the wrong point in time

A problem has been identified that could lead to the deletion of a deleted hash table file while it was still being referenced. This has been fixed.

Bug19276 - Possible unnecessary post-merge full scan of indexes

After an index merge, a task sometimes runs to perform maintenance of the data structures used to track deleted triples. This operation requires a full scan of indexes. Previously, this full scan was performed even if it was unnecessary. This has been fixed.

Changes to the Lisp API

Rfe9595 - Reduce unnecessary type-checking from Lisp API functions

The exported Lisp API functions (like get-triples or add-triple) were performing unnecessary type checks. This commit removes these checks and thereby increases the speed of code that uses this API.

This change also means that the Lisp API no longer supports using triple-store names to designate stores. That is, functions now require a triple-store instance and will no longer accept a triple-store name.

Bug19173 - Corrected bug in Lisp API for remote-triple-stores

Previously, cursors for remote-triple-stores could produce an error when a query returned too many unique strings. This error will no longer occur.

Bug19177 - Fix broken interpretation of namespaces in Sesame transaction documents

The 'name' and 'prefix' attributes of the addNamespace and removeNamespace tags were interpreted the wrong way. ('name' should hold the URL, and 'prefix' the short name).

Documentation

Rfe9339 - 3.x to 4.x code conversion document added

At the request of users, we have provided a document that lists the changes in the AllegroGraph Lisp, Python, and Java APIs in the upgrade from AllegroGraph 3.x to AllegroGraph 4.x versions.

Rfe9432 - Updates to Python API document

There were several updates to the Python API documnent to better delineate the distinction between public and private methods in 4.0.3.

Rfe9504 - Document TempDir configuration directive

The location where the server creates temporary files can now be controlled by specifying a TempDir option in a configuration file. This option is now described in the Daemon Configuration web page.

Rfe9540 - Audited the Python API doc for currency

The Python API document has been updated to reflect recent changes in the Python API.

Rfe9599 - Update Java section of 3-to-4 conversion document.

The v3-to-v4 code conversion document, convert3to4.md, has been update to enlarge on the information about the new AllegroGraph Java Client.

Bug19320 - Include links to upgrading information in documentation

Added links from the navigation section of each document to the "converting from AllegroGraph 3 to 4" documentation.

SPARQL

Rfe9561 - Improvement in SPARQL recycling of storage-layer cursors.

The SPARQL query engine now more aggressively discards and recycles cursors over on-disk indices. This improves the behavior of the system under load.

Gruff

New overview/scroller panes

The View menu has three new commands for using an overview/scroller pane.

New submenu File | Export Displayed Data As ...

A new "Export" child menu on the File menu allows exporting the currently-displayed triples or nodes to a file in one of several standard formats.

Child menu names are now included in the documentation.

Until now, a menu command that's on a cascading child menu, such as "Drawing Options | Derived Node Labels | Capitalize First Word", appeared in the docs as simply "Drawing Options | Capitalize First Word", which could make it hard to find on the menus. The docs now include the full path.

New freetext command "Add | Edit the Set of Registered Predicates".

There are new commands for registering some or all predicates for freetext queries at one time, and for unregistering some in agraph4.

Rfe9090 - Initial port to the agraph4 freetext API

The freetext commands now work on agraph 4.

Export commands failed when bridge lines were present

The new File | Export commands failed when bridge lines are present. This is now fixed.

Welcome to AllegroGraph 4.0.2

Welcome to AllegroGraph 4.0.2. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9084 - Return Prolog parse errors as bad requests, not internal server errors.

When issuing a Prolog with a syntax error, the response no longer indicates an internal server error, but is returned as a bad request error, tagged "MALFORMED QUERY".

Rfe9313, Bug18592 - Improved clause reordering in select queries

Modified the clause size estimation code to make it work better with AllegroGraph 4.0. The change greatly increases the speed of some Prolog-based select queries.

Rfe9345 - Ensure that distribution does not contain extra debug scaffolding

We now ensure and test that certain developer tools and debugging aids are not part of the distributed AllegoGraph. Removing these from the released code will have a small performance gain in some parts of the system.

Rfe9351 - Don't write deleted hash table if no triples have been deleted

Previously, the shared memory hash table that is used to track deleted triples was written to a file at every checkpoint. With this change, the table is only written when deletion of triple actually has happened. This speeds up trivial checkpoints as well as database open and close operations by about 35%.

Rfe9404 - Improve error message when bulk loading N-Triple data

The error signaled when loading invalid N-Triple data has been made clearer.

Rfe9435 - Support offset argument to geospatial query services

For consistency, all services that support a limit argument now also have an offset argument, which allows results to be skipped.

Rfe9440 - Improve cursor code efficiency in some situations

Improved some low-level cursor implementation details that provide a small but measurable speed and space boost for some queries.

Rfe9447 - Improve detection of SPARQL Negation-As-Failure idiom.

This change allows for SPARQL's idiom for Negation-As-Failure to be detected more reliably, and thus executed more optimally.

In particular, this allows for the OPTIONAL component to contain FILTERs, so long as they refer only to variables bound within the OPTIONAL itself.

One such query might be:

PREFIX foo: <http://www.example.com/foo/foo_model#>  
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>  
SELECT ?foo ?type  
WHERE {  
   ?foo a foo:Bar .  
   ?foo a ?type .  
   OPTIONAL {  
    ?subType rdfs:subClassOf ?type .  
    FILTER(?subType != ?type)  
   }  
   FILTER(!BOUND(?subType))  
} 

Rfe9448 - Add upi< and upi<= functors to prolog select

Added additional functors to the Prolog-based select language to make some operations easier to use.

Rfe9462 - Improved database shutdown process

The shutdown process that AllegroGraph uses when a database is closed has been improved to be more orderly and reliable.

Rfe9464 - Support adding and removing of namespaces from Sesame XML transactions

A client program that uses the and tags has been shown to exist (TopBraid), so they have been implemented.

Rfe9493 - Added mixed workload to events test

A -m parameter was added to the events test to run a Phase 6 mixed workload after the normal phases complete. This runs a mixture of queries, adds, and deletes until it is interrupted.

Bug19070 - /etc/init.d/agraph 'status' doesn't work

Previously, running "/etc/init.d/agraph" resulted in an error message instead of reporting the status on the agraph service. This has been fixed.

Bug19078 - Improve error message for invalid use of create or open-triple-store

The error message signaled when creating (or opening) a triple-store with incorrect parameters has been improved. The documentation of both functions is also clearer.

Bug19089 - Make sure daemon doesn't hang when it encounters an error

Turns off a debugging switch that was, unintentially, switched on by default, causing the service daemon to enter the debugger when encountering an error.

Bug19090 - Improve handling of graph-filtered-stores

Graph filtered triple-stores are used in various places in AllegroGraph's query engine. Prior to this fix, they did not implement some of the necessary API and could generate an error.

Bug19091 - Prevent server shutdown from locking up when a session is open.

The server would lock up when sent a sigterm if an HTTP session was still open. This is now fixed.

Bug19101 - Error detection for with-variables and CONSTRUCT queries.

AllegroGraph's SPARQL engine raises an error when imposed bindings (the :with-variables keyword argument to run-sparql) are not a subset of the variables mentioned in the body of the query.

This error checking failed to include variables mentioned in a CONSTRUCT template or DESCRIBE variable list, which meant that query invocations which imposed bindings for variables mentioned in the CONSTRUCT template alone would raise an error.

The resolution for this bug corrects the issue.

Bug19121 - Fix bug in namespace caching for HTTP back-ends.

Occasionally (depending on the system clock rolling over to the next second at the wrong moment) a newly defined (or deleted) namespace wasn't being seen by a back-end, causing 'unknown namespace' errors. This is now fixed.

Bug19123 - Allow reasoning-triple-stores to create new-blank-nodes

Reasoning-triple-stores (and other encapsulated-triple-store classes) were unable to mint new blank IDs. This prevented them from loading any RDF data that contained blank nodes. This fixes the problem.

Bug19145 - sendfd library does not handle "out of file descriptors" condition gracefully

When a client process experienced file descriptor shortage while connected to an Allegrograph store, memory corruption could occur. This has been corrected.

Bug19151 - Refcounts on open indexes not updated aggressively

Previously, if regular query operations were performed against a database that was continually growing, file handle exhaustion could occur due to incorrect reference counts on indexes. This problem has been fixed.

Bug19155 - Correct printing of invalid-characters-in-name condition

This fixes a syntax error in the formatting of the condition used to signal an invalid triple-store name or catalog. Previously, the error message produced was difficult to interpret; now it is clearer.

AllegroGraph Server: HTTP Interface

Rfe9227: Allow an HTTP proxy to be configured for the server

The HTTPProxy configuration directive can now be used when the server has to use an HTTP proxy to connect to the internet (for example, to load data from a URL).

Rfe9318 - Remove persistent federations from the HTTP protocol

Accessing federations of stores is now done by opening a session for such a federation, not by creating a persistent federated store and opening that.

Rfe9450 - Allow committing and rolling back through HTTP headers

As a performance optimization, it is now possible to have commit or rollback commands piggyback on other HTTP requests, using the x-commit and x-rollback HTTP headers.

Rfe9458 - Expose a full-featured textual free-text query language over HTTP

The 'pattern' argument to the /freetext service now understands strings like 'foo | bar' (foo OR bar), 'foo*' (everything starting with foo), '"foo bar"' (matching the exact string "foo bar") and even 'quux (foo | bar)' (quux AND (foo OR bar)).

Rfe9458 - Tweak free-text query expression format

When searching from the Lisp API, a phrase search is now done with '(phrase "foo bar") rather than "\"foo bar\"", and a wildcard match is done with '(match "wild") rather than just "wild".

Rfe9483: Add continueOnError parameter to HTTP statement loading

For N-Triples and N-Quads data, this argument can be used to make the load continue when malformed data is encountered, skipping the broken triples.

Rfe9496 - Support clearNamespaces tag in Sesame XML transaction documents

This tag was overlooked in the original implementation. It is now supported.

Rfe9499 - Support prepared queries in the HTTP protocol

It is now possible to 'prepare' SPARQL queries, either as a side-effect of executing them, or as a separate action. The requests for executing the prepared query will be more lightweight, since they don't have to pass in the query itself and most of its arguments, and skip the query-parsing stage.

Bug19088 - Allow a default namespace to be defined over HTTP

Due to the server stripping off trailing slashes from URLs and redirecting, the PUT request to ../namespaces/, which is required to create a namespace without a name, was redirected to ../namespaces, and didn't work. The redirecting behaviour is now turned off for this URL.

Changes to the Lisp API

close-triple-store has new 'commit' keyword

close-triple-store has a new 'commit' keyword argument.If true, then the commit-triple-store will be called before the store is closed.

user-reasoner keyword removed

The deprecated use-reasoner keyword argument has been removed from make-tutorial-store; use apply-reasoner-p instead.

sync-triple-store has been removed

The deprecated sync-triple-store has been removed; use commit-triple-store instead.

Rfe9576 - :expected-size argument to open-triple-store and create-triple-store no longer supported

To specify the expected size of a new database, edit the catalog configuration file before creating the database. To change the expected size of an existing database, change the ExpectedStoreSize parameter in the parameters.dat file of the database.

Bug19046 - Fix error generated by future-part printer

If a namespace mapping did not exist, then the future-part printer could signal an error instead of correcty printing the part. [Lisp API]

Bug19081 - Change return type of ego-group-layers from vector to list

The ego-group-layers function was incorrectly returning a vector of results instead of a list. Its behavior now matches the documentation: it returns a list of lists where each sublist contains nodes discovered at that level. [Lisp API]

Documentation

Improved Lisp reference manual documentation

Made several minor formatting changes and added documentation for the upi<= function and functor.

Document new free-text index HTTP API.

The creation and configuration of multiple text indices over HTTP is now described in the protocol document.

Rfe9169 - Add links to Javadocs from AllegroGraph documentation

Added links from the navigation bar of each AllegroGraph documentation webpage to the Javadocs.

Rfe9314 - Update Lisp API docs to reflect changes in freetext-indexing API

Indexing is no longer managed by keeping a list of predicates, but by creating one or more indices, each with multiple configurable parameters.

Rfe9318 - Update the HTTP protocol documentation for changed federation API

The services for creating and deleting persistent federations are gone, a more flexible way to open sessions, which also allows federation of stores, has been added.

Rfe9377 - Update free-text indexing example in Python tutorial

The 12th example in the Python tutorial now shows how to create multiple indexes and search for both text literals and URIs. The example introduces these new methods:

createFreeTextIndex  
getFreeTextIndexConfiguration()  
evalFreeTextSearch() 

Rfe9378 - Update Python documentation to reflect new federation API

The Python tutorial has been updated to reflect recent changes in the Python API for creating federated sessions.

Rfe9383 - Document new-style federation in the java client

The 16th examples in the Java tutorial has been updated to reflect the new API for creating a federated repository.

Rfe9500 - Update free-text-index entries in Python API html doc.

The connection object methods for free-text indexing have been updated to reflect recent changes in the API.

Java Client

Rfe9318 - Support session-based (rather than persistent) federation

The methods in AGServer for creating and deleting federations have been replaced by a single method, federate, which takes any number of repository objects, and returns a 'virtual' repository that federates all of them.

Rfe9392 - Use accept-encoding gzip to get back compressed results

Get and Post requests now use the Accept-Encoding: gzip header to request that responses be compressed, to improve performance when results are large.

Modified both tutorial examples files to call closeAll prior to deleteRepository in exampele1, as not doing so can result in orphaned sessions on the toClose list.

Rfe9436 - Make session lifetime configurable

Session lifetimes can now be configured for each connection, and should be set prior to making a method call that initiates a dedicated session. The default session lifetime is 1 hour. Tutorial example 22 sets the session lifetime to 120 seconds to demonstrate.

Rfe9442 - Add basic information to the jar manifest

The Java client jar manifest now includes information about the current implementation, including name, version, vendor name, vender URL, time of creation, etc.

Rfe9499 - Support query preparation

Calls to prepareTupleQuery, prepareGraphQuery, and prepareBooleanQuery now return query objects that will, during the next call to evaluate(), request that the server save the query for future reuse (in dedicated sessions only), eliminating some parsing and other overhead in subsequent calls to evaluate(). Tutorial example 14 has been modified to use non-autocommit mode so that it demonstrates query preparation in a dedicated session and multiple calls to evaluate() with different variable bindings.

Bug19086 - JenaTutorialExamples now uses Graph#getName()

The tutorial examples now use Graph#getName() to get the graph's name, rather than using Graph#toString(), which now prints out additional human readable information about the graph.

Bug19092 - Jena ResultSet and PrefixMapping compliance

AGPrefixMapping now supports methods qnameFor, removeNsPrefix, samePrefixMappingAs, setNsPrefixes, shortForm, expandPrefix, lock, and withDefaultMappings. The toString method now returns human readable information about the prefix mapping.

AGResultSet now supports methods getResourceModel, getResultVars, and remove.

AGQuerySolution now supports methods contains, get, getLiteral, getResource, and varNames.

Python Client

Rfe8610 - Help finding pairs of UPIs

The 23rd example in the Python tutorial has been extended to demonstrate how to use inequalities to reduce the number of permutation matches in SPARQL and Prolog queries.

Rfe9311 - Expose revised FTI API in the Python client

Text-indexing is no longer configured by adding predicates, but by creating (and updating or deleting) multiple separate indices.

Rfe9318 - Move to session-based federation API.

To federate stores, one now calls AllegroGraphServer.openFederated, which opens a session that federates the given stores, rather than creating a persistent federated store and opening that.

Rfe9357 - Support wordFilter option to text-indices in Python client

The createFreeTextIndex and modifyFreeTextIndex methods of a connection now allow a wordFilters argument to be specified, which allows accent removal and word stemming to be configured.

Text Indexing

Rfe8775 - Support multiple text-indices per repository, add configuration parameters.

Multiple free-text indices can now be active in a repository, and there is an API to create, query, modify, and delete them. New parameters supported are: stop words, minimum word length, whether to index resources, which literals to index, indexing non-object parts, and the possibility to index all triples, regardless of predicate.

Rfe8775 - Add triples to free-text index as soon as they are added.

A commit used to be necessary to be able to find newly added triples through the free-text index. They can now be queried (by the current transaction) as soon as they are added.

Rfe9357 - Add word-filters to text-index configuration (accent removal, stemming)

It is now possible (both in the Lisp API and over HTTP) to specify a list of word filters when defining a freetext index. These are operations that are performed on every word that is indexed, and every word that is queried, allowing various 'normalisations' of words. Currently allows two filters, "drop-accents" to remove accents from letters, and "stem.english" to apply basic, English-language word stemming.

Changes to WebView

Rfe8775 - Allow free-text reconfiguration through WebView.

WebView now allows the creation, modification, and deletion of free-text indices, exposing a UI for all the new configuration parameters.

Rfe9408 - Expose free-form sessions in WebView

There is now, for users with the "session" permission flag, a control on the main WebView page for starting and entering a session over a federated, reasoning, filtered, and/or remote store.

Rfe9438 - Show server version in WebView

WebView now shows the exact version of the server in its header.

Rfe9453 - Support bulk-adding of namespaces in WebView

Clicking 'bulk input' when adding a namespace now pops up a new input dialog, where a set of namespaces can be typed or pasted all at once.

Bug19105 - Restore Safari-compatibility for WebView

WebView was broken on recent versions of the Safari browser. This has now been remedied.

Welcome to AllegroGraph 4.0.1

Welcome to AllegroGraph 4.0.1. This version of AllegroGraph contains updates to the AllegroGraph 4.0 release, including:

• Cursor optimizations for increased performance
• Improved caching of RDFS++ reasoning results
• Improved SPARQL range-query efficiency
• Heuristic optimization of negation as failure in SPARQL queries
• SPARQL query performance tuning documentation added
• Hardware Profiles added to Performance-Tuning Guide

This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9137: Heuristic optimization for negation as failure in SPARQL queries.

SPARQL queries which use the common OPTIONAL/FILTER (!bound(?foo)) idiom to express negation as failure are now handled more efficiently, computing fewer unnecessary intermediate results.

This optimization can be disabled by setting the non-exported variable sparql.algebra::rewrite-naf-p to nil.

Rfe9320: Improved caching of RDFS++ reasoning results

Improved RDFS++ reasoner efficiency by selectively caching intermediate results. This makes many queries involving the reasoner between 2 and 10 times faster.

Rfe9352, Bug19067, Bug19068: Improve internal cursor operations

Optimizing how AllegroGraph treats empty cursors (i.e., cursors known to return no results) so that operations with them are faster.

Rfe9374: Improved SPARQL range-query efficiency in some cases

Improved the detection of range queries that will return no results which allows SPARQL to optimize other cursor operations.

Bug18978: Optimizing cursor creation and filtering

Improved the way AllegroGraph internally manages cursor creation and triple-filtering to increase performance.

Documentation

Hardware Profiles added to Performance-Tuning Guide.

The performance-tuning guide has a new section describing the kind of hardware Franz recommends for users who will be deploying applications on the scale of 3-4 billion triples.

Rfe9374: SPARQL query performance tuning documentation added.

The performance tuning guide has been extended to describe possible approaches to handling typed literals to improve SPARQL query performance for numeric and date FILTERs.

Welcome to AllegroGraph 4.0

Welcome to the new AllegroGraph v4.0. This latest version of AllegroGraph is designed take full advantage of symmetric multi-processing hardware and brings enterprise database features to the world of Semantics and RDF. AllegroGraph, now incorporates:

• Full ACID database compliance.
• Full Recoverability.
• Online backup and restore.
• Automatic Indexing.
• HTTP-based client/server libraries supporting: Java, Sesame, Jena and Python.

This release of AllegroGraph v4.0 is focused on enhancements to database backup and restore, Java Jena client library compliance, and overall product stability improvements.

AllegroGraph Server: General Changes

Add a Lisp function to list the catalogs on a server.

The function #'catalogs, exported from the db.agraph package, can now be used to find out which catalogs a server provides.

Added ego-group-layers function

ego-group-layers returns a nodes ego-group organized by the depth at which each node was found. [Lisp API]

Documentation: ego-group-layers (node depth generator)

Return a list of lists of the nodes in nodes ego group.

Each element of the result is a list of the nodes discovered at that list's depth.

These are the nodes in the graph that can be reached by following paths of at most length depth starting at node. The paths are determined using the generator to select neighbors of each node in the ego-group. See the description of SNA generators for additional details.

Added support for auto-commit to the bulk load functions

The commit parameter to load-ntriples, load-rdf/xml and load-trix can be one of

• nil - do not commit
• t - commit at the end of the load
• a number N - commit every N-triples

Changed behavior of breadth-first-path-search when maximum-depth is given.

Previously, when the breadth-first-path-search functions were given a maximum depth, AllegroGraph would return the shortest path or paths found that were no longer than that depth. Now, AllegroGraph returns all paths found that are no longer than maximum depth.

This can be very expensive for graphs with high connectivity. To return only the shortest path, use a maximum depth of nil. [Lisp API].

Since the Prolog SNA functors use the underlying SNA functions, their behavior has also changed.

Change the keyword value used to identify typed literals

Previously, AllegroGraph used :typed-literal to specify a typed literal. To make things more consistent with :literal :literal-language, this has been changed to :literal-typed. [Lisp API]

Define an error type to signal failures to connect to the AllegroGraph server.

Any operation (opening, creating, deleting, probing) that has to talk to the server will now raise a condition of type db.agraph:could-not-connect-to-server-error if it fails to find a server on the specified port.

Fix mistake in instance-subprocess communication protocol.

This would cause the instance to go into an infinite loop and become unresponsive when one of its children died.

Make delete-duplicates-on-merge setting persistent.

The flag was not properly restored when opening a triple store.

Rfe7381 - Add a db keyword argument to several API commands Added the db argument to the following commands

• part->string
• part->value
• part->ntriples
• part->long
• part->concise
• part->terse
• print-triple

[Lisp API]

Rfe8765 - Improved Prolog query performance

Many Prolog queries execute more than 30% more quickly.

Rfe8860 - Geospatial functions now accept strings

The geospatial functions in AllegroGraph previously required a subtype object. They now accept strings which name a subtype object. This makes them easier to use. [Lisp API]

Rfe9146 - Allow 'merging' of store metadata on concurrent change.

Before, two sessions both adding a type-mapping and then committing would cause a conflict error. The metadata system is now clever enough to merge such changes.

Rfe9209 - Don't log process termination messages before cleanup.

Both instance processes and their children used to output '... terminated' before they cleaned up, which in some cases led to them hanging or crashing after they claimed to have terminated.

Rfe9212 - Periodically clean up deleted triple data structures

Previously, the data structures which track deleted triples could grow without bounds. Now, they are periodically examined and obsolete portions are purged.

Rfe9291 - Use XSD instead of internal type-codes for /mappings

The /mapping protocol now returns lists of

mapping-kind datatype/predicate XSD-type 

rather than

mapping-kind datatype/predicate numeric-type-code 

This makes both debugging and general use of the mappings much simpler for clients.

Rfe9294 - Added SNA path functors to bind only the first matching path

Added breadth-first-search-path and bidirectional-search-path functors to correspond with the existing depth-first-search-path functor. These all run SNA path finding algorithms and stop after finding the first matching path.

Rfe9315 - Do early probe to see if file system supports >4GB file offsets

Allegrograph requires that the underlying file systems support offsets longer than 32 bits. Some file systems (i.e. NFSv2) do not support long offsets, which caused file system related failures in varying situations. Allegrograph now probes each of the file systems specified in a configuration to make sure that long file offsets are supported.

Rfe9322 - Changes in encoded date and time UPIs.

Corrected a problem where the fractional seconds stored in an encoded UPI could be lost. The update fixes the stored representation and invalidates existing triple-stores that have any time, date or date-time encoded UPIs.

Rfe9329 - Improved loading of a required shared library.

Previously, AllegroGraph required loading of a particular shared library (librt) that did not always exist by the same name on all Linux distributions. This problem has been corrected.

Rfe9330: Set up selinux security context on shared library during installation

Previously, on systems with SELinux enabled, the agraph server would not start up properly unless the operator issued the following command first:

chcon -t textrel_shlib_t <agraph lib dir>/libacl*.so 

Now, this command is automatically performed during RPM package install or when using the install-agraph script that comes with the tarball package.

Rfe9340 - Disallow overlapping catalog directories.

Having, for example, one catalog in /tmp/foo, and another in /tmp/foo/bar, or two catalogs in the exact same directory, is no longer allowed, since this is bound to cause stores to be corrupted.

Rfe9362: Output a log message when deleting a triple-store.

Deleting a store wasn't visible in the log before. It now notes which store was deleted.

Spr36556 - Reduce number of duplicate rdf:type inferences

AllegroGraph's type inference was returning a triple for each possible inference path rather than filtering out the duplicates. I.e., if a rdf:type B could be inferred two ways, then the reasoner would return two triples. This no longer happens.

Spr 36565 - Standardize the printing of encoded types

AllegroGraph would sometimes print typed literals non-syntactically. This no longer occurs.

Bug18679 - Made part->value function more consistent

The part->value function now returns type-codes for both UPIs and future-parts. Previously, it returned a type-code for UPIs and a keyword (like :literal or :anon) for future-parts. [Lisp API].

Bug18679 - Removed the deprecated part-value

The deprecated part-value function has been removed. Use part->value in its place. [Lisp API]

Bug18738 - Make query-planner work on remote-triple-stores

Previously, select queries that needed to re-order their clauses would instead signal an error on remote-triple-stores. This patch corrects this problem and lets remote-triple-stores work correctly with the query-planner. [Lisp API]

Bug18840, Bug19003 - Inconsistent state after duplicate purging

Total triple counts were wrong after duplicate purging. By default, AllegroGraph purges duplicate triples from the index during merge operations. Previously, when duplicate triples were purged, the total count of triples in the store was not updated. Also, in most cases, one of the seven triple indexes would not be purged of duplicates, resulting in wasted storage and potentially inconsistent query results. These problems have been corrected.

Bug18879 - Corrected add-triple so that it could accept large strings

Unlike the bulk loaders, the add-triple function was unable to accept strings longer than 8092 characters and would signal an error. Now add-triple can accept strings of arbitrary size.

Bug18800 - Unbound prolog variables could pollute select query results

Queries like

(select (?s ?t)  
 (q ?s !<http://www.franz.com/sna#marriage>  
!<http://www.franz.com/sna#peruzzi>)  
 (optional (q ?s !rdf:type ?t))) 

could return unbound expessions like `{Unbound 14a170c9}` instead of `nil`. Now these queries correctly return `nil`.

Bug18880 - Corrected serialization of RDF/XML

The serializer sometimes added extra angle brackets around typed literals. Now it correctly outputs them as, e.g.,

"42"^^<http://www.w3.org/2001/XMLSchema#int> 

Bug18947 - Make service daemon responsible for deleting stores

delete-triple-store used to only check whether the store was open in the current image, and thus deleted stores even if another process had them open. This moves the responsibility for deleting stores to the service-daemon, which actually knows whether anyone has the store open.

Bug18949 - Fix geospatial type mapping in federated stores

AllegoGraph was not handling geospatial type-mappings in federated-triple-stores and could fail to return all of the triples from every leaf store. This is corrected.

Bug18972 - Correct printing of times with fractional seconds

When a time or dateTime with fractional seconds had leading zeros in the fractional seconds part, AllegroGraph could fail to print it. I.e., a time like 10:31:55.028192 might print as 10:31:55.281900. This printing problem has been corrected.

Bug19010 - Fix 'store in use' bug when accessing store over HTTP.

Fixes a bug where a store accessed over HTTP could not be deleted or replaced through the direct Lisp API until its handle had timed out.

Bug19018 - Make sure instance processes are cleaned up when breaking

Fixes a few cases where unexpected errors would cause the daemon to get confused about the stores it had open (instance crash, client dying while opening or creating).

Bug19024 - Show catalog name in store-status error messages

Errors like 'store exists' and 'store not found' used to only contain the store name, now they also have the catalog. Trying to perform an operation on a non-existent catalog now raises an error that clearly indicates the catalog doesn't exist.

Bug19045 - Use . as the fractional seconds separator in times and dateTimes

Previously AllegroGraph used the comma (,) as the separator in its printed representation of XSD times and dateTimes. This is acceptable ISO 8601 notation but not acceptable XSD notation. This bug fixes the problem so that fractional seconds are separated from whole seconds by a period (.).

Bug19055: Fix potential hang when committing

Under certain circumstances, commits could hang indefinitely while waiting for chunk merging to complete, but chunk merging would never commence due to a counter having an invalid value. This problem has been corrected.

Bug19061: upi->String table race condition fix

Previously, under high load conditions, errors could occur during string table operations. This problem has been fixed.

AllegroGraph Server: HTTP Interface

Allow nested lists in JSON-shaped Prolog-query output.

application/json results for Prolog queries can now contain nested arrays of results, to make it easier to use some of our SNA functionality.

Allow the creation of sessions with federated, graph-filtered, or reasoning stores.

It is now possible to spawn HTTP sessions on a store built up by applying an arbitrary combination reasoning, federation, and graph-filtering to concrete triples.

Cause output of --http-trace switch to actually get flushed to the file.

Before, it was buffered, and would only be visible after a large amount was written, or the server was shut down.

Clean up namespaces when a repository is deleted.

Before, deleting and re-creating a repository would leave the old namespaces intact.

Enable streaming of XML bodies when loading statements.

A problem in our SAX parser made this difficult before, but that has since been fixed. This makes it possible to load large files with less overhead.

Fix a bug that prevented session permissions to show up in WebView user-contol UI.

(No further text.)

Fix a bug with namespace caching.

Sometimes, back-ends wouldn't notice a user's namespaces had changed when they were updated twice within the same second.

Fix bug in remote geospatial queries.

There was a problem that broke remote-triple-stores when more than one geospatial subtype occurred in a query response. This is now fixed.

Fix user-access bug in HTTP protocol.

Fix a bug that prevented users with access to a store in the root catalog, but not the whole catalog, from accessing this store over HTTP.

Make user-access-configuration interface in WebView more obvious

It was easy to mistake the widget for adding access for an indication that access had already been granted.

Support changing user passwords through the HTTP interface

There is now /users/[NAME]/password, to which one can POST a new password.

Support resetting namespaces to the defaults.

There is now a reset argument to the namespace clearing interface, which, when true, will cause the default namespaces to be restored.

Rfe9131 - Add defaultGraphName parameter to SPARQL query interface

This is used to provide a way to refer to the default graph inside a SPARQL query using a URI.

Rfe8999 - Use a unified format for serializing blank nodes.

Before, each component had its own format. This moves the HTTP server to the unified format.

Rfe9183 - Silence AServe's socket-reset warnings

The log is no longer polluted with spurious connection-reset messages.

Rfe9185 - Support delete-duplicates-during-merge remotely

Remote-triple-stores now support the :delete-duplicates-during-merge initarg, and the db-delete-duplicates-during-merge accessor.

Rfe9217 - Add more help text to the WebView interface.

Potentially confusing input fields in WebView (such as those where resources or literals are entered) now have a question-mark icon that pops up an explanation.

Rfe9236 - Support SPARQL/Update over HTTP.

POST requests to a repository can now issue a SPARQL/Update query, if the user has write access to the store.

Rfe9247 - Expose server processes in HTTP interface

It is now possible to list, inspect, and kill the server's processes through the HTTP interface. Additionally, one can start up telnet servers in the processes, in order to directly debug them.

Rfe9247(2) - Allow process control from WebView

Superusers can now view the server processes, and open telnet servers in them, through the WebView UI. (See the 'Processes' link in the navigation bar at the top-level page.)

Rfe9295 - Add a commit argument to PUT/POST /statements.

This supports auto-committing every X triples, to make uploading huge files viable.

Rfe9310 - The PUT /repository/[name] now overwrites by default.

The PUT /repository/[name] service in the HTTP protocol now overwrites existing repositories by that name unless override=false is passed.

Bug19002 - Fix bug in HTTP protocol namespace caching.

Fix a bug that caused problems with namespaces after deleting and re-creating (over HTTP) a store that had namespaces defined.

Bug19005 - Corrected problem in client lookups of individual datatype mappings.

Previously, a request for an individual datatype mapping would return nil regardless of how the mapping was defined. Now it returns the correct value.

Documentation

Server Installation Page Updated to be Ubuntu-Friendly

The server-installation instructions have been updated to make them more friendly to people who don't know how to install Python, python-pycurl, and python-cjson packages. This included some specific suggestions for Ubuntu users for RPM installation and Python client installation.

Rfe9322 - Added client tutorials to the documentation

The documentation now contains tutorials on how to use the Python and Java clients. Please see the links off of the main web pages.

Java Client

Expanded Coverage of Literal Values in Java Tutorial.

The Java tutorial has been revised and expanded to document the behavior of literal typed values (in example5()). This includes the default behavior of untyped literals, as well as their use in getStatements queries as well as filtered and direct SPARQL queries. The section covers strings, floats, ints, untyped values, dates and times.

Java Tutorial Describes Managing User Accounts

The Java tutorial has a new section on "Creating AllegroGraph Users with WebView." It describes how to create a manage new AllegroGraph user accounts.

Rfe8812 - Jena compliance improvements

Added compliance tests for Jena GraphMaker and Model interfaces and corrected several non-compliances.

Properly handle the case of getting a namespace uri for a prefix that does not exist, and add support for expanding a QName.

Add GraphMaker support for creating anonymous graphs, listing graphs, checking existence of a graph, and returning existing graph objects rather than creating new ones when possible.

Throw AlreadyExistsException when attempting to create an existing graph in strict mode, and throw a DoesNotExist exception when attempting to remove or open a non-existent graph in strict mode.

Rfe8812 - Jena Graph compliance improvements

Added compliance tests for the Jena Graph interface and corrected several non-compliances.

AGBulkUpdateHandler's removeAll method properly clears a graph.

Added a custom AGCapabilities class declaring handlesLiteralTyping to be false, as AGGraph does not currently support D-entailment.

Add support for the executeInTransaction method in AGTransactionHandler, though using begin/commit/rollback explicitly is preferred.

AGTripleIterators are now tied to AGGraphs, calling remove on the iterator will delete from the graph.

Rfe9359 - Jena Prolog support.

Added support for Prolog select queries over a Jena model. Queries must be written to select for variables that are bound to RDF parts, rather than to arbitrary Lisp objects.

Python Client

Removed Common Logic

Removed the experimental and undocumented client side Common Logic query language. Please contact [email protected] for information if you were using this language.

Removed JDBC result sets

Removed the ability to get JDBC result sets. Please contact [email protected] if you need ways to get the result data without instantiating the corresponding Literal, URI, or BNode objects.

Namespace implementation is now server-side

Removed the client-side namespace implementation in favor of a server-side namespace management.

"Duplicate Triples" added to Python Tutorial.

The Python Tutorial has a new section on "Duplicate Triples", based on example23(). It describes the sources of duplicate triples and repetitive query results, and prescribes techniques for bringing them under control.

Python Tutorial Describes Managing User Accounts.

The Python tutorial has a new section on "Creating AllegroGraph Users with WebView." It describes how to create a manage new AllegroGraph user accounts.

Expanded Coverage of Literal Values in Python Tutorial.

The Python tutorial has been revised and expanded to document the behavior of literal typed values (in example5()). This includes the default behavior of untyped literals, as well as their use in getStatements queries as well as filtered and direct SPARQL queries. The section covers strings, floats, ints, untyped values, dates and times.

Python Examples from Command Line

The Python tutorial example file (tutorialexamples40.py) can now be run from the command line.

$ python tutorial_examples_40.py  runs all tests.  
$ python tutorial_examples_40.py all  runs all tests.  
$ python tutorial_examples_40.py 1 5 22   runs tests 1, 5, and 22 

Rfe9295 - Changes addStatements and addFile

addStatements and addFile can now auto-commit triples based on a count for large loads of triples. Set the add commit size to a positive integer by calling setAddCommitSize or setting the addcommitsize property on the RepositoryConnection for size-based commits, or set to None or 0 for regular commit semantic on the add methods.

Using this feature can decrease the server's memory usage during loading.

Rfe9316 - RepositoryConnection.setRuleLanguage removed

RepositoryConnection.setRuleLanguage has been removed. Both addRules and loadRules already take an language parameter which defaults to QueryLanguage.PROLOG as the only language currently supported for rules.

Bug19001 - Changes to RepositoryConnection.clearNamespaces API

RepositoryConnect.clearNamespaces now supports a reset parameter (which defaults to True). clearNamespaces deletes all namespaces in the repository for the current user. If the reset argument is True, the user's namespaces are reset to the default set of namespaces, otherwise all namespaces are cleared.

Bug19004 - Make sure Python clients ping their session.

Previously, the thread to keep a dedicated session alive was created but not started, making it possible for Python clients to have their session time out on them.

Welcome to AllegroGraph 4.0m2

General

The M2a release includes the following major functionality:

• Full database backups and restores.

• Text Indexer Enhancement for databases with more than 1 Billion text elements and strings with international characters.

• Infiniband testing and performance document.

• Deleted Triples Data structure improvement that removes the previous hard-coded limit on the number of triples that can be deleted per transaction.

• Cluster Manager Design Document.

• Jena functionality that was outlined as being highest priority.

• Federation of AllegroGraph 4 databases

• Support for Social Network Analysis

• Support for two dimensional geospatial analysis.

• Support for 48-bit triple IDs.

• User definable placement of the transaction log.

There were many minor enhancements and bug fixes.

Note that AllegroGraph databases created with release m1 or m2 will have to be converted to a new format before use with version m2a. See the Upgrade Guide.

Performance and Robustness

• Improve logging by making some messages more concise and adding additional messages to improve diagnostics.

• Improve and make string tables more robust

  • better consistency checks

  • compress large strings (>= 256 chars) in the string table

• Added conversion tools for moving m1 triple-stores to m2

• Support SIGUSR1-based reload in agraph-control script

• After merging UPI tables, schedule a checkpoint to delete files which are no longer needed.

• Safer file deletion for UPI tables to prevent problems with untunded databases.

• Optimize UPI table merging, do not merge table portions that can't have common entries.

• Improve client handling of instance process death

• Improve upgrade checkpoint handling

• Improved database versioning framework

• Reference count indexes so that they do not disappear out from under cursors

• Rmprove transaction log upgrade handling

• Writes to transaction log files are now aligned to the filesystem block size to allow for optimal performance when used with certain storage subsystems.

• Control placement of transaction log (with TransactionLogDir)

• The maximum number of triples that can be added to a triple store is now 2^48 (formerly 2^32).

• Added TransactionLogDir config file parameter. This specifies where transaction log files should go. If not specified, defaults to the value of Main.

• The agraph server process now responds to SIGUSR1 by reloading the server configuration file. The agraph-control script and /etc/init.d/agraph scripts now accept the 'reload' command and will send SIGUSR1 to the agraph server process.

• Enhanced in-the-field debugging support. When requested by Franz tech support, you can send SIGUSR2 an AllegroGraph subprocess to get a backtrace for all threads.

Other Enhancements

• configure-agraph now turns off terminal echo before prompting for the super user password. Also, since the password is no longer echoed, configure-agraph prompts for the password twice to make sure it has been entered properly.

• support fractional seconds in time and dateTime

• improve support for duplicate deletion during merge operations

• improve and unify the printing of blank nodes

• make the opening and re-opening of federated-triple-stores more robust

• document group centrality

• add API to auto-commit (on a per store basis)

• add depth-first-search-path functor

• add :plain encoding

• improve handling of polygons in the geospatial sub-system

• cleanup part printing

• Remove line breaks in server log files to make automated processing and filtering easier.

• Crash instance process if one of its helper processes crashes.

• Detect death of instance process in client. Generate clear error message instead of hanging the client.

• Add database versioning framework. Implement stepwise upgrade paths for databases that have been created with different software versions.

Fixed bugs and other corrections

• Fix for bug18831: Agraph.cfg "overwrite" option appends to previous file instead of overwriting it as it should.

• Fixed bug18774: Running "agraph-control start" twice would overwrite a valid pid file. This has been fixed.

SPARQL

• Significant SPARQL performance improvements throughout, particularly in Left Joins (OPTIONAL), DISTINCT and REDUCED, and certain numeric FILTERs.

• Massive performance increase for queries with both ORDER BY and a small LIMIT.

• Imposed bindings (:with-variables) now apply to the output portion of SPARQL CONSTRUCT and DESCRIBE queries.

• Some queries that use the SPOGI cache as an optimization could return duplicates. Fixed.

• Similarly, the free-text index now returns distinct results in all cases, which avoids some unnecessary duplicate results.

• Add the ability to use a graph URI, on both input and output, as a signifier for the default graph UPI of the current store.

• Complete xsd:string/simple literal entailment. This allows you to seamlessly switch between using xsd:strings and plain literals without query changes. Plain literals are more efficient for the storage layer, and encouraged in all circumstances.

• Improvements (in both efficiency and correctness) for SPARQL comparisons and XQuery functions when handling encoded values.

• RDF/XML serializer can now handle on-the-fly namespace abbreviation, at the cost of more verbose output.

• SPARQL extensions (e.g., geospatial forms) are now permitted by default.

• URIs can now be used to specify geospatial subtypes, as well as the more verbose and less portable UUID form.

Java Client

• Added Social Network Analysis capability.

• Updated the Java Tutorial and tutorial examples to include Social Network Analysis.

• Added Geospatial analysis capability.

• Updated the Java Tutorial and tutorial examples to include Geospatial Analysis.

• Established a Jena semantic framework API.

• Created the Jean Tutorial, and a file of Jena tutorial examples.

Python Client

• Added back support for SPOGI cache enabling/disabling

• Enabled returning SPARQL-results+json or SPARQL-results+xml result from a query rather than a binding set

• Introduced a boolean argument to openSession and session API for whether or not to load the initfile in the process handling the session

• Federation support.

• Introduced several options and better command-line option parsing for the load stress script

• Minor updates to Python Tutorial and examples.