Table of Contents

AllegroGraph 4.13.1 user-visible changes

AllegroGraph Server: General Changes

Bug22255 - Concurrent full index optimization requests

AllegroGraph 4.13 user-visible changes

AllegroGraph Server: General Changes

Rfe12645 - Add a command line interface to AllegroGraph's materializer

Rfe12642: Improve handling of datafile free space information

Rfe12590 - Shared backend scheduling improvements

Rfe9523 - Logging improvements

Bug22220 - Race condition while determining last written tlog id

Bug22184 - Instance shutdown and replication

Bug22156 - date-string-to-upi doesn't handle zero time zones consistently

Bug22111 - Database names may not contain special characters

Bug22100 - More strict, faster parsing of some xsd datetimes

Bug22086 - Materializer deletes triples with the same subject, predicate and object

Bug22032 - Control whether or not external references are followed during RDF/XML import

Bug21953 - Allow import of triples to reasoning-triple-store

Bug21656 - Cleaner daemon shutdown

Bug20773 - Support federations of RDFS++ stores

HTTP Client

Rfe12763 - Update HTTP API for the materializer

SPARQL

Rfe12767 - Improved the efficiency of single group aggregate queries

Rfe12762 - Optimizing level 2 index optimization

Rfe12760 - Improve efficiency of some SPARQL Property Path queries

Rfe12757 - Improve efficiency of some property path queries

Rfe12717 - Handle RDFterm-equality of strings consistently

Rfe12699 - Improve efficiency of SPARQL equality tests on resources

Rfe12681 - Optimize interprocess communication

Rfe12679 - Use query local namespace abbreviations in result serialization

Rfe12670 - Allow /SPARQL access to AllegroGraph as a SPARQL endpoint

Rfe12657 - More efficient handling for the VALUES clause

Rfe12656 - Improve execution of some nested joins

Rfe12498 - Improve full-scan warning heuristics

Rfe11981 - New query option chunkProcessingMemory

Rfe11515 - Fully support the SPARQL 1.1 HTTP Protocol

Bug22214 - Encoded IDs failed to compare properly in SPARQL queries

Bug22204 - decimal to decimal cast loses precision

Bug22203 - Fix decimal parsing in SPARQL queries

Bug22153 - Some wild card property path queries could generate incorrect paths

Bug22147 - Add support for ObjectId's in the _id field.

Bug22142 - Expressions on aggregates fail for empty group

Bug22134 - SPARQL aggregates on expressions with errors could compute incorrect results

Bug22123: Empty construct template will cause an error

sparql< and sparql> for date and dateTime comparisons

Bug22074 - DELETE templates with unknown variables caused an error

Bug21561 - Remote-triple-stores do not support the table results format

AGWebView

Bug22161 - Adding security filters in Firefox

Changes to the Lisp API

Rfe12632: Improvements in Lisp API to AllegroGraph materializer

Bug22132 - calling run-sparql with the :with-variable argument with a timezoned upi

Prolog

Bug22171: Select0 should return UPIs when it can

Bug22157 - Temporal functors could signal an error when used with all bound values

Documentation

Python Client

Java Client

AllegroGraph 4.12.2 user-visible changes

AllegroGraph Server: General Changes

Rfe12734 - More aggressive level 2 index optimization

Rfe12726 - Index optimization increasing database size on disk

Rfe12725 - Faster level 2 index optimization

Rfe12636 - Make config-file argument to agraph-backup backup-settings optional.

Rfe12635 - Major revision to agraph-backup operating modes.

Rfe12621 - xsd:time improvements

Rfe12580 - Don't create repo if client/server uids don't match

Rfe12456 - Time zone roundtrip for xsd:dateTime and xsd:time

Rfe11303 - Fix partial repo creation on agraph-backup restore operations

Rfe11044 - Add -p short form of --port to agraph-backup.

Rfe10629 - Add --supersede argument to agraph-backup

Bug22099 - Ensure UUIDs are unique during agraph-backup restores.

Bug22090 - Timezone minutes in xsd types

Bug22083 - xsd:date and xsd:dateTime years

Bug22073 - ORDER BY and non-encoded decimals

Bug22072 - Validate xsd:decimals

Bug22068 - Reasoning and constant triple patterns

Bug22066 - Printing of NaN, INF and -INF

Bug22057 and Bug22054- agraph-backup backup-settings works as documented.

Bug22045 - 14 hour datetime comparison logic

Bug22044 - Fix xsd:date xsd:dateTime comparisons

Bug22041 - Comparisons to decimal zero

Bug22039 - agload character encoding fixes

Bug22034 - part->concise on encoded dates and decimals

HTTP Client

SPARQL

Rfe12624 - Standardize some low-level SPARQL equality tests

Bug22092 - Fix for SPARQL queries that use the STR() function with a constant

Bug22078 - SPARQL insert did not always encode typed literals

Bug22077 - Improve efficiency of certain SPARQL sub-queries

AGWebView

Changes to the Lisp API

Bug21608 - Include ag-call in remote Lisp client.

Prolog

Documentation

Python Client

Java Client

AllegroGraph 4.12 user-visible changes

AllegroGraph Server: General Changes

Rfe12576 - Improve RDF/XML parser efficiency

Rfe12522 - Global configuration directive for query options

Rfe12429 - More flexible parsing for InstanceTimeout

Rfe11010 - Require the i index for freetext queries

Bug22024 - Serialization of decimals

Bug21978 - serialize-* functions in Lisp client default to UTF-8.

Bug21976 - Complex GRAPH queries could produce incorrect graph bindings

Bug21975 - Increase AllegroGraph Load speed

Bug21966 - Possible "UPI not in string table" problem with remote-triple-stores

Bug21956 - Improve memory usage for zero-length property path queries

Bug21954 - Problem generating soundexes for phrases in Callimachus

Bug21949 - Improve bookkeeping of already processed variables during query execution

Bug21937 - Zoneless xsd:dateTime roundtrip

Bug21934 - String table mutex not reinitialized after recovery

Bug21920 - Soundex should ignore punctuation

Bug21915 - Make upi->value on xsd:dates return integers

Bug21914 - xsd:dates on the first day of the month

Bug21909 - agraph-backup could fail to generate unique UUID on restore

Bug21903 - Free text indexing with chunk-at-a-time processing

Bug21899 - Shared memory server fails to clean up

Bug21873 - Link to docs in rpm install

Bug21872 - agload will write output to agload.log when in --debug mode

Bug21869 - Fix problem with registering cartesian predicate mappings

Bug21853 - Connection Pool session port connection failure

Bug21846 - Long running federated queries could fail

Bug21816 - Fix session multiplexing file descriptor leak

Bug21784 - Safer auto-close for garbage db's in the Lisp direct client

Bug18813 - Reasoning and subclasses of owl:TransitiveProperty

HTTP Client

SPARQL

Rfe12537 - Improve zero length property path clause estimation

Rfe12520 - The SNA members and size magic properties work with RDF lists

Rfe12506 - Improve handling of some queries that use GRAPH clauses

Rfe12481 - Improve SPARQL DELETE efficiency

Rfe12442 - Efficiency improvements for SPARQL aggregation

Rfe12400 - Minor property-path query optimization

Rfe12398 - Clarify semantics of the identity clause reorderer

Rfe12392 - Optimize certain complex join combinations

Rfe12385 - Improve BGP reordering in some cases

Rfe12380 - Improve efficiency of some DISTINCT or REDUCED queries

Rfe12377 - Add intervalContainsTime magic property

Rfe12372 - Make BGP reordering more intuitive

Rfe12368 - Use constraints to limit bindings for GRAPH queries

Rfe12352 - Provide a sort order for SNA path and group gensyms

Rfe12192 - Optimize some wild-card property-path queries

Rfe12032 - Improve property-path expansion

Rfe11948 - Add PREFIX option for enabling reasoning in a SPARQL query

Rfe11625 - Improve algebraic optimizations of joins and left-joins

Rfe11146 - Handle simple OPTIONAL clauses more efficiently

Rfe11068 - Improve literal equality constraint handling

Bug22030 - ORDER BY expressions with non-query variables could cause errors

Bug22021: Correct SPARQL TSV printing problem for encoded UPIs

Bug22016 - Correct SPARQL 1.1 bug with IN FILTERs

Bug22015 - Fix possible error when using the SPOGI cache and SPARQL 1.1

Bug21986 - Using distinct aggregation with GROUP_CONCAT and a separator could fail

Bug21980 - Too many results in DISTINCT and REDUCED queries

Bug21974 - Assertion failure involving 'n-valid-partials'

Bug21962 - Improve estimates for certain query patterns

Bug21961 - Incorrect results with EXISTS and NOT EXISTS

Bug21955 - Sorting on unbound variables could lead to a string dictionary error

Bug21948 - Improve bookkeeping of bound variables during sub-query

Bug21947 - Improve efficiency of VALUES in some cases

Bug21941 - Queries with magic properties could fail in a remote-triple-store

Bug21940 - Using magic properties with filters could lose solutions

Bug21933 - Externally introduced variables could fail in nested sub-queries

Bug21921 - BGP triple filters with Chunk-at-a-Time processing

Bug21902 - Fix incompatibility between magic predicates and the identity reorderer

Bug21901 - Minor improvements to some range query clause orderings

Bug21892 - Query engine selection failed to take OFFSET into account

Bug21880 - Using with-variables in SPARQL describe caused an error

Bug21865 - Comparisons of xsd:dateTimes with fractional seconds

Bug21862 - Some FILTERs and BGP orderings could lead to empty bindings

Bug21858 - Distinct, SERVICE and sub-query could lead to planning failure

Bug21852 - Improve bound variable tracking

Bug21828 - VALUES could fail to correctly handle OPTIONAL bindings

Bug21815 - Fix property-path memory leak

Bug21803 - Exclusive ranges of encoded non-integer types

Rfe12591: Improve efficiency of some Basic Graph Pattern evaluations

AGWebView

Bug21997 - Agwebview and sessions

Changes to the Lisp API

Rfe12562 - upi->value round-trip

Rfe12557 - run-sparql can now use a UPI or future-part for the default base

Rfe12508 - normalizing with-variables should convert $var to ?var

Bug21953 - Support server-side loading via the Lisp client.

Bug21891 - db.agraph.log:log-output captured in dumplisp

Bug21860 - Make Lisp clients find bundled liblzo2.so.2

Bug21841 - db-get-triple leaks sockets

Bug21848 - load-turtle could use the wrong triple-store

Bug21810 - Polygon-vertexes did not work with future-parts

Bug21611 - Load patches in Lisp remote client

Bug21070 - Fix corruption hazard in direct Lisp client.

Bug21045 - create-freetext-index function failed when called with UPIs

Prolog

Rfe12222 - AllegroGraph Prolog unifies UPIs and future-parts

Documentation

Python Client

Java Client

Bug21953. Fix generation of some POST method requests.

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

Rfe12309 - Improve algebraic query manipulation

Rfe12305 - Relative paths in the configuration file

Rfe12280 - Simplify geospatial type-mapping API

Rfe12260 - Automatically map temporal predicate data to xsd:dateTimes

Rfe12259 - Optimize certain combinations of JOINs and FILTERs

Rfe12182 - More restrictive umask

Rfe12175 - Audit log

Rfe12151 - Remove support for loading N-Triples into a federated triple-store

Rfe12147 - Better manage memory especially for large solutions

Rfe12138 - Agload: reduced memory footprint and other enhancements

Rfe12125 - Improve performance of unordered queries with small limits

Rfe12111 - Improve memory checking efficiency

Rfe12089 - Improve cursor resource management

Rfe12063 - Allow in-memory-triple-stores to use blank nodes

Rfe11973 - Improve usage of UPI-map

Rfe11955 - :commit argument added to delete-duplicate-triples

Rfe11886 - Use statistics more often when reordering query clauses

Rfe11585 - Some queries unnecessarily accumulated partial results

Rfe11511 - Encode xsd:decimals

Rfe11197 - Serialize triple IDs as resources rather than literals

Rfe10292 - Handle filename limit for unix domain sockets

Bug21793 - Fix race in inter process communication

Bug21782 - Potential double close of cursors

Bug21779 - Fix low-level cursor resourcing bug

Bug21761 - Prolog q- functor could be treated as q

Bug21745 - Fix crash related to aggregate functions

Bug21744 - Some Prolog select queries could return duplicate results

Bug21740 - Rare hangs under high load

Bug21735 - Corrupted distinct results

Bug21702 - XSD casting in a BIND could convert to xsd:integer incorrectly

Bug21693 - Avoid forking too many blank node servers in agload.

Bug21691 - Reasoning over subproperties of symmetric properties

Bug21680 - Some BIND forms could be evaluated incorrectly.

Bug21670 - FILTER and EXISTS could interact incorrectly.

Bug21663 - Redo of new index creation confuses instance process

Bug21651 - Prolog ?? and reasoning

Bug21642 - Agload buffer overflow

Bug21640 - Race condition when reaping abandoned sessions

Bug21625 - Large result sets could produce bogus triples with INSERT and CONSTRUCT

Bug21596 - Serialize-rdf/xml could fail on some predicates

Bug21586 - Possible divide by zero error in index optimizer

Bug21580 - Improve external variable binding bookkeeping

Bug21570 - Fix 'No session ports available' error message

Bug21553 - Fix MemoryCheckWhen without MemoryReleaseThreshold

HTTP Client

Rfe12136 - Optimize front-end http server

SPARQL

Rfe12296 - Support AllegroGraph direct reification in SPARQL

Rfe12246 - Enhancements to SPARQL freetext queries

Rfe12243 - Support AllegroGraph Social Networking Analysis in SPARQL

Rfe12231 - Improve SPARQL geospatial integration

Rfe12171 - Improve variable GRAPH clause handling in some situations

Rfe12057 - Manage disk and memory resources better for CONSTRUCT queries

Rfe12035 - Improve handling of some SPARQL queries with a GRAPH clause

Rfe11319 - Support SPARQL federated queries (the SERVICE clause)

Rfe8063 - Support AllegroGraph temporal reasoning in SPARQL

Bug21757 - SPARQL geospatial queries could fail

Bug21742 - SPARQL parser failed on quad patterns with collections

Bug21725 - Correct potential error when SPARQL Update touches too many rows

Bug21718 - Unused variable optimization could produce incorrect results

Bug21573 - SPARQL queries could intern strings unnecessarily

Bug21011, Bug21571 - handle nested SPARQL BGP syntactic sugar correctly

Improve Pattern estimation where there is no :p index

AGWebView

Rfe12325 - Add audit log query interface to AGWebView

Changes to the Lisp API

Bug21708 - Fix some low-level Lisp API geospatial functions

Bug21689 - Lisp client printer variable leakage

Documentation

Python Client

Java Client

Rfe12118 - AGRepository with connection pooling

Rfe10513 - Configurable blankNodesPerRequest

Rfe9869 - Support SPARQL/Update from Jena

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

Rfe12004 - Ag4 plugins for TBC 4.0.0

Rfe11986 - Improve efficiency of some property-path queries

Rfe11937 - Agload: remove blank node cache size guessing code

Rfe11839 - Improve eager evaluation of impossible query constraints

Rfe11827 - Improved memory footprint of joins with unbound variables

Rfe11233 - Simplify property path queries when possible

Rfe9699 - Dynamically grow string table files

Bug21592 - Improve cursor resource management

Bug21583 - see Bug21575 below.

Bug21582 - Complex nested unions could fail to return some bindings

Bug21575, Bug21583 - property path optimizations and better bookkeeping

Bug21570 - Fix 'No session ports available' error message

Bug21550 - Complex FILTER EXISTS expressions could cause parsing errors

Bug21549 - Nested sub-queries could fail to project variables correctly

Bug21534 - Garbled error message on memoryCheckWhen parsing failure

Bug21506 - Correct possible error when mixing UNION and GRAPH

Bug21505 - Unused variables from a sub-query could confuse AllegroGraph

Bug21498 - Do not signal error when all ORDER BY expressions are redundant

Bug21497 - Filters could be misapplied in some situations

Bug21496 - Improve memory management for CONSTRUCT queries

Bug21493 - Bind is too aggressive in assuming single values

Bug21486 - Correct handling of some IN LINE data syntax

Bug21476 - Some blank nodes could fail to print in a 32-bit Lisp client

Bug21470 - Corrected Turtle parser relative URI normalization

Bug21458 - Standardize AllegroGraph exit code

Bug21454 - URI parsing was too strict

Bug21379 - Reasoning triple-store confusion about unspecified graph fixed

Bug21365 - Fix lookup of user access rights when authorizing access to repositories in the root catalog.

Bug21353 - Correctly handle aggregation on empty datasets

Bug21158 - Catch errors during filter analysis

HTTP Client

Rfe11609 - Support text/table content-type in the HTTP protocol

Bug20622 - "Mode" was not optional for DELETE /statements/duplicates

SPARQL

Rfe11920 - Signal error when aggregate expressions are used in the wrong place

Rfe11494 - Update semantics of SPARQL property path queries

Bug21599 - Variable GRAPH queries that used the graph could lose results

Bug21581 - Improve bookkeeping when using property paths

Bug21562 - Insert & remove-graph-uri(s) should not override WITH

Bug21551 - Deletes via SPARQL Update did not correctly handle encoded UPIs

Bug21548 - Nested unions with fixed patterns could miss results

Bug21546 - Correct off-by-one error in some SPARQL constraints

Bug21515 - Improve SPARQL parser speed

Bug21514 - Fix possible error caused by nested sub-query

Bug21507 - Fix problem with property path query clause estimation

Bug21478 - Fix spurious parsing error when mixing sub-query and aggregation

Bug21469 - Relative URIs were not being correctly normalized by the SPARQL parser

Bug21464 - Some combinations of nested UNION and OPTIONAL could fail

AGWebView

Changes to the Lisp API

Bug21535 - Db registry memory leak in the Lisp client

Bug21512 - With-variables not handled consistently with sub-queries

Documentation

Python Client

Java Client

Rfe11527 - Support sessions that use the main server port

Rfe8384 - Limiting the duration of a query

AllegroGraph 4.9 user-visible changes

AllegroGraph Server: General Changes

Rfe11922 - Add materiazer. See Materializer for details.

Rfe11879 - Improve error message on invalid license

Rfe11845 - Optimize blank node printing

Rfe11827 - Reduce memory footprint a bit

Rfe11784 - Release memory back to the operating system after query failure

Rfe11762 - Improve the efficiency of queries with unbound results

Rfe11756 - Improve the chunk processing behavior of ORDER BY and LIMIT queries

Rfe11728 - Improve handling of large DISTINCT or ORDERed results

Bug21531 - Non-simple EXISTS FILTERs could cause query failure

Bug21415 - Queries could return invalid solutions

Bug21410 - Duplicate suppression not handling deleted triples correctly

Bug21407 - An error in a SELECT expression prevented processing of subsequent ones

Bug21404 - Config file EOL format

Bug21398 - Variables in EXISTS filter could be incorrectly bound

Bug21390 - N-Triples serializer does not handle embedded angle brackets

Bug21377 - Updates with both DELETE and INSERT

Bug21376 - Wrong results with ORDER BY

Bug21367 - Expressions in ORDER BY could cause query failure

Bug21349 - Eager evaluation of boolean FILTERS could lead to failures

Bug21345 - Possible problem with BIND inside of an OPTIONAL

Bug21344 - Bind could fail to use external variable bindings

Bug21342 - Eager evaluation in FILTERs can lead to problems using variables

Bug21341 - Non-simple IF statements could cause an error

Bug21326 - Geospatial query failure

Bug21325 - Lost results in sub-queries and some joins

Bug21272 - No results with BIND

Bug21270 - Sub-queries with ORDER BY can fail to bind results

Bug21260 - XPath Constructor Functions fail on strings to numbers

Bug21256 - Parser problem with some queries that mixed aggregation and sub-query

Bug21245 - Allegrograph's turtle parser performs URI normalization incorrectly

Bug21243 - Non-QNameable URIs cause turtle serializer error

Bug21226 - agload max blank node cache size not obeyed in all cases

Bug21215 - Improve cursor resource usage of Prolog SELECT queries

Bug20906 - Remove duplicate triples fails on a store with zero triples

HTTP Client

Rfe11755 - Add support for insert-graph-uri, remove-graph-uri, etc.

Rfe11743 - Add baseURI to the SPARQL HTTP Protocol

Bug21408 - Retrieving JSON statements over HTTP

Bug21280 - Query failures cause spurious errors

SPARQL

Rfe11862 - Match default dataset behavior for SPARQL Update and Query

Rfe11817 - SPARQL parser should signal an error on badly formed requests

Rfe11793 - Improve handling of numeric and date literal equality filters

Rfe11787 - Support SPARQL 1.1 UUID and STRUUID functions

Rfe11779 - Support SPARQL 1.1 change to inline data syntax

Rfe11757 - Provide select-N strategy for ordered results with small limit

Rfe11555 - Manage memory used to ensure unique results from CONSTRUCT and UPDATE

Bug21401 - SPARQL Update must intern strings from variable bindings

Bug21391 - Queries against in-memory-triple-stores could return incorrect results

Bug21372 - Possible error when requesting sparql-xml output for Prolog select

Bug21371 - IF confusion with free variables

Bug21361 - Improve SPARQL BASE URI handling

Bug21357 - SPARQL blank-node parsing was too strict

Bug21354 - Update STRAFTER() and STRBEFORE() to match changes in the standard

Bug21294 - Sparql XML results for ASK queries missing head clause

Bug21277 - Improve cursor resource usage of SPARQL 1.1 queries

Bug21266 - Queries with only a geospatial clause could fail to gather solutions

Bug21249 - Incorrect return value of COPY GRAPH <X> TO GRAPH <X>

Bug21239 - Concurrent queries in shared backends

Bug21237 - Computations in aggregate expressions could fail

Bug21233 - SPIN functions could fail when used with constants

Bug21222 - Failing SPARQL UPDATE HTTP request should error

Bug21163 - Add new default-dataset-behavior: rdf

AGWebView

Rfe11910 - Add HTTP handlers for materialization

Rfe11856 - WebView - new UI Beta for Graph

Rfe11676 - Webview: in Scripts editor, use CodeMirror JavaScript

rfe11596 - WebView: new UI Beta for Query

Rfe11582 - Webview: upgrade Query Editor to CodeMirror v2.32

Bug21250 - Webview: remove limit from download

Changes to the Lisp API

Rfe11454 - get-triples-list (Lisp API) returns cursor only on request

Bug21381 - (Lisp API) SPARQL introspection functions required engine argument

Bug21378 - (Lisp API) sparql-1.1 engine does not implement destination-db

Documentation

Python Client

Rfe11911 - Add materializer calls to python client

Java Client

Rfe11906 - Java support for the materializer

Rfe11771 - Upgrade embedded JavaScript implementation

Rfe11241 - Support more query/update protocol params

AllegroGraph 4.8 user-visible changes

AllegroGraph Server: General Changes

Rfe11711 - Improve cursor resource usage of Geospatial Prolog queries

Rfe11653 - Rm requests can be processed out of order

Rfe11649 - Manage memory for ORDER BY and DISTINCT queries

Rfe11631 - Improve query optimizer when inference is enabled

Rfe11416 - Provide greater control of query memory usage

Bug21235 - Resource manager sync-files effectively ignores msync errors

Bug21224 - Crashes while processing queries with DISTINCT

Bug21214 - Correct scope of BINDings introduced on the LHS of union

Bug21187 - Stack exhaustion with shared cursor

Bug21182 - Unbound values in distinct aggregrates could lose results

Bug21145 - agload: message corrected for insufficient buffer size

Bug21136 - Free BGP patterns can lead to very large memory allocation requests

Bug21124 - Zero length path between non-matching constants

Bug20853 - Updated the soundex algorithm

HTTP Client

Rfe11681 - HTTP: Allow statements queries using the POST method

Rfe11680 - HTTP: Add /unique/[column] service

SPARQL

Rfe11682 - Improve performance of SPARQL OPTIONAL queries

Rfe11591 - Modified the Lisp run-sparql API to better handle query failure (e.g., for when a query runs out of time or other system resources).

Rfe11581 - handle repeated Prologues in SPARQL UPDATE queries

Rfe11559 - Add logQuery parameter to the HTTP SPARQL protocol

Bug21220: SPARQL Update LOAD should require only write permission (not eval)

Bug21203 - SPARQL Update insert data fails to treat blank nodes correctly

Bug21192 - Upi not in string table error when mixing BIND, SPARQL Update and FTI

Bug21168 - Support SPARQL 1.1 COUNT(DISTINCT *)

AGWebView

Changes to the Lisp API

Documentation

Python Client

Rfe11697 - Added RepositoryConnection.prepareUpdate

Java Client

Rfe11243 - Support X.509 authentication

Bug21225 - With no requestEntity, post using setRequestBody

AllegroGraph 4.7 user-visible changes

AllegroGraph Server: General Changes

Rfe11587 - Change InstanceTimeout defaults

Rfe11536 - Transitive inference queries could be slow

Rfe11285 - Improve query constraint analysis

Bug21114 - Instances lingering too long

Bug21094 - Queries against federations could signal UPI-not-in-string-table errors

Bug21089 - agload division by 0

Bug21077 - Expressions can be analyzed too eagerly

Bug21069 - A warm-standby client now exits non-zero if error occurs during commit replay

Bug21059 - Improve efficiency of all predicates cache when RDFS++ is enabled

Bug21057 - Fix log rotation causing crashes

Bug21040 - Turtle data import should use UTF-8 encoding

Bug21022 - Group expressions with unbound elements could be dropped

Bug21008 - Correct XSD datatype for certain division operations

Bug21007 - Error creating a session with non-default graph for inferred triples

HTTP Client

Rfe11601 - HTTP and WebView: MongoDB interface config

SPARQL

Rfe11558 - Add query logging support

Rfe11492 - Add support for SPARQL CSV and TSV

Rfe11486, Rfe11538 - improve SPARQL 1.1 engine geospatial integration

Bug21117 - Some SPARQL 1.1 path queries could parse incorrectly

Bug21093 - SPARQL UPDATEs could fail to save newly created strings

Bug21082 - Optional cross-product with no bindings could fail

Bug21078, bug21083 - Handle sub-query SELECT expresions and filters correctly

Bug21072 - unused variable check could remove variables in EXISTS filters

Bug21064 - Unneeded graph variables with DISTINCT query could error

Bug21046 - Multiple MINUS clauses could lead to query execution error

Bug21037 - Construct queries with Turtle results format can cause string lookup error

Bug21031 - Remove unstable optimization

Bug21023 - Correct datatype of the seconds() function

Bug21019 - Filter EXISTS with fixed triple patterns could fail

Bug21016 - UPDATE with blank nodes could lead to error

Bug20994 - Correct (in)equality test between plain and typed literals

Bug20840 - Improve comparisons between xsd:dates and xsd:dateTimes

Bug20736 - Federated-triple-stores can fail to find equivalent literals

Bug20770, Bug21020 - improve SPARQL 1.1 aggregation support

Bug20165 - Unnecessary variable in a BIND clause could cause error

AGWebView

Rfe11601 - MongoDB interface config

Rfe11557 - WebView: serve doc files

Rfe11474 - Execute query without changing text

Rfe11398 - Update query editor to CodeMirror v2.24

Bug21085 - Webview: Recent queries should not add items redundantly

Bug21062 - Recent queries in multiple browser tabs

Bug21050 - WebView: CodeMirror Query editor steals arrow keys

Bug21036 - Editor empty after 40 recent queries

Bug20998 - Webview: failures in IE7

Bug20982 - Query limit and planner selections ignored

Changes to the Lisp API

Bug21042, Bug21043 - standardize remote-triple-store SPARQL cursors

Bug20067 - Make session ping threads more resilient

Documentation

Bug21080 - Doc layout bad in IE intranet-mode

Python Client

Rfe11244: Python Support for client X.509 certificate authentication

Java Client

Rfe11523 - Upgrade to Sesame 2.6.5 jars

Rfe11358 - Upgrade Clojure to 1.4.0

Bug21099: java pool holds sockets open

Bug21092 - Clojure: errors in tutorial

AllegroGraph 4.6 user-visible changes

AllegroGraph Server: General Changes

Rfe11449 - Timeout for database instances

Rfe11445 - Queries with large intermediate result sets could generate segfaults

Rfe11440 - Remove trailing slash from server URL

Rfe11418 - improve efficiency of AllegroGraph's turtle parser

Rfe11404 - Optimized opening stores

Rfe11177 - Agload --relax-syntax flag

Rfe11126 - SSL client certificate and CRL support

Rfe8881 - Optionally suppress duplicates at commit time

Bug20997 - Solve latency issue while starting replication.

Bug20984: Turtle serializer emitted invalid blank node identifiers

Bug20981 - Exhausted cursor recycling happens too early

Bug20968 - Improve string dictionary cache performance

Bug20943 - Free text index bug under replication

Bug20913: Xsd date and time coercion functions could signal error

Bug20910 - Queries with variable predicates could lose results

Bug20896 - Hang when max chunks for an index reached

Bug20887 - AllegroGraph XML parsers should not follow external references

Bug20877 - xs:date could fail on some short plain literals

Bug20833 - openFederated should handle remote stores

Bug20822 - Cascading error can occur if database create/open fails

Bug20819 - problem with some combinations of multiple OPTIONAL clauses

Bug20814 - Make session urls unique

Bug20806 - fix loading of initfile and sitescripts

Bug20799 - turtle-parser was case sensitive when parsing numbers

Bug20793 - RDFS++ queries with free predicate could infer invalid triples

Bug20757 - Problems when starting agraph while another agraph is running

Bug20628 - Changes to warmstandby API

Bug20533 - Agload --rapper flag documentation

Bug20508 - Remote triple store can't use the RDFS++ reasoner

HTTP Client

Rfe10812 - Improve http tracing

Bug20798 - Improved http error handling

SPARQL

Rfe11469: add line number to SPARQL parser error messages

Rfe11357 - enhance SPARQL 1.1 engine for changes to standard

Bug20951 - SPARQL could fail to match plain and typed strings

Bug20939 - CLEAR only recognized if it's the first verb

Bug20936 - OFFSET 0 and LIMIT did not mix

Bug20912 - Inequality filters could signal an error

Bug20911 - SPARQL could lose precision on very small numbers

Bug20898 - Complex queries could fail on federated triple-stores

Bug20891 - Improve SPARQL comparator selection

Bug20884: improve Lisp client remote-triple-store SPARQL interface

Bug20867: REPLACE function in SPARQL 1.1 used incorrect syntax for matches

Bug20863 - Some SPARQL XSD numeric computations could fail

Bug20859 - Some filter combinations could cancel each other out

Bug20852 - remote-triple-store and SPARQL UPDATE don't mix

Bug20851 - SPARQL FILTERs could be too strict

Bug20838 - SPARQL CONSTRUCT queries could fail for large result sets

Bug20830 - Some inequalities could use the wrong comparisons

Bug20821 - EXISTS or NOT EXISTS and additional filters did not mix

Bug20805 - encodeforuri could produce incorrect results

AGWebView

Rfe10885 - Setting duplicate suppression from AGWebView

Bug20996 - Webview: js error creating a new repo

Bug20993 - Webview: Internet Explorer (IE) compatibility mode

Bug20963 - Fix session url in AGWebView

Bug20894 - AGWebView not working with ie9 on intranet

Bug20843 - AGWebView sometimes clears the editor

Changes to the Lisp API

Bug20884: improve Lisp client remote-triple-store SPARQL interface

Bug20883: improve the Lisp client's remote-triple-store caching interface

Bug20185 - Remote lisp client used wrong value for restriction reasoning

Documentation

Python Client

Rfe11313 - Remove export*, use response capture

Java Client

Rfe11493 - Support writing N-Quads from an AGModel

Rfe11438 - Provide N-Quads parser/writer META-INF/services in agraph.jar

Rfe11436 - Add Sesame 2.6.2 src jar

Rfe11397 - Support RepositoryFactory and Repository configuration

Rfe11240 - Support user/role management

Rfe11230 - Support masquerading as another user

Bug20946 - IE javascript error in Mold: innerHTML is null

Bug20811 - Allow prolog queries from javascript

AllegroGraph 4.5 user-visible changes

AllegroGraph Server: General Changes

Rfe11338 - CJK bigram tokenizer

Rfe11298 - Spread access to the same triple store across shared backends

Rfe11257 - Backup/restore user namespace settings

Rfe11099 - Add Apache Solr support to AllegroGraph

Bug20750 - Unused store closer holds lock too long

Bug20736 - Federated-triple-store could mistreat xsd:strings in queries

Bug20717 - Fix the UPI not in string table query issue

Bug20712 - Implement xs:date and xs:time XPath constructor functions

Bug20697 - Zero length property path with future parts

Bug20690 - Property path results cardinality

Bug20616 - Fix race condition in triple store termination

Avoid Logging a backtrace during unclean shutdown

HTTP Client

Rfe11308 - Support UPDATE via POST /repository/ID/statements

Rfe11213 - Allow superusers to run queries with another user's security filters

Bug20671 - List the Hub process in the HTTP API

SPARQL

Rfe11320 - Trying to use SERVICE should signal an error

Rfe11302 - Support clauses with no variables

Rfe11262 - Improve SPARQL 1.1 filter transformation

Rfe11254 - Improve SPARQL 1.1 sub-query implemenation

Rfe11245 - Optimize Queries where the right-hand side of a join is a BGP

Rfe11135 - Implement SPARQL 1.1 DISTINCT aggregation

Rfe11098 - Integrate Apache Solr with SPARQL 1.1 query engine

Rfe11070 - Improve performance of variable predicate queries

Bug20778 - Several fixes for various SPARQL 1.1 issues

Bug20760 - Several improvements in SPARQL aggregation.

Bug20756 - Multiple OPTIONAL clauses for the same variable could fail

Bug20752 - Queries with both filters and property paths could fail

Bug20732 - SPARQL 1.1 property path improvements

Bug20723 - Improve handling of SPARQL filter predicates

Bug20722 - SPARQL 1.1 parser treated whitespace differently

Bug20721 - Improve handling of certain OPTIONAL edge cases

Bug20711 - Improve some date and dateTime comparison issues

Bug20695 - Problem when computing range-queries on dateTimes

Bug20689 - Parsing of SPARQL property path expressions where a group is followed by a + as in

AGWebView

Bug20751 - Agwebview could submit queries twice in some browsers

Bug20738 - AGWebView could not start remote sessions

Changes to the Lisp API

Documentation

Rfe11236 - Fix command line documentation of the agdebug tool

Python Client

Rfe11229 - Add support for user masquerading to python client

Java Client

Rfe11329: AGConnPool, remove unnecessary dependency on Jena

Rfe11310: AGConnPool should rollback when borrowing

Rfe11242: Upgrade to Sesame 2.6.2 jars

Bug20795: AGConnPool does not close connections

AllegroGraph 4.4.0.1 user-visible changes

AllegroGraph Server: General Changes

Bug20667 - XSD float exponent fix

Bug20615 - TrxLogRetain replication now works on secondaries

HTTP Client

SPARQL

Rfe11219 - Add query option support for all SPARQL constructs

Rfe11218 - Improve maximum solution size handling

Rfe11202: SPARQL Update example with batch update

Bug20768 - SPARQL 1.1 property paths could return incorrect results

Bug20692 - Sparql 1.1 CONTAINS function can fail

Bug20676 - Geospatial datatype mappings could cause query errors

Bug20665 - SPARQL-1.1 UNION doesn't see with-variable bindings on the RHS

Bug20664 - SPARQL-1.1 CONCAT function signaling errors with typed literals

AGWebView

Rfe11198 - Improve WebView's bulk loading UI

Bug20662 - Reload triple count in WebView if load fails

Bug20641 - Improve restoring backups from WebView

Bug20637 - Fix problem where SPARQL Update did not work with AGWebView

Changes to the Lisp API

Bug20640 - Fix printing of encoded-ids in remote lisp client

Documentation

Python Client

Java Client

Rfe11202: SPARQL Update example with batch update

AllegroGraph 4.4 user-visible changes

AllegroGraph Server: General Changes

Rfe11082: Transactional duplicate deletion

Rfe11039 - Include AllegroGraph version in ps output

Rfe11017 - Add an agdebug command line utility

Rfe10984 - Fix serialization issues with encoded IDs

Rfe10981 - Improve upgrading/migration process

Rfe10948 - Log secondary index creation

Rfe10853 - Support fti charset in Python client

Rfe10783 - Add per triple-store control of secondary indices

Rfe10568, Rfe11172 - Set the default query engine via agraph.cfg

Rfe10179, Rfe10540 - SNA optimizations

Rfe10398 - Improved speed of low-level AllegroGraph operations

Rfe10090, Rfe10541 - optimizations to AllegroGraph secondary indices

Rfe9786 - Make shm server global

Bug20619 Optionally not inherit trx log arch params during restore

Bug20460 - Encoded IDs and reasoning triple-stores mix not

Bug20443 - Encoded IDs did not work with the TriX serializer

Bug20355 - Durably save transactions on warm-standby client

Bug20332 - Trix parser can get confused and try to encode plain literals

Bug20230 - Fix deletion slowdown under replication

Bug19957 - Turtle parser should support type-mapping

Spr38044 - Restrict possible values for TransactionLogRetain

HTTP Client

SPARQL

Rfe10985 - Spin functions and magic-properties

Rfe10737 - SPARQL 1.1 BIND support

Rfe10736, Rfe10743 - SPARQL 1.1. exists, not exists, and minus

Rfe10700 - SPARQL 1.1 assignment

Rfe10699 - SPARQL 1.1 UPDATE support

AGWebView

Rfe11131: Allow triggering deletion in WebView

Rfe10631 - Allow loading files via agload in WebView

Bug20492 - Improve code editor in WebView

Changes to the Lisp API

Rfe11082: Transactional duplicate deletion (repeated from the General section)

Documentation

Python Client

Rfe11122 - Add "delete duplicates" API function to python client

Java Client

Rfe11121 - Support transactional duplicate deletion

Rfe11117: java api for SPIN (adds to rfe10988)

Rfe10622 - Http response handlers now have their own package

AllegroGraph 4.3.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10984 - Fix serialization issues with encoded IDs

Rfe10967 - Nicer CSV output option

Bug20460 - Encoded IDs and reasoning triple-stores failure

Bug20451 - Fix encoded ID API on remote lisp client

Bug20443 - Encoded IDs did not work with the TriX serializer

Bug20416 - Use proper n-triples syntax for text/csv

HTTP Client

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

Rfe10983 - Query execution with initial bindings

Welcome to AllegroGraph 4.3.2

AllegroGraph 4.3.2 user-visible changes

AllegroGraph Server: General Changes

HTTP Client

Bug20424 - Disable 5-minute HTTP request timeout

SPARQL

AGWebView

Changes to the Lisp API

Documentation

Python Client

Java Client

Welcome to AllegroGraph 4.3.1

AllegroGraph 4.3.1 user-visible changes

AllegroGraph Server: General Changes

Rfe10889 - Provide guaranteed duplicate removal behavior

Rfe10850 - Ag3 plugins for TBC 3.5

Bug20408 - Implement encoded-ids for lisp remote-triple-store

Bug20397 - Load triples from files faster on remote lisp client

Bug20378 - Improve duplicate reduction in UPI-map based cursors

Bug20372 - Fix part printing for literals and resources with same name

Bug20355 - Durably save transactions on warm-standby client

Bug20336 - Fix inconsistency on warm standby secondaries

Bug20230 - Fix deletion slowdown under replication

HTTP Client

Rfe10893 - Allow "unlimited" concurrent HttpClient connections

SPARQL

Bug20382 - Sparql queries that returned * could confuse remote-triple-stores

AGWebView

Rfe10924 - Enable download of server log file from WebView

Changes to the Lisp API

Documentation

Python Client

Java Client

Bug19798 - Illegal resource in Jena reified statements

Welcome to AllegroGraph 4.3

AllegroGraph 4.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10790 - improve logic for finding the sort-upi-map binary

Rfe10665 - AG4 plugins for TBC 3.5.0

Rfe10636, Bug20160 - signal an error if a blank node ID is too big, support for more blank nodes per store

Rfe10632 - Enable per-user security-filtering of stores.

Rfe10090, Rfe10541 - Optimizations to AllegroGraph secondary indices

Rfe10508 - Japanese full text search

Rfe10020 - add support for owl:SymmetricProperty

Rfe8278 - Add ability to customize the tokenizer used by freetext indexing

Bug20289 - Do not allow stored procedures to enter the debugger

Bug20273 - Suboptimal index queries broken

Bug20160, Rfe10636 - signal an error if a blank node ID is too big, support for more blank nodes per store

Bug20133 - Select queries should create filtered triple-stores only as needed

Bug19834 - Resource manager connection management bugs

Bug20037 - allow trailing slash on Server URL in AG4 dialogs

HTTP Client

Rfe10839 - Freetext index charsets and tokenizer in HTTP API

Bug20250 - Fix federated remote stores in HTTP API

SPARQL

Bug20305 - Encoded IDs were incompatible with SPARQL CONSTRUCT queries

Bug20288 - Encoded IDs don't work with SPARQL/XML and SPARQL/JSON output

AGWebView

Rfe10728 - Improve WebView store overview user interface

Bug20304 - Fix codemirror for Internet Explorer 9

Bug20302 - Fix loading more results in Internet Explorer

Bug20296 - Fix query result downloads in Firefox 3.

Bug20203 - Fix WebView for newer Internet Explorer versions.

Bug20069 - Uploading file into a context from WebView is broken

Changes to the Lisp API

Bug20146 - Export UPI hash-table functions from db.agraph.user

Bug20145 - (Lisp API only) loading AllegroGraph should provide module :agraph4.

Bug20144 - Fix interference from remote-triple-store ping

Documentation

Bug20149 - Correct documentation of datatype and predicate mappings (Lisp)

Python Client

Java Client

Rfe10135 - Improve javadocs

Rfe10745 - Add Jena text indexing code example

Rfe10624 - AGQuery limit and offset modifiers

Rfe10623 - Jena AGQueryExecution count methods

Rfe10492 and Rfe10493 - javadoc for AGRepositoryConnection

Rfe10452 - support N-Quads responses in the Java client

Rfe10450 - Jena Prolog and transaction code examples

Rfe10447 - add AGGraphQuery#count() method

Rfe10446 - add count() examples to the Java tutorial

Rfe10445 - add example/test getting triple ids via prolog

Rfe10411 - Streaming large query results

Rfe9445 - Upgrade slf4j, funnel through jcl, and add source jars

Welcome to AllegroGraph 4.2.1c

AllegroGraph 4.2.1c user-visible changes

AllegroGraph Server: General Changes

Clarify what retain for replication means

Bug20257 - fix convertdb for some rare cases

Bug20236 - Fix startup when HTTPProxy is specified in config

Bug20223 - Fix agload for Turtle files

HTTP Client

SPARQL

AGWebView

Bug20200 - Clarify format support in import forms in WebView.

Changes to the Lisp API

Rfe10790 - improve logic for finding the sort-upi-map binary

Documentation

Python Client

Java Client

Welcome to AllegroGraph 4.2.1a

AllegroGraph Server: General Changes

Rfe10689 - Increase unique string limit to ~4B

Rfe10562 - Change encoded-id marker to @@ (from @#)

Rfe10532 - Return encoded ids in ascending order

Rfe10521 - Deleted hash table management based on generation

Rfe10200 - Regulate speed of mergers

Welcome to AllegroGraph 4.2.1.

AllegroGraph Server: General Changes

Rfe10483, Rfe10528, Bug20012 - Inadequate transaction log durability

Rfe10475 - Improve I/O behavior under memory pressure

Rfe10439 - RDSF++ reasoner now does not create duplicate inferred triples

Rfe10434 - Rename Lisp API function to duplicate-deletion-strategy

Rfe10413 - Add check for 64-bit Linux

Rfe10400 - Minor cursor implementation changes

Rfe10399 - Verify lisp client and server version match

Rfe10363 - Improved error message when a file cannot be found

Rfe10354 - Redesigned --quiesce feature of agtestclient

Rfe10345 - Don't log warnings when opening NFS triple-stores.

Rfe10342 - Command line wrapper for the lubm-generator Lisp module

Rfe10337 - Improve cursor resource management

Rfe10335 - Improve logging of (adjusted) store parameters

Rfe10308 - AllegroGraph can use UPI-maps to run queries in more cases

Rfe10243 - opening database should honor existing merge mode as default

Rfe10184 - Improve custom hash table efficiency

Rfe10114 - Change default for TransactionLogRetain to "recovery"

Rfe9338 - Improved error message due to session port connection failure

Bug20122 - Handle disk full on transaction log writing

Bug20118 - graph type-mapping could lead to incorrect data

Bug20089 - encoded-ids not working in non-object fields for clients

Bug20050 - Improve speed of bulk load mode in agload

Bug20031 - agraph-backup backs up the same file twice

Bug19996 - Repair durability guarantees for NFS stores.

Bug19969 - Correct problem printing times with fractional seconds

Bug19954 - Commit/rollback could sometimes result in crash

Bug19950 - Handling of tranaction log files corrected

Bug19933 - Made agload recognize "nq" file format (nquads).

Bug19916 - Improve :terse output

Bug19836 - Date and datetime with negative year

Bug19682 - Federated-triple-stores error for type-mappings

Bug19116 - Datatype-mappings are not applied to graph

HTTP Client

Rfe10360 - Better HTTP error messages for encoded-id errors.

Bug20124 - Make number of initial HTTP workers configurable.

Bug20022 - Add location-rewriting to the AG reverse-proxy.

Bug20013 - Requests to back-ends with Expect: headers fail

Bug19948 - Allow full HTTP delete-duplicates control

SPARQL

Rfe10486 - SPARQL query planning should not require the PSOGI index

Rfe10374 - Improve efficiency of some SPARQL queries

Rfe10371 - Improve SPARQL's handling of DISTINCT LIMIT queries

Rfe10348 - Iri= for subject and predicates (generalize SPARQL constraints)

Rfe10344 - Improve SPARQL statistical BGP reordering logic

Rfe10312 - Optimize IRI= for object fields in SPARQL queries

Rfe10118 - Added interface to SPARQL query analysis to Python client

Rfe9507 - Add initial support for SPARQL subqueries

Bug20070 - fix problem with geospatial encoded UPIs and some output formats

Bug20047 - Can't download query results with missing values as CSV

Bug19989 - Sparql CONSTRUCT could generate invalid blank nodes

Bug19832 - Correct bug in low level SPARQL equality code

AGWebView

Rfe10370 - Use only relative URLs in WebView

Bug20051 - Downloading query results from WebView was broken

Bug20016 - Fix horizontal scrolling in WebView for Firefox

Bug20009 - 4.2 WebView can't handle some data saved by 4.1

Bug19972 - Webview breaks when it doesn't find JSON support

Changes to the Lisp API

Rfe10420 - Add lisp client function optimize-indices

Rfe10066 - Add option to use rapper for load-turtle

Rfe9715 - Lisp client package now includes Lisp tutorial examples

Bug19672 - Triple-store-exists-p leaks server information

Bug19133 - get-triple now applies geospatial mappings

Documentation

New JavaScript document for programming the server

New Security document describing data security

Rfe10341 - Improve documentation search

Python Client

Rfe10505 - use session to get new blankNodeIds

Rfe10418 - Call indices/optimize from Python

Rfe10326 - Support deleteDuplicates setting from Python client

Rfe10325 - Added evalJavaScript to Python client

Java Client

Rfe10389 - Upgrade to sesame 2.3.2

Rfe10373 - Support deleteDuplicates mode in java

Rfe10372 - Support periodic commit during add/load

Rfe10257 - Support script loading in sessions

Rfe10256 - Support loadInitFile switch in java

Rfe10201 - Support adding N-Quads from java

Rfe10177 - Support getting statements by id

Rfe10174 - Support adding Turtle from Sesame and Jena

Rfe10059 - SPOGI triple cache management

Rfe9436 - Session lifetime management

Rfe8362 - Allow geospatial queries against graph/context field

Bug19906 - Writing a JavaScript NaN value as JSON loops forever

Bug19342 - Java LUBM Prolog and SPARQL queries

Welcome to AllegroGraph 4.2

AllegroGraph Server: General Changes

Rfe10293 - Warn against using ReiserFS.

Rfe10267 - Do not request duplicate removal if there are no duplicates detected

Rfe10255 - Allow minimum text-index word size of 1

Rfe10216 - Support alternative tlog synchronization methods.

Rfe10191 - Have install-agraph pass its arguments to configure-agraph

Rfe10182 - Improve efficiency when waiting for an in-memory chunk

Rfe10173 - Upgrade plugins to support TBC 3.4.0-Beta1

Rfe10167 - Log euid and runas information at daemon startup.

Rfe10162 - Perform a clean shutdown on agraph-control stop

Rfe10156, Rfe10157 - Improve lock acquisition and store creation on NFS.

Rfe10149 - Support setting a graph for inferred triples in HTTP sessions

Rfe10147 - Export 'custom-service' macro from the db.agraph package

Rfe10143 - support bulkMode in connections

Rfe10141, Rfe10142 - Use UPI-maps more often inProlog queries

Rfe10117 - implement run-time (dynamic) query index analysis

Rfe10069 - AllegroGraph v4.0.6 introduced a dependency on a patch to update message digest functions. Now, loading the AllegroGraph client will fail early if this patch is missing, and will report an actionable error.

Rfe10021 - Support Soundex freetext indices

Rfe9877 - Lazier collection of FTI results

Rfe9861 - Improve method of sending an x-rdftransaction

Rfe9746 - Various improvements to agload

Rfe9657 - Unattended execution of configuration script

Rfe9631, Rfe10028 - db-get-triples improvements, UPI map usage

Rfe9622 - support getting the server's version information

Rfe9517 - Unify AServe and AGraph messages in the AGraph log

Rfe9515 - Drop deleted triples from text index during merge.

Rfe9470 - Reduce madvise calls during compressed index lookup

Rfe9296 - Transactionally safe duplicate triple elimination

Rfe8414 - Add encoded-ids to AllegroGraph

Bug19910: agraph-recover now returns 0 when successful, 1 otherwise

Bug19897 - SHM hangs if socket creation fails

Bug19891 - Fix log level test for log-backtrace and when-debugging.

Bug19888 - Prevent logging gratuitous errors on forced shutdown.

Bug19864 - Some cursors could fail to correctly free resources

Bug19848 -- Improve behavior of qm when UPI-maps are not available

Bug19842 - Error message server-pid-and-client-pid-must-match-error

Bug19813 - Fix store parameter merging for configured values.

Bug19816 - FTI Merger sometimes produces corrupt chunks

Bug19800 - support empty enumerations in owl:oneOf

Bug19791 - Fix checkdb's --port argument.

Bug19786 - Checkdb now exits with 0 on success.

Bug19785 - Convertdb now exits with 0 on success.

Bug19777 - Replication to multiple secondaries improved

Bug19766 - Sub-optimal index selection

Bug19773 - Dropping an index is not crash recovery proof

Bug19765 - Detect more cases of string table corruption.

Bug19764 - N-Quad output was incorrect

Bug19697 - improve performance of part printing

Bug19657 - Fix race-condition in text-index merger

Bug19600 - direct-upi wire format does not handle nil values

Bug19587 - Suboptimal encoding of some characters in shm-queue.

Bug19578 - Clearnamespaces XMLTransaction tag is broken

Bug19550 - Fix reification in rdf/xml parser

Bug19549 - RDF/XML parser loses the trailing /

Bug19530 - Fix off-by-one error in text-index code.

Bug19522 - Prevent spurious warning in some Prolog select queries

Bug19517 - improve Lisp API for remote-triple-stores

Bug19515 - Correct precision loss in printing of RDF time and dateTimes

Bug19508 - Prevent text-index from leaking files on crashes

HTTP Client

Rfe10278 - Alphabetize store and catalog lists in HTTP responses

Rfe10234 - Inefficient limited Prolog queries over HTTP

Rfe10230 - Accept application/x-turtle as a synonym for text/turtle

Rfe10220 - Provide an HTTP interface for controlling bulk mode

Rfe10210 - Don't backtrace on unsupported-db-version error

Rfe10165 - Expose turtle loading over HTTP.

Rfe10078 - Make HTTP protocol no longer hide inaccessible repositories.

Rfe10009 - Implement Turtle loading

Rfe9986 - Support Sesame 2.2-style XML transactions

Rfe9912 - Remove SharedMemSize parameter from HTTP protocol and HTTP docs

Rfe9910 - Reuse sockets for backend-frontend communication

Rfe9838 - Use HTTP keep-alive in the remote Lisp client.

Bug19909 - Inter-server HTTP communication should use local addresses

Bug19871 - HTTP server does not unescape \u in resources

Bug19846 - Ensure the HTTP client explicitly frees SPARQL cursors

Bug19801 - HTTP character encoding should not depend on system locale

Bug19661 - HTTP JSON output for query results truncates row on first unbound result.

Bug19616 - Allow multiple contexts to be specified in HTTP /size

SPARQL

Rfe10207 - Efficiency improvement for some SPARQL graph queries

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing

Rfe10030 - Share statistical planner between Prolog and SPARQL

Bug19809 - fti:matchExpression is sometimes treated like fti:match

Bug19681 - fix SPARQL problem in queries using both FROM and FROM NAMED

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing (repeated from Rfe10061 listing)

AGWebView

Rfe10300 - Allow optimizing of stores from WebView

Rfe10280 - Expose scripting in WebView

Rfe10277 - Improve help text for WebView part input

Rfe10244 - Add JavaScript server-scripting support

Rfe10236 - Disable query planner selector in WebView for Prolog queries

Rfe10185 - Allow the user to specify a limit for queries in WebView

Rfe10181 - Leave WebView widgets in the default browser style

Rfe10155 - Allow a query planner to be selected in WebView

Rfe10128 - Rework WebView namespace management

Rfe10126 - Show PIDs in WebView's process list

Rfe10116 - Show running sessions in WebView

Rfe10039 - Support backup and restore through WebView.

Bug19880: Resizing the query input field in WebView is broken

Bug19868 - Webview tries to write to the installation dir

Bug19858 - AGWebView accidentally shows Turtle as export format

Bug19762 - Fix problem in process page in WebView

Bug19741 - Add example to context input field help in AGWebView import

Bug19523 - Fix several bugs in WebView's query result display.

Changes to the Lisp API

Rfe10296 - Check minimum build id on client startup.

Rfe7148 - Optimize away keyword scanning in get-triples

Bug19890 - load-rdf/xml-from-string (Lisp API) could cause memory error

Bug19678 - serialize-rdf-n3 fails with encoded UPIs

Bug19069 - improve error handling of N-Triples parser

Documentation

Rfe9657 - server-installation.html document changed to describe non-interactive running of configure-allegrograph.

Python Client

Rfe10221 - BulkMode support added to Python client

Java Client

Rfe10303 - Support revised bulk-load mode interface

Rfe10197 - Support for encodable-ids and generated URIs

Rfe10189 - Java Interface for Stored Procedures

Rfe10119 - Add analyze() method to provide query analysis

Rfe9971 - JAVA API for HTTP protocol checkVariables

Rfe9858 - Support ResultSet nextSolution() method

Rfe9846 - JENA graph unions and graph-scoped reasoning

Rfe9534 - Add source jar to the client download

Welcome to AllegroGraph 4.1.1

AllegroGraph Server: General Changes

Rfe9796 - Make tlog-write-checkpoint-indicator use :allocate-temporary

Rfe8994 - Change behavior when a filesystem is full

Rfe9319 - Freetext index configuration and search support

Rfe9534 - Add sources for jena, arq, and json jars

Rfe9969 - Update clojure api

Rfe9999 - Auto-adjust MinMergeChunks

Rfe10119 - Add analyze() method to provide query analysis

Bug19658 - Agload needs bin directory symlink

Bug19674 - After-the-fact index creation should mind the amount of files it opens

Bug19698 - Changes to prepareGraphQuery and prepareBooleanQuery

Bug19720 - Fix "don't attach as root" check

Bug19739 - Compressed index files not unmapped when closed

HTTP Client

Rfe9756 - Return 400, not 500, when trying to delete an open store over HTTP.

Rfe9971 - Java api for http protocol checkVariables

Rfe10120 - Make sure custom-service exists in agraph4.fasl

SPARQL

Bug19157 - SPARQL error if ORDER variable isn't in query

Bug19618 - Properly escape strings in SPARQL/JSON output.

Bug19619 - Fix typed literal writing in SPARQL/JSON output.

AGWebView

Rfe9492 - Add export to AGWebview

Rfe9587 - Show progress indication in WebView uploads

Rfe9898 - Add drop-down menus to WebView's navigation bar.

Rfe9900 - Optimize loading of WebView scripts and style-sheets.

Rfe9950 - Provide a way to start exploring a store in WebView

Rfe9951 - In WebView user management, show drop-downs for stores.

Rfe9964 - Ask confirmation before letting user un-superuser himself in WebView

Rfe9990 - Do not display \uHHHH character escapes in WebView

Rfe10025 - More robust recent-query saving in WebView

Rfe10026 - WebView now shows a spinning animation when loading a file

Rfe10027 - In WebView's graph view, fix nodes after dragging

Rfe10050 - Make specifying a graph when adding from WebView easier.

Rfe10051 - Allow a graph/context to be specified when importing in WebView.

Rfe10052 - Add a general delete-triples UI to WebView

Rfe10127 - Allow init files to be loaded in sessions created from WebView

Bug19581 - Newly registered users in WebView don't get enough access

Bug19536 - In WebView's graph view, clicking on a node was broken.

Bug19552 - Can't start a session from catalog page in WebView

Documentation

Rfe9574 - Lisp client documentation

Rfe9750 - Improve document navigation

Rfe9883 - Javadoc included in the distribution

Rfe9929 - User manual for 4.1.1 AGWebView

Rfe9936 - Document TopBraid Composer plugin

Rfe10046 - AllegroGraph Text Indexes

Rfe10077 - Add user-selected triple indices to tutorials

Rfe10108 - AllegroGraph Triple Indexes

Rfe10139 - Emphasize "agraph" user vs. AG superuser

Rfe10159 - Rapper documentation

Bug19620 - Fix typos in documentation of create-freetext-index

Welcome to AllegroGraph 4.1

Rfe9535 - Use managed files throughout the storage layer

Rfe9772 - More advanced server scripting

Rfe9784 - Keep track of which pages of deleted bitmap have been touched

Rfe9785 - Revise backup and restore

Rfe9789 - Support callback parameter when writing JSON output

Rfe9798 - Allow relative directories in config files

Rfe9805 - Point in time recovery

Rfe9835 - Use stream API to allocate blocks in sparse deleted bitmap

Rfe9840 - Improve "disk full" behavior in agraph-backup

Rfe9842 - Agraph-backup: merge parameters.dat while restoring

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

Rfe9889 - Allow resource manager to forget a file

Rfe10055 - Properly expose the no-commit flag

Rfe10056 - Require authentication for warm-standby connections.

Bug19391 - Agraph-install no longer outputs paths with double slashes

Bug19420 - Chunk writers leak disk space when db crashes

Bug19446 - Catalog parsing problem - confusing error message

Bug19487 - Solve ':void external-format' errors in logs.

Bug19526 - Back up tlog file only up to backup checkpoint

Bug19684 - Include the triple-id index in the standard indices

Lisp Client

Bug19442 - Make sure remote-triple-store accepts :server argument.

SPARQL

Bug19474 - Improve SPARQL describe queries

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

AGWebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

Rfe9899 - Fix typo and wrong word choice in WebView.

Bug19500 - Fix problem with displaying geospatial results in WebView

Bug19525 - Fix failure when opening a federated session in WebView.

Documentation

Spr37148 - Question about connecting using java

Rfe9783 - AllegroGraph's isolation model

Rfe9806 - Enhance SPARQL examples in tutorials

Rfe9821 - Make SPARQL examples more robust

Rfe9831 - Python/Java tutorials, illustrate use of conn.size() with contexts.

Rfe9915 - Java and Python tutorials demonstrate SPARQL and RDFS++

Rfe10104 - Point-in-time recovery document

Rfe10105 - Replication and warm-standby document

Rfe10106 - Update backup/restore document

Rfe10107 - Transaction Log Archiving document

Bug19454 - Python tutorial explains how "int" becomes "long"

Document MinimumCheckpointInterval and MaxRecoveryTime parameters

Welcome to AllegroGraph 4.0.6a

AllegroGraph Server: General Changes

Rfe9837 - Avoid log warning when opening an existing repo

Rfe9980 - getCatalog now works with the root catalog

Rfe9983 - Support the AllegroGraph Lisp Client in ANSI Lisp

Rfe10041 - Implement owl:oneOf

Rfe10075 - Implement query index use analyzer, version 1

Bug19650 - Fix problem federating repositories from the root catalog

Welcome to AllegroGraph 4.0.6

AllegroGraph Server: General Changes

Rfe9535 - Use managed files throughout the storage layer

Rfe9620 - Increase limit on number of index chunks

Rfe9731 - Change transaction log aligment calculation

Rfe9762 - Add someValuesFrom and allValuesFrom restriction reasoning

Rfe9769 - Log merge statistics

Rfe9772 - More advanced server scripting

Rfe9776 - Standardize blank node serialization

Rfe9789 - Support callback parameter when writing JSON output

Rfe9803 - Bulk loading support

Rfe9812 - Add client API for triple-store index manipulation

Rfe9889 - Allow resource manager to forget a file

Rfe9907 - Add with-indices support to agload

Rfe9913 - Improve single-float printing

Rfe9930 - Support adding, dropping, listing repository indices

Rfe9953 - Share transaction log buffer across attachees

Rfe9966 - Front-end server-socket backlog parameter is too low

Rfe9970 - Upgrade path for configurable indices

Rfe9972 - Updates to database upgrade code

Rfe9992 - Backup/restore improvements

Rfe10012 - Consistency checker for sstab

Rfe10019 - Setup automatic datatype-mapping for triple-ids

Bug19171 - Importing RDF files with long text literals is much faster.

Bug19181 - Correct newline parsing for n-triple imports

Bug19253 - Accept resources for :base-uri in load-rdf/xml.

Bug19266 - Correct cursor re-use problem

Bug19391 - Agraph-install no longer outputs paths with double slashes

Bug19407 - Allow the direct Lisp client to log to the server log file

Bug19420 - Chunk writers leak disk space when db crashes

Bug19442 - Make sure remote-triple-store accepts :server argument.

Bug19446 - Catalog parsing problem - confusing error message

Bug19460 - Issue installing agraph 4.0.5 over 4.0.4

Bug19569 - Java client now helps delete orphaned saved queries

Bug19474 - Improve SPARQL describe queries

Bug19487 - Solve ':void external-format' errors in logs.

Bug19489 - Directory where pidfile resides is chown'd to agraph user

Bug19497 - Fix convertdb

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Bug19509 - Agload could load inconsistently on virtual machines

Bug19539 - Support dynamic catalogs

Bug19548 - Correctly import default namespaces in rdf/xml literals.

Bug19551 - Fix handling of blank nodes in RDF/XML parser.

Bug19560 - Provide sstab upsizing upgrade path

Bug19562 - Improve error message when AllegroGraph loaded on an unsupported platform

Bug19565 - Crash can occur when file descriptor limit is large

Bug19567 - Log handles remain open after SIGHUP

Bug19595 - Repair createGraph() for anonymous graphs

Bug19602 - String table fails to recover short strings from transaction log

Bug19614 - Bus error during prolog query

Bug19632 - N-Quads serializer was emitting the default-graph incorrectly

HTTP Interface

Rfe9885 - Fix HTTP connection keep-alive for 204 responses.

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

Rfe9906 - Support index inspection and manipulation over HTTP.

Bug19506 - Make HTTP server more cautious about closing stores

Documentation

Rfe9936 - User document for Top Braid Composer and AG

Lisp Client

Bug19452 - Race condition in lisp client startup

Python Client

Rfe9906 - Add support for index changes to the Python client

Rfe9965 - Python miniclient now supports retry on ECONNRESET

Java Client

Rfe9845 - Support owl:Restriction reasoning in the Java client

SPARQL

Bug19474 - Improve SPARQL describe queries

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

WebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

Bug19500 - Fix problem with displaying geospatial results in WebView

Bug19525 - Fix failure when opening a federated session in WebView.

Welcome to AllegroGraph 4.0.5d

AllegroGraph Server: General Changes

Rfe9966 - Front-end server-socket backlog parameter is too low

Welcome to AllegroGraph 4.0.5c

AllegroGraph Server: General Changes

Rfe9887 - Exclude inferences from Sesame queries by default

Bug19484 - Jena always uses reasoning

Bug19485 - setAutoCommit(true) causes exception

Bug19491 - Jena reasoning not on full graph

Welcome to AllegroGraph 4.0.5b

AllegroGraph Server: General Changes

Rfe9866 - Increase limit on string file sizes

Bug19503 - Reclaim cursor resources more aggressively

Welcome to AllegroGraph 4.0.5

AllegroGraph Server: General Changes

Rfe9334 - License support for agraph with triple-limit and expiration

Rfe9348 - Added offset keyword argument to getStatements/evalFreeTextSearch

Rfe9471 - Limit maximum checkpoint frequency by configuration variable

Rfe9564 - Alternate mode for checkpoint triggering

Rfe9604 - Suppress unnecessary checkpoints

Rfe9608 - Have HTTP /eval service handle multiple returned values

Rfe9691 - Remove race condition in internal API

Rfe9709 - Improved automated test for database recovery

Rfe9713 - Document load-nquads

Rfe9726 - Minor agload improvements

Rfe9729: Make it possible to disable interval based checkpointing

Rfe9732 - Disable interval based checkpointing when recovery time based checkpointing is enabled

Rfe9766 - Ensure that the AllegroGraph server and the Lisp client use the same UID

Bug19101 - with-variables->bindings-array is too strict

Bug19168, Bug19332 - Ensure that user functions are compiled correctly

Bug19209 - Fix bug with reasoner and rollback

Bug19339 - Update UPI serialization

Bug19429 - Problems with catalog option FilePlacementRule in agraph.cfg

Bug19431 - Wrong parameters logged during database startup

Changes to the Lisp API

Rfe9643 - Remove synchronize-automatically from the Lisp API

Bug19188 - [Lisp API] create-triple-store :if-exists :error problem

Documentation

Rfe9475 - Document catalog parameters vs database parameters

Rfe9721 - Restore Lisp freetext tutorial

Bug19411 - Configuration and performance documentation updated

Java Client

Rfe9743 - Upgrade to Sesame 2.3.1

Jena Client

Rfe9744 - Upgrade to Jena 2.6.2

Welcome to AllegroGraph 4.0.4

AllegroGraph Server: General Changes

Rfe8792 - Clean up partially created triple stores

Rfe9123 - Extend test functions to start and stop db server

Rfe9292 - AllegroGraph Loader

Rfe9367 - AllegroGraph log should include the config file

Rfe9451 - Pause text-index merges during checkpoints

Rfe9497 - Ensure that select queries release temporary indices

Rfe9508 - Add configuration options for manual control of file placement

Rfe9518 - SHM server handles client failures

Rfe9579 - Include catalog name in conditions signaled by the storage layer

Rfe9580 - Use less strict permissions on UPI map and transaction log files

Rfe9602 - Optimize various UPI coercion functions

Rfe9642 - Transaction logging adjustments

Rfe9642 - Transaction logging adjustments for warm standby

Rfe9654 - Remove Aux configuration parameter

Rfe9710 - Make warning about uploading big files through WebView clearer.

Bug19199 - Lock database directories

Bug19203 - Restore the :use-planner option of the select query planner

Bug19221 - Durability issue related to file deletion

Bug19251 - Spurious warning message about nonexistent file ctl0

Bug19256 - Database recovery fails with partially written checkpoint

Bug19263 - Server log contains cut-off lines

Bug19264 - Opening databases is slower than it needs to be

Bug19279 - Deleted triple data structure maintenance only happens once

Bug19282 - Improve parsing of XSD single and double floats

Bug19303 - Improve algorithm used to find sort-upi-map program

Bug19308 - Make namespaces available to code evaluated server-side.

Bug19328 - Make remote stores raise an error when created while open

Changes to the Lisp API

Bug19298 - Improve the printing of future-parts

Documentation

Rfe9339 - Improve 3-to-4 code conversion document

Rfe9549 - Many improvements in the Lisp documentation

Rfe9690 - Documentation for AllegroGraph Loader

Rfe9711 - More detail in Java client installation instructions

WWW495 - Add LUBM Benchmarks to documentation navigation list

Text Indexing

Rfe8932 - Allow sorting of freetext-searches by relevance.

Rfe9356 - Add fuzzy text-index searching.

WebView

Rfe9504 - Improve handling of WebView file uploads to use less memory.

Welcome to AllegroGraph 4.0.3

AllegroGraph Server: General Changes

Rfe8999 - Use full blank-node IDs in RDF/XML output.

Rfe9363 - Revise use of fsync and fdatasync and move to checkpoint time

Rfe9490 - increase application's initial heap size

Rfe9505 - Improve efficiency of string dictionary lookup

Rfe9541 - Server installation script avoids writing double-slashes to agraph.cfg

Rfe9560 - Truncated cursors free resources more aggressively

Rfe9567 - Don't print a backtrace on socket timeout errors.

Rfe9586 - Prevent unneeded file reads in metadata-rollback

Rfe9593 - Optimize string-dictionary

Rfe9598 - Improve efficiency of string table intern operations.

Rfe9665 - Adjusted merge invocation algorithm

Rfe9673 - Make sort-upi-map operation more verbose

Bug19041 - Delete-triple-store only deletes files from the main directory (not the aux)

Bug19085 - Fixed race condition in delete-list.dat handling

Bug19178 - Fix race condition in metadata system

Bug19182 - Performance regression with concurrent writes.

Bug19192 - Merger may not notice that the database is shutting down

Bug19189 - Prevent HTTP back-ends from holding their lock too long

Bug19201 - Catch and log errors raised in the process-services thread.

Bug19206 - Unrecoverable database due to checkpoint file id reuse

Bug19222 - Do not fsync temporary files during checkpoint processing

Bug19223 - Recovery hangs

Bug19236, Bug19237 - improve geospatial subtype mapping support

Bug19244 - Delete old deleted-shmem-files

Bug19257 - Deleted hash file deleted at the wrong point in time

Bug19276 - Possible unnecessary post-merge full scan of indexes

Changes to the Lisp API

Rfe9595 - Reduce unnecessary type-checking from Lisp API functions

Bug19173 - Corrected bug in Lisp API for remote-triple-stores

Bug19177 - Fix broken interpretation of namespaces in Sesame transaction documents

Documentation

Rfe9339 - 3.x to 4.x code conversion document added

Rfe9432 - Updates to Python API document

Rfe9504 - Document TempDir configuration directive

Rfe9540 - Audited the Python API doc for currency

Rfe9599 - Update Java section of 3-to-4 conversion document.

Bug19320 - Include links to upgrading information in documentation

SPARQL

Rfe9561 - Improvement in SPARQL recycling of storage-layer cursors.

Gruff

New overview/scroller panes

New submenu File | Export Displayed Data As ...

Child menu names are now included in the documentation.

New freetext command "Add | Edit the Set of Registered Predicates".

Rfe9090 - Initial port to the agraph4 freetext API

Export commands failed when bridge lines were present

Welcome to AllegroGraph 4.0.2

AllegroGraph Server: General Changes

Rfe9084 - Return Prolog parse errors as bad requests, not internal server errors.

Rfe9313, Bug18592 - Improved clause reordering in select queries

Rfe9345 - Ensure that distribution does not contain extra debug scaffolding

Rfe9351 - Don't write deleted hash table if no triples have been deleted

Rfe9404 - Improve error message when bulk loading N-Triple data

Rfe9435 - Support offset argument to geospatial query services

Rfe9440 - Improve cursor code efficiency in some situations

Rfe9447 - Improve detection of SPARQL Negation-As-Failure idiom.

Rfe9462 - Improved database shutdown process

Rfe9464 - Support adding and removing of namespaces from Sesame XML transactions

Rfe9493 - Added mixed workload to events test

Bug19070 - /etc/init.d/agraph 'status' doesn't work

Bug19078 - Improve error message for invalid use of create or open-triple-store

Bug19089 - Make sure daemon doesn't hang when it encounters an error

Bug19090 - Improve handling of graph-filtered-stores

Bug19091 - Prevent server shutdown from locking up when a session is open.

Bug19101 - Error detection for with-variables and CONSTRUCT queries.

Bug19121 - Fix bug in namespace caching for HTTP back-ends.

Bug19123 - Allow reasoning-triple-stores to create new-blank-nodes

Bug19145 - sendfd library does not handle "out of file descriptors" condition gracefully

Bug19151 - Refcounts on open indexes not updated aggressively

Bug19155 - Correct printing of invalid-characters-in-name condition

AllegroGraph Server: HTTP Interface

Rfe9227: Allow an HTTP proxy to be configured for the server

Rfe9318 - Remove persistent federations from the HTTP protocol

Rfe9450 - Allow committing and rolling back through HTTP headers

Rfe9458 - Expose a full-featured textual free-text query language over HTTP

Rfe9458 - Tweak free-text query expression format

Rfe9483: Add continueOnError parameter to HTTP statement loading

Rfe9496 - Support clearNamespaces tag in Sesame XML transaction documents

Rfe9499 - Support prepared queries in the HTTP protocol

Bug19088 - Allow a default namespace to be defined over HTTP

Changes to the Lisp API

close-triple-store has new 'commit' keyword

user-reasoner keyword removed

sync-triple-store has been removed

Rfe9576 - :expected-size argument to open-triple-store and create-triple-store no longer supported

Bug19046 - Fix error generated by future-part printer

Bug19081 - Change return type of ego-group-layers from vector to list

Documentation

Improved Lisp reference manual documentation

Document new free-text index HTTP API.

Rfe9169 - Add links to Javadocs from AllegroGraph documentation

Rfe9314 - Update Lisp API docs to reflect changes in freetext-indexing API

Rfe9318 - Update the HTTP protocol documentation for changed federation API

Rfe9377 - Update free-text indexing example in Python tutorial

Rfe9378 - Update Python documentation to reflect new federation API

Rfe9383 - Document new-style federation in the java client

Rfe9500 - Update free-text-index entries in Python API html doc.

Java Client

Rfe9318 - Support session-based (rather than persistent) federation

Rfe9392 - Use accept-encoding gzip to get back compressed results

Rfe9436 - Make session lifetime configurable

Rfe9442 - Add basic information to the jar manifest

Rfe9499 - Support query preparation

Bug19086 - JenaTutorialExamples now uses Graph#getName()

Bug19092 - Jena ResultSet and PrefixMapping compliance

Python Client

Rfe8610 - Help finding pairs of UPIs

Rfe9311 - Expose revised FTI API in the Python client

Rfe9318 - Move to session-based federation API.

Rfe9357 - Support wordFilter option to text-indices in Python client

Text Indexing

Rfe8775 - Support multiple text-indices per repository, add configuration parameters.

Rfe8775 - Add triples to free-text index as soon as they are added.

Rfe9357 - Add word-filters to text-index configuration (accent removal, stemming)

Changes to WebView

Rfe8775 - Allow free-text reconfiguration through WebView.

Rfe9408 - Expose free-form sessions in WebView

Rfe9438 - Show server version in WebView

Rfe9453 - Support bulk-adding of namespaces in WebView

Bug19105 - Restore Safari-compatibility for WebView

Welcome to AllegroGraph 4.0.1

AllegroGraph Server: General Changes

Rfe9137: Heuristic optimization for negation as failure in SPARQL queries.

Rfe9320: Improved caching of RDFS++ reasoning results

Rfe9352, Bug19067, Bug19068: Improve internal cursor operations

Rfe9374: Improved SPARQL range-query efficiency in some cases

Bug18978: Optimizing cursor creation and filtering

Documentation

Hardware Profiles added to Performance-Tuning Guide.

Rfe9374: SPARQL query performance tuning documentation added.

Welcome to AllegroGraph 4.0

AllegroGraph Server: General Changes

Add a Lisp function to list the catalogs on a server.

Added ego-group-layers function

Added support for auto-commit to the bulk load functions

Changed behavior of breadth-first-path-search when maximum-depth is given.

Change the keyword value used to identify typed literals

Define an error type to signal failures to connect to the AllegroGraph server.

Fix mistake in instance-subprocess communication protocol.

Make delete-duplicates-on-merge setting persistent.

Rfe7381 - Add a db keyword argument to several API commands Added the db argument to the following commands

Rfe8765 - Improved Prolog query performance

Rfe8860 - Geospatial functions now accept strings

Rfe9146 - Allow 'merging' of store metadata on concurrent change.

Rfe9209 - Don't log process termination messages before cleanup.

Rfe9212 - Periodically clean up deleted triple data structures

Rfe9291 - Use XSD instead of internal type-codes for /mappings

Rfe9294 - Added SNA path functors to bind only the first matching path

Rfe9315 - Do early probe to see if file system supports >4GB file offsets

Rfe9322 - Changes in encoded date and time UPIs.

Rfe9329 - Improved loading of a required shared library.

Rfe9330: Set up selinux security context on shared library during installation

Rfe9340 - Disallow overlapping catalog directories.

Rfe9362: Output a log message when deleting a triple-store.

Spr36556 - Reduce number of duplicate rdf:type inferences

Spr 36565 - Standardize the printing of encoded types

Bug18679 - Made part->value function more consistent

Bug18679 - Removed the deprecated part-value

Bug18738 - Make query-planner work on remote-triple-stores

Bug18840, Bug19003 - Inconsistent state after duplicate purging

Bug18879 - Corrected add-triple so that it could accept large strings

Bug18800 - Unbound prolog variables could pollute select query results

Bug18880 - Corrected serialization of RDF/XML

Bug18947 - Make service daemon responsible for deleting stores

Bug18949 - Fix geospatial type mapping in federated stores

Bug18972 - Correct printing of times with fractional seconds

Bug19010 - Fix 'store in use' bug when accessing store over HTTP.

Bug19018 - Make sure instance processes are cleaned up when breaking

Bug19024 - Show catalog name in store-status error messages

Bug19045 - Use . as the fractional seconds separator in times and dateTimes

Bug19055: Fix potential hang when committing

Bug19061: upi->String table race condition fix

AllegroGraph Server: HTTP Interface

Allow nested lists in JSON-shaped Prolog-query output.

Allow the creation of sessions with federated, graph-filtered, or reasoning stores.

Cause output of --http-trace switch to actually get flushed to the file.

Clean up namespaces when a repository is deleted.

Enable streaming of XML bodies when loading statements.

Fix a bug that prevented session permissions to show up in WebView user-contol UI.

Fix a bug with namespace caching.

Fix bug in remote geospatial queries.

Fix user-access bug in HTTP protocol.

Make user-access-configuration interface in WebView more obvious

Support changing user passwords through the HTTP interface

Support resetting namespaces to the defaults.

Rfe9131 - Add defaultGraphName parameter to SPARQL query interface

Rfe8999 - Use a unified format for serializing blank nodes.

Rfe9183 - Silence AServe's socket-reset warnings

Rfe9185 - Support delete-duplicates-during-merge remotely

Rfe9217 - Add more help text to the WebView interface.

Rfe9236 - Support SPARQL/Update over HTTP.

Rfe9247 - Expose server processes in HTTP interface

Rfe9247(2) - Allow process control from WebView

Rfe9295 - Add a commit argument to PUT/POST /statements.

Rfe9310 - The PUT /repository/[name] now overwrites by default.

Bug19002 - Fix bug in HTTP protocol namespace caching.

Bug19005 - Corrected problem in client lookups of individual datatype mappings.

Documentation

Server Installation Page Updated to be Ubuntu-Friendly

Rfe9322 - Added client tutorials to the documentation

Java Client

Expanded Coverage of Literal Values in Java Tutorial.

Java Tutorial Describes Managing User Accounts

Rfe8812 - Jena compliance improvements

Rfe8812 - Jena Graph compliance improvements

Rfe9359 - Jena Prolog support.

Python Client

Removed Common Logic

Removed JDBC result sets

Namespace implementation is now server-side

"Duplicate Triples" added to Python Tutorial.

Python Tutorial Describes Managing User Accounts.

Expanded Coverage of Literal Values in Python Tutorial.

Python Examples from Command Line

Rfe9295 - Changes addStatements and addFile

Rfe9316 - RepositoryConnection.setRuleLanguage removed

Bug19001 - Changes to RepositoryConnection.clearNamespaces API

Bug19004 - Make sure Python clients ping their session.

Welcome to AllegroGraph 4.0m2

General

Performance and Robustness

Other Enhancements

Fixed bugs and other corrections

SPARQL

Java Client

Python Client

AllegroGraph 4.13.1 user-visible changes

AllegroGraph Server: General Changes

Bug22255 - Concurrent full index optimization requests

Previously, if index optimization was requested with a full index optimization already in progress, the database instance would log an error and hang. This bug has been fixed.

AllegroGraph 4.13 user-visible changes

AllegroGraph Server: General Changes

Rfe12645 - Add a command line interface to AllegroGraph's materializer

AllegroGraph now includes a command line tool to materialize the inferred triples in a triple-store. The tool is named agmaterialize. It can currently be run only on the same machine where the AllegroGraph server is running. The command line interface is described in the Materializer documentation.

Rfe12642: Improve handling of datafile free space information

Previously, datafile free space information was maintained in a fixed sized data structure which could run out of room after a certain level of datafile fragmentation was reached. This has been corrected.

Rfe12590 - Shared backend scheduling improvements

Previously, when all shared backends were busy, further requests were held until a backend became available. Such pending requests were scheduled for execution in basically random order with latency up to 1s.

With this change, pending requests are scheduled rather than essentially random. The scheduling is basically FIFO (first in, first out, meaning the oldest request is processed first), but FIFO may be violated for efficiency reasons. (Strict FIFO behavior is undesirable, because it greatly reduces AllegroGraph's ability to schedule requests optimally under stress.)

Also with this change, scheduling latency is now minuscule and shared backends are now allowed to execute slightly more queries in parallel if all backends are busy, which allows greater efficiency in most cases.

Rfe9523 - Logging improvements

This change adds a centralized logging facility to the AllegroGraph daemon. The new logger guarantees no interleaving of messages and is able to deal with full filesystem conditions gracefully. Lisp direct clients have now the option of sending log messages to an AllegroGraph server with the with-log-routed-to-server macro. See the documentation for more. Previously, log messages were sent to the AllegroGraph server belonging to the first direct connection opened in the client.

Bug22220 - Race condition while determining last written tlog id

Previously, a rare error could be encountered when starting replication or while switching roles, indicating that replication failed due to a missing transaction log that had not yet been written to disk. This problem has been fixed. Now, only transaction logs on disk are used to start replication.

Bug22184 - Instance shutdown and replication

In rare circumstances, when the instance belonging to a replicated repository was shut down, writing the transaction log could fail. This bug has been fixed.

Bug22156 - date-string-to-upi doesn't handle zero time zones consistently

The date-string-to-upi functor was not treating a Zulu time zone specified as "Z" consistently with the rest of AllegroGraph's date and time parsers. This has been corrected.

Bug22111 - Database names may not contain special characters

Because it is impossible to reliably use databases (also called repositories) in session specifications if their names contained characters that are used in those specifications, the list of characters which must not be used in names has been expanded. The following characters are now disallowed in database (repository) names:

\ / ~ : Space $ { } < > * + [ ] | 

Previously, only the following four characters were disallowed in database (repository) names:

\ ~ / : 

Note that this change only effects new databases (repositories); existing databases (repositories) with any of the newly invalid characters in their names will continue to be accessible. It may, however, result in difficulties when attempting to use them in more complex session specifications.

Bug22100 - More strict, faster parsing of some xsd datetimes

The parsers for xsd:date, xsd:dateTime and xsd:time used to be lenient and accept things like:

20131118         # missing separator  
2013-11-18T12:27 # missing seconds 

and many other convenient but non-standard formats. The new parser only accepts what the XML Schema specification mandates which also allows it to be faster. See XSD Date and Time datatypes for more information on these datatypes.

Bug22086 - Materializer deletes triples with the same subject, predicate and object

The initial version of AllegroGraph's materializer deleted all triples with the same subject, predicate and object. This limitation has now been lifted: the materializer now only deletes triples with matching subject, predicate, object and graph.

Bug22032 - Control whether or not external references are followed during RDF/XML import

Previously, AllegroGraph ignored external references in RDF/XML source files. The Lisp API now includes a new keyword argument to load-rdf/xml:

agload also has a new command-line switch. Supplying either of -x or --external-references will cause agload to follow external references. See Data Loading.

Bug21953 - Allow import of triples to reasoning-triple-store

A bug was introduced in v4.12 that prevented import of triples via load-* routines (e.g. load-rdf/xml) on reasoning-triple-stores, when the source argument was passed a URI. This problem has been fixed. This change also provides a better error message when attempting to import statements to a federated-triple-store, which is not allowed.

Bug21656 - Cleaner daemon shutdown

When the AllegroGraph daemon was stopped, despite the claim of doing a 'clean shutdown' in agraph.log, the shutdown of the instances was not clean and the corresponding repositories would potentially require crash recovery the next time they were opened. Also, such a shutdown lost any index optimization work performed since the most recent checkpoint.

With this change, instance processes perform a checkpoint if necessary which makes shutdown slower, but startup faster and it makes sure the index optimization results persist.

Bug20773 - Support federations of RDFS++ stores

Previously, queries against a federation of RDFS++ stores would cause an error during query planning. This has been corrected.

HTTP Client

Rfe12763 - Update HTTP API for the materializer

The REST API for materializeEntailed now includes an inferredGraph argument that can be used to specify which graph newly entailed triples are added to (or deleted from). The materializer is described in Materializer.

SPARQL

Rfe12767 - Improved the efficiency of single group aggregate queries

The code generated for single group aggregate queries is now slightly more efficient.

Rfe12762 - Optimizing level 2 index optimization

With this change, level 2 index optimization (the default) no longer tries to optimize access for triples added since optimization was started. This allows index optimization to finish even if triples are added continuously at a high rate. The new code is faster on average, has the same worst case disk usage (about two times the original size), but it has a higher average disk usage.

Rfe12760 - Improve efficiency of some SPARQL Property Path queries

AllegroGraph now builds some property path data structures as needed rather than computing all of them up front. Depending on the data, this can greatly enhance the efficiency of some queries.

Rfe12757 - Improve efficiency of some property path queries

In some circumstances, AllegroGraph could choose the less efficient of two strategies when evaluating a property path query. AllegroGraph now ensures that it makes the better choice.

Rfe12717 - Handle RDFterm-equality of strings consistently

Previously, AllegroGraph implemented sameTerm() using a strict interpretation of RDF literal equality when comparing plain literals and typed literal strings but implemented = using a more relaxed interpretation. These tests are now done consistently using the relaxed interpretation. In particular,

'hello' is RDFterm-equal to 'hello'^^xsd:string 

This is in accordance with both common usage and the RDF 1.1 specification.

Rfe12699 - Improve efficiency of SPARQL equality tests on resources

The efficiency of equivalency tests in SPARQL filter expressions has been improved.

Rfe12681 - Optimize interprocess communication

The performance of message passing between different AllegroGraph server processes has been slightly improved.

Rfe12679 - Use query local namespace abbreviations in result serialization

AllegroGraph now uses the namespaces defined by a SPARQL query's prefixes when serializing output in RDF/XML. Previously, temporary namespaces were used instead.

Rfe12670 - Allow /SPARQL access to AllegroGraph as a SPARQL endpoint

Previously, access to an AllegroGraph triple-store as a SPARQL endpoint was via sending a query to

SERVER:PORT/respositories/NAME?query=... 

This REST API has been augmented so that sending a query to the triple-store's URL with a trailing /sparql also works. E.g.,

SERVER:PORT/respositories/NAME/sparql?query=... 

This is not required by the SPARQL standard but is in conformance with many other existing endpoints.

Rfe12657 - More efficient handling for the VALUES clause

AllegrGraph now rearranges queries so that VALUES clauses are evaluated first when possible. This improves query efficiency as VALUES clauses are generally small.

Rfe12656 - Improve execution of some nested joins

AllegroGraph now evaluates nested joins more efficiently.

Rfe12498 - Improve full-scan warning heuristics

AllegroGraph warns when a SPARQL query executes a full-scan. Heretofore, this meant that a query like

select * { ?s ?p ?o } limit 100 

could have produced a full-scan warning even though it is benign. AllegroGraph now saves its warnings for when they may actually indicate a problem with the query.

Rfe11981 - New query option chunkProcessingMemory

Previously when chunk-at-a-time processing was enabled (see query option chunkProcessingAllowed), the query option chunkProcessingSize controlled memory consumption by limiting the number of intermediate results stored in memory. The total memory consumption depended on the number of variables and the complexity of the query, so it was rather difficult to know how much a query will actually use. With this change, chunkProcessingSize is deprecated and chunkProcessingMemory can be specified instead. The unit of chunkProcessingMemory is a byte but the usual abbreviations are also accepted ("8G" (the default), "2M").

Rfe11515 - Fully support the SPARQL 1.1 HTTP Protocol

AllegroGraph now fully supports the five different HTTP request types specified in the SPARQL 1.1 HTTP Protocol. These are:

In addition, AllegroGraph continues to support SPARQL update using POST where the update command is encoded in the query portion of the HTTP request.

Bug22214 - Encoded IDs failed to compare properly in SPARQL queries

Inequality comparisons between encoded IDs were always returning true. This has been corrected.

Bug22204 - decimal to decimal cast loses precision

When in a SPARQL query an xsd:decimal was cast to xsd:decimal it lost precision as it was converted to a single float internally. This bug has been fixed.

Bug22203 - Fix decimal parsing in SPARQL queries

In SPARQL query strings, decimals in the (-1,0) interval were parsed with the wrong sign. For example, -0.9 was parsed as 0.9. This bug has been fixed.

Bug22153 - Some wild card property path queries could generate incorrect paths

In rare circumstances, the data structures used to cache property path information could be invalidated leading to too many path results. This has been corrected.

Bug22147 - Add support for ObjectId's in the _id field.

The SPARQL MongoDB magic predicate now works with documents containing ObjectId's in the _id field.

Bug22142 - Expressions on aggregates fail for empty group

If a query wrapped an aggregate with an expression like

select (xsd:float(avg(?age)) as ?avgAge) ... 

and the result set contained an empty group (i.e., a group with no members), then AllegroGraph would signal error. This has been corrected.

Bug22134 - SPARQL aggregates on expressions with errors could compute incorrect results

Other than COUNT, a SPARQL aggregate on an expression should return unbound if that expression evaluates to a type error during computation. AllegroGraph was incorrectly returning the aggregate of any bindings produced after the last type error. This has been corrected.

As an example, if AllegroGraph needed to compute the MIN( ?p * 2 ) and ?p was bound to 4, 7, 15, "hello", 6, and 1, then AllegroGraph would previously have returned 2 (the minimum of 12 and 2). Now, it will return no binding for that expressions.

Bug22123: Empty construct template will cause an error

If a SPARQL CONSTRUCT had an empty template like

CONSTRUCT {} WHERE { ?s a ?o } 

then AllegroGraph would signal an error. This has been corrected.

Bug22094 - Correct sparql< and sparql> for date and dateTime comparisons

Because there is no defined operator mapping for inequality comparisons between xsd:date and xsd:dateTime, comparing them should throw a SPARQL type-error. AllegroGraph was incorrectly returning nil instead. This has been corrected. See XSD Date and Time datatypes for more information on these datatypes.

Bug22074 - DELETE templates with unknown variables caused an error

AllegroGraph would signal an error if a SPARQL DELETE/WHERE command's DELETE clause contained variables that were not bound by the WHERE clause. This has been corrected. An example of such a command would be:

DELETE { graph ?unknownGraph { ?s ?p ?o } }  
WHERE { ?s ?p ?o }  
 

Bug21561 - Remote-triple-stores do not support the table results format

Remote-triple-stores did not support returning tables from SPARQL queries. This has been corrected.

AGWebView

Bug22161 - Adding security filters in Firefox

Previously, with Firefox, clicking the 'add' button in the security filters table of the user management page (Admin/users in the menu) of AGWebView did not submit the form. This bug has been fixed.

Changes to the Lisp API

Rfe12632: Improvements in Lisp API to AllegroGraph materializer

Materialization previously occurred only for triples in the default-graph of the triple-store. It now occurs for all of the graphs in the store. Materialization on subsets of the graphs can be accomplished by creating a graph-filtered triple-store and calling materialize-entailed-triples on it.

Materialization previously placed inferred triples into a special graph that could not be utilized by the rest of AllegroGraph (the special graph behaved similarly to but not exactly like the usual AllegroGraph default-graph). It now places inferred triples into a graph that can be specified in the call to materialize-triples using the materialized-graph argument. It defaults to .

Two new arguments were added to delete-materialized-triples. These are:

Materialization now logs its start and end to the AllegroGraph log file. The overall efficiently of the materializer has also been improved.

Bug22132 - calling run-sparql with the :with-variable argument with a timezoned upi

When the Lisp function run-sparql was used in conjunction with a :with-variable argument that bound a variable to an encoded xsd:dateTime or xsd:time, then in some cases comparisons involving that variable failed to match equivalent representations of the same xsd:dateTime or xsd:time with a different timezone. This bug has been fixed.

Prolog

Bug22171: Select0 should return UPIs when it can

Previously, the Prolog select0 query macro could return future-parts rather than UPIs. For example, this query would return a future-part.

(select0 (?a) (= ?a !rdf:type)) 

This could cause problems in the unlikely event that namespace abbreviations changed when running the same query multiple times. select0 now returns UPIs so that the above behavior is not possible. Note that if a future-part represents a triple that is not currently in the triple-store, then a query can return UPIs that cannot be printed (because the UPI will not be in the string dictionary).

Bug22157 - Temporal functors could signal an error when used with all bound values

Previously, AllegroGraph used real numbers to represent times, dates and dateTimes. Now, the internal representation includes time-zone information which means that comparing two values can no longer use simple inequalities. The Prolog temporal functors like point-before-datetime had not been updated to reflect these changes. This oversight has been corrected. This change means that the Prolog functors will use the same XSD logic used by the SPARQL date and time functions.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

AllegroGraph 4.12.2 user-visible changes

AllegroGraph Server: General Changes

Rfe12734 - More aggressive level 2 index optimization

In some cases, level 2 index optimization (the default) left a small part of the index unoptimized. With this change, the entire index is optimized in all cases.

Rfe12726 - Index optimization increasing database size on disk

Space in some garbage on-disk data structures is now reclaimed in a more timely manner which makes the increase in database size during index optimization less severe.

Rfe12725 - Faster level 2 index optimization

Level 2 index optimization (which is the default level) has been made faster.

Rfe12636 - Make config-file argument to agraph-backup backup-settings optional.

Previously, the agraph-backup backup-settings command required a config-file argument. This argument has been eliminated in favor of an optional --config option that can be used in place of the --port argument. If both --port and --config are supplied, then the port argument takes precedence. Also, agraph-backup could fail in the past if there were discrepencies between a --port argument specified on the command line, and a Port directive contained in a config file. Now, if the --port argument is specified, it will take precedence over any Port directives read from a config file. See Backup and Restore.

Rfe12635 - Major revision to agraph-backup operating modes.

The various backup and restore modes of agraph-backup have been overhauled.

Previously, backup mode created an archive file containing only repository data. Now, it creates an archive directory that contains both repository data and settings. This archive can be restored via restore or restore-all. The --ext-backup-command option is supported. Previously, restore mode operated on an archive file. Now, it can also operate on an archive directory created by backup or backup-all. In either case, it will restore a single repository (data and settings) from the archive.

The layout of archive directories created by backup-all are slightly changed, and support for the --ext-backup-command option is added. The restore-all mode will now fail if repositories being restored to already exist. Use the --supersede option to force an overwrite of any such repositories.

Previously, the backup-settings mode created an archive containing both repository and system settings. It also supported the --ext-backup-command option. Now, it only archives system settings and the --ext-backup-command option is no longer supported. Adds a restore-settings option that restores system settings, overwriting any existing settings. It operates on any valid archive directry (though ones created by backup will likely not have any system settings data).

Please refer to the updated Backup and Restore documentation for full details.

Rfe12621 - xsd:time improvements

Zoneless xsd:times are no longer converted to zero timezone when added, and timezone offsets roundtrip to and from the database. Also, with this change xsd:times and xsd:dateTimes are no longer comparable. This puts xsd:time on equal footing with xsd:dateTime. See the date and time documentation for more information.

Rfe12580 - Don't create repo if client/server uids don't match

Previously, if the uid of the direct Lisp client and the AllegroGraph server differed, then a create-triple-store call would fail with a `server-pid-and-client-pid-must-match-error` indicating that the creation failed. However, the repository would actually be created, possibly overwriting an existing repository. Now, the uid check is performed earlier in the create repository process, so that the repository is not created when the uids do not match.

Rfe12456 - Time zone roundtrip for xsd:dateTime and xsd:time

AllegroGraph previously represented xsd:times and xsd:dateTimes with Zulu (i.e. Greenwich Mean) time internally and converted data to that time zone as it was imported. On output, any xsd:time or xsd:dateTimes would be displayed in Zulu time and would lose the original time zone information. For example,

"2014-02-23T08:18:59+05:00"^^xsd:dateTime 

would become

"2014-02-23T03:18:59Z"^^xsd:dateTime 

AllegroGraph now keeps the original data's time zone information. Note that the additional computation involved with coercing time information during query means that queries involving xsd:dateTimes or xsd:times may run slightly (a few percentage points) more slowly. There is also a new query option, presentationTimeZone, that controls the display of time zones in the output of SPARQL queries. For example, adding

PREFIX franzOption_presentationTimeZone: <franz:-04:00> 

will cause all xsd:times and xsd:dateTimes to be printed at the -04:00 time zone. See the documentation for more information. Since the database format has been changed, the usual upgrade process involving a backup/restore cycle (see Backup and Restore) is required. Please note the following:

1: There are no changes for xsd:dates. They still support what the XML Schema describes as 'recoverable timezone'.

2: "2014-02-23T08:18:59+08:00"^^xsd:dateTime is DISTINCT from "2014-02-23T00:18:59Z"^^xsd:dateTime.

3: Joins can be made over dates that represent the same point on the same timeline even if they have different timezones. Which (equivalent) representation is returned by the join is unspecified.

See the date and time documentation for more information.

Rfe11303 - Fix partial repo creation on agraph-backup restore operations

When performing an agraph-backup restore operation, if you passed a directory name as the archive argument, the restore would fail, but not before a repository was created. Further, the repository would remain in 'restore' mode, so it could not be opened, and its existence would prevent a subsequent valid agraph-backup restore operation from succeeding. agraph-backup now first checks that the argument filename names an existing file which is not a directory before creating any repositories. Also, error messages when agraph-backup** is passed invalid archive file arguments have been improved. See Backup and Restore.

Rfe11044 - Add -p short form of --port to agraph-backup.

With agraph-backup, users may use either -p or --port to specify the port on which an AllegroGraph server is running. See Backup and Restore.

Rfe10629 - Add --supersede argument to agraph-backup

agraph-backup now accepts a --supersede option, which (if specfied) allows overwriting of existing archives (for the backup command) and directories (for the backup-all command), and databases (for the restore and restore-all commands). See Backup and Restore.

Bug22099 - Ensure UUIDs are unique during agraph-backup restores.

Previously, when restoring archive backups to an AllegroGraph server, it was trivially possible to create multiple repositories with the same uuid. Now, agraph-backup will error if a restore operation will cause a uuid conflict. See Backup and Restore.

Bug22090 - Timezone minutes in xsd types

Specifying timezone minutes is no longer optional in xsd:dateTimes and xsd:times, so the following examples are invalid:

"1978-01-03T00:00:00+05"^^xsd:dateTime INVALID  
   Correct form: "1978-01-03T00:00:00+05:00"^^xsd:dateTime  
"00:00:00+05"^^xsd:time INVALID  
   Correct for: "00:00:00+05:00"^^xsd:time 

Previously, only xsd:dates checked for the presence of timezone minutes. See the documentation for more information.

Bug22083 - xsd:date and xsd:dateTime years

AllegroGraph now correctly parses both positive and negative dates with more than four digit years. It also signals an error if the date is too large to be encoded into a UPI. Previously, dates like these were silently truncated. See the date and time documentation for more information.

Bug22073 - ORDER BY and non-encoded decimals

When non-encoded decimals were being ordered, they were first converted to IEEE 54 double floats (which have a precision of 15-17 significant digits) which potentially caused them to be sorted incorrectly. This bug has been fixed.

Bug22072 - Validate xsd:decimals

AllegroGraph now signals an error if an xsd:decimal's exponent in scientific notation is outside of the range [-63, 63] because these values cannot be encoded into a UPI. Previously, xsd:decimals like these were silently truncated.

Bug22068 - Reasoning and constant triple patterns

With reasoning enabled, some queries involving triple patterns without variables failed with 'Bus error'. This has been corrected.

Bug22066 - Printing of NaN, INF and -INF

Serialization of xsd single and double float NaN, INF and -INF was broken (for instance, INF became "#.inEinity-double"). This bug has been fixed.

Bug22057 and Bug22054- agraph-backup backup-settings works as documented.

The agraph-backup backup-settings command was incorrectly backing up data as well as settings rather than just backing up settings. Now it just backs up settings. Also, the backup-settings command now can work by finding the settings location from the config file (so the server need not be running). See Backup and Restore where a note describes all the agraph-backup revisions.

Bug22045 - 14 hour datetime comparison logic

For two xsd:dateTimes, one with and one without a timezone, all comparisons must return false if they differ by less than 14 hours. Similar logic applies to xsd:dates and xsd:times. Previously, support for this 14 hour comparison logic was spotty. It worked for some combinations (such as comparisons in a FILTER with a literal) but didn't work for others. With this fix, comparisons for all three types follow the XML Schema specification. See the date and time documentation for more information.

Bug22044 - Fix xsd:date xsd:dateTime comparisons

AllegroGraph previously compared xsd:dates and xsd:dateTimes by converting the date to a dateTime at its first instance. Because dates are actually 24-hour intervals, this logic is suspect so AllegroGraph now signals a type error if this comparison is made (i.e., FILTERs will not longer match and BIND would leave its variable unbound). The previous behavior can still be had by using an explicit cast from date to dateTime. See the date and time documentation for more information.

Bug22041 - Comparisons to decimal zero

Due to a bug in the encoding, decimal 0 appeared to be greater than all decimals less than 1.0. The usual upgrade process involving a backup/restore cycle (see Backup and Restore) is required. All misencoded decimal 0s will be converted to the correct value. If any such corrections are made during the upgrade, then indices are resorted and background index optimization is begun. The index optimization may continue even after the upgrade process has completed. The resulting indices are then as optimized as if all the triples were just imported into the database. This means if the backup was of a fully optimized repository, then after the upgrade running level 2 (the default) index optimization is necessary to get back to that state.

Bug22039 - agload character encoding fixes

Previously, the --encoding argument to agload was ignored. This has been corrected. See Data Loading for information on agload.

Bug22034 - part->concise on encoded dates and decimals

For xsd:dates, xsd:dateTimes, xsd:times and xsd:decimals, the Lisp functions part->concise and part->terse returned the stringified internal representation (like "172839/500" for decimals, and "( . )" for datetimes) which didn't fulfill the promise of being in a "human readable form". With this change, the above functions will return the value in a readable string format without the type. An example:

(part->concise (value->upi "2013-01-03T10:34:00-05:00" :date-time))  
=> "2013-01-03T10:34:00-05:00" 

See XSD Date and Time datatypes.

HTTP Client

No significant changes.

SPARQL

Rfe12624 - Standardize some low-level SPARQL equality tests

The efficiency and conformance of low-level tests used to compute RDFterm-equality and SPARQL ordering has been improved. One non-backwards compatible change is that AllegroGraph previously allowed ordered comparisons between plain literals that looked like numbers and typed numeric literals (e.g., "6.21" and "6.23"^^xsd:integer). This comparison now correctly throws a SPARQL type error.

Bug22092 - Fix for SPARQL queries that use the STR() function with a constant

If a SPARQL query contained a FILTER that used the STR() function applied to a constant, then the query could signal an error at planning time. For example, a FILTER like

FILTER( STR( <http:://example.com> ) = ?var ) 

would not succeed. This has been corrected.

Bug22078 - SPARQL insert did not always encode typed literals

Triples added to a triple-store using SPARQL inserts that used computed values did not always have their typed literals encoded into UPIs. For example, an insert like:

insert { <ex:a> <ex:b> ?c . } where { bind( '1'^^xsd:byte as ?c ) } 

would store the literal in string form rather than encoding it directly. This has been corrected.

Bug22077 - Improve efficiency of certain SPARQL sub-queries

Bound variable information could be lost while processing certain SPARQL sub-query forms which would then result in less efficient query execution. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug21608 - Include ag-call in remote Lisp client.

Previously ag-call was omitted from the remote Lisp client. This has been corrected.

Prolog

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

No significant changes.

AllegroGraph 4.12 user-visible changes

AllegroGraph Server: General Changes

Rfe12576 - Improve RDF/XML parser efficiency

Improve the lower-level efficiency of AllegroGraph's RDF/XML parser so that it is between 5 and 10% faster.

Rfe12522 - Global configuration directive for query options

A new global configuration directive named QueryOption was added. It can be used multiple times. Each specification is equivalent to a query prefix option. The global configuration directive "QueryOption NAME=VALUE" is the same as the query prefix option "PREFIX franzOption_NAME: ". See Server Configuration for information on configuration directives.

Rfe12429 - More flexible parsing for InstanceTimeout

With this change, InstanceTimeout configuration parameter values in the configuration file can be specified as times rather than only as a number of seconds. For example, you can use 12s, 2m or 3h to specify 12, 120 or 10,800 seconds. The configuration file is described in Server Configuration.

Rfe11010 - Require the i index for freetext queries

AllegroGraph's native freetext index requires the presence of the triple-id (i) index in order to operate efficiently. A common problem was to try to use freetext queries when there was no i index. This change automatically adds an i index whenever a freetext index is created and also signals an error if a freetext query is attempted when there is no such index.

Bug22024 - Serialization of decimals

xsd:decimals with zeroes right before the decimal point were serialized with the decimal point misplaced. This bug has been fixed. The database contents were not affected.

Bug21978 - serialize-* functions in Lisp client default to UTF-8.

Use UTF-8 as the external-format when a filename is passed to one of the serialize-* functions in the Lisp client instead of the default external format, which can vary depending on the system's locale.

Bug21976 - Complex GRAPH queries could produce incorrect graph bindings

Depending on the dataset, queries with deeply nested structure that used EXISTS or NOT EXISTS FILTERS with embedded GRAPH clauses could generate incorrect bindings for the graph variable. This has been corrected.

Bug21975 - Increase AllegroGraph Load speed

Low-level efficiency enhancements improve AllegroGraph load speed by roughly 10%.

Bug21966 - Possible "UPI not in string table" problem with remote-triple-stores

It was a possible for a remote-triple-store to lose track of the strings cached on the client after a rollback operation. This could lead to a "UPI not in string table" error. This has been corrected.

Bug21956 - Improve memory usage for zero-length property path queries

A zero-length property path query connects all subjects and objects on the right and left hand sides of the path to themselves. If the store is large and the path produces many matches on one or both sides, AllegroGraph was using more memory than necessary. This has been greatly improved though some queries may experience a slight performance degradation.

Bug21954 - Problem generating soundexes for phrases in Callimachus

If a store was operating in Callimachus mode and a phrase was soundexed that had words in it with no soundex (e.g., a number) then AllegroGraph would generate garbage soundex data. This has been corrected

Bug21949 - Improve bookkeeping of already processed variables during query execution

AllegroGraph was not updating the list of bound variables when evaluating a BIND clause during query execution. This did not change the final results of a query but did mean that AllegroGraph would not always evaluate the query in the most efficient manner. This has been corrected.

Bug21937 - Zoneless xsd:dateTime roundtrip

Adding a triple with an xsd:dateTime without a timezone (e.g. "2013-07-30T23:59:59") was stored as if it had a timezone of zero. With this bug fix, zoneless dateTimes are stored and serialized without timezone. This is in agreement with the XML schema specification and similar to how xsd:dates behave.

Bug21934 - String table mutex not reinitialized after recovery

A mutex used in the string table data structure was not being reinitialized after database restart. Under certain circumstances this could result in a database hanging after restart. This problem has been corrected.

Bug21920 - Soundex should ignore punctuation

AllegroGraph's soundex function (which is used by Callimachus) was not ignoring punctuation properly. This has been corrected.

Bug21915 - Make upi->value on xsd:dates return integers

Due to an undocumented change in v4.11, upi->value on a upi encoded as :date returned a cons of the integer representing the instant and a flag indicating whether the timezone was present. With this change only the integer part is returned (like before v4.11). The flag is returned as the second value by upi->values.

Bug21914 - xsd:dates on the first day of the month

Adding a triple with an xsd:date with a positive timezone that falls on the first day of a month resulted in error "illegal date 0". This bug has been fixed.

Bug21909 - agraph-backup could fail to generate unique UUID on restore

When using the --newuuid option, it was possible for agraph-backup to create multiple triple-stores with the same UUID. This has been corrected. See Backup and Restore for information on agraph-backup.

Bug21903 - Free text indexing with chunk-at-a-time processing

Depending on the execution plan, a query that used both fti:match and chunk-at-a-time processing (see franzOption_chunkProcessingAllowed) could produce too few results. This bug has been fixed.

Bug21899 - Shared memory server fails to clean up

Under low memory conditions, the shared memory server can fail to allocate blocks of shared memory on behalf of a database instance. Under certain circumstances the shared memory server would fail to clean up from partial allocations, thereby exacerbating the low memory condition. This problem has been fixed.

Bug21873 - Link to docs in rpm install

When AllegroGraph was installed from an rpm package, the documentation links in AGWebView were broken. This bug has been fixed.

Bug21872 - agload will write output to agload.log when in --debug mode

Along with the other effects of using the --debug command line argument to agload, all output will be logged to agload.log. See Data Loading.

Bug21869 - Fix problem with registering cartesian predicate mappings

There was a problem which prevented the shortened form of cartesian predicate mappings from working. This has been corrected.

Bug21853 - Connection Pool session port connection failure

Under certain circumstances, the AllegroGraph Java client would throw an exception like the following:

com.franz.agraph.http.exception.AGHttpException:  
Possible session port connection failure 

Setting the SessionPorts configuration parameter is discussed here in Server Installation and also in Server Configuration. But this exception could occur even with a proper SessionPorts setting. This problem has been resolved.

Bug21846 - Long running federated queries could fail

The query canceling interface introduced in AllegroGraph 4.11 could lead to query failures when using federated triple-stores. This has been corrected. Note that it is not currently possible to cancel queries running on a federated store. This will be improved upon in a future release of AllegroGraph.

Bug21816 - Fix session multiplexing file descriptor leak

When requests to sessions are multiplexed through the server's main port, the frontend didn't close the http socket used to communicate with the session process. This bug has been fixed. Note that session multiplexing can be enabled for the java client by setting the system property:

com.franz.agraph.http.useMainPortForSessions=true  
 
 

Bug21784 - Safer auto-close for garbage db's in the Lisp direct client

The db's are now marked for closure when it is noticed they are no longer used and closed at a later specific time.

Bug18813 - Reasoning and subclasses of owl:TransitiveProperty

With this fix the reasoner is able to infer that a property whose type is a subclass of owl:TransitiveProperty is a transitive property. That is,

:p rdfs:subClassOf owl:TransitiveProperty .  
:q rdf:type :p .  
:a :q :b .  
:b :q :c . 

entails

:a :q :c .  

HTTP Client

No significant changes.

SPARQL

Rfe12537 - Improve zero length property path clause estimation

This change improves the heuristics AllegroGraph uses to estimate the number of rows that a zero length property path query can generate which improves the way AllegroGraph processes a BGP.

Rfe12520 - The SNA members and size magic properties work with RDF lists

You can now use the and Magic Properties to query RDF lists stored in your triple-store. For example, if points to the head of the RDF list "1", "2", "3", then this query would return the three elements:

prefix sna: <http://franz.com/ns/allegrograph/4.11/sna/>  
select ?item {  
  <ex:list> <ex:pointsTo> ?list .  
  ?item sna:members ?list .  
} 

Would return "1", "2" and "3". See SPARQL Magic Properties for SNA.

Rfe12506 - Improve handling of some queries that use GRAPH clauses

SPARQL is defined such that a query like

GRAPH ?g { A } 

should behave as if A was evaluated with ?g bound to each graph in the dataset in turn and then the results of all of these evaluations were unioned together. While correct, this implementation can be very inefficient. AllegroGraph uses various heuristics to determine if a given GRAPH clause can be evaluated more directly and does so when it can. This change improves these heuristics so that more queries will be able to use the more efficient implementation.

Rfe12481 - Improve SPARQL DELETE efficiency

This change greatly increases the efficiency of the DELETE portion of a SPARQL DELETE... INSERT... WHERE... command. The actual increase is dependent on the data and the delete template but it can be more than 40 times faster.

Rfe12442 - Efficiency improvements for SPARQL aggregation

Optimize the code used for SPARQL aggregate queries in general and for count aggregates in particular.

Rfe12400 - Minor property-path query optimization

Improve the underlying efficiency of some property-path queries.

Rfe12398 - Clarify semantics of the identity clause reorderer

AllegroGraph has two clause reorderers which try to preserve the ordering of triple-patterns in a BGP (basic graph pattern). Previously, these were named:

To avoid future confusion, these options are being renamed such that the first method is called :weak-identity and the second is called :identity.

The default query clause reorderer continues to be statistical so queries that do not specify a reorderer will behave exactly as they did before this change.

Rfe12392 - Optimize certain complex join combinations

The query optimizer now converts a query with an algebra like:

(:join (:join A :bgp1) :bgp2) 

into the equivalent:

 (:join A :bgp1-and-bgp2) 

This BGP (basic graph pattern) merger improves AllegroGraph's ability to reorder queries for efficient execution. AllegroGraph also converts the expression:

(:join (:join :bgp1 A) :bgp2) 

into

(:join :bgp1-and-bgp2 A) 

unless the two BGPs do not overlap (in which case the transform would produce a cross-product).

Rfe12385 - Improve BGP reordering in some cases

In some cases, the SPARQL query planner now does a better job reordering the triple-patterns in a BGP (basic graph pattern).

Rfe12380 - Improve efficiency of some DISTINCT or REDUCED queries

AllegroGraph now collects fewer intermediate results for some DISTINCT or REDUCED queries.

Rfe12377 - Add intervalContainsTime magic property

Added an additional temporal magic property that can be used to determine if a time is contained within an interval. See SPARQL Magic Properties.

Rfe12372 - Make BGP reordering more intuitive

Previously, AllegroGraph would switch the execution of patterns in a BGP when the BGP contained only two patterns and these patterns had the same statistical clause estimates. E.g., in a query like:

?a ?b ?c .  
?c ?d ?e . 

AllegroGraph would execute the second pattern first. AllegroGraph now executes the first pattern first in the case of ties which is more intuitive.

Rfe12368 - Use constraints to limit bindings for GRAPH queries

Improve AllegroGraph's use of FILTERs in limiting the graphs considered in GRAPH clauses with variable graphs. For example,

GRAPH ?g { A }  
FILTER( ?g = <ex:test> ) 

will now be evaluated more efficiently since AllegroGraph will only consider the graph <ex:test> rather than every graph in the dataset.

Rfe12352 - Provide a sort order for SNA path and group gensyms

Social Network Analysis (SNA) Magic Properties can create node identifiers to keep track of paths and groups. These act like blank nodes and so had no sort order defined for them. This was inconvenient because it made it impossible to, for example, list all of the nodes of a path together. This change adds a sort order to these node identifiers so that they are easier to use in queries.

Rfe12192 - Optimize some wild-card property-path queries

AllegroGraph now computes property-path queries that use the * or + operators more efficiently.

Rfe12032 - Improve property-path expansion

AllegroGraph now expands more property-path queries into their equivalent BGPs (basic graph patterns). This can improve the performance of the query optimizer. For example, AllegroGraph previously expanded

?s :a/:b ?o .  
into 

?s :a :b0 . :b0 :b ?o .

but did not expand paths with non-simple operators like wild-cards or alternation. It now expands these as well so that a path like

?s :a/:b/(:c|:d)/:e/:f ?o 

becomes

?s   :a _:b0 .  
_:b0 :b _:b1 .  
_:b1 (:c|:d) _:b2 .  
_:b2 :e _:b3 .  
_:b3 :f _?o .  

Rfe11948 - Add PREFIX option for enabling reasoning in a SPARQL query

AllegroGraph's SPARQL 1.1 engine now supports the inferenceMode query option to control the kind of inference used by an individual query. The mode is selected using the query option PREFIX notation. For example, the following PREFIX would cause a query to be evaluated using AllegroGraph's RDFS++ reasoner:

PREFIX franzOption_inferenceMode: <franz:rdfs++> 

The following modes can be supplied:

Rfe11625 - Improve algebraic optimizations of joins and left-joins

Improve the logic used in the algebraic transform of

(:join (:left-join A B) C) into (:left-join (:join A B) C) 

Which is the textual equivalent of moving the optional clause down in a query like

A  
optional { B }  
C 

so that it becomes

A  
C  
optional { B } 

which can be evaluated more efficiently unless A and C share no bindings.

Rfe11146 - Handle simple OPTIONAL clauses more efficiently

AllegroGraph now handles OPTIONAL clauses that contain only a single triple-pattern more efficiently. This means that queries with clauses that look like

 ?s :predicateA ?o .  
 OPTIONAL { ?s :predicateB ?b }  
 OPTIONAL { ?s :predicateC ?c } 

will be processed more efficiently.

Rfe11068 - Improve literal equality constraint handling

AllegroGraph now handles filter expressions involving literal equality more efficiently. For example, in a query like

select * {  
  ?a :predicate ?b .  
  filter( ?b = 'literal' )  
} 

AllegroGraph previously would scan all triples with the given predicate and apply the filter afterwards. Now AllegroGraph determines the set of possible matches for 'literal' and queries for them directly.

In addition, AllegroGraph now performs a similar operation if the filter is a literal equality on an expression using the SPARQL str function. For example:

filter( str(?variable) = 'searching' ) 

As with regular literal equality constraints, such an expression is converted into the equivalent tests for specific values.

Bug22030 - ORDER BY expressions with non-query variables could cause errors

If a SPARQL query used an ORDER BY expression that contained variables that were not found anywhere in the query, then an error could result. An example of this would be a query like:

select * {  
  ?s a ?o .  
} order by desc( str( ?notFoundInQuery )) 

This issue has been corrected.

Bug22021: Correct SPARQL TSV printing problem for encoded UPIs

When encoded UPIs were serialized from a SPARQL query using the SPARQL TSV (tab sepated values) output format, they were printing without the enclosing quotes. This has been corrected.

Bug22016 - Correct SPARQL 1.1 bug with IN FILTERs

It was possible for the query engine to produce incorrect results when using FILTER with either the SPARQL IN operator or multiple equality filters on the same variable. This has been corrected.

Bug22015 - Fix possible error when using the SPOGI cache and SPARQL 1.1

In rare circumstance it was possible for a SPARQL 1.1 query to signal an error when the SPOGI cache was enabled. This has been corrected.

Bug21986 - Using distinct aggregation with GROUP_CONCAT and a separator could fail

If a SPARQL query used the GROUP_CONCAT aggregator along with both the DISTINCT and SEPARATOR modifiers, then the solution bindings for that variable could be left empty. This has been corrected.

Bug21980 - Too many results in DISTINCT and REDUCED queries

Under extremely rare circumstances, queries involving DISTINCT or REDUCED could return too many results. This bug has been fixed.

Bug21974 - Assertion failure involving 'n-valid-partials'

When chunk-at-a-time processing was enabled (see query prefix franzOption_chunkProcessingAllowed, described here in the SPARQL Reference), the code keeping track of the number of intermediate results used to run into an assertion failure under very rare circumstances (when two consecutive intermediate results were the same). This bug has been fixed.

Bug21962 - Improve estimates for certain query patterns

AllegroGraph was mis-estimating the sizes of query patterns that resulted from the inlining of filters that used the IN expression. For example,

?s ?predicate ?object .  
FILTER( ?predicate IN ( ex:a, ex:b, ex:c ) 

This could result in sub-optimal BGP re-ordering. This has been improved.

Bug21961 - Incorrect results with EXISTS and NOT EXISTS

Some queries involving EXISTS or NOT EXISTS filter expressions returned too few results. This bug has been fixed.

Bug21955 - Sorting on unbound variables could lead to a string dictionary error

If a query used an ORDER BY expression involving a computation and some of the variables in the expression were unbound, then the expression could signal a UPI not in string dictionary error. This has been corrected.

Bug21948 - Improve bookkeeping of bound variables during sub-query

When executing a sub-query with a limit, AllegroGraph failed to update the list of variables known to be bound. This prevented some queries from executing as efficiently as possible. This has been corrected.

Bug21947 - Improve efficiency of VALUES in some cases

AllegroGraph now does a better job of using the information in a VALUES clause to restrict alternate solutions. This can greatly increase the speed of some queries.

Bug21941 - Queries with magic properties could fail in a remote-triple-store

Queries that used magic properties could cause an error when run from a remote-triple-store. This has been corrected.

Bug21940 - Using magic properties with filters could lose solutions

If the circumstances were just right, a query that used a magic property that produced multiple bindings and that used a filter on those bindings could fail to return all solutions. This has been corrected.

Bug21933 - Externally introduced variables could fail in nested sub-queries

If a query used externally introduced variables (e.g., via the Lisp APIS's with-variables argument or via the variable bindings in an HTTP REST request), and these variables were used in FILTER clauses in a nested sub-query, then the query could fail to find any matches. This has been corrected.

Bug21921 - BGP triple filters with Chunk-at-a-Time processing

In rare cases, a filtering triple-pattern could fail to remove some solutions when using Chunk-at-a-Time query processing. This has been fixed.

Bug21902 - Fix incompatibility between magic predicates and the identity reorderer

If the identity BGP clause reorderer was being used in a query with magic predicates, then the magic predicates were being sent to the end of the BGP. This has been corrected.

Bug21901 - Minor improvements to some range query clause orderings

Improve the statistical estimates used to reorder BGP patterns that use range queries.

Bug21892 - Query engine selection failed to take OFFSET into account

AllegroGraph's decision on whether to use the single-set or the chunk-at-a-time query execution engine relied only on the LIMIT of the query rather than the LIMIT and the OFFSET. This has been corrected.

Bug21880 - Using with-variables in SPARQL describe caused an error

Using SPARQL describe with an external variable binding would lead to an error at query execution time. For example, this query would fail:

 describe ?x 

(where ?x was bound externally). This has been corrected.

Bug21865 - Comparisons of xsd:dateTimes with fractional seconds

Comparisons (including ORDER BY) of xsd:dateTimes with fractional seconds were broken. This bug has been fixed.

Bug21862 - Some FILTERs and BGP orderings could lead to empty bindings

In rare cases, AllegroGraph could fail in its bookkeeping of which variables were bound while it executed a query. This could lead to the introduction of spurious rows with null bindings. This has been corrected.

Bug21858 - Distinct, SERVICE and sub-query could lead to planning failure

If a query used a sub-query in a SERVICE clause and asked for unique results, then AllegroGraph would signal an error during query planning. This has been corrected.

Bug21852 - Improve bound variable tracking

AllegroGraph should assume that any variable supplied via external settings (e.g., via the with-variables parameter to run-sparql) is bound. Failure to do so could cause some queries to fail. This has been corrected.

Bug21828 - VALUES could fail to correctly handle OPTIONAL bindings

Using the VALUES clause could incorrectly drop solutions whose rows contained unbound values. This has been corrected.

Bug21815 - Fix property-path memory leak

Some queries involving property-path expressions could run out of memory, because the intermediate result set was not freed as soon as it was consumed. This bug has been fixed.

Bug21803 - Exclusive ranges of encoded non-integer types

Previously, filter expressions on all non-integer types (floats, decimals, etc, excluding xsd:date and xsd:dateTime) involving exclusive ranges ('<' and '> ') behaved as if an inclusive range ('<=', '> =') had been specified. This bug has been fixed.

Rfe12591: Improve efficiency of some Basic Graph Pattern evaluations

AllegroGraph now makes better use of intermediate result sets when evaluating basic graph patterns (BGPs) in nested groups of joins and left-joins.

AGWebView

Bug21997 - Agwebview and sessions

Due to a thread safety issue, AGWebView's 'Create Session' or any subsequent action in that session could produce garbage results ranging from 'Not found' to garbled output. This bug has been fixed.

Changes to the Lisp API

Rfe12562 - upi->value round-trip

For dates and date-times a complete representation is a (<universal-time> . <timezone-information>) cons. While this would make a good candidate for the return value of upi->value, to maintain backward compatibility by default upi->value must return a single number representing the universal time.

However, this means that during the upi->value/value->upi round-trip the timezone information is lost.

This change adds a new keyword argument :complete? (defaults to nil). With :complete? t, upi->value returns the complete representation and the roundtrip works in the sense of:

(upi= (value->upi (upi->value upi :complete? t) (upi-type-code upi))  
      upi) 

Currently, for UPIs other than dates and date-times, :complete? has no effect.

Rfe12557 - run-sparql can now use a UPI or future-part for the default base

Previously the Lisp function run-sparql only allowed a string to specify the default-base. It can now take a string or a UPI or future-part.

Rfe12508 - normalizing with-variables should convert $var to ?var

The with-variables parameter in the Lisp API to run-sparql allows for variable names to be identified as strings or symbols. The name can be a plain string like "person" or a string whose first character is #\? or #\$ like "?person" or "$person". All three of these representations refer to the same query variable: ?person. Prior to this change, AllegroGraph could misinterpret variable names that used $ as the first character. This has been corrected.

Bug21953 - Support server-side loading via the Lisp client.

When using the Lisp client and connected to a remote-triple-store, passing a URI or URI-like string to one of the bulk-load functions (e.g. load-ntriples) will cause the resource to be loaded directly by the server instead of being delivered via the client.

Bug21891 - db.agraph.log:log-output captured in dumplisp

If dumplisp (a function used to save a running image) was used after opening a triple store, the stream used for logging would be captured. Upon restore of the dumped image, subsequent logging attempts (performed by internal AllegroGraph code) could result in confusing errors or file corruption. This has been fixed.

Bug21860 - Make Lisp clients find bundled liblzo2.so.2

Previously, Lisp clients resorted to using the liblzo library located in the acl directory ("sys:") or somewhere the OS dynamic library loader can find it. With this change the bundled liblzo2.so.2 is loaded.

Bug21841 - db-get-triple leaks sockets

The Lisp function db-get-triple didn't close the socket it used for communication with a remote triple store which resulted in leaking sockets in CLOSE_WAIT state. This bug has been fixed.

Bug21848 - load-turtle could use the wrong triple-store

The load-turtle command could fail to respect the value of the db parameter and instead load triples into the store currently bound to db. This has been corrected.

Bug21810 - Polygon-vertexes did not work with future-parts

If called with a future-part rather than a UPI, the polygon-vertexes function would return no results. This has been corrected.

Bug21611 - Load patches in Lisp remote client

With this fix the Lisp remote client loads patches and enables pretty printing of parts and triples just as the direct client does.

Bug21070 - Fix corruption hazard in direct Lisp client.

Prevent possible database corruption that could occur when interrupting a direct Lisp client app that is modifying the transaction log.

Bug21045 - create-freetext-index function failed when called with UPIs

The create-freetext-index function (in the Lisp API) would fail if called with UPIs (rather than, e.g., future-parts). This has been corrected.

Prolog

Rfe12222 - AllegroGraph Prolog unifies UPIs and future-parts

When AllegroGraph is loaded, the semantics of Prolog unification is extended such that:

This enhancement does not change the capabilities of Prolog queries in AllegroGraph, but allows simpler, less-verbose coding that looks a little more declarative and less procedural. For example, the following

(select (?manager ?employee)  
  (member ?employee (!acme:joe !acme:jill !acme:jim !acme:jerry))  
  (q ?employee !acme:hasManager ?manager)  
  (member ?manager (!acme:bill !acme:bob !acme:betty !acme:benjamin))) 

would not work as intended before this change because ?manager would be unified with a UPI, and a UPI would not unify the future-parts in the last clause. The last clause would have had to be coded as something like

(member ?manager1 (!acme:bill !acme:bob !acme:betty !acme:benjamin))  
(part= ?manager manager1) 

Using Prolog is AllegroGraph is documented here in the Lisp Reference.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug21953. Fix generation of some POST method requests.

AGHTTPClient would write query parameters to the body of POST requests when the content-type was not application/x-www-form-urlencoded. This can result in http servers not finding the query parameters and processing the request incorrectly. This bug corrects the problem by writing POST request query parameters to the URL, unless the request has no body and the content type is application/x-www-form/urlencoded. AGRepositoryConnection.load() (see Javadocs) now works for importing server-side statements.

AllegroGraph 4.11 user-visible changes

AllegroGraph Server: General Changes

Rfe12309 - Improve algebraic query manipulation

AllegroGraph now notices patterns of the form

(:join (:join A (:bgp B)) (:bgp C)) 

and transforms them into the equivalent

(:join A (:bgp B C)) 

(I.e., it merges the two BGPs). This transform brings more triple-patterns together which lets AllegroGraph order the query operations better.

Rfe12305 - Relative paths in the configuration file

Relative file or directory paths in the configuration file were always interpreted relative to the directory containing the configuration file. In this rfe, a BaseDir configuration directive was added that allows another path to be specified for this purpose. See the documentation in Server Configuration for more.

Rfe12280 - Simplify geospatial type-mapping API

Geospatial subtype mappings can now be set up with the regular datatype and predicate mappings rather than needing to use the specialized functions. The old functions still work but are now unnecessary. They will be removed in some future release.

Rfe12260 - Automatically map temporal predicate data to xsd:dateTimes

Triple-stores created in this version of AllegroGraph will include the three AllegroGraph temporal predicates in the list of automatically mapped predicates. I.e., the objects in triples with these predicates:

will be converted into encoded xsd:dateTime literals. This encoding makes range queries against them faster and makes it easier to use AllegroGraph's temporal functionality.

Rfe12259 - Optimize certain combinations of JOINs and FILTERs

The query engine now recognizes certain patterns of JOINs and FILTERs and rewrites them to be more efficient. For example, a pattern like

(:join A (:filter B f)) 

can often be better evaluated as

(:filter (:join A B) f)  

Rfe12182 - More restrictive umask

With this change, users who aren't the owner of the file, or a member of the group of the file (that is, "other" users in chmod manpage terms) have no read, write or execute permission on files created by AllegroGraph.

On existing installations, it's recommended, but in no way enforced, to change permissions the same way with 'chmod -R o-rwx <directories>' where <directories> is the list of directories holding AllegroGraph data (LogDir, SettingsDirectory, catalog directories, etc).

Rfe12175 - Audit log

When the 'Auditing' global configuration directive is set to 'yes' (it defaults to 'no'), events that have implications on security or performance will be logged to the new system database in the new system catalog. Note that regardless of permissions only superuser can read or write the system catalog.

See Auditing for more information.

Rfe12151 - Remove support for loading N-Triples into a federated triple-store

AllegroGraph 3 introduced some techniques that optimized loading of N-Triples into a federated-triple-store. These have been superseded in AllegroGraph 4 because agload is significantly faster and easy to use.

Rfe12147 - Better manage memory especially for large solutions

AllegroGraph now manages large query result sets (with or without DISTINCT or ORDER BY) more efficiently.

Rfe12138 - Agload: reduced memory footprint and other enhancements

Reduced memory consumption of agload processes. Particularly in the case where there are many input sources.

Agload now analyzes input sources on the fly. This means you will not necessarily encounter errors about invalid input sources prior to modification of a triple-store. Use --dry-run command line argument if you wish to pre-analyze an agload job.

Rfe12125 - Improve performance of unordered queries with small limits

AllegroGraph now uses chunk-at-a-time query processing for queries with small LIMITs that do not use ORDER BY more frequently. In particular, the chunkProcessingAllowed query option can now take three values:

The default value for chunkProcessingAllowed has been changed from no to possibly.

Rfe12111 - Improve memory checking efficiency

Improve the efficiency of AllegroGraph's memory tracking code slightly.

Rfe12089 - Improve cursor resource management

Improve AllegroGraph's low-level cursor memory management.

Rfe12063 - Allow in-memory-triple-stores to use blank nodes

The in-memory-triple-store implementation for the remote-lisp-client was missing several methods required for full blank node support. This has been corrected.

Rfe11973 - Improve usage of UPI-map

Improved the way AllegroGraph manages the use of secondary indexes (UPI-maps).

Rfe11955 - :commit argument added to delete-duplicate-triples

The :commit argument was added to delete-duplicate-triples to allow for the deleting a very large number of duplicates.

Rfe11886 - Use statistics more often when reordering query clauses

The query engine now uses store statistics in more places when working to reorder the clauses in a query's basic graph patterns (BGPs). This should produce better plans more often with less need for customer query tweaking.

Rfe11585 - Some queries unnecessarily accumulated partial results

AllegroGraph was sometimes accumulating results unnecessarily. For example, the query

select (count(*) as ?total) where {?s ?p ?o} 

was accumulating the actual results even though only the summary was necessary.

Rfe11511 - Encode xsd:decimals

xsd:decimals used to be stored as strings. With this change, they are automatically encoded directly in the UPI which potentially brings significant savings in storage space and computation in queries. The most significant 20 digits are preserved by the encoding.

Note that this change only affects the encoding of new triples. To convert old triples and take advantage of the more efficient storage, the database contents must be reloaded from a serialized format. We recommend exporting in nquad format and then reimporting it with agload. Restoring from a backup is not sufficient.

Rfe11197 - Serialize triple IDs as resources rather than literals

Previously, a triple ID would have been serialized as a typed literal like

"13"^^<http://franz.com/ns/allegrograph/4.0/triple-id> 

This format prevented the use of triple IDs in the subject of a triple so we have changed the serialization to a URI like

<http://franz.com/ns/allegrograph/4.0/tripleId#13> 

Any triple IDs already in a triple-store will remain the same (though they will print differently). This change does mean, however, than any files that contain triple IDs in the old format will need to be converted in order to be re-loaded (note, however, that the IDs assigned to triples are not part of any serialization format which means that they cannot reliably round-trip in any case).

Rfe10292 - Handle filename limit for unix domain sockets

Linux has a limit of 107 characters for filenames of UNIX domain sockets. With this change, when bumping into that limit, a meaningful error message is printed and the workaround of setting the UnixDomainSocketDirectory or the SettingsDirectory configuration directive to a shorter path is suggested.

Bug21793 - Fix race in inter process communication

Under extremely rare circumstances, multiple threads could end up listening on the same communication channel which can lead to errors and deadlocks. This was duly detected by an assertion that appeared in the log as:

the assertion (endp (intersection  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::r)  
                     (db.agraph.spawn::receiver-for  
                      db.agraph.spawn::receiver)))  
failed. 

This bug has been fixed.

Bug21782 - Potential double close of cursors

An inadvertent double close of cursors was discovered that could potentially result in improper sharing of a data structure. This has been corrected.

Bug21779 - Fix low-level cursor resourcing bug

Under heavy load, the secondary index (UPI-map) creation function could leak a cursor leading to potential resource problems. This has been corrected.

Bug21761 - Prolog q- functor could be treated as q

If a reasoning triple-store can use secondary indexes (UPI-maps) then it was possible for AllegroGraph to treat some q- functors as q functors in a given Prolog select query. This has been corrected.

Bug21745 - Fix crash related to aggregate functions

Under rare circumstances, queries involving aggregate functions may cause the backend to crash. This bug has been fixed.

Bug21744 - Some Prolog select queries could return duplicate results

If using secondary indices (i.e., UPI-maps), it was possible for Prolog select queries that returned raw UPIs to overwrite one solution with another. This problem could only occur with UPI only queries and never with queries that returned strings. The problem is now fixed.

Bug21740 - Rare hangs under high load

The message broker (called the "Hub" process) may get into a state where forwarding a message blocks, preventing all other messages from being handled. This bug has been fixed.

Bug21735 - Corrupted distinct results

Under rare circumstances, queries involving DISTINCT could produce results with the bindings assigned to the wrong variables. This bug has been fixed.

Bug21702 - XSD casting in a BIND could convert to xsd:integer incorrectly

Using an XSD cast in a BIND could loss the datatype and switch it to xsd:integer. E.g.,

BIND(xs:long(?test) AS ?testLong) 

would bind ?testLong to a value whose datatype was xsd:integer rather than xsd:long. This has been corrected.

Bug21693 - Avoid forking too many blank node servers in agload.

Improves agload performance under default conditions when loading ntriples or nquads files which are small and numerous.

Bug21691 - Reasoning over subproperties of symmetric properties

AllegroGraph's RDFS++ reasoner partially failed to infer triples from sub-properties of a symmetric property. This has been corrected.

Bug21680 - Some BIND forms could be evaluated incorrectly.

A BIND form at the start of a BGP could fail to get values from earlier in the query evaluation.

For example, this form

?a a ?b .  
{ BIND( ?b as ?otherB  
  ?otherB a ?c .  
} 

could lose the binding for ?otherB. This has been corrected.

Bug21670 - FILTER and EXISTS could interact incorrectly.

Depending on the shape of the rest of the query, it was possible for a FILTER clause to be applied to the inside of an EXISTS or NOT EXISTS filter rather than being applied to the main body of the query. This has been corrected.

Bug21663 - Redo of new index creation confuses instance process

If a database with a newly-created index has been shut down uncleanly, restart of the database may crash. This has been corrected.

Bug21651 - Prolog ?? and reasoning

On a reasoning triple store, the prolog ?? construct in the predictate position used to fail with a type error. This bug has been fixed.

Bug21642 - Agload buffer overflow

Due to a buffer overflow, agload could corrupt memory and potentially crash. This has been fixed.

Bug21640 - Race condition when reaping abandoned sessions

Previously, if concurrent processes performed the operation to reap abandoned sessions, shared memory state could be corrupted, resulting in hangs or crashes. This has been fixed. (This bug was actually fixed in version 4.10.)

Bug21625 - Large result sets could produce bogus triples with INSERT and CONSTRUCT

In cases where there were more than 100,000 results, the machinery that prevents INSERT and CONSTRUCT queries from producing duplicate results could create invalid answers. This has been corrected.

Bug21596 - Serialize-rdf/xml could fail on some predicates

AllegroGraph's RDF/XML serializer would fail if a predicate ended with something other than a letter, a digit, an underscore or a dash. This conformed to an older version of the Extensible Markup Language but not to the most recent version. This has been corrected.

Bug21586 - Possible divide by zero error in index optimizer

Under some circumstances an index optimizer could attempt to perform a division by zero when logging statistics to agraph.log. This would result in a non-fatal error message being logged instead of the statistics. This has been corrected.

Bug21580 - Improve external variable binding bookkeeping

The values for externally supplied variables could be lost for some complex interactions of EXISTS (or NOT EXISTS) filters and UNIONS.

This has been corrected.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Also, the http error code was changed from 400 ("Bad request") to 503 ("Service unavailable") to indicate a temporary condition.

Bug21553 - Fix MemoryCheckWhen without MemoryReleaseThreshold

If the memoryReleaseThreshold configuration directive was not specified but memoryCheckWhen was, then query execution failed with type errors trying to compare memory footprint to NIL. With this fix, AllegroGraph refuses to start in these circumstances.

HTTP Client

Rfe12136 - Optimize front-end http server

The timeout for reading headers was lowered to 5s which makes the front-end http server make better use of its available resources and more resilent when clients with http keep-alive don't close the connection in a timely manner.

Also, a warning is logged if the number of idle http worker threads falls below 2, because it can easily cause a performance drop.

SPARQL

Rfe12296 - Support AllegroGraph direct reification in SPARQL

AllegroGraph now supports direct reification in SPARQL queries using the <http://franz.com/ns/allegrograph/4.0/tripleId> magic property. This property allows you to bind the ID of a triple to a SPARQL variable which can then be used in reification. For example, the following query first adds two triples and then adds four additional triples that describe the first two:

prefix : <eh://>  
prefix franz: <http://franz.com/ns/allegrograph/4.0/>  
insert data {  
  :gary :likes :dogs .  
  :gary :likes :cats .  
} ;  
 
insert {  
  ?id :mentioned ?now .  
  :gary :believes ?id .  
} where {  
  ?id franz:tripleId (:gary :likes ?what) .  
  bind( now() as ?now )  
} 

See the documentation for additional details on direct reification and its advantages and limitations.

Rfe12246 - Enhancements to SPARQL freetext queries

The fti:match and fti:matchExpression predicates can now optionally retrieve the object of the matching triples and select the freetext index to use. (These enhancements are currently available only for AllegroGraph's native freetext indexes. Support for Solr and MongoDB will be forthcoming).

The new features rely on SPARQL magic properties to provide a syntactic shortcut for list patterns. As an example, this query retrieves the subject (into ?subject) and text (into ?text) for the freetext query that matches "nepal":

select ?subject ?text {  
  (?subject ?text) fti:match 'nepal' .  
} 

This query retrieves only subjects and uses only the freetext index named comments.

select ?item {  
  ?item fti:match ('upgrade' 'comments')  
} 

More examples and information can be found in the documentation.

Rfe12243 - Support AllegroGraph Social Networking Analysis in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable Social Networking Analysis including centrality measures, path finding and cliques. See SPARQL Magic Properties for more details.

Rfe12231 - Improve SPARQL geospatial integration

AllegroGraph now supports several magic properties to make SPARQL / geospatial integration easier. For example, this query finds all of the points within a circle centered at 10, 2:

select ?who ?where {  
  (?who ?where) geo:inCircleXY ( ex:location 10 2) .  
} 

The following magic properties are provided:

geo:inBoundingBoxXY  
geo:inBoundingBox  
geo:inCircle  
geo:inCircleKilometers  
geo:inCircleMiles 

(where geo is the usual Geospatial namespace abbreviation for <http://franz.com/ns/allegrograph/3.0/geospatial/>). See SPARQL Magic Properties for additional details on using these properties.

Rfe12171 - Improve variable GRAPH clause handling in some situations

SPARQL specifies that a graph clause like

 GRAPH ?g { ... } 

must be implemented as if it iterated over each graph in the dataset and took the union of the resulting bindings. There are sometimes more efficient ways to get the same results and AllegroGraph uses these when it can. AllegroGraph was not, however, using information about any constraints on the variable ?g that were provided by other parts of the query. For example, a query like:

?s :p ?g .  
GRAPH ?g { ... } 

might produce a single binding for ?g but AllegroGraph was still iterating over all of the named graphs of the dataset. This has been optimized so that AllegroGraph will use the additional information when it can determine that the number of bindings for ?g are smaller than the number of graphs in the dataset.

Rfe12057 - Manage disk and memory resources better for CONSTRUCT queries

When there are many results, SPARQL CONSTRUCT queries buffer their results to disk to conserve memory. The cutoff value for when to buffer to disk was unnecessarily small which meant that CONSTRUCT went to disk more often then necessary. This has been corrected.

Rfe12035 - Improve handling of some SPARQL queries with a GRAPH clause

By definition, SPARQL must handle a GRAPH clause as if the query engine evaluated the clause once for each graph in the dataset. However, it is often possible to evaluate the clause more efficiently.

This change improves the heuristics AllegroGraph uses to determine when the more efficient approach will achieve the correct results which means that some queries with a GRAPH clause will be executed more quickly.

Rfe11319 - Support SPARQL federated queries (the SERVICE clause)

AllegroGraph now supports SPARQL federated query using the SERVICE clause. The initial implementation is functional but not highly optimized.

Rfe8063 - Support AllegroGraph temporal reasoning in SPARQL

AllegroGraph's SPARQL engine now supports a set of magic properties to enable temporal reasoning. See SPARQL Magic Properties for more details.

Bug21757 - SPARQL geospatial queries could fail

Depending on the set up of a triple-store's geospatial subtype mappings, it was possible for a SPARQL query using geospatial extensions to fail to find results. This has been corrected.

Bug21742 - SPARQL parser failed on quad patterns with collections

The SPARQL parser could lose track of triple-patterns that arose from abbreviations using either nested blank nodes or collections. This has been corrected.

Bug21725 - Correct potential error when SPARQL Update touches too many rows

If a SPARQL DELETE ... INSERT ... WHERE request changes many rows, then AllegroGraph caches the results of the WHERE expression to disk to reduce memory pressure. In some cases, this could result in a error when the cached results were re-read. This only occurred for Update requests that touched more than 100,000 rows. This has been corrected.

Bug21718 - Unused variable optimization could produce incorrect results

AllegroGraph analyzes SPARQL queries that use DISTINCT in order to remove the collection of bindings for variables that are not needed to produce the final result. In some cases, it was possible for this analysis to cause the execution of a code path that added solutions incorrectly. This has been corrected.

Bug21573 - SPARQL queries could intern strings unnecessarily

SPARQL queries that used the FROM or FROM NAMED clause always tried to intern the graphs into the triple-store's string table. This was both unnecessary and could cause an error if the triple-store was read-only or a federation.

This has been corrected.

Bug21011, Bug21571 - handle nested SPARQL BGP syntactic sugar correctly

Some SPARQL triple-patterns that involved nested brackets and object lists were being parsed incorrectly. This has been corrected. Some examples of queries that work correctly now but that failed without this change include:

PREFIX : <ex:>  
SELECT * {  
    :x :y [:z (:b [:d (:e :f),(:g :h)])]  
}  
 
PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c :d], [:e :f] .  
} 

and

PREFIX : <ex:>  
SELECT * {  
    ?psi :a :b, [:c [:d :e]] .  
} 

Improve Pattern estimation where there is no :p index

Improved query pattern estimators for triple-stores with no predicate index (e.g., :posgi, :psogi, etc.). This will help reorder query clauses more intelligently.

AGWebView

Rfe12325 - Add audit log query interface to AGWebView

There is now an audit menu item in the admin menu. This provides access to the audit log query page which lets you filter audit log events by user, class and date.

Changes to the Lisp API

Bug21708 - Fix some low-level Lisp API geospatial functions

Some low-level Lisp API functions had not been enhanced to account for AllegroGraph's geospatial encoding. This has been corrected.

Bug21689 - Lisp client printer variable leakage

On a remote store, the run-sparql function failed with an 'Illegal sharp character' error if print-length was set to a small value. This bug and more generally the leakage of printer variables into run-sparql has been fixed.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe12118 - AGRepository with connection pooling

With this change, an AGRepository can now be configured to use a connection pool via the setConnPool method, enabling Sesame apps to transparently benefit from connection pooling. See the javadoc for setConnPool for details.

Rfe10513 - Configurable blankNodesPerRequest

With this change, the Java client supports configuring the number of blank nodes automatically fetched per request to the server by the AGRepository's AGValueFactory. The default remains at 100, but it can be configured by setting the property:

com.franz.agraph.repository.blankNodesPerRequest

Applications can also configure it directly, see the javadoc for AGValueFactory#setBlankNodesPerRequest for details.

Rfe9869 - Support SPARQL/Update from Jena

With this change the Jena adapter supports an execUpdate method in AGQueryExecution for executing SPARQL Updates. Jena Tutorial example13 has been augmented to demonstrate its use.

AllegroGraph 4.10 user-visible changes

AllegroGraph Server: General Changes

Rfe12004 - Ag4 plugins for TBC 4.0.0

With this change, the AG plugins have been updated to work with TBC 4.0.0. For installation details, see Top Braid Composer Plugin.

Rfe11986 - Improve efficiency of some property-path queries

AllegroGraph now evaluates property path queries whose initial or final endpoints come from a small set of values more efficiently. For example, a query like

prefix gnO:<http://www.geonames.org/ontology#>  
select ?id ?par ?parname {  
  bind (URI('http://sws.geonames.org/104515/') as ?id)  
 
  ?id rdf:type gnO:Feature .  
  ?id gnO:parentFeature+ ?par .  
  optional {  
    ?par gnO:name ?parname  
  }  
} 

will be evaluated much more quickly than before because the subject of the path query is fixed.

Rfe11937 - Agload: remove blank node cache size guessing code

In agload, blank node cache size guessing code did not work very well. It was removed and the blank node cache size defaults to 1,000,000 which works reasonably well on small and large memory machines.

This change will only be noticed when loading ntriple or nquad files with blank nodes.

Rfe11839 - Improve eager evaluation of impossible query constraints

AllegroGraph tries to evaluate FILTER constraints as early as possible. In particular, a constraint may become impossible or a tautology before the query even runs if external variable bindings are supplied. AllegroGraph was already handling tautologies eagerly but was not doing the same for impossible constraints. This has now been remedied.

Rfe11827 - Improved memory footprint of joins with unbound variables

Previously, queries involving joins had a worst case memory footprint of O(2^N) where N is the number of join variables. This is now O(1), but the time complexity is O(N). In practice, this means that the client gets a timeout instead of a memory exhaustion warning.

Rfe11233 - Simplify property path queries when possible

Property path queries that do not use the zero or more or one or more operators and that do not have complex alternation can be expanded directly into simpler forms. This lets them be executed using the regular query machinery which can often be faster than the specialized path query machinery.

Rfe9699 - Dynamically grow string table files

Prior to this change string tables files were large sparse files. The apparent size of these files frequently led to user confusion. In addition, full filesystem errors were not handled gracefully when modifying the sparse files. Now string table files start at a minimum size and grow as needed. If growth of a string table fails due to a full filesystem error, a complaint will be logged to agraph.log and the operation will be retried periodically until it succeeds.

Bug21592 - Improve cursor resource management

Some cursors were failing to close all of their file handles which could lead to errors. This has been corrected.

Bug21583 - see Bug21575 below.

Bug21582 - Complex nested unions could fail to return some bindings

Given the right data, a complex query involving multiple nested unions could misapply an optimization meant for one part of the union to a different part. This could lead to some variables in the query being unbound.

This has been corrected.

Bug21575, Bug21583 - property path optimizations and better bookkeeping

Improve the way that AllegroGraph implements some property path queries.

Correct a problem with property path queries and over-zealous resource management.

Improve the bookkeeping of unused variables when using external bindings.

Bug21570 - Fix 'No session ports available' error message

When trying to start a dedicated session and no more free ports were available, AllegroGraph failed with an internal error (calling @UUID on NIL) instead of producing an informative error message. This bug has been fixed.

Bug21550 - Complex FILTER EXISTS expressions could cause parsing errors

Complex FILTERs that used EXISTS (or NOT EXISTS) could signal errors during parsing. This has been corrected.

Bug21549 - Nested sub-queries could fail to project variables correctly

A query with nested sub-queries that used * to return all variables in some of the sub-queries could incorrectly reason about variable scope which could cause queries to fail. This has been corrected.

Bug21534 - Garbled error message on memoryCheckWhen parsing failure

When the memoryCheckWhen configuration directive had an unsyntactic value ("memoryCheckWhen junk:1"), the error message was wrong. This bug has been fixed.

Bug21506 - Correct possible error when mixing UNION and GRAPH

Under some circumstances, a query with both UNION and GRAPH clauses could signal an error during execution. This has been corrected.

Bug21505 - Unused variables from a sub-query could confuse AllegroGraph

AllegroGraph removes query variables that are not used if a query has either the DISTINCT or REDUCED modifiers. The unused variables logic was not correctly handling the case of a sub-query whose projected variables were not used elsewhere. This could lead to an error during query processing. This has been corrected.

Bug21498 - Do not signal error when all ORDER BY expressions are redundant

AllegroGraph removes ORDER BY expressions that do not depend on the bindings of a row because they cannot have any bearing on the outcome. For example, the query:

SELECT * { ?s a ?o } ORDER BY ?pineapple 

is equivalent to the query

SELECT * { ?s a ?o } 

Unfortunately, AllegroGraph was incorrectly signaling an error when all of the expressions in an ORDER BY clause were irrelevant and therefore removed.

This has been corrected.

Bug21497 - Filters could be misapplied in some situations

AllegroGraph attempts to move FILTERS closer to where they apply in order to increase query speed. If a FILTER was applicable on both sides of a UNION and also applied to single triple-patterns then it was possible for AllegroGraph to think that the FILTER only needed to be used on the left-hand side of the UNION. For example, a query like this could fail to apply the FILTER to ?c on the right-hand side:

select * {  
  { ?a ex:b ?c . } union { ?a ex:c ?c . }  
  filter (?c*2 = 2)  
 } 

This has been corrected.

Bug21496 - Improve memory management for CONSTRUCT queries

AllegroGraph now tracks memory more accurately and manages it more efficiently for CONSTRUCT queries.

Bug21493 - Bind is too aggressive in assuming single values

AllegroGraph was over-optimizing assignment (BIND) such that assignments to the same variable name in different pieces of a UNION could fail to work correctly. For example, a query like

SELECT * {  
  { BIND( :person1 as ?person ) }  
  UNION  
  { BIND( :person2 as ?person ) }  
} 

would return only one binding for ?person rather than two.

This has been corrected.

Bug21486 - Correct handling of some IN LINE data syntax

AllegroGraph was not correctly reading all of the supplied values when using the abbreviated simple-variable syntax. E.g., a query like

select ?person (concat(?first, ' ', ?last) as ?name) {  
  values ?person { ex:person1 ex:person2 ex:person10 }  
  ?person ex:first-name ?first .  
  ?person ex:last-name ?last .  
} 

was processing only the first supplied value. This has been corrected.

Bug21476 - Some blank nodes could fail to print in a 32-bit Lisp client

It was possible for blank-nodes to raise an error when AllegroGraph tried to print them in N-Triples format. This has been corrected.

Bug21470 - Corrected Turtle parser relative URI normalization

AllegroGraph was incorrectly applying sections 6.2.2 and 6.2.3 of RFC3986 when it merged a base URI with a relative one. This has been corrected.

Bug21458 - Standardize AllegroGraph exit code

AllegroGraph's license check can be resolved in three ways:

  1. Normal start up - a license is present and correct.

  2. Free start up - no license is present so the free limit is used.

  3. Limited start up - only partial license information is present so AllegroGraph runs in free mode.

In all of these cases, the exit code returned will be zero (0).

If the license code is invalid or if there are any other problems with server start up, then AllegroGraph will fail to start and return a non-zero exit code.

Bug21454 - URI parsing was too strict

AllegroGraph was being stricter than necessary when parsing BASE URIs such as foo:bar:/baz. This has been corrected.

Bug21379 - Reasoning triple-store confusion about unspecified graph fixed

A reasoning triple-store failed to distinguish between a query where the graph was left off (i.e., left as a wild card) and a query where the graph was explicitly set to the default-graph of the current triple-store. This could cause it to infer incorrect triples.

This has been corrected.

Bug21365 - Fix lookup of user access rights when authorizing access to repositories in the root catalog.

AllegroGraph would deny authorization requests on repositories in the root catalog under some conditions, despite specific user access right allowing such (read/write access on catalog "/", repository ""), or global rights (read/write access on catalog "", repository "*"). This has been corrected.

Bug21353 - Correctly handle aggregation on empty datasets

AllegroGraph was failing the Data Access Working Group (DAWG) "agg empty group" test because it was not returning a row for a query asking for aggregation on an empty dataset. This has been corrected.

Bug21158 - Catch errors during filter analysis

AllegroGraph was signaling an error during query planning if the eager analysis of a FILTER expression resulted in a error. The correct behavior is to treat the FILTER as an impossible constraint.

This is now handled correctly.

HTTP Client

Rfe11609 - Support text/table content-type in the HTTP protocol

AllegroGraph will now return a simple ASCII table if an HTTP request uses the content-type text/table. For example, a query might return a table like:

  -----------------------------------  
  | person    | name     | location |  
  ===================================  
  | mo:Bob    | "Bob"    | ---      |  
  | mo:Robert | "Robert" | 23       |  
  -----------------------------------  
 

Bug20622 - "Mode" was not optional for DELETE /statements/duplicates

The mode parameter, which is used to indicate which components of each triple must be equivalent to count as duplicates of each other, is now optional and defaults to "spog" for the DELETE /repositories/[name]/statements/duplicates HTTP protocol request.

SPARQL

Rfe11920 - Signal error when aggregate expressions are used in the wrong place

AllegroGraph did not signal a useful error when aggregate expressions were used incorrectly in a SPARQL query. E.g.,

SELECT * {  
  ?a ex:foo ?value .  
  BIND( SUM(?value) AS ?sumValue )  
} 

This has been corrected.

Rfe11494 - Update semantics of SPARQL property path queries

AllegroGraph's property path queries now match the updated semantics of the SPARQL 1.1 specification. In particular, wild-card operators like * and + no longer count their matches. AllegroGraph still supports the now deprecated counting operations, e.g.,

{2,4} 

but no longer counts the unique paths found. I.e., you can use these non-standard operators to find if there is a path but not to count the number of paths.

Note that AllegroGraph now passes all of the Data Access Working Group (DAWG) tests with the exception of

 (pp37) Nested (*)* 

This test uses the path

((:p)*)* 

and AllegroGraph incorrectly returns some duplicate results.

Bug21599 - Variable GRAPH queries that used the graph could lose results

A SPARQL GRAPH query that used a variable graph and included the graph variable in the body of the query could under some circumstances lose results. For example, a query like

SELECT * {  
  GRAPH ?g {  
    ?g a <FoodGroup> .  
    ?g <covers> ?r .  
  }  
} 

could possibly have returned too few results depending on the data in the store.

This has been corrected.

Bug21581 - Improve bookkeeping when using property paths

If AllegroGraph can determine that a variable in a SPARQL query is not used, then it will enable certain optimizations. For example, if a query is returning DISTINCT results and a variable is not used in any joins nor returned by the query, then its values do not matter.

This logic was not correctly rewriting property paths to account for the unused varaibles.

This has been corrected.

Bug21562 - Insert & remove-graph-uri(s) should not override WITH

If insert-graph-uri(s) or remove-graph-uri(s) were passed as query parameters to a SPARQL DELETE...INSERT...WHERE statement, then AllegroGraph was using these parameters even if the statement used WITH to specify a GRAPH. This is now corrected. SPARQL Update statements that use WITH will not be overridden via insert or remove graph URI(s).

Bug21551 - Deletes via SPARQL Update did not correctly handle encoded UPIs

SPARQL DELETE DATA and DELETE...INSERT...WHERE commands could fail to remove some triples due to AllegroGraph's internal representation of typed literals. The failure would occur only if the literal was specified directly (i.e., when the literal was a variable, AllegroGraph behaved correctly).

Bug21548 - Nested unions with fixed patterns could miss results

If a SPARQL query contained nested unions and these unions contained fixed triple-patterns (i.e., patterns with no variables or patterns where all of the variables are bound via some external binding), then some results from these unions could be lost. This has been corrected.

Bug21546 - Correct off-by-one error in some SPARQL constraints

Depending on the data and the query, some SPARQL constraint analysis could lead to a range error. This has been corrected.

Bug21515 - Improve SPARQL parser speed

Compliance changes in the SPARQL parser of AllegroGraph v4.9 reduced the speed of parsing significantly for very large queries and update requests.

This has been corrected so that parse speed is now equivalent to what it was in AllegroGraph v4.8.

Bug21514 - Fix possible error caused by nested sub-query

The data structures used for SPARQL sub-queries were sometimes being cleaned up too early which could lead to a bus error. This has been corrected.

Bug21507 - Fix problem with property path query clause estimation

AllegroGraph was incorrectly estimating the size of clauses involving property paths in SPARQL queries which could lead to poor plans. This has been corrected.

Bug21478 - Fix spurious parsing error when mixing sub-query and aggregation

AllegroGraph's SPARQL parser could become confused if a query mixed sub-query and aggregation and the sub-query used * to refer to all variables.

This has been corrected.

Bug21469 - Relative URIs were not being correctly normalized by the SPARQL parser

Section 5.2 of RFC3986 specifies how "." and ".." path components of relative URIs should be normalized when merging with a BASE URI. The original SPARQL parser implementation skipped this step in the interests of efficiency. This has now been corrected.

Bug21464 - Some combinations of nested UNION and OPTIONAL could fail

The SPARQL 1.1 engine could fail to correctly keep track of solution bindings for queries with certain patterns of UNIONs and OPTIONALs. This has been corrected. #### Bug21501 - Queries with cross-products and empty RHS could emit incorrect solutions

If a query had a cross-product and the right-hand side of the cross-product was empty, then it was possible for AllegroGraph to not discard the solutions on the left-hand side. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

Bug21535 - Db registry memory leak in the Lisp client

If a triple store object isn't closed properly and becomes garbage, the finalizer is supposed to close it. Fixed a bug that rendered this mechanism ineffective, causing leakage of memory.

Bug21512 - With-variables not handled consistently with sub-queries

In some cases, external bindings imposed from with-variables could be lost when using sub-queries. This has been corrected.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe11527 - Support sessions that use the main server port

With this change, the Java client can now be configured to have sessions use the main server port rather than a dedicated port. By default, a dedicated port will be used. Setting the system property com.franz.agraph.http.useMainPortForSessions=true will configure the client to use the main port for sessions.

Rfe8384 - Limiting the duration of a query

With this change, the Java client now supports limiting the duration of a query. When Sesame applications use the AGQuery#setMaxQueryTime method, the AG server will timeout the query appropriately during evaluation and clean up server resources, and the client library will throw a QueryInterruptedException back to the client application.

AllegroGraph 4.9 user-visible changes

AllegroGraph Server: General Changes

Rfe11922 - Add materiazer. See Materializer for details.

Rfe11879 - Improve error message on invalid license

AllegroGraph now prints information to the console when there are problems with the license settings in the configuration file.

For example, if the configuration file is missing the LicenseExpires parameter, then AllegroGraph will print:

AllegroGraph Server Edition 4.9, built on October 01, 2012 10:04:33 GMT-0700  
Copyright (c) 2005-2012 Franz Inc.  All Rights Reserved.  
current-time: Monday, October 01, 2012 10:43:09 AM  
 
Daemonizing...  
Server started: LicenseExpires parameter missing. Running with triple-limit of 5,000,000.  
 

Rfe11845 - Optimize blank node printing

Blank nodes now print slightly faster and more efficiently than they did before.

Rfe11827 - Reduce memory footprint a bit

Result arrays are now freed right when they become unused. Previously, garbage accumulated until a certain limit was reached. This should have a small positive effect on high water-mark memory footprint.

Rfe11784 - Release memory back to the operating system after query failure

Prior to this change, AllegroGraph would continue to claim memory used for queries even after the query would complete. AllegroGraph will now return unused memory back to the OS when a query memory failure occurs.

Rfe11762 - Improve the efficiency of queries with unbound results

Queries that use UNION or OPTIONAL can have unbound columns in the result set. This change improves the way AllegroGraph does joins when there are unbound columns in a result set.

Rfe11756 - Improve the chunk processing behavior of ORDER BY and LIMIT queries

The chunk-at-a-time processor automatically reduces the size of the chunks being processed based on the limit (it makes no sense to process a chunk of 500,000 rows if you are only interested in the first one!). In the case of ORDER BY queries, however, it makes sense to keep the larger limit since all of the results must be examined. This is now the case.

Rfe11728 - Improve handling of large DISTINCT or ORDERed results

This enhancement improves the efficiency of the on-disk merging operations so that the size of ORDERed or DISTINCT result sets is limited only by disk space and not by available RAM.

Bug21531 - Non-simple EXISTS FILTERs could cause query failure

AllegroGraph was incorrectly processing non-simple graph patterns in an EXISTS or NOT EXISTS FILTER. This meant that queries like

SELECT * {  
   ...  
   FILTER( EXISTS { ... } ||  
           EXISTS { ... FILTER( ... ) } )  
} 

could cause an error rather than being correctly processing. This has been corrected.

Bug21415 - Queries could return invalid solutions

It was possible for solutions with no bindings for some of their variables to escape from the right-hand side of a join. This would cause too many solutions to be returned from the query.

This has been corrected.

Bug21410 - Duplicate suppression not handling deleted triples correctly

AllegroGraph was handling deleted triples improperly when duplicate suppression was enabled. This has been corrected.

Bug21407 - An error in a SELECT expression prevented processing of subsequent ones

An error in a SELECT expression would prevent processing of the rest the expressions in the query for the current solution. For example, depending on the data, in a query like:

SELECT  
        ?s  
        ?str  
        (STRAFTER(?str,"b"@cy) AS ?abcy)    # could cause a type error  
        (STRAFTER(?str,"ab") AS ?aab)  
WHERE {  
        ?s :str ?str  
} 

The evaluation of STRAFTER() with a second argument of "b"@cy could signal a type-error. AllegroGraph was stopping further computation at that point which meant that the binding for ?aab could be left empty when it should not have been.

This has been corrected: now, each expression is computed separately.

Bug21404 - Config file EOL format

Previously, non-unix newlines (CR+LF, LF+CR) in the license section of the config file could prevent the server from starting up. This has been fixed and any mix of Unix, Mac and DOS style newlines is accepted.

Bug21398 - Variables in EXISTS filter could be incorrectly bound

It was possible for variables that appeared only in an EXISTS (or NOT EXISTS) filter to be incorrectly bound during query evaluation which could lead to incorrect results.

This has been corrected.

Bug21390 - N-Triples serializer does not handle embedded angle brackets

Even though valid URIs are not allowed to contain embedded angle brackets, it is still possible to add them to a triple-store. For example, the valid N-Triples file:

<ex://h> <ex://p> <ex://invalid\u003E> 

would add a triple whose object is ex://invalid>.

AllegroGraph was serializing such URIs without performing the necessary \u escaping which meant that the data exported could not be re-read by AllegroGraph's parsers.

This has been corrected.

Note that this was only a problem for data with invalid URIs.

Bug21377 - Updates with both DELETE and INSERT

Due to a race condition in a shared backends, updates that involved both DELETE and INSERT could produce bogus results. This bug has been fixed.

Bug21376 - Wrong results with ORDER BY

Due to a rare race, in shared backends, queries with ORDER BY could produce bogus results. This has been fixed.

Bug21367 - Expressions in ORDER BY could cause query failure

Queries with expressions in an ORDER BY could cause query failure during string look up. This bug was introduced in AllegroGraph v4.8 and is now corrected.

Bug21349 - Eager evaluation of boolean FILTERS could lead to failures

If a query used FILTERS consisting of boolean combinations of simpler filters and also used external variable bindings such that some of the FILTER clauses could be evaluated immediately, then it was possible for the FILTER evaluator to signal an error.

This has been corrected.

Note that bug21342 corrected this problem for the simpler case of negation. This bug fixes the more general case.

Bug21345 - Possible problem with BIND inside of an OPTIONAL

If a query included a BIND clause that stood by itself and there was a join, it was possible to lose the value of the binding. In particular, a query like

SELECT * { ?a ?b ?c . OPTIONAL { bind('hello' as ?x } } 

could leave ?x unbound. This has been corrected.

Bug21344 - Bind could fail to use external variable bindings

If a query used the BIND form and the value of the expression depended on variables set via an external binding (e.g., via the with-variables argument in the Lisp API or via HTTP parameters in the HTTP interface), then it was possible for the BIND expression to ignore those values. This has been corrected.

Bug21342 - Eager evaluation in FILTERs can lead to problems using variables

If a query uses a FILTER and also variable bindings such that an expression in the FILTER could be evaluated before actually running the query and if the expression was used in other boolean combinations (e.g., with not, and, or or), then it was possible for AllegroGraph to signal an error. For example, if the query was

select * { ?a ?b ?c . FILTER( ! bound( ?d )) } 

and if ?d was bound using a variable binding, then AllegroGraph would signal an error instead of correctly realizing that the FILTER could never succeed.

This has been corrected.

Bug21341 - Non-simple IF statements could cause an error

It was possible for non-simple IF statements in an expression to lose track of their variables which led to an error at query planning time.

This has been corrected.

Bug21326 - Geospatial query failure

Fixed a bug that, under rare circumstances, caused geospatial queries to fail with bus error.

Bug21325 - Lost results in sub-queries and some joins

When an internal buffer holding intermediate results was grown in a sub-query or in some joins, results could have been lost. This bug has been fixed.

Bug21272 - No results with BIND

Sometimes, as in the query below, instead of no results, a single result was returned when BIND was involved.

select * {  
  bind('o' as ?c)  
  ?a ?b ?c .  
  ?d ?e ?f .  
  filter( ?a < ?d && ?d < ?a )  
} 

This bug has been fixed.

Bug21270 - Sub-queries with ORDER BY can fail to bind results

Depending on the data and what other queries have been evaluated, a query that used an ORDER BY clause in a sub-query could fail to correctly bind the result variables. For example, a query like:

select ?age {  
  { select ?age {  
        ?who <http://people.example/age> ?age .  
    } order by ?age  
  }  
} 

could return results but leave the bindings for ?age unbound.

This has been corrected.

Bug21260 - XPath Constructor Functions fail on strings to numbers

The XPath Constructor Functions should convert typed string literals into the requested numeric type when the value of the string can be so coerced. For example:

xs:int('2'^^xsd:string) ==> 2 

AllegroGraph was raising a type error in this case. This has been corrected.

Bug21256 - Parser problem with some queries that mixed aggregation and sub-query

AllegroGraph was incorrectly signaling a parser error on some queries that mixed aggregation and sub-query. For example,

select (max(?a) as ?foo) {  
  select distinct ?a ?b {  
    ?a a ?c .  
    ?c a ?b .  
  }  
} 

This has been corrected.

Bug21245 - Allegrograph's turtle parser performs URI normalization incorrectly

Percent encoded URIs should not be unescaped when loading data in the Turtle format. AllegroGraph was incorrectly doing so. This has been corrected.

Bug21243 - Non-QNameable URIs cause turtle serializer error

If a triple contained a URI that could not be expressed as a QName, then the turtle serializer would signal an error that it could not split the URI rather than just emitting the entire URI and continuing. For example, a triple containing the URI <<ttp://example/> would cause this error:

Cannot split <http://example/> into XML QName. 

This has been corrected.

Bug21226 - agload max blank node cache size not obeyed in all cases

Agload could over-allocate memory in a certain case. This has been fixed.

Bug21215 - Improve cursor resource usage of Prolog SELECT queries

AllegroGraph previously closed some of the cursors created while running a Prolog query lazily. It now closes them when the query completes which improves overall resource utilization.

Bug20906 - Remove duplicate triples fails on a store with zero triples

An error would be raised if duplicate deletion was attempted on a store with zero triples.

This has been corrected.

HTTP Client

Rfe11755 - Add support for insert-graph-uri, remove-graph-uri, etc.

Version 6 of the Sesame HTTP Protocol includes several graph parameters that can be used to control the behavior of SPARQL Update statements. These are:

These four parameters are now supported by AllegroGraph

Rfe11743 - Add baseURI to the SPARQL HTTP Protocol

The SPARQL HTTP protocol now accepts baseURI as an HTTP query request parameter. This can be used to shorten the rest of a SPARQL query.

Bug21408 - Retrieving JSON statements over HTTP

Fixed a bug that caused the statement list to be truncated when an 'application/json' encoded response was requested via the HTTP protocol.

Bug21280 - Query failures cause spurious errors

AllegroGraph 4.8 modified the internal API used to signal query failures (e.g., due to timeout or memory limits). Unfortunately, the code in the HTTP layer of AllegroGraph was not modified to accommodate the change which led to spurious bus errors when a query failed. This has been corrected.

SPARQL

Rfe11862 - Match default dataset behavior for SPARQL Update and Query

SPARQL Update was using a default dataset that contained only triples with unnamed graphs (the default graph in AllegroGraph terminology) in the FROM portion and put triples with graphs into each of their graphs in the FROM NAMED. This was different from the default dataset for SPARQL Query which put all of the triples into the FROM portion and treated the FROM NAMED portion similarly.

AllegroGraph now uses the same default dataset behavior for both Query and Update. This dataset is as described for query above. Note that this means that an Update command like

DELETE DATA { <eh://a> <eh://b> <eh://c> } 

will delete the triples

{ <eh://a> <eh://b> <eh://c> } 

from every graph in which it appears whereas the previous behavior would have been to delete it only from the default graph.

Note that triples added during an Update command which do not specify a graph will be added to the triple-store's default graph.

Rfe11817 - SPARQL parser should signal an error on badly formed requests

Previously, the SPARQL parser would silently accept nonsense queries like:

<http://www.foo.com> <http://www.bar.com> 'banana' . 

because it would silently ignore tokens that came before the SPARQL prologue (i.e., before PREFIX and BASE) and convert such a query into an empty UPDATE request (i.e., a non-operation).

These queries will now signal a SPARQL parse error. Because the parser is now more strict, some queries that previously parsed may now signal an error. For example, this invalid query was previously accepted and now will not be:

# WRONG: bad prefix because it ends with a period (".")  
PREFIX ex: <http://example.com> .  
SELECT * { ?s ex:testsFor ?o . }  

Rfe11793 - Improve handling of numeric and date literal equality filters

Improve the SPARQL 1.1 query planner's handling of numeric and date literal equality filters. In particular, AllegroGraph will now optimize queries with FILTER clauses like:

?s ex:p1 ?o .  
FILTER( ?o = 3 ) 

which would previously have required a scan of all triples with predicate ex:p1.

Rfe11787 - Support SPARQL 1.1 UUID and STRUUID functions

Adds support for the SPARQL 1.1 UUID() and STRUUID() functions.

Rfe11779 - Support SPARQL 1.1 change to inline data syntax

SPARQL 1.1 recently switched from a syntax for inline data that used BINDINGS to one that uses VALUES. The standard also made it possible to use inline data in more places. AllegroGraph now fully supports this feature and passes all of the Data Access Working Group (DAWG) tests for it.

Note that the BINDINGS syntax is still supported in this version of AllegroGraph. It will be deprecated and removed in some future release.

Rfe11757 - Provide select-N strategy for ordered results with small limit

Typically, a SPARQL query with an ORDER BY clause must accumulate all of the results before sorting them. If, however, the query has a small LIMIT, then it can be more efficient to retain the top LIMIT results and only sort those.

Rfe11555 - Manage memory used to ensure unique results from CONSTRUCT and UPDATE

The data structures AllegroGraph uses to ensure that results of a SPARQL CONSTRUCT or UPDATE are unique have been modified so that result sets can be processed correctly regardless of size (up to the amount of available disk space).

Bug21401 - SPARQL Update must intern strings from variable bindings

If variable bindings are used in a SPARQL Update query and these bindings are part of the template being used to insert new triples into the store, then the strings from the variable values must be added to the triple-store's string table. This was not being done which could lead to a UPI not in string table error.

This has been corrected.

Bug21391 - Queries against in-memory-triple-stores could return incorrect results

The Lisp SPARQL API allows for the creation of in-memory-triple-stores from CONSTRUCT and DESCRIBE queries. These suffered from a bug such that queries that specified more than two of subject, predicate, object and graph fields could return incorrect triples. As examples, a query like:

ASK { ex:foo ex:bar ?o } 

would work correctly whereas a query like

ASK { ex:foo ex:bar ex:baz } 

could return yes when it should have returned no.

This has been corrected.

Bug21372 - Possible error when requesting sparql-xml output for Prolog select

An error could be signaled if the default query engine was sparql-1.0 and a results format of sparql-xml was requested for a Prolog select query.

This has been corrected.

Bug21371 - IF confusion with free variables

SPARQL queries with free variables could produce incorrect results when using IF. For example, the following query

SELECT ?result {  
  BIND( IF(!bound(?dataType), "first", "second") as ?result)  
} 

would return second instead of first. This occurred because the query engine was not correctly analyzing the free variable ?dataType (this variable is free because the query provides not binding for it).

This has been corrected.

Bug21361 - Improve SPARQL BASE URI handling

AllegroGraph was not always correctly merging base and relative URIs. For example, a query like

BASE <http://localhost:8080/>  
SELECT * { ?a ?b </> } 

was being incorrectly run as if it was

SELECT * { ?a ?b <http://localhost:8080//> } 

This and other similar issues have been corrected.

Bug21357 - SPARQL blank-node parsing was too strict

The SPARQL parser was not allowing the full range of blank node names to be used. Thus, queries like:

SELECT * {  
  ?a ?b _:123 .  
  _:123 ?c _:fv-0 .  
} 

would fail to parse correctly. This has been corrected.

Bug21354 - Update STRAFTER() and STRBEFORE() to match changes in the standard

Recent changes to the SPARQL 1.1 standard changed the output of STRAFTER() and STRBEFORE() when no match is found. Previously, they were specified to return an empty literal with language or datatype that matched the first argument. Now they should produce an empty literal with neither language nor datatype. AllegroGraph now matches the standard.

Bug21294 - Sparql XML results for ASK queries missing head clause

The SPARQL XML generated by AllegroGraph was missing the <head/> clause. Since this clause is generally empty, only systems that validate the results against an external DTD were effected by the missing clause.

This has been corrected.

Bug21277 - Improve cursor resource usage of SPARQL 1.1 queries

AllegroGraph previously closed some of the cursors created while running SPARQL 1.1 queries lazily. It now closes them when the query completes which improves overall resource utilization.

Bug21266 - Queries with only a geospatial clause could fail to gather solutions

If a SPARQL query contained only a geospatial clause, then the SPARQL 1.1 engine could fail to add the solutions it found to the answer set. This has been corrected.

Bug21249 - Incorrect return value of COPY GRAPH <X> TO GRAPH <X>

A SPARQL Update statement like

COPY DEFAULT TO DEFAULT 

should have no effect on the triple-store and should succeed. AllegroGraph implemented the behavior correctly but was returning failure instead of success. This has been corrected.

Bug21239 - Concurrent queries in shared backends

With the sparql-1.1 engine, queries running in the same shared backend concurrently had unexpected interactions that lead to various errors. This has been fixed.

Bug21237 - Computations in aggregate expressions could fail

If a complex expression was used inside of a SPARQL aggregation function, the value could be left unbound. A work-around was to express

(min(2 * ?x) as ?smallestDoubled) 

as

(min(?x) as ?smallest) (2*?x as ?smallestDoubled) 

This is now corrected and the work-around is no longer necessary.

Bug21233 - SPIN functions could fail when used with constants

SPARQL queries that used SPIN functions could fail if the arguments to the functions were constants not already in a triple-store's string-table. This has been corrected.

Bug21222 - Failing SPARQL UPDATE HTTP request should error

Previously SPARQL UPDATE requests that failed would return false in SPARQL/XML format. In order to be better conform with other SPARQL endpoints, AllegroGraph now returns a 412 status code (precondition failed) instead.

Bug21163 - Add new default-dataset-behavior: rdf

SPARQL leaves things up to the implementation when no explicit dataset is specified (i.e., when no FROM or FROM NAMED clauses are part of the query). AllegroGraph now provides three choices for dataset behavior:

AllegroGraph's standard behavior continues to be :all. You can change this on a per-query basis using the defaultDatasetBehavior query option:

PREFIX franzOption_defaultDatasetBehavior: <franz:rdf>  

AGWebView

Rfe11910 - Add HTTP handlers for materialization

Added the materializeEntailed service.

A PUT request will materialize entailed triples based on the with, without, useTypeSubproperty, and commit query parameters.

A DELETE request will remove the materialized triples.

Rfe11856 - WebView - new UI Beta for Graph

In WebView next to the Documentation menu is a link to "WebView Beta" a new experimental UI for Query and Graph. A link at the top will returns to the classic WebView.

A link at the top will open the new Graph View UI.

Also Node queries have a View menu in the results to open the node in Graph view.

Administration features of WebView are not yet included within the new UI.

Rfe11676 - Webview: in Scripts editor, use CodeMirror JavaScript

The WebView Scripts editor now supports syntax highlighting and indentation by including the CodeMirror JavaScript mode.

rfe11596 - WebView: new UI Beta for Query

In WebView next to the Documentation menu is a link to WebView Beta a new experimental User Interface (UI) for Query. The new UI provides a modern style and easier navigation. A link at the top will returns to the classic WebView.

All existing features of WebView Query are implemented:

Administration features of WebView are not yet included within the new UI.

Rfe11582 - Webview: upgrade Query Editor to CodeMirror v2.32

The query editor for SPARQL and Prolog, implemented by CodeMirror, has been upgraded from v2.24 to v2.32.

Bug21250 - Webview: remove limit from download

The download link for Query results no longer uses the limit parameter in the query UI.

Changes to the Lisp API

Rfe11454 - get-triples-list (Lisp API) returns cursor only on request

The Lisp get-triples-list function previously returned the cursor it created as a second value. Now, however, it returns a cursor if either one was passed in initially (using the cursor keyword argument) or if the new keyword argument return-cursor is true.

Bug21381 - (Lisp API) SPARQL introspection functions required engine argument

The functions get-allowed-results-formats and get-allowed-rdf-formats returned nil if no query engine was provided. This has been corrected.

Bug21378 - (Lisp API) sparql-1.1 engine does not implement destination-db

The Lisp API allows SPARQL Update commands to specify the triple-store that they modify. This feature was not included in the sparql-1.1 engine. This has been corrected.

Documentation

No significant changes.

Python Client

Rfe11911 - Add materializer calls to python client

Added materializeEntailed and deleteMaterialized calls to RepositoryConnection.

Java Client

Rfe11906 - Java support for the materializer

With this change, the Java client now supports materialization, allowing inferences to be generated and added to a repository in a transactional fashion. See the Javadoc for AGMaterializer#newInstance() to acquire a configurable materializer, and AGRepositoryConnection#materialize to run the materializer.

Rfe11771 - Upgrade embedded JavaScript implementation

Fixes several small bugs in the way uploaded JavaScript scripts are interpreted. The most visible one was that the !== operator was acting like the < operator.

Rfe11241 - Support more query/update protocol params

With this change, the Java client now sends HTTP protocol params for update, baseURI, timeout, remove-graph-uri, insert-graph-uri, using-graph-uri, using-named-graph-uri, and logQuery.

AllegroGraph 4.8 user-visible changes

AllegroGraph Server: General Changes

Rfe11711 - Improve cursor resource usage of Geospatial Prolog queries

AllegroGraph previously closed cursors created while running a Prolog geospatial query lazily. It now closes them when the query completes which improves overall resource utilization.

Rfe11653 - Rm requests can be processed out of order

The order in which file management requests were processed caused unexpected errors under heavy load. The bug is now fixed.

Rfe11649 - Manage memory for ORDER BY and DISTINCT queries

Previously AllegroGraph needed to accumulate all results in memory for queries that used DISTINCT or ORDER BY. This could greatly increase the overall memory footprint. AllegroGraph now handles large result sets in available memory.

Rfe11631 - Improve query optimizer when inference is enabled

One part of query optimization is reordering the triple-patterns in each BGP. This reordering is based on the statistics collected by the store and these statistics do not take inference into account. This meant that the plan produced could be quite bad.

This has been corrected by modifying the estimation code to take RDFS++ into account.

Rfe11416 - Provide greater control of query memory usage

Added a memoryLimit query option that controls how much memory a query can use. If a query needs to allocate more memory than this value, it will fail.

Bug21235 - Resource manager sync-files effectively ignores msync errors

If the filesystem on which a database resided filled up during a checkpoint, this fact was logged but the checkpoint process would continue running as if the error didn't happen, which would result in database corruption. This bug has been fixed. Now when a full filesystem is detected during a checkpoint, AllegroGraph stops operating in order to protect committed data.

Bug21224 - Crashes while processing queries with DISTINCT

Under rare circumstances, the internal data structures used while processing DISTINCT queries could get corrupted leading to segmentation faults. This bug has been fixed.

Bug21214 - Correct scope of BINDings introduced on the LHS of union

The binding of a variable on the left-hand side of a UNION should not influence its value on the right-hand side. For example, the following query should produce two rows with ?a bound to 1 in one of them and left unbound in the other.

select ?a {  
  { bind(1 as ?a) } union { bind(2 as ?b) }  
} 

AllegroGraph was retaining the binding on the LHS across to the RHS of the union and so was answering with two rows each of which had ?a bound to 1. This has been corrected.

Bug21187 - Stack exhaustion with shared cursor

With lots of deleted triples a query could run out of stack when creating cursors. This has been fixed.

Bug21182 - Unbound values in distinct aggregrates could lose results

If a variable binding used in a distinct aggregate was unbound, it could cause other aggregrates in the same row to fail to update properly. For example, a query like:

SELECT (count( distinct ?taste) as ?tasteCount ) (count(?who) as ?whoCount) {  
	   ?who <http://people.example/age> ?age .  
	   OPTIONAL { ?who <http://people.example/taste> ?taste . }  
} 

could lose data for the ?whoCount bindings if the ?taste binding from the OPTIONAL was unbound. This has been corrected.

Bug21145 - agload: message corrected for insufficient buffer size

The error message for insufficient ntriples/nquads buffer size was reporting buffer as size -1. This message has been corrected and now reports actual buffer size in order to abate confusion about the condition.

Note: This condition is usually triggered when an ntriple of greater than 4MB is encountered while using the default dispatch strategy.

Bug21136 - Free BGP patterns can lead to very large memory allocation requests

Queries with free patterns (like ?s ?p ?o) in a BGP on the right-hand side of a query could lead the query size estimator to believe that it should reserve a very large space to save the putative result set. For example, a query like

SELECT ?s WHERE {  
    { ?s ?p ?o }  
    MINUS  
    { ?other ?p ?o . ?other ?someProp ?s }  
} 

would lead the query engine to try and allocate 1024-Petabytes. This has been corrected.

Bug21124 - Zero length path between non-matching constants

AllegroGraph was incorrectly finding a path between two different constants which led to queries like the following to return a single (empty) result when they should have returned no results at all.

select * {  
  ex:one ex:connectsTo{0} ex:two .  
} 

This has been corrected. Other zero-length path queries were behaving correctly.

Bug20853 - Updated the soundex algorithm

AllegroGraph's freetext indexing word filter was using an older variant of the Soundex algorithm that did not match the latest version (cf. http://www.archives.gov/research/census/soundex.html). The algorithm has now been updated.

Freetext indexes created with the soundex word-filter should be dropped and re-built.

HTTP Client

Rfe11681 - HTTP: Allow statements queries using the POST method

Add a [store]/statements/query URL that corresponds to /statements, except that it isn't limited to the GET method, and thus can be used to do complex queries without building up overlong URLs.

Rfe11680 - HTTP: Add /unique/[column] service

Add a [store]/unique/[column] URL that allows clients to fetch the set of unique parts that occur in a column in the database, optionally restricting other columns.

SPARQL

Rfe11682 - Improve performance of SPARQL OPTIONAL queries

If a query has the form A OPTIONAL B (where A and B share variables so that it is a join) and the A part has no rows, then it is not necessary to evaluate the B part at all.

The SPARQL 1.1 query engine now recognizes this situation and skips the evaluation of the right-hand side.

Rfe11591 - Modified the Lisp run-sparql API to better handle query failure (e.g., for when a query runs out of time or other system resources).

Previously run-sparql would return (as multiple values):

Now run-sparql returns four values. In normal execution, the first three values are the same and the fourth value will be nil. When a query fails, the results will be:

Code that requests a cursor should be prepared to handle a condition of type db.agraph.query.base:cancel-query-condition.

Rfe11581 - handle repeated Prologues in SPARQL UPDATE queries

SPARQL 1.1 UPDATE allows the Prologue (BASE and PREFIX definitions) to be repeated between each UPDATE command. Previously, AllegroGraph only allowed BASE and PREFIX to be used at the beginning of a SPARQL statement. This is now corrected.

Rfe11559 - Add logQuery parameter to the HTTP SPARQL protocol

SPARQL queries can now use the logQuery parameter in their HTTP requests. The parameter can be set to one of:

The log output can be cryptic; please contact Franz Inc. for support in interpreting the messages.

Bug21220: SPARQL Update LOAD should require only write permission (not eval)

A user would need to have evaluate permission on a triple-store in order to use SPARQL Update's LOAD command to load data from a file. This has been changed so that having write permission is sufficient.

Bug21203 - SPARQL Update insert data fails to treat blank nodes correctly

The INSERT DATA and MODIFY SPARQL Update commands failed to use the small blank node identifier in the INSERT template. This caused multiple blank nodes to be created when there should have been only one.

Bug21192 - Upi not in string table error when mixing BIND, SPARQL Update and FTI

SPARQL Update queries that produced new strings (e.g., using BIND) on a triple-store with free-text indexing turned would lead to UPI not in string table errors. Note that in this case a triple would have been added to the store but its strings would not be added to the string table which could cause subsequent queries to fail until the errant triple was removed.

This has been corrected.

Bug21168 - Support SPARQL 1.1 COUNT(DISTINCT *)

The SPARQL 1.1 query engine was not correctly handling the COUNT(DISTINCT *) form. This has been corrected.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

Rfe11697 - Added RepositoryConnection.prepareUpdate

Added RepositoryConnection.prepareUpdate for SPARQL 1.1 Update requests. The call to evaluate on the returned UpdateQuery object returns True or False.

Java Client

Rfe11748 - Upgrade to sesame 2.6.8

With this change, the Java client has been upgraded to use openrdf sesame 2.6.8 jars.

Rfe11243 - Support X.509 authentication

With this change, the Java client now supports X.509 server and client authentication. For set up details, see the javadoc for AGServer.

Bug21225 - With no requestEntity, post using setRequestBody

Previously, AGHTTPClient post method used setQueryString to pass params when there was no requestEntity provided. This burdened the server when a param became very large. Now params are passed using setRequestBody when no requestEntity has been provided.

AllegroGraph 4.7 user-visible changes

AllegroGraph Server: General Changes

Rfe11587 - Change InstanceTimeout defaults

The configure-agraph script now supports instance timeouts in interactive mode and via the --instance-timeout command line option. It defaults to one week.

Also, the default value the InstanceTimeout configuration directive (effective when there is no global or catalog level InstanceTimeout) has been changed from 0 (= disabled) to one hour.

Rfe11536 - Transitive inference queries could be slow

The RDFS++ reasoning for transitive queries could be slower than necessary for some datasets. The code has been reimplemented to be significantly faster.

Rfe11285 - Improve query constraint analysis

AllegroGraph now does a better job of applying filter constraints throughout a query. For example, previous versions of AllegroGraph would not have transformed either of the two BGPs (on either side of UNION) into range queries (which are more efficient).

SELECT * {  
  ?a ex:d ?c .  
  { ?a ex:b ?c . } UNION { ?a ex:c ?c . }  
  FILTER (?c > 2)  
} 

Now it will.

Bug21114 - Instances lingering too long

Instances with non-zero InstanceTimeout only timed out when there was activity involving the service daemon. This has been corrected. Instances with an InstanceTimeout of zero (the default) are not affected.

Bug21094 - Queries against federations could signal UPI-not-in-string-table errors

It was possible for queries against federated triple-stores to lead to UPI-not-in-string-table errors. This has been corrected.

Bug21089 - agload division by 0

Progress reporting failed with division by 0 if the operation completed within 1ms.

Bug21077 - Expressions can be analyzed too eagerly

It was possible for constant expressions to be analyzed to their final values too quickly which could lead to an error if the expressions were used in a BIND. For example, this query would fail:

select * { bind(str('hello') as ?x } 

rather than producing a single binding of ?x to the string hello. This has been corrected.

Bug21069 - A warm-standby client now exits non-zero if error occurs during commit replay

Replication secondary (warm-standby client) now exits non-zero if error occurs during commit replay. This is to ensure transactional behavior in the replica.

Bug21059 - Improve efficiency of all predicates cache when RDFS++ is enabled

The query engine keeps track of the predicates in the triple-store to optimize queries which contain clauses where the predicate position is a variable. If RDFS++ was enabled and the cache was not already fresh, then AllegroGraph would try to find all of the store's predicates with reasoning enabled. Since new predicates cannot be inferred, this could be hugely inefficient. This has been corrected.

Bug21057 - Fix log rotation causing crashes

When the SIGHUP handler closes a log stream, attempts to log could fail with a 'stream closed' error potentially crashing the server. This bug has been fixed.

Bug21040 - Turtle data import should use UTF-8 encoding

The Turtle specification mandates that Turtle files are encoded using UTF-8. AllegroGraph was not enforcing this in all cases which could mangle Unicode data on import. This has been corrected.

Bug21022 - Group expressions with unbound elements could be dropped

AllegroGraph could lose rows in a aggregation query if the GROUP BY expression resulted in unbound columns. This has been corrected.

Bug21008 - Correct XSD datatype for certain division operations

When performing division of two xsd:integers or two xsd:decimals, AllegroGraph was emitting the result as an xsd:float rather than an xsd:decimal. This has been corrected.

Bug21007 - Error creating a session with non-default graph for inferred triples

The syntax for starting a reasoning session that uses a different graph for its inferred triples is

<store>[rdfs++#<graph>] 

AllegroGraph was failing to parse this specification correctly and signaled an error instead. This has now been corrected.

HTTP Client

Rfe11601 - HTTP and WebView: MongoDB interface config

Added resources to HTTP protocol for MongoDB parameters:

GET /repositories/[name]/mongoParameters  
POST /repositories/[name]/mongoParameters 

See Mongo Interface, The HTTP Protocol, GET mongoParameters, and POST mongoParameters.

SPARQL

Rfe11558 - Add query logging support

To add in query debugging and tuning, SPARQL queries using the 1.1 query engine can now be logged to the AllegroGraph log file. To log a query, use the prefix option notation by pre-pending this prefix to your query:

prefix franzOption_logQuery: <franz:yes> 

The log output will go directly to the AllegroGraph log. In this release, the output is relatively cryptic. Please consult with Franz (support@franz.com) to interpret the data.

Rfe11492 - Add support for SPARQL CSV and TSV

Added CSV (comma separated values) and TSV (tab separated values) as native SPARQL output formats. Also corrected problems with using CSV output on result sets with unbound values.

Rfe11486, Rfe11538 - improve SPARQL 1.1 engine geospatial integration

The SPARQL 1.1 query engine did not implement the POINT extension for SPARQL FILTER functions so that queries like:

SELECT * {  
   ...  
} ORDER BY geo:haversine-km(?loc,POINT(105.842328, 21.0184040)) 

would fail. This has been corrected. Additionally, the geospatial query analyzer has been improved so that more computations are carried out at query analysis time rather than at query execution time. This can speed up some geospatial queries tremendously.

Bug21117 - Some SPARQL 1.1 path queries could parse incorrectly

The SPARQL 1.1 parser could combine some combinations of alternation and sequences incorrectly. For example, this query

SELECT ?xxx ?yyy {  
    ?xxx (dc:aaa|dc:bbb/dc:ccc) ?yyy .  
} 

would have been parsed as if it read (dc:aaa|dc:bbb|dc:ccc). This has been corrected.

Bug21093 - SPARQL UPDATEs could fail to save newly created strings

If run with chunkProcessingAllowed, a SPARQL UPDATE query that generated new strings could fail to save some of these new strings in the underlying triple-store. This has been corrected.

Bug21082 - Optional cross-product with no bindings could fail

If the final step of a query was a LEFT-JOIN and that LEFT-JOIN was also a cross-product (i.e., the right hand side shared no variables with the left hand side) and the right hand side produced no bindings of its own (e.g., it was all constant or its variables could be elided because they were not used anywhere else in the query), then it was possible for AllegroGraph to incorrectly remove answers. This has been corrected.

Bug21078, bug21083 - Handle sub-query SELECT expresions and filters correctly

SELECT expressions in sub-queries were not being handled correctly by AllegroGraph. This has been corrected so that queries like

SELECT ?a (?a * 2 as ?b) ?c ?d {  
  BIND(1 as ?a)  
  { SELECT ?c (2* ?c as ?d) {  
    BIND(20 as ?c)  
  } }  
} 

now perform properly.

It was also possible for AllegroGraph to incorrectly optimize FILTER expressions in sub-queries that returned all variables (*). This too has been corrected.

Bug21072 - unused variable check could remove variables in EXISTS filters

The SPARQL 1.1 query planner removes variables that are not used in the query from the set of bindings maintained during query execution. Unfortunately, the check to see whether or not a variable was used could fail to properly descend into EXISTS (and NOT EXISTS) filters. For example, the following query would have failed:

PREFIX :    <http://example/>  
PREFIX  rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  
SELECT DISTINCT (?s1 AS ?subset) (?s2 AS ?superset)  
WHERE  
{  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        ?s1 :member ?x .  
        FILTER ( ?s1 = ?s2 || NOT EXISTS { ?s2 :member ?x . } )  
} 

This has been corrected.

Bug21064 - Unneeded graph variables with DISTINCT query could error

If the variable in a GRAPH clause was not needed by the query (because, e.g., the query asks for DISTINCT results and the graph variable is not used anywhere else in the query), then the query planner will remove the variable from the plan. Unfortunately, the query executor could still try to assign a value to the graph variable which would lead to an error at query execution time. This has been corrected.

Bug21046 - Multiple MINUS clauses could lead to query execution error

A SPARQL query with multiple MINUS clauses could cause the SPARQL 1.1 query engine to incorrectly determine the variables on which to join the clauses. For example, this query would fail:

SELECT (?s1 AS ?subset) (?s2 AS ?superset) {  
    # All pairs of sets  
    ?s2 rdf:type :Set .  
    ?s1 rdf:type :Set .  
 
    MINUS {  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        # Assumes ?s1 has at least one member  
        ?s1 :member ?x .  
        # If we want to exclude A as a subset of A.  
        # This is not perfect as "?s1 = ?s2" is not a  
        # contents based comparison.  
        FILTER ( ?s1 = ?s2 || NOT EXISTS { ?s2 :member ?x . } )  
    }  
    MINUS {  
        # If we don't want the empty set being a subset of itself.  
        ?s1 rdf:type :Set .  
        ?s2 rdf:type :Set .  
        # Choose the pair (empty set, empty set)  
        FILTER ( NOT EXISTS { ?s1 :member ?y . } )  
        FILTER ( NOT EXISTS { ?s2 :member ?y . } )  
    }  
} 

This has been corrected.

Bug21037 - Construct queries with Turtle results format can cause string lookup error

If a SPARQL CONSTRUCT query used expressions to create strings that were not already in the underlying triple-store and if the RDF format requested with Turtle, then a UPI not in string table error could be generated. This has been corrected.

Note that the other RDF formats do not exhibit this behavior.

Bug21031 - Remove unstable optimization

The SPARQL 1.1 query engine attempted to reduce the amount of work needed to evaluate certain joins. Unfortunately, it was possible to create queries that exposed a conflict between this optimization and other optimizations which would lead to an error at query evaluation time. This optimization has been temporarily removed. It will be added again once the conflicts it exposed have been resolved.

Bug21023 - Correct datatype of the seconds() function

The SPARQL 1.1 seconds() function should return a value of type xsd:decimal but AllegroGraph was returning xsd:float. This has been corrected.

The date and time functions (like month() and seconds()) were also not behaving correctly on literals of type xsd:time. This has been corrected.

Bug21019 - Filter EXISTS with fixed triple patterns could fail

If a SPARQL query used FILTER (NOT) EXISTS and the EXISTS clause contained fixed triple-patterns, then AllegroGraph could signal an error. For example, this query would fail:

SELECT * {  
  ?a a ex:Fruit .  
  FILTER EXISTS { ex:apple a ex:Plant . }  
} 

This has been corrected.

Bug21016 - UPDATE with blank nodes could lead to error

A SPARQL UPDATE query that included blank nodes in the INSERT template could lead to an error. This has been corrected.

Bug20994 - Correct (in)equality test between plain and typed literals

Some comparisons between plain literals and typed literals could return false rather than signaling a SPARQL type error. This meant that a FILTER that tested for (in)equality could succeed where it should have failed. This has been corrected.

Bug20840 - Improve comparisons between xsd:dates and xsd:dateTimes

The SPARQL operator mapping table is quite strict in the sort of comparisons that are allowed. AllegroGraph was allowing inequality comparisons between xsd:dates and xsd:dateTimes even though these datatypes are not strictly comparable. These sort of comparisons will now signal a type error leading FILTER conditions to fail. I.e., a query like

... FILTER ( ?aDate > ?aDateTime ) ... 

will now fail (correctly) even though it may have resulted in results previously.

Bug20736 - Federated-triple-stores can fail to find equivalent literals

In SPARQL filters, typed-literal equality is based on the value of the literal, not its lexical form. I.e., these literals are all the same:

To reduce the amount of work during query execution, the SPARQL engine checks to see whether any triples exist with literals of these (encoded) types. This check was being handled incorrectly for federated triple-stores which could lead to answers being left out of federated result sets. This has now been corrected.

Bug20770, Bug21020 - improve SPARQL 1.1 aggregation support

SPARQL 1.1. allows assignment in a GROUP BY clause. For example:

... GROUP BY (str(?name) as  ?theName) ... 

AllegroGraph did not allow for this syntax and required the work-around of:

... BIND(str(?name) as ?theName) ... } GROUP BY ?theName ... 

This omission has been filled.

Secondly, AllegroGraph did not allow aggregate expressions in the HAVING clause or SELECT expression involving more than one aggregate. For example, these queries would fail:

SELECT (avg(?value) as ?avg) ((min(?value) + max(?value)) / 2 as ?half {  
  ...  
} 

and

SELECT * {  
  ...  
}  
group by ?value  
having(sum(?value) > 20) 

Both of these omissions have also been corrected.

Bug20165 - Unnecessary variable in a BIND clause could cause error

If the variable in a BIND clause was not needed by the query (because, e.g., the query asks for DISTINCT results and the variable is not used anywhere else), then the query planner will remove the variable from the plan. Unfortunately, the query executor could still try to assign a value to the variable which would lead to an error at query execution time. This has been corrected.

AGWebView

Rfe11601 - MongoDB interface config

Added support in WebView for setting MongoDB parameters, in the Repository Overview page. See Mongo Interface.

Rfe11557 - WebView: serve doc files

AllegroGraph documentation, a collection of HTML files installed with the server, is now served by WebView, and the Documentation menu in WebView links to these local files in addition to those on www.franz.com.

Rfe11474 - Execute query without changing text

In the Query editor, a query may now be executed without editing the text.

Rfe11398 - Update query editor to CodeMirror v2.24

The query editor for SPARQL and Prolog, implemented by CodeMirror, has been upgraded from v2.15 to v2.24, including javascript and css.

Bug21085 - Webview: Recent queries should not add items redundantly

WebView Recent Queries now keeps the most recently used queries. If a previous query (including the same planner and limit) is repeated, that query is moved to the top of Recent Queries, with the number remaining the same. Limit and planner are displayed on the Recent Queries page.

Bug21062 - Recent queries in multiple browser tabs

Using WebView in multiple tabs of a browser caused a javascript error leading to a blank screen below the menu. This has been fixed by consistently using localStorage.

Note for using multiple browser tabs/windows: the last tab to execute query will overwrite localStorage of "recent queries" that will be seen if a new browser tab is opened.

Bug21050 - WebView: CodeMirror Query editor steals arrow keys

When the cursor is in Query Editor, Alt-LeftArrow and Alt-RightArrow will now perform the default browser behavior instead of navigating within the editor page.

Bug21036 - Editor empty after 40 recent queries

WebView holds up to 40 recent queries, but incorrectly restored the text when more than 40 were made. This is fixed.

Note that logout and login will clear the list of recent queries.

Bug20998 - Webview: failures in IE7

Many links to navigate between WebView pages did not work in Internet Explorer 7 (IE7) due to usage of browser history. In the Query editor page, limit and planner options did not take effect in IE7. These problems have been fixed.

Bug20982 - Query limit and planner selections ignored

In the Query editor screen, the limit and planner selections were ignored. Now, they are both used for Execute and Download. Also, they will remain set after Execute, and following a link from the Recent Queries page will remember these settings for that query (but they are not stored for a Saved Query).

Changes to the Lisp API

Bug21042, Bug21043 - standardize remote-triple-store SPARQL cursors

The cursors returned by run-sparql for remote-triple-stores behaved differently from ones returned for other triple-store classes. In particular, SELECT queries could not request a results-format of :cursor and the rows returned by cursors were lists of bindings rather than arrays of bindings.

The cursors for remote-triple-stores are now standardized to behave in the same way as cursors in the rest of AllegroGraph. This item is repeated above under AllegroGraph Server: General Changes, where it is also pointed out this applies to Lisp clients only.

Bug20067 - Make session ping threads more resilient

Remote clients ping their sessions to keep them from timing out on the server. Intermittent network errors could cause the ping thread to fail. This fix makes pinging attempts continue until the session timeout passes without successful pings.

Documentation

Bug21080 - Doc layout bad in IE intranet-mode

AllegroGraph docs now use html5 doctype and style. For Internet Explorer (IE), the highest mode available will be used because IE7 layout is not optimal.

Python Client

Rfe11244: Python Support for client X.509 certificate authentication

To support X.509 certificate authentication with SSL, AllegroGraphServer objects now have cainfo and sslcert keyword arguments for indicating the client security information. verifyhost and verifypeer can also be used.

These arguments correspond to the pycurl.CAINFO, pycurl.SSLCERT, pycurl.SSLVERIFYHOST, and pycurl.SSLVERIFYPEER settings on the Curl object if you want to read about them in the pycurl documentation for more information.

user and password keyword arguments can be omitted when using certificate authentication.

See src2/franz/openrdf/tests/tests.py's test_ssl for example use.

Java Client

Rfe11523 - Upgrade to Sesame 2.6.5 jars

With this change, the Java client has been updated to use the Sesame 2.6.5 jars.

Rfe11358 - Upgrade Clojure to 1.4.0

The Clojure client API now uses Clojure 1.4.0 and Leiningen 1.7.1.

Bug21099: java pool holds sockets open

Problem: the AGConnPool used one MultiThreadedHttpConnectionManager per AGRepositoryConnection, but only closed it with the pool rather than with the repo. This used multiple connections to the main AGraph port and left http socket connections open for them and a session port after the AGRepositoryConnection was closed. Session ports are not recycled by the AGraph server if SessionPorts is not specified.

Fix: use a SimpleHttpConnectionManager for each AGRepositoryConnection and close both at the same time. This minimizes socket connections and avoids the secondary pool within MultiThreadedHttpConnectionManager.

Bug21092 - Clojure: errors in tutorial

Enhanced repo-federation to work without rcon-args as was used in the tutorial. This fixes tutorial example16.

Enhanced value-factory to work with either Repository or RepositoryConnection. This reduces reflection warnings in the tutorial.

AllegroGraph 4.6 user-visible changes

AllegroGraph Server: General Changes

Rfe11449 - Timeout for database instances

Support has been added for making database instances linger for some amount of time after their last close. This allows for reducing the likelihood that a subsequent access to an otherwise idle database will require a long startup time. The time to linger is determined by the global and per catalog InstanceTimeout directive whose default is 0s.

InstanceTimeout is from the instant the last session or backend that has the store open closes it.

New function ensure-not-lingering of the same argument as open-triple-store.

close-triple-store has a new keyword arg :ensure-not-lingering.

Rfe11445 - Queries with large intermediate result sets could generate segfaults

Queries with very large intermediate result sets could generate segmentation faults. This has been corrected.

Rfe11440 - Remove trailing slash from server URL

With this change, AGServer now removes any trailing slash from the specified server URL. Previously, an errant trailing slash on the server URL resulted in an obscure error message about "No suitable response format available" that was difficult to diagnose.

Rfe11418 - improve efficiency of AllegroGraph's turtle parser

Improved the memory usage and speed of AllegroGraph's turtle parser making it more than twice as fast as previous versions.

Rfe11404 - Optimized opening stores

By reducing inter-process communication latency, opening triple stores has been made significantly faster.

Rfe11177 - Agload --relax-syntax flag

agload now uses --relax-syntax flag instead of --relax-for-btc flag. This flag now allows for underscores and dashes to be used in anonymous node strings for N-Triples and N-Quads. agload is documented in Data Loading.

Rfe11126 - SSL client certificate and CRL support

New https-verification keyword argument to open-triple-store and create-triple-store.

Rfe8881 - Optionally suppress duplicates at commit time

AllegroGraph now has a switch duplicate-suppression-strategy, which will prevent new duplicate triples (both duplicates already in the store, and duplicates within the same commit) from being committed to the store.

Bug20997 - Solve latency issue while starting replication.

Fixed a bug where transaction logs (tlogs) were repeatedly (and unnecessarily) examined when performing replication. Now each tlog is examined once. This results in the time required to be O(n) instead of O(n*n) (that is, time is linear with the number of tlogs rather than increasing by the square of the number of tlogs).

Bug20984: Turtle serializer emitted invalid blank node identifiers

The rdf-n3 (turtle) serializer was emitting blank node identifiers without prefixing them with ":_". This has been corrected.

Bug20981 - Exhausted cursor recycling happens too early

Exhausted cursors were being recycled too early which could sometimes result in program crashes. This has been fixed.

Bug20968 - Improve string dictionary cache performance

When heavily used, AllegroGraph's string dictionary cache could become bogged down in bookkeeping resulting in very poor performance. This change prevents this excessive bookkeeping and provides the expected near-constant cache performance.

Bug20943 - Free text index bug under replication

When a new free-text index was created while replication was enabled, it caused an error in the standby process. This is now fixed.

Bug20913: Xsd date and time coercion functions could signal error

If given invalid input, the type coercion functions for dates, times and dateTimes could signal an error other than a SPARQL type error. This would lead to a query error rather than a filter failure. For example, this query:

prefix xsd: <http://www.w3.org/2001/XMLSchema#>  
select ?ds ?d {  
  bind( '2012-12-04 is my favorite day' as ?ds )  
  bind( xsd:date(?ds) as ?d )  
} 

would signal an error rather than returning a single result for which the binding for ?d is empty.

This has been corrected.

Bug20910 - Queries with variable predicates could lose results

If AllegroGraph finds a query pattern with variable predicates and distinct results (and certain other conditions that are data dependent are also true), then it will optimize the pattern by converting it into an iteration over all of the distinct predicates in the store. A typical query of this form is

select distinct ?p {  
  ?s a ex:Animal .  
  ?s ?p ?o  
} 

Because computing all of the distinct predicates is expensive, AllegroGraph caches this information. Previously, AllegroGraph could fail to update the cache correctly when new triples were added. This could lead to queries returning too few results.

This has been corrected.

Bug20896 - Hang when max chunks for an index reached

As part of their operation, the automatic index optimization processes will sometimes merge some number of input chunks into a single output chunk in order to keep the total chunk count below a certain threshold. However, under some circumstances, some indexes would never undergo the optimization process so this chunk reduction would not occur. Eventually the chunk limit would be reached and the database would hang. This problem has been corrected.

Bug20887 - AllegroGraph XML parsers should not follow external references

AllegroGraph's RDF/XML and TriX parsers could try to follow external references in input files and fail. This has been corrected.

Bug20877 - xs:date could fail on some short plain literals

The xs:date constructor function could fail to correctly create a date if the argument passed to it was a small plain literal. E.g., constructs like

BIND (xsd:date( concat(str(?date), '-01-01')) as ?begin_date) 

could pass a string whose length was less than 10 to the xs:date function and this could lead to the failure.

This has been corrected.

Bug20833 - openFederated should handle remote stores

AllegroGraphServer.openFederated now works on remote stores. Previously it always attempted to open supplied arguments as local stores.

The openFederated API still has some limitations. To open a federated session with all federation minilanguage features such as reasoning and graph filtering, pass a properly formatted spec string to AllegroGraphServer.openSession directly.

Bug20822 - Cascading error can occur if database create/open fails

Previously, if there was an error during database open/create, another error could occur during the cleanup process, obscuring the actual problem. This has been fixed.

Bug20819 - problem with some combinations of multiple OPTIONAL clauses

Some query patterns with OPTIONALs whose left-hand side was empty could lead to incorrect results. An example of a query that could have failed was:

select distinct ?value ?type ?name {  
  OPTIONAL {  
    ?value <http://example.com#isa> ?type .  
  }  
  OPTIONAL {  
    ?value <http://example.com#name> ?name .  
  }  
  FILTER( REGEX(STR(?name),'Sa') && BOUND(?type) )  
} 

This has been corrected.

Bug20814 - Make session urls unique

In order to avoid accidental sharing of sessions, include a unique session id in the session-url. Behind the scenes, what previously looked like http://localhost:55555 is now something like: http://localhost:55555/sessions/7e8df8cd-26b8-26e4-4e83-0015588336ea.

Bug20806 - fix loading of initfile and sitescripts

Initfiles and sitescripts longer than 4096 characters were presented with 4096 character long blocks in reversed order. This bug is now fixed.

Bug20799 - turtle-parser was case sensitive when parsing numbers

AllegroGraph's turtle parser was failing to parse floating point numbers that used an upper-case E before the exponent. This is now corrected.

Bug20793 - RDFS++ queries with free predicate could infer invalid triples

If a query involving RDFS++ reasoning used an unbound predicate, the reasoner could use inverse property reasoning to infer invalid triples. This has been corrected.

Bug20757 - Problems when starting agraph while another agraph is running

Previously if you tried to start AllegroGraph when it was already running, the second start would fail but it would cause the already-running AllegroGraph to stop operating correctly. This has been fixed.

It has always been acceptable to run two or more AllegroGraph servers simultaneously as long as they, at the very least, used different settings for the following parameters in their configuration files:

The operating system already enforced a unique Port setting. This change adds enforcement of unique SettingsDirectory and GlobalShmServerDirectory settings. Therefore some (dangerous) side-by-side AllegroGraph configurations that used to work may no longer start up. An instructive error message will be displayed in this case.

Bug20628 - Changes to warmstandby API

PUT /repositories/[name]/warmstandby

If one attempts to start a replication task that already exists, an http 409 Conflict response will be returned.

Bug20533 - Agload --rapper flag documentation

The --rapper flag is now specified in the group "Other options". Previously it was not given a group in the command line help. agload is documented in Data Loading.

Bug20508 - Remote triple store can't use the RDFS++ reasoner

Added support for apply-rdfs++-reasoner on a remote client triple-store.

HTTP Client

Rfe10812 - Improve http tracing

HttpTrace is now a global configuration directive of the same meaning as the --http-trace command line option. The global directive HttpTraceOptions and the command line option --http-trace-options were added, the value of which is a comma separated list of options. Options starting with the character + turn on the corresponding log category. Those starting with - turn them off. Use the max-message-size=<N> option to truncate overly long messages. The default is +log,max-message-size=1000.

Bug20798 - Improved http error handling

Previously, if the client that was performing an http request died and the backend couldn't transmit the response, then the backend was killed and the request was retried on another backend that also failed because the client socket was closed which wasted resources. The new behavior is to let the backend live on (without retrying), if the error is really from the client side.

SPARQL

Rfe11469: add line number to SPARQL parser error messages

The SPARQL parser will now indicate the line at which an error is encountered. The error message has also been reformatted to make it easier to understand the parsing problem.

Rfe11357 - enhance SPARQL 1.1 engine for changes to standard

The SPARQL 1.1 standards committee recently made several small changes to the standard. To accommodate these, AllegroGraph's SPARQL engine has:

There are several other changes to the standard which AllegroGraph will support in a future release. These include:

For additional information, please see the SPARQL release notes.

Bug20951 - SPARQL could fail to match plain and typed strings

SPARQL specifies that a plain literal (like "hello") is equivalent to a literal with type xsd:string (like "hello"^^xsd:string). The SPARQL 1.1 query engine was failing to provide this entailment in some cases. This has been corrected.

Bug20939 - CLEAR only recognized if it's the first verb

The SPARQL 1.1 UPDATE verb CLEAR was only recognized correctly if it was the first verb in a SPARQL statement. This has been corrected.

Bug20936 - OFFSET 0 and LIMIT did not mix

If a SPARQL query used an explicit OFFSET of zero, the LIMIT would be ignored. I.e., a query like:

SELECT * { ?a a ex:Animal . }  
LIMIT 10 OFFSET 0 

would return all ex:Animals rather than just 10. Leaving the OFFSET off entirely would produce the correct behavior. This has been corrected so that OFFSET 0 is treated as expected.

Bug20912 - Inequality filters could signal an error

Inequality filters in SPARQL queries can be transformed into fast range queries. If a query used a filter like

FILTER( ?a < 0.0 ) 

then the SPARQL 1.1 query planner could attempt to decrement an unsigned number below zero which would lead to a segmentation fault.

This has been corrected.

Bug20911 - SPARQL could lose precision on very small numbers

SPARQL numeric conversion routines could treat double-floats and decimals as single-floats which would lose precision. For example, a query with a filter like

FILTER( ?x > 4.5e-45 ) 

would have been interpreted as

FILTER( ?x > 0 ) 

This has been corrected.

Bug20898 - Complex queries could fail on federated triple-stores

Because of the way statistics were calculated on federated triple-stores, SPARQL queries with many clauses in a basic graph pattern (BGP) could fail when the query engine attempted to reorder them. This has been corrected.

Bug20891 - Improve SPARQL comparator selection

SPARQL specifies difference comparison operations depending on the type of the things being compared. The SPARQL 1.1 query engine could select the wrong operation if both the left and right hand sides were variables. This has been corrected.

Bug20884: improve Lisp client remote-triple-store SPARQL interface

The remote-triple-store implementation of the Lisp client was unable to execute SPARQL queries with non-trivial filter clauses. This has been fixed. (Repeated in the Lisp API section below.)

Bug20867: REPLACE function in SPARQL 1.1 used incorrect syntax for matches

The SPARQL 1.1 REPLACE function was implemented using a different syntax for pattern replacement than that specified by the standard. Replacements are now correctly specified using the $<number> syntax.

Bug20863 - Some SPARQL XSD numeric computations could fail

Depending on the datatypes of the numbers involved, it was possible for a SPARQL numeric computation to signal a type error rather than correctly returning a value. This has been corrected.

Bug20859 - Some filter combinations could cancel each other out

If a SPARQL query used IN (or several FILTERS like ?uri = constant) and if there were other filters on variables that overlapped with the variables in the IN construct, then it was possible for AllegroGraph to lose track of the additional filters in some circumstances. This happened due to two separate query optimizations handling the filters inconsistently. This problem has been corrected.

Bug20852 - remote-triple-store and SPARQL UPDATE don't mix

The Lisp remote-triple-store client could signal an error after evaluating a SPARQL UPDATE statement. The UPDATE would occur but the result would be mis-interpreted and signal an error. This has been corrected.

Bug20851 - SPARQL FILTERs could be too strict

Certain combinations of triple patterns in a BGP and FILTERs could cause AllegroGraph to misapply the FILTERs. For example, depending on the underlying triple-store, this query

SELECT ?first_film ?actor ?second_film {  
  ?first_film dbpp:starring dbpr:Jack_Black , ?actor .  
  ?second_film dbpp:starring dbpr:Joan_Cusack , ?actor .  
  FILTER ( ?actor != dbpr:Joan_Cusack )  
  FILTER ( ?second_film != ?first_film )  
  FILTER ( ?actor != dbpr:Jack_Black )  
} 

could have misapplied the ?actor filter and produced fewer results than expected.

This has been corrected.

Bug20838 - SPARQL CONSTRUCT queries could fail for large result sets

Due to limitations in the internal mechanisms used to keep track of intermediate results, SPARQL CONSTRUCT queries that returned more than 100,000 rows would cause a segmentation fault. This has been corrected.

Bug20830 - Some inequalities could use the wrong comparisons

Some SPARQL 1.1 queries could select the wrong comparison code when comparing two variable clauses with either >= or <=. This could lead to incorrect results. This has been fixed.

Bug20821 - EXISTS or NOT EXISTS and additional filters did not mix

The new SPARQL 1.1 EXISTS and NOT EXISTS filters failed to parse correctly when combined with other FILTER clauses. This has been fixed.

Bug20805 - encodeforuri could produce incorrect results

The SPARQL 1.1 encode_for_uri could produce incorrect results for some UNICODE characters. This has been corrected.

AGWebView

Rfe10885 - Setting duplicate suppression from AGWebView

Added "Suppress duplicate statements" feature to the repository overview page. This sets the duplicate-suppression-strategy switch, which can be set to prevent new duplicate triples (both duplicates already in the store, and duplicates within the same commit) from being committed to the store.

Bug20996 - Webview: js error creating a new repo

This fixes a javascript error when creating a repository from the Catalog page.

Bug20993 - Webview: Internet Explorer (IE) compatibility mode

AGWebView now includes an html meta header for IE9 so that IE9 will use its documentMode instead of IE7 compatibility. Also tested with IE8.

Also using html5 doctype to get standards mode in all browsers.

Bug20963 - Fix session url in AGWebView

Urls for sessions are changed in Bug20814 (described under AllegroGraph Server: General Changes above). This fixes AGWebView and changes the url for its use of sessions to include the uuid.

Bug20894 - AGWebView not working with ie9 on intranet

When connecting to AGWebView on an intranet site IE may automatically use Compatibility Mode. A syntax error causes it to only display the "Loading" message and not continue. Work-around is to use a full url, or otherwise not use compatibility mode. The syntax error is now fixed so an intranet does not cause the error.

Bug20843 - AGWebView sometimes clears the editor

The execute button would clear the query editor (and the results area). This happened when the same query had been executed sometime before in the same browser followed by logout/login. The browser history mechanism had not been fully cleared. Recent queries are cleared after logout. This is now fixed.

Changes to the Lisp API

Bug20884: improve Lisp client remote-triple-store SPARQL interface

The remote-triple-store implementation of the Lisp client was unable to execute SPARQL queries with non-trivial filter clauses. This has been fixed. (Repeated in the SPARQL section above.)

Bug20883: improve the Lisp client's remote-triple-store caching interface

The Lisp client's remote-triple-store implementation did not support the spogi-cache-enabled-p or spogi-cache-size methods. This has been corrected.

Bug20185 - Remote lisp client used wrong value for restriction reasoning

For a remote reasoning store, this method used the wrong string internally, leading to an error when a query was made:

(setf (restriction-reasoning-enabled-p *db*) t) 

For a remote reasoning store, this method enabled restriction reasoning if the key :restriction-reasoning-enabled was present regardless of the key's value. Now respects t or nil.

(apply-rdfs++-reasoner :restriction-reasoning-enabled nil)  

Documentation

No significant changes.

Python Client

Rfe11313 - Remove export*, use response capture

RepositoryConnection.export and exportStatements have been deprecated. They used client side serialization of statements that was buggy and not up to spec. The functionality has been replaced with a way to capture server responses of any supported mime type.

A new context manager function on RepositoryConnection is called saveResponse, and can be used as follows:

with open('out', 'w') as response:  
    with conn.saveResponse(response, 'application/rdf+xml'):  
        # The server response is written to response  
        conn.getStatements(None, None, None) 

You can reference the function testsaveresponse in src2/franz/openrdf/tests/tests.py for examples.

Java Client

Rfe11493 - Support writing N-Quads from an AGModel

With this change, the Jena adapter now supports writing to N-Quads via AGModel.write(out, "N-QUADS").

Rfe11438 - Provide N-Quads parser/writer META-INF/services in agraph.jar

With this change, the agraph.jar now provides N-Quads implementations for org.openrdf.rio.RDFParserFactory and org.openrdf.rio.RDFWriterFactory in META-INF/services (Sesame doesn't yet provide N-Quads implementations).

Rfe11436 - Add Sesame 2.6.2 src jar

With this change, the openrdf-sesame-2.6.2-onejar-src.jar has been added to eclipse project classpaths (repo and distribution) for convenience.

Rfe11397 - Support RepositoryFactory and Repository configuration

With this change, the Java client now supports Sesame's RepositoryFactory api and Repository configuration from RDF data. See javadoc for the new com.franz.agraph.repository.config package for details.

Rfe11240 - Support user/role management

With this change, AllegroGraph superusers can now dynamically manage users and roles. See the AGServer javadoc for details.

Rfe11230 - Support masquerading as another user

With this change, an AllegroGraph superuser can now masquerade as another user when issuing requests. See the javadoc for AGRepositoryConnection#setMasqueradeAsUser for details.

Bug20946 - IE javascript error in Mold: innerHTML is null

Work-around: the error dialog can be dismissed and the search results are still displayed. Fixed.

Bug20811 - Allow prolog queries from javascript

To run javascript the user must have the "execute arbitrary code" permission which should be enough to run prolog queries too. The bug that was fixed prevented the user from running prolog queries from javascript.

AllegroGraph 4.5 user-visible changes

AllegroGraph Server: General Changes

Rfe11338 - CJK bigram tokenizer

A new tokenizer :simple-cjk for Chinese/Japanese/Korean (CJK) text has been added. It uses bigrams for consecutive CJK characters as words. To use this tokenizer, specify :simple-cjk as the tokenizer option when creating or modifying a freetext index.

The bigram tokenizer can be used for CJK mixed text, but its simplicity may result in false positives. It also tends to index a much larger number of words compared to other tokenizers. See Free-text Indices and create-freetext-index.

Rfe11298 - Spread access to the same triple store across shared backends

Previously, when using shared backends, operations for a particular triple store tended to be routed to a single shared backend, even if it was busy. Now concurrent operations for a particular triple store are routed to a shared backend that is not already processing a request for that same store. Additional backends (up to the value of the "Backends" configuration parameter) will be started if necessary to achieve this goal. This is expected to increase concurrency in such situations.

Dedicated sessions are unaffected by this change.

Since this change is most beneficial when there are many shared backends available, the default setting of the "Backends" configuration parameter has been changed from 3 to 10.

Rfe11257 - Backup/restore user namespace settings

agraph-backup is enhanced in two ways:

The user namespace settings are now handled by the backup-all/backup-settings/restore-all subcommands.

The backup-settings subcommand can take a path to the executable of an older AllegroGraph's version of agraph-backup, and call it to archive the store data. With this feature, archiving an entire repository from an older AllegroGraph repository becomes a single command.

See Backup and Restore.

Rfe11099 - Add Apache Solr support to AllegroGraph

Now you can connect an external Solr text search engine from AllegroGraph server. With new prolog functions you can issue a query to the Solr engine to retrieve associated document IDs. See Solr Text Indices.

Bug20750 - Unused store closer holds lock too long

The thread which closes idle triple stores in backends held an important lock while trying to acquire locks on individual stores. If a lock on a store was being held by another thread (e.g., one running a long query), this could cause the whole backend to become unresponsive to other unrelated requests. This has been corrected.

Bug20736 - Federated-triple-store could mistreat xsd:strings in queries

In some cases a federated-triple-store could fail to account for the equality of plain literals and literals with a datatype of xsd:string. This has been corrected.

Bug20717 - Fix the UPI not in string table query issue

Depending on the result format and the client, queries that generated new strings (e.g., using SPARQL 1.1's BIND clause) could lead to a UPI not in string table error. This has been corrected.

Bug20712 - Implement xs:date and xs:time XPath constructor functions

The constructor functions for xs:date and xs:time are now implemented.

Bug20697 - Zero length property path with future parts

Zero length property path patterns now correctly match future parts that are not yet in the db:

?x :p{0} :a . 

Note that the same pattern with a literal worked even without this fix:

?x :p{0} "o" . 

Bug20690 - Property path results cardinality

In the presence of cycles, property path expressions did not always return results with the right cardinality. This has been fixed.

Bug20616 - Fix race condition in triple store termination

AllegroGraph now waits for a triple-store's subprocesses to exit when the last connection is closed instead of killing them while they could potentially be executing a clean shutdown.

Avoid Logging a backtrace during unclean shutdown

Under certain circumstances (such as when 'killall agraph' was used), there was a race condition during process termination that could result in an error and backtrace being logged to agraph.log. This has been corrected.

HTTP Client

Rfe11308 - Support UPDATE via POST /repository/ID/statements

The Sesame 2 protocol allows statements to be added to a repository by sending a POST request to /repository/ID/statements. AllegroGraph previously only supported update via POST requests to /repository/ID.

Now SPARQL update statements may be directed to either location. You can also POST an RDF document or data in the form of an application/x-rdftransaction to /repository/ID/statements.

Rfe11213 - Allow superusers to run queries with another user's security filters

AllegroGraph sessions belonging to superusers now allow their owners to use the X-Masquerade-As-User header in HTTP requests to activate another user's security filters.

Bug20671 - List the Hub process in the HTTP API

AllegroGraph now lists the Hub process in the HTTP API's process list and in WebView. The Hub process can not be killed or asked to perform a backtrace.

SPARQL

Rfe11320 - Trying to use SERVICE should signal an error

AllegroGraph's SPARQL engine does not support federated query (http://www.w3.org/TR/sparql11-federated-query/). Previously queries using the SERVICE keyword would fail silently. Now these queries will signal an error.

Rfe11302 - Support clauses with no variables

The SPARQL 1.1 engine now supports basic graph patterns where one or more of the clauses have no free variables. Previously, a query like

select * {  
   :a :p1 :c .  
      :p2 ?value .  
} 

could produce incorrect results. This has been corrected.

Rfe11262 - Improve SPARQL 1.1 filter transformation

AllegroGraph's SPARQL 1.1 engine is able to move some filters out of the query processor and into the storage layer. For example, a query like

select * {  
  ?a rdf:type ?class .  
  filter( str(?class) > 'z' )  
} 

will move the inequality into a filter involving only the first clause. In some cases, however, the query processor could leave behind a stub filter which would unnecessarily add to total query time (without altering the results). This is now corrected.

Several other minor efficiency improvements have also been added to the query engine.

Rfe11254 - Improve SPARQL 1.1 sub-query implemenation

A problem in the sub-query implementation led to two failures in the DAWG test-suite. These have been corrected and additional tests have been added to the regression tests

Rfe11245 - Optimize Queries where the right-hand side of a join is a BGP

The SPARQL 1.1 engine now handles queries where the right-hand side of the join is a BGP more efficiently. Previously, it would compute the RHS separately and then join the two sets. Now, it merges the RHS BGP into the existing LHS set directly. This can provide some speed and memory improvements for some queries.

Rfe11135 - Implement SPARQL 1.1 DISTINCT aggregation

AllegroGraph's SPARQL 1.1 engine now implements DISTINCT in its aggregates. This means that a query like

select ?gender (count(distinct ?profession) as ?pCount) {  
  ?person k:profession ?profession .  
  ?person k:sex ?gender .  
} group by ?gender order by ?pCount 

will now correctly return a count of the distinct professions for each gender rather than a count of all professions.

Rfe11098 - Integrate Apache Solr with SPARQL 1.1 query engine

Assuming that you have setup Apache Solr with an AllegroGraph triple-store, you can now query it using the SPARQL 1.1 query engine. See the description of using Solr in Solr Text Index. When used with SPARQL 1.1, there are two storage strategies: addinng a triple that associates the text with a Solr document ID and telling Solr to associate the text with the triple-id directly. Both are described in the referenced document.

Rfe11070 - Improve performance of variable predicate queries

AllegroGraph's SPARQL engine now recognizes queries that have variables in the predicate position and provides custom methods to optimize them. An example of such a query is:

SELECT ... {  
 ... clauses ...  
 ?s ?p ?o . 

The optimizations are only possible when AllegroGraph has already computed the complete list of predicates in the store. This can be accomplished using the ComputeStatistics query option. For example

PREFIX franzOption_ComputeStatistics: <franz:yes>  
SELECT ... 

Bug20778 - Several fixes for various SPARQL 1.1 issues

There were numerous issues in the initial release of the SPARQL 1.1 query engine. Some of the problems were caught by customers and others by internal review. This release of AllegroGraph fixes:

There remain some known issues with SPARQL; please see the SPARQL release notes for details.

Bug20760 - Several improvements in SPARQL aggregation.

Bug20756 - Multiple OPTIONAL clauses for the same variable could fail

The SPARQL 1.1 query engine could fail to find some bindings when multiple OPTIONAL clauses referred to the same variable. For example, a query like

SELECT ?first ?last {  
  ?s ab:lastName ?last .  
  OPTIONAL { ?s ab:nick ?first . }  
  OPTIONAL { ?s ab:firstName ?first . }  
} 

could fail to find bindings for ?first that come from the second OPTIONAL. This is now corrected.

Bug20752 - Queries with both filters and property paths could fail

If a SPARQL 1.1 property path query used FILTERs and these FILTERs were moved into the triple patterns, then it was possible for the filters to be ignored. This has been corrected.

Bug20732 - SPARQL 1.1 property path improvements

In some cases, SPARQL 1.1 property path queries could do too much work and return too many results. This has been corrected.

Bug20723 - Improve handling of SPARQL filter predicates

Some of the SPARQL 1.1 filter predicates were failing to function properly when used in some contexts because their return value was not being correctly transformed from an internal representation to an external one. This has been corrected.

Bug20722 - SPARQL 1.1 parser treated whitespace differently

The SPARQL 1.1 parser required whitespace after a period and before an additional SPARQL keyword. This meant that a query like

select * { ?a ?b ?c.filter( ?c > 3) } 

would signal a parse error. This has been corrected.

Bug20721 - Improve handling of certain OPTIONAL edge cases

Depending on the data in the store, the SPARQL 1.1 engine could sometimes fail to process queries like:

select ?a ?b ?c ?d {  
  optional { ?a a ?b . }  
  { ?c <urn:a:b> ?d . }  
} 

where the OPTIONAL could lead to an empty group of bindings on the left-hand-side during query processing. This has been corrected.

Bug20711 - Improve some date and dateTime comparison issues

The SPARQL 1.0 and 1.1 engines behaved inconsistently with respect to one another for certain comparisons between xsd:dates and xsd:dateTimes. This could lead to differences in query behavior. These issues have been reconciled.

Bug20695 - Problem when computing range-queries on dateTimes

Both the SPARQL 1.0 and the SPARQL 1.1 engine could fail to correctly compare dateTime datatypes with fractional seconds. This has been corrected.

Bug20689 - Parsing of SPARQL property path expressions where a group is followed by a + as in

?x (ex:p)+ ?y 

no longer results in a error.

AGWebView

Bug20751 - Agwebview could submit queries twice in some browsers

On some browsers AGWebView could submit queries twice. This led to unnecessary work for SPARQL or Prolog SELECT queries and to duplicate data for SPARQL UPDATE queries. This has been corrected.

Bug20738 - AGWebView could not start remote sessions

A recent change in AllegroGraph broke AGWebView's ability to start sessions on remote stores. This is now corrected.

Changes to the Lisp API

No significant changes.

Documentation

Rfe11236 - Fix command line documentation of the agdebug tool

The agdebug tool now correctly indicates a non-optional argument.

Python Client

Rfe11229 - Add support for user masquerading to python client

Added support for user masquerading to Python client by adding RepositoryConnection.runAsUser(username).

Java Client

Rfe11329: AGConnPool, remove unnecessary dependency on Jena

If using AGConnPool with Sesame API, but not Jena, the Jena libraries are no longer required.

Rfe11310: AGConnPool should rollback when borrowing

Pool will now always rollback when borrowing a connection. This makes the transaction catch up to the most recent.

The 'testOnBorrow' property for the pool is now redundant, so generally not recommended.

Rfe11242: Upgrade to Sesame 2.6.2 jars

With this change, the Java client has been updated to use the Sesame 2.6.2 jars, including support for AGUpdate and AGRepositoryConnection#prepareUpdate methods. Java Sesame Tutorial example13 has also been updated to reflect use of prepareUpdate.

Bug20795: AGConnPool does not close connections

Closing the AGConnPool now closes sessions properly.

AllegroGraph 4.4.0.1 user-visible changes

AllegroGraph Server: General Changes

Bug20667 - XSD float exponent fix

Due to rounding issues when encoding, exponents of floats in the :sparql-xml result format were - very seldom - off by one. This has been fixed.

Bug20615 - TrxLogRetain replication now works on secondaries

The catalog directive: TransactionLogRetain replication now archives properly for replication secondaries.

HTTP Client

No significant changes.

SPARQL

Rfe11219 - Add query option support for all SPARQL constructs

AllegroGraph's SPARQL parser supports specifying query options as part of the query string. These take the form

prefix franzOption_OPTIONNAME: <franz:OPTIONVALUE> 

In particular, you can tell AllegroGraph which query engine to use with the queryEngine parameter. For example, to use the SPARQL 1.0 engine in a query, you would add the prefix

prefix franzOption_queryEngine: <franz:sparql-1.0> 

This extension is now supported for all SPARQL constructs: ASK, CONSTRUCT, DESCRIBE, SELECT and UPDATE.

Rfe11218 - Improve maximum solution size handling

Because the SPARQL 1.1 engine operates on the entire query solution simultaneously it can be more memory intensive than the SPARQL 1.0 engine for some queries. There was an internal check to warn if query solution sets were becoming too large. This check was causing some queries to fail rather than simply signaling a warning. This has been changed so that these queries will write a warning to the log rather than failing.

Rfe11202: SPARQL Update example with batch update

With this change, Java tutorial example 13 has been augmented to show how to evaluate SPARQL Update queries, including one that batches updates in a single request. [This item is repeated in the Java Client section.]

Bug20768 - SPARQL 1.1 property paths could return incorrect results

AllegroGraph was failing to filter non-matching rows for property path queries like

select ?a {  
  ?a owl:sameAs{1} ?a .  
} 

meaning that it would emit results when it should not. This has been fixed.

Bug20692 - Sparql 1.1 CONTAINS function can fail

Corrected a problem with some instances of the SPARQL 1.1 CONTAINS function.

Bug20676 - Geospatial datatype mappings could cause query errors

Changes to some query optimization code in the AllegroGraph SPARQL engines (both sparql-1.0 and sparql-1.1) were incompatible with geospatial datatype mappings (geospatial predicate mappings were not effected). This could lead to parse errors during query processing. This has been fixed.

Bug20665 - SPARQL-1.1 UNION doesn't see with-variable bindings on the RHS

The SPARQL 1.1 engine was losing track of the variable bindings on the right-hand side of a UNION. This could cause queries like

select ?s1 ?s2 {  
  { ?s1 a ?p . }  
  union  
  { ?s2 a ?p . }  
} 

(with a variable binding for ?p) to produce incorrect results. This has been corrected.

Bug20664 - SPARQL-1.1 CONCAT function signaling errors with typed literals

The SPARQL-1.1 CONCAT function could signal an error instead of generating the correct result when one or both of the arguments with a typed literal. This has been corrected.

AGWebView

Rfe11198 - Improve WebView's bulk loading UI

WebView no longer conflates bulk load mode with agload usage. It's now possible to use agload's multi-core optimized loading mechanism and to optionally turn bulk load mode on. Some aspects of the file loading UI in WebView have also been optimized: The format selection element is now a combo box, and is located closer to the file input field.

Bug20662 - Reload triple count in WebView if load fails

WebView will now display the correct triple count in a store after an import got stopped by an error (e.g. the triple license limit).

Bug20641 - Improve restoring backups from WebView

When restoring from a backup in the WebView catalog overview, AllegroGraph will now automatically upgrade the newly-created store to the newest version. Additionally, there is now an option to set the store up for use as a replication secondary.

Bug20637 - Fix problem where SPARQL Update did not work with AGWebView

Changes to SPARQL UPDATE caused some UPDATE queries to fail in AGWebView. This has been corrected.

Changes to the Lisp API

Bug20640 - Fix printing of encoded-ids in remote lisp client

AllegroGraph's remote lisp client now correctly translates encoded IDs' UPIs back into strings. It can also correctly translate variable-width encoded IDs into UPIs now.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe11202: SPARQL Update example with batch update

With this change, Java tutorial example 13 has been augmented to show how to evaluate SPARQL Update queries, including one that batches updates in a single request. [This item is repeated in the SPARQL section.]

AllegroGraph 4.4 user-visible changes

AllegroGraph Server: General Changes

Rfe11082: Transactional duplicate deletion

INCOMPATIBLE CHANGE: AllegroGraph no longer supports duplicate deletion at index optimization time. The database creation parameters indicating duplicate deletion strategy are no longer accepted.

Instead, AllegroGraph now allows deletion of duplicates using the delete-duplicate-triples API function, which is also accessible via the HTTP API call DELETE /statements/duplicates.

Rfe11039 - Include AllegroGraph version in ps output

AllegroGraph subprocesses modify their command line so that they can supply identifying information to programs like 'ps' and 'top'. Now, in addition to the name of the subprogram, the version of AllegroGraph is included. Also, for the AllegroGraph Service Daemon process, the port number that it is listening on is included.

Rfe11017 - Add an agdebug command line utility

AllegroGraph now ships with a command line utility called agdebug that will output a summary the contents of the transaction log files in a store. This is primarily useful for diagnosing problems with transactions and the database's storage system.

Rfe10984 - Fix serialization issues with encoded IDs

Both the turtle and the RDF/XML serializers failed to correctly handle encoded IDs. This has been corrected.

Rfe10981 - Improve upgrading/migration process

The agraph-backup tool has two new commands, backup-all and restore-all, to help migrating existing databases to the new version of AllegroGraph or copying all catalogs and repositories to a new location. See Backup and Restore and Database Upgrade.

Rfe10948 - Log secondary index creation

Adds log entries to the AllegroGraph log when a secondary index (also called a UPI map) is created.

Rfe10853 - Support fti charset in Python client

Keyword arguments innerChars, borderChars, and tokenizer have been added to createFreeTextIndex and modifyFreeTextIndex.

getFreeTextIndexConfiguration also returns these new configuration properties and values. See put-freetext-index.

Rfe10783 - Add per triple-store control of secondary indices

Modified the Lisp and HTTP APIs so that each triple-store can specify whether or not it should use existing secondary indices and/or create new secondardy indices during query. Since these indices must be maintained and updated when modifying the triple-store, automatic creation should be turned off until the store is quiescent. The default is to use secondary indices when they exist but not to create new ones. Use [create-secondary-indices-p[]/(setf create-secondary-indices-p), and use-secondary-indices-p/(setf use-secondary-indicies-p)* to control the behavior.

Rfe10568, Rfe11172 - Set the default query engine via agraph.cfg

AllegroGraph supports two SPARQL query engines. The algebra engine has been available for several years and is well tested, stable and robust. The newer, set-based, engine, named sparql-1.1 can be significantly faster than the algebra engine but may contain bugs that the algebra engine does not.

Out of the box, AllegroGraph uses the sparql-1.1 engine. This can be changed by using the QueryEngine parameter in the configuration file. If you include

QueryEngine algebra 

Then AllegroGraph will use the algebra engine. If you leave the setting off or use

QueryEngine sparql-1.1 

Then AllegroGraph will continue to use the set based query engine.

The value of this configuration is read each time a triple-store is created or opened.

Rfe10179, Rfe10540 - SNA optimizations

Improved the efficiency of SNA operations in several ways. All SNA generator functions are now slightly faster. SNA generators that use select now save the select plan and re-use it. This can make these generators hundreds of times faster than before. The SNA system will also use secondary indices (UPI-maps) when available.

Rfe10398 - Improved speed of low-level AllegroGraph operations

Optimized the code used to copy UPIs and parts of triples. This produces a query speedup in SPARQL queries and most Prolog select queries. Some count-only Prolog select queries can run slightly slower.

Rfe10090, Rfe10541 - optimizations to AllegroGraph secondary indices

Improves both the creation of secondary indices (UPI-maps) and their efficiency in use. Queries that make heavy use of secondary indices will operate more quickly.

Rfe9786 - Make shm server global

A new configuration directive GlobalShmServerDirectory is added to specify a directory where the global shared-memory server creates its socket. When omitted, the value of SettingsDirectory is used.

Bug20619 Optionally not inherit trx log arch params during restore

The configuration file catalog directive TransactionLogArchive is no longer inherited from a backup database during a restore. In addition, when the --replica flag is set during agraph-backup restore, TransactionLogRetain and TransactionLogReplicationJobname are no longer inherited from the backup and must be set manually in the config file catalog section prior to restore. This is so that a replication primary and replication secondary can have independent transaction log archiving behavior. The flag --replica also ensures --nocommit is set and --newuuid is not set, and should be used as a replacement for --nocommit when restoring backups for replication client purposes.

Bug20460 - Encoded IDs and reasoning triple-stores mix not

If reasoning was used during a query, then encoded IDs were not correctly recognized which led to incorrect results. This has been corrected.

Bug20443 - Encoded IDs did not work with the TriX serializer

The TriX serializer was incorrectly emitting encoded IDs as if they were plain literals instead of resources. This has been corrected.

Bug20355 - Durably save transactions on warm-standby client

The warm-standby secondary machine now saves transaction log entries in a way that ensures they've been flushed to disk durably as soon as the entire commit has been read.

Bug20332 - Trix parser can get confused and try to encode plain literals

Under some circumstances, the TriX parser could attempt to encode plain literals as if they were typed literals which could cause a data import to fail. This has been corrected.

Bug20230 - Fix deletion slowdown under replication

The replication server used a connection to the replicated triple store that ensured it would never expunge deleted triples, slowing down operations if many deletes had occurred. Now, the replication server correctly expunges these triples and does not slow down.

Bug19957 - Turtle parser should support type-mapping

The turtle parser was not respecting a triple-store's type-mapping settings. This meant that triples added via turtle would not have their objects converted into encoded UPIs. This has been corrected.

Spr38044 - Restrict possible values for TransactionLogRetain

AllegroGraph will now signal an error when invalid values for TransactionLogRetain are encountered in the config file.

HTTP Client

No significant changes.

SPARQL

Rfe10985 - Spin functions and magic-properties

SPIN functions and magic-properties have been implemented. The SPIN implementation is described in Spin document.

Rfe10737 - SPARQL 1.1 BIND support

Add support for SPARQL 1.1's BIND language construct. In particular, queries like

select ?title ?costPlusTax {  
?book <http://purl.org/dc/elements/1.1/title> ?title .  
?book <http://example.org/ns#price> ?cost .  
?book <http://example.org/ns#tax> ?tax .  
bind( ?cost * (1 + ?tax) as ?costPlusTax )  
} 

are now supported.

Rfe10736, Rfe10743 - SPARQL 1.1. exists, not exists, and minus

AllegroGraph's SPARQL engine now handles the MINUS operator of SPARQL 1.1. It also handles the new EXISTS and NOT EXISTS filter operators.

Rfe10700 - SPARQL 1.1 assignment

SPARQL 1.1 assignment is now implemented so that queries like

select ?title ?cost ?discount (?cost * ?discount as ?totalCost)  
where {  
?book :title ?title .  
?book :price ?cost .  
?book :discount ?discount .  
} 

are now supported.

Rfe10699 - SPARQL 1.1 UPDATE support

The SPARQL parser and implementation now support the UPDATE specification at http://www.w3.org/TR/2011/WD-sparql11-update-20110512/.

Note: This change is incompatible with some existing uses of SPARQL UPDATE.

AGWebView

Rfe11131: Allow triggering deletion in WebView

It is now possible to invoke duplicate deletion from WebView.

Rfe10631 - Allow loading files via agload in WebView

WebView now has a "Bulk Load" option to speed up loading large files. Since this option uses the agload utility internally, it also allows specifying an error handling strategy other than "Cancel".

Bug20492 - Improve code editor in WebView

The text editor in WebView (used for e.g., queries) has been improved for browser compatibility (mainly with Internet Explorer) and functionality. It now expands automatically to fit the text contained, displays line numbers and automatically indents code.

Changes to the Lisp API

Rfe11082: Transactional duplicate deletion (repeated from the General section)

INCOMPATIBLE CHANGE: AllegroGraph no longer supports duplicate deletion at index optimization time. The database creation parameters indicating duplicate deletion strategy are no longer accepted.

Instead, AllegroGraph now allows deletion of duplicates using the delete-duplicate-triples API function, which is also accessible via the HTTP API call DELETE /statements/duplicates.

Documentation

No significant changes other than new features documented.

Python Client

Rfe11122 - Add "delete duplicates" API function to python client

RepositoryConnection.deleteDuplicates:

Delete all duplicates in the store. Must commit.

mode - "spog" or "spo"

Java Client

Rfe11121 - Support transactional duplicate deletion

With this change, the Java client now provides the method AGRepositoryConnection#deleteDuplicates(comparisonMode) for deleting spog and spo duplicates in a transactional manner. See the javadoc for more details.

See Change History for user-visible changes to earlier releases.

Rfe11117: java api for SPIN (adds to rfe10988)

New methods added to AGRepositoryConnection: listSpinFunctions() listSpinMagicProperties()

New classes: AGSpinFunction, AGSpinMagicProperty

Rfe10622 - Http response handlers now have their own package

With this change, the Java client's HTTP response handlers have been refactored and moved into a com.franz.agraph.http.handler package.

AllegroGraph 4.3.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10984 - Fix serialization issues with encoded IDs

Both the turtle and the RDF/XML serializers failed to correctly handle encoded IDs. This has been corrected.

Rfe10967 - Nicer CSV output option

AllegroGraph now supports another CSV output format (MIME type application/processed-csv) that uses the UTF-8 encoding, prints literals with numeric types as numbers, and doesn't wrap string literals in quote marks.

Bug20460 - Encoded IDs and reasoning triple-stores failure

If reasoning was used during a query, then encoded IDs were not correctly recognized which led to incorrect results. This has been corrected.

Bug20451 - Fix encoded ID API on remote lisp client

The remote lisp client was missing a few functions in the encoded ID API: The functions that were missing are next-encoded-upi-for-prefix, encoded-id-prefix-registered-p and encoded-id-prefix-in-use-p. These are now implemented and available on the remote lisp client.

Bug20443 - Encoded IDs did not work with the TriX serializer

The TriX serializer was incorrectly emitting encoded IDs as if they were plain literals instead of resources. This has been corrected.

Bug20416 - Use proper n-triples syntax for text/csv

The CSV generator (when requesting MIME type text/csv) for query results will now always generate valid CSV such that the entries imported into a spreadsheet or a database are in correct n-triples part syntax.

HTTP Client

No significant changes.

SPARQL

No significant changes.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Rfe10983 - Query execution with initial bindings

With this change, the Java client's Jena adapter now supports creating query executions with initial bindings using the method: AGQueryExecutionFactory#create(query,model,initialBinding);

Welcome to AllegroGraph 4.3.2

AllegroGraph 4.3.2 user-visible changes

AllegroGraph Server: General Changes

No significant changes.

HTTP Client

Bug20424 - Disable 5-minute HTTP request timeout

AllegroGraph now allows HTTP API requests to run (and generate results) for longer than 5 minutes.

SPARQL

No significant changes.

AGWebView

No significant changes.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug20272 - NPE when no response stream is available

Previously, the Java client's HTTP response handler could throw an opaque NullPointerExeption if no response stream is available (this might occur due to a server crash). With this change, the response handler checks for a null stream and throws an exception advising the user that no response stream is available and to inspect the server log file for possible explanations.

Welcome to AllegroGraph 4.3.1

AllegroGraph 4.3.1 user-visible changes

AllegroGraph Server: General Changes

Rfe10889 - Provide guaranteed duplicate removal behavior

As long as no triples are concurrently being added, completion of a level 2 database optimization will now ensure that all duplicate triples (as defined by the current duplicate triple removal mode) are deleted.

Rfe10850 - Ag3 plugins for TBC 3.5

TBC 3.5 no longer provides built-in support for AG3 databases. With this change, AG3 plugins are made available to work with TBC 3.5. The plugins are not compatible with previous versions of TBC due to conflicts with the built-in support for AG3.

AG3 plugins for TBC 3.5 will be available from this Eclipse update site:

http://www.franz.com/ftp/pub/agraph/eclipse/tbc/3.5

Bug20408 - Implement encoded-ids for lisp remote-triple-store

It is now possible to register, unregister and list encoded-id prefixes when using the remote lisp client.

Bug20397 - Load triples from files faster on remote lisp client

The remote lisp client now can load large files into the server, and will no longer run out of memory when doing so.

Bug20378 - Improve duplicate reduction in UPI-map based cursors

AllegroGraph reduces the number of triples examined during SPARQL queries by skipping subsequent duplicates when iterating over a cursor. This duplicate reduction was not occuring for cursors based on secondary indices (UPI-maps) which could lead to decreased query performance. This has been corrected.

Bug20372 - Fix part printing for literals and resources with same name

AllegroGraph would incorrectly print literals that had the same name as a resource. This is now fixed.

Bug20355 - Durably save transactions on warm-standby client

The warm-standby secondary machine now saves transaction log entries in a way that ensures they've been flushed to disk durably as soon as the entire commit has been read.

Bug20336 - Fix inconsistency on warm standby secondaries

Replication secondary instances for AllegroGraph would sometimes query and attempt to display inconsistent data while replication was copying transactions from the primary instance. This has now been fixed: AllegroGraph ensures that only data from transactions that have been fully been replayed is used.

Bug20230 - Fix deletion slowdown under replication

The replication server used a connection to the replicated triple store that ensured it would never expunge deleted triples, slowing down operations if many deletes had occurred. Now, the replication server correctly expunges these triples and does not slow down.

HTTP Client

Rfe10893 - Allow "unlimited" concurrent HttpClient connections

Previously, the Java client's AGServer class would default to using an HttpClient that limited the number of concurrent connections per host to 20 (up from the HttpClient's default of 2, per RFC 2616 sec 8.1.4), which could still result in unnecessary confusion and socket churn when interacting with application level connection pool management. With this change, the defaultMaxConnectionsPerHost and maxTotalConnections are both increased to Integer.MAX_VALUE to avoid conflict with application level connection pool management.

SPARQL

Bug20382 - Sparql queries that returned * could confuse remote-triple-stores

If a SPARQL query returned all of its variables (e.g.. SELECT * ...), then the remote-triple-store could fail to determine the actual variables returned by the query. This could in turn lead to incorrect results from clients that use this information (e.g., Gruff). This is now corrected.

AGWebView

Rfe10924 - Enable download of server log file from WebView

WebView now allows viewing and downloading the agraph.log server file for diagnostic purposes. To open the log, select the logfile entry in the Admin menu.

Changes to the Lisp API

No significant changes.

Documentation

No significant changes.

Python Client

No significant changes.

Java Client

Bug19798 - Illegal resource in Jena reified statements

Previously, the Jena adapter's createReifiedStatement method could create reified statements containing illegal blank node ids that were based on generated UUIDs, manifesting in invalid part errors from the server. With this change, these reified statements are now created using legal resources. AGGraphMaker instances can now be created with a specified ReificationStyle (the default style is ReificationStyle.Minimal).

Welcome to AllegroGraph 4.3

AllegroGraph 4.3 user-visible changes

AllegroGraph Server: General Changes

Rfe10790 - improve logic for finding the sort-upi-map binary

There were cases when the Lisp client could fail to locate the path to the sort-upi-map external program. This has been corrected.

Rfe10665 - AG4 plugins for TBC 3.5.0

With this change, the AG4 plugins have been modified to work with TBC 3.5.0. The plugins are not backwards-compatible with previous versions of TBC due to changes in its API.

AG4 plugins for TBC 3.5 will be available from the update site: http://www.franz.com/ftp/pub/agraph/eclipse/tbc/3.5

Rfe10636, Bug20160 - signal an error if a blank node ID is too big, support for more blank nodes per store

AllegroGraph previously supported only 2^32 blank-nodes per triple-store. This is now increased to 2^56. Furthermore, if AllegroGraph tries to create a blank-node with an ID larger than the maximum supported, it will now signal an error rather than silently reusing existing nodes.

Rfe10632 - Enable per-user security-filtering of stores.

AllegroGraph now has a new class, security-filtered-triple-store, which allows specifying criteria that triples must fulfill in order to be accessible in that store.

AllegroGraph also automatically filters stores for users of a remote-triple-store if these users (or any of their roles) have security-filters defined.

Rfe10090, Rfe10541 - Optimizations to AllegroGraph secondary indices

Improves both the creation of secondary indices (UPI-maps) and their efficiency in use. Queries that make heavy use of secondary indices will operate more quickly.

Rfe10508 - Japanese full text search

An alternative text indexer for Japanese is provided. From the Lisp client, it is available by supplying :japanese as the :tokenizer keyword argument to create-freetext-index. When you use the Japanese tokenizer, we recommend you set :minimum-word-size to 2 or even 1 (the default is 3, but there are many Japanese words that consist of 2 characters).

The following is an example configuration for indexing Japanese text in literals of the object part of triples whose predicate is !<<description>.

(create-text-index "ja" :predicates '(!<description>)  
                        :stop-words nil  
                        :index-fields '(:object)  
                        :tokenizer :japanese  
                        :minimum-word-size 2)  

Rfe10020 - add support for owl:SymmetricProperty

AllegroGraph's RDFS++ reasoner now supports owl:SymmetricProperty so that it will make inferences like:

P rdf:type owl:SymmetricProperty  
a P b  
===> b P a 

In the process of implementing this addition, several other small efficiency improvements were made so that the reasoner is now usually faster even though it is doing more work. For example, the time to run all 14 LUBM-50 queries went from 10.04 seconds down to 9.44 seconds.

Rfe8278 - Add ability to customize the tokenizer used by freetext indexing

The Lisp API create-freetext-index takes two additional keyword arguments, border-chars and inner-chars, to fine-tune the word tokenization behavior. Each argument takes a freetext-charset object, specifying a set of characters that can appear at the borders of a word, or inside a word. A freetext-charset object can be created by a new API freetext-charset.

The following example creates a freetext index in which only alphanumeric characters and a hyphen #\- consists a word; that is, #. and #_ breaks words.

(create-freetext-index "custom" :predicates '(!<description>)  
                       :inner-chars (freetext-charset '(:alphanumeric #\-))) 

Bug20289 - Do not allow stored procedures to enter the debugger

If a stored-procedure written in lisp calls the function error with a non-error condition, the AllegroGraph backend process running the stored procedure would enter the debugger. This resulted in an unresponsive backend.

Now, the backend will catch invocations of the debugger, report the failure, and continue running.

Bug20273 - Suboptimal index queries broken

Under certain circumstances, suboptimal index queries could return triples that did not match the query pattern. This has been fixed.

Bug20160, Rfe10636 - signal an error if a blank node ID is too big, support for more blank nodes per store

AllegroGraph previously supported only 2^32 blank-nodes per triple-store. This is now increased to 2^56. Furthermore, if AllegroGraph tries to create a blank-node with an ID larger than the maximum supported, it will now signal an error rather than silently reusing existing nodes.

Bug20133 - Select queries should create filtered triple-stores only as needed

Prolog select queries were sometimes creating temporary graph-filtered triple-stores unnecessarily which wasted resources and time. This is now corrected: graph-filtered triple-stores are created only when necessary.

Bug19834 - Resource manager connection management bugs

Previously, global variables were used to maintain connection information between Lisp clients and one of the AllegroGraph subprocesses. This could cause problems if a single Lisp client made connections to two separate AllegroGraph servers or if an AllegroGraph server was stopped and restarted. This has been fixed.

Bug20037 - allow trailing slash on Server URL in AG4 dialogs

Previously, putting a trailing slash on the Server URL in AG4 dialogs would result in an error message that was hard to debug. With this change, dialogs are now robust to trailing slashes.

HTTP Client

Rfe10839 - Freetext index charsets and tokenizer in HTTP API

The HTTP API for freetext index management now supports the optional arguments "tokenizer" as well as "borderChars" and "innerChars", which specify the tokenizer and character sets for word boundaries.

Bug20250 - Fix federated remote stores in HTTP API

The /session resource didn't correctly create sessions when specified using the <http://example.com/repositories/test> syntax. Now, POSTing to /session correctly opens a session on a remote triple store.

SPARQL

Bug20305 - Encoded IDs were incompatible with SPARQL CONSTRUCT queries

SPARQL CONSTRUCT queries will not produce triples whose subject or predicate are literals. The check for this was incorrectly excluding triples with encoded IDs in the subject (or predicate) because these IDs have a different UPI type-code that the one used for resources. This has been corrected.

Bug20288 - Encoded IDs don't work with SPARQL/XML and SPARQL/JSON output

The SPARQL/XML results format would crash when used with encoded IDs; The SPARQL/JSON results format generated output but emitted encoded IDs as typed literals rather than as resources. Both of these problems are corrected.

AGWebView

Rfe10728 - Improve WebView store overview user interface

The AllegroGraph WebView store overview page now is better structured and allows easier access to commonly-used functions.

Bug20304 - Fix codemirror for Internet Explorer 9

Internet Explorer 9 users can now use WebView's text editor with syntax highlighting.

Bug20302 - Fix loading more results in Internet Explorer

WebView can now load more than 100 results (using the [more] link) in Internet Explorer.

Bug20296 - Fix query result downloads in Firefox 3.

Fix a JavaScript incompatibility that caused downloading query results from WebView to fail in Firefox version 3.

Bug20203 - Fix WebView for newer Internet Explorer versions.

Internet Explorer 8 and 9 support a new history tracking mechanism, and deprecated features that WebView used to keep track of history before. Now, WebView makes use of this new mechanism when it's available. This allows Internet Explorer to correctly run AllegroGraph WebView again.

Bug20069 - Uploading file into a context from WebView is broken

The error " is not a valid part" would always appear. This is now fixed.

Changes to the Lisp API

Bug20146 - Export UPI hash-table functions from db.agraph.user

In the transition from AllegroGraph 3 to AllegroGraph 4, the UPI hash-table functions ceased being exported from db.agraph.user. This is now corrected.

Bug20145 - (Lisp API only) loading AllegroGraph should provide module :agraph4.

AllegroGraph previously only provided the module :agraph which meant that using

(require :agraph4) 

could load agraph4.fasl unnecessarily. This is fixed.

Bug20144 - Fix interference from remote-triple-store ping

To ensure that a remote-triple-store remains opened, AllegroGraph's lisp API sends a ping message in the background in regular intervals. Previously, it would use the same session object to record responses from the server as a user's requests. This would cause errors when a ping message interrupts a long-running request.

Now, AllegroGraph uses a separate session object on the client's side to send these ping messages.

Documentation

Bug20149 - Correct documentation of datatype and predicate mappings (Lisp)

The Lisp reference manual had several typos in its discussion of datatype and predicate mappings. These have been fixed.

Python Client

No significant changes.

Java Client

Rfe10135 - Improve javadocs

Added overview pages, more docs for classes and methods, and links to AllegroGraph documentation for introduction, catalog, federation, etc.

Rfe10745 - Add Jena text indexing code example

With this change, a Jena text indexing code example is now available in the distribution in JenaTutorialExamples.java as example 12; this will eventually also be written up in the Jena tutorial document, but the focus here is on making the code example available as it is useful on its own.

Rfe10624 - AGQuery limit and offset modifiers

With this change, the Sesame and Jena AGQuery classes now allow limit and offset to be specified by the query object, outside of any limit or offset specified in the query string, making it easier for apps to override these modifiers.

Rfe10623 - Jena AGQueryExecution count methods

With this change, the Jena api now supports counts queries via the AGQueryExecution#countSelect() and countConstruct() methods. Solutions are counted server-side and only the count is returned to the client.

Rfe10492 and Rfe10493 - javadoc for AGRepositoryConnection

Add javadoc that points to SessionPorts for relevant methods. Add javadoc for type mappings, linking to http and lisp docs.

Rfe10452 - support N-Quads responses in the Java client

With this change, the Java client now supports N-Quads responses from the server. The property -Dcom.franz.agraph.http.defaultRDFFormat=NQUADS can now be used to configure the client to request N-Quads format by default for RDF statement responses; when this property is not present, the default format is TriX. The method AGHttpRepoClient#setPreferredRDFFormat() can also be used to override the default RDFFormat on a per connection basis.

Rfe10450 - Jena Prolog and transaction code examples

With this change, Jena Prolog and transaction examples are now available in the distribution in JenaTutorialExamples.java as examples 17, 18, and 22; they will eventually also be written up in the Jena tutorial document, but the focus here is on making the code examples available as they are useful on their own.

Rfe10447 - add AGGraphQuery#count() method

With this change, the java client now supports counting for graph queries (e.g CONSTRUCT and DESCRIBE) with the AGGraphQuery#count() method. Also added javadoc for the AGTupleQuery#count() method.

Rfe10446 - add count() examples to the Java tutorial

With this change, the Java tutorial shows how to use the AGTupleQuery#count() and AGGraphQuery#count() methods to to obtain query result counts without returning results (counting is done server-side, only the count is returned).

Rfe10445 - add example/test getting triple ids via prolog

With this change, example17 in TutorialExamples.java shows how to get triple ids via Prolog. TripleIdTests has also been augmented to get triple ids via Prolog.

Rfe10411 - Streaming large query results

When AGTupleQuery.evaluate() is used for sparql and prolog queries, the results are returned by the server in sparql/xml format. This is parsed by the Sesame library using a SAX XML parser and the results are collected in an ArrayList before the TupleQueryResult is returned. For very large result sets, this will cause an OutOfMemoryError.

Alternatively, AGTupleQuery.evaluate(handler) can be used where the handler methods are called as results are parsed. This can handle any length results without a memory error, but the API is less convenient. A new example has been added to the tutorial to demonstrate this: TutorialExamples.example3a()

A new class AGStreamTupleQuery can be used to wrap an AGTupleQuery so the TupleQueryResult will be streamed back. This provides the familiar iteration API rather than a handler, but the http response is only parsed as the iterator is advanced. The usage of this API is explained further in the javadocs: AGTupleQuery query = conn.prepareTupleQuery(QueryLanguage.SPARQL, "SELECT ..."); query = new AGStreamTupleQuery(query); TupleQueryResult results = query.evaluate();

There is also a new method in AGRepositoryConnection that will cause the connection to automatically wrap all tuple queries to use streaming: AGRepositoryConnection.setStreamResults(boolean streamResults)

Instead of SAX, the AGStreamTupleQuery uses an XMLStreamReader for parsing. Initial performance results suggest that in addition to being resistant to OutOfMemoryError this uses less overall memory than the SAX parser for small and medium result sets (10-500).

This approach of streaming results has not yet been applied to getStatements().

Rfe9445 - Upgrade slf4j, funnel through jcl, and add source jars

With this change, the java client now uses slf4j-api-1.6.1, and uses slf4j-jcl-1.6.1 to funnel sesame and jena logging through commons-logging-1.1.1 (used by httpclient-3.1). Source jars are also included for javadoc and to faciliate debugging.

Welcome to AllegroGraph 4.2.1c

AllegroGraph 4.2.1c user-visible changes

AllegroGraph Server: General Changes

Clarify what retain for replication means

If the TransactionLogRetain is set to replication then all transaction logs will be saved until replication starts at which point those log files no longer needed will be archived. This behavior may be further modified in a future release.

Bug20257 - fix convertdb for some rare cases

The convertdb utility failed to upgrade some stores that had transaction logs with a certain size. Now it can correctly upgrade these stores.

Bug20236 - Fix startup when HTTPProxy is specified in config

AllegroGraph now starts up correctly when a HTTP proxy specification is present in the configuration file.

Bug20223 - Fix agload for Turtle files

A change in agload broke loading Turtle-formatted files with agload. Now, agload can correctly load the Turtle format again.

HTTP Client

No user-visible changes.

SPARQL

No user-visible changes.

AGWebView

No user-visible changes.

Bug20200 - Clarify format support in import forms in WebView.

AllegroGraph now supports loading .gz-compressed files (and autodetecting their file format) from the server's local file system. The file import dialogues in WebView also have a note about the file name format for autodetection.

Changes to the Lisp API

Rfe10790 - improve logic for finding the sort-upi-map binary

There were cases when the Lisp client could fail to locate the path to the sort-upi-map external program. This has been corrected.

Documentation

No user-visible changes.

Python Client

No user-visible changes.

Java Client

No user-visible changes.

Welcome to AllegroGraph 4.2.1a

AllegroGraph Server: General Changes

Rfe10689 - Increase unique string limit to ~4B

The limit on the number of unique strings in the string table was previously 1,431,655,765. This limit has been increased to 4,294,967,295 entries.

Rfe10562 - Change encoded-id marker to @@ (from @#)

The marker used to indicate that a URI represents an encoded-ID has been changed from @# to @@.

Rfe10532 - Return encoded ids in ascending order

The .../encodedIds?prefix=""&amount=# http request was returning the ids in reverse order. This has been fixed to return them in ascending sequential order.

Rfe10521 - Deleted hash table management based on generation

Deleted hash tables are now cleaned up as soon as they are no longer needed, preventing the store from accumulating unnecessary files.

Rfe10200 - Regulate speed of mergers

Triple index mergers now regulate their speed based on the number of triples to be merged and the rate at which triples are added after a merge has started.

Welcome to AllegroGraph 4.2.1.

AllegroGraph Server: General Changes

Rfe10483, Rfe10528, Bug20012 - Inadequate transaction log durability

When using the default TlogSyncMethod of "odirect", the durability of transaction log files was not correctly ensured. This release fixes that issue by preallocating transaction log files and fsyncing them before using them. Additional transaction log files are created as needed. In addition, transaction log files that are no longer needed are now recycled instead of deleted.

This change will have an impact on database creation time due to the initial preallocation. To compensate for this, the default TransactionLogSize parameter has been changed from 1 gigabyte to 256 megabytes.

There is a new configuration parameter DesiredTlogFiles which specifies the number of transaction log files which should be preallocated at database creation time. The default value is 2. Specifying a larger value helps lower the probability of additional transaction log files being created during commits. This parameter is documented in Server Configuration.

Rfe10475 - Improve I/O behavior under memory pressure

When under memory pressure, Linux can sometimes generate far more I/O than expected when minor page faults occur during accesses to memory mapped files. This can result in very poor performance, especially when the I/O device is already congested. This change instructs the operating system not to perform these extra I/Os.

Rfe10439 - RDSF++ reasoner now does not create duplicate inferred triples

In some cases, the RDFS++ reasoner could follow multiple owl:sameAs paths and generate many duplicate inferred triples. This increased AllegroGraph's memory footprint and decreased RdFS++ reasoning performance.

The reasoner now generates only a single inferred triple for each owl:sameAs path when searching for sameAs statements with a fixed subject and predicate or for ones with a fixed predicated and object.

Rfe10434 - Rename Lisp API function to duplicate-deletion-strategy

The Lisp API function duplicate-deletion-strategy alters the way AllegroGraph manages duplicate triples during index optimization operations. It can be one of:

Previously, this function was called delete-duplicates-during-merge-p.

This setting is also available via the HTTP interface.

Rfe10413 - Add check for 64-bit Linux

Added a check to install-agraph and configure-agraph scripts to ensure that they're running on a 64-bit Linux. This should reduce confusion in the case where AllegroGraph Server was accidentally installed on a 32-bit Linux host.

Rfe10400 - Minor cursor implementation changes

Some changes were made to enhance AllegroGraph's cursor mechanisms. These do not alter existing cursor behavior but help pave the way for future improvements.

Rfe10399 - Verify lisp client and server version match

As the lisp client is very sensitive to changes in the backend API exposed by the AllegroGraph server, it now checks that it is connecting to the correct version of the server.

Rfe10363 - Improved error message when a file cannot be found

The error message generated when a database file is not found has been improved to make it more clear what is causing the problem.

Rfe10354 - Redesigned --quiesce feature of agtestclient

The quiesce functionality in agtestclient was unnecessarily complicated and inefficient. The feature has been redesigned to require running on the host of the secondary server in question and also require the expected count and a timeout. This is sufficient for the purposes for which the quiesce function is used. Expected count must either be queried from the primary or the triple-count must be known beforehand.

A bug was also fixed with the order of the arguments in the delete function. (They had been reversed.) Also, help documentation was created, which can be accessed by running agtestclient with no arguments. Finally, the --add argument was renamed to the more appropriate --add-random.

Rfe10345 - Don't log warnings when opening NFS triple-stores.

Opening a triple-store on NFS caused unnecessary warnings to be issued. These warnings are no longer logged.

Rfe10342 - Command line wrapper for the lubm-generator Lisp module

Note: lubmgen can easily contaminate or erase a production database if used without clear understanding of its purpose.

Rfe10337 - Improve cursor resource management

Improve the management of the cursors AllegroGraph uses internally for triple counting and triple deletion.

Rfe10335 - Improve logging of (adjusted) store parameters

AllegroGraph will now log human-readable configuration values when opening a store if a human-readable presentation is available. (The system determines whether a value has a human-readable presentation or not. That is not under user control. Values which do not have a human-readable presentation are also logged but not in a human-readable way.)

Rfe10308 - AllegroGraph can use UPI-maps to run queries in more cases

Increased the number of times when AllegroGraph can use UPI-maps (secondary indices) to run queries. This improves the enhancements made in Rfe10141 and Rfe10142.

Rfe10243 - opening database should honor existing merge mode as default

It may have been possible that the duplicate removal mode is rolled back accidentally by reopening the database. The problem is fixed.

Rfe10184 - Improve custom hash table efficiency

Improve the way AllegroGraph handles some internal hash table computations. This will provide some very small performance improvements to general AllegroGraph operations. Note that none of these operations occur in inner loops so the performance gains will not be markedly noticeable.

Rfe10114 - Change default for TransactionLogRetain to "recovery"

In prior releases, the default value for the TransactionLogRetain setting was "all", which meant that, unless configured otherwise, all transaction log files since the the database was created would be saved. This release changes the default to "recovery", which means that only the transaction log files required for instance crash recovery are retained. See Transaction Log Archiving.

Rfe9338 - Improved error message due to session port connection failure

When using methods that start a session (setAutoCommit, addRules, and registerSNAGenerator), if there is a connection error, the error message points to the documentation about SessionPorts.

Bug20122 - Handle disk full on transaction log writing

Previously, encountering a full filesystem while writing to a transaction log would result in a crash. Now, AllegroGraph will (if possible) log a message to the logfile and to stderr noting the problem and will periodically retry the write operation until it succeeds.

Bug20118 - graph type-mapping could lead to incorrect data

If a triple was added with a graph field that was mapped to an encoded UPI (for example, a geospatial point or a typed literal) and the object field was not mapped (for example, a plain literal or a resource), then AllegroGraph would fail to correctly store the triple's object. This has been fixed.

Bug20089 - encoded-ids not working in non-object fields for clients

AllegroGraph was failing to translate encoded-IDs in fields other than the object field for any non-direct Lisp client. This has been fixed.

Bug20050 - Improve speed of bulk load mode in agload

agload was incorrectly handling the --bulk parameter which caused it to perform significantly more slowly when bulk mode was turned on. With this change, bulk mode is 6 to 7 times faster than it was before.

Bug20031 - agraph-backup backs up the same file twice

Previously, agraph-backup was backing up one of the database files twice. Though this did not result in a corrupt backup, it was wasting time and space. This problem has been fixed.

Bug19996 - Repair durability guarantees for NFS stores.

The change to implement rfe10216 in AllegroGraph 4.2 accidentally broke durability guarantees on NFS stores: Transaction logs on NFS were opened in an asynchronous write mode, which could lead to commits getting lost in the case of a crash.

Bug19969 - Correct problem printing times with fractional seconds

If a fractional second had a value less than 0.001, then AllegroGraph would incorrectly print the time (or dateTime) using scientific notation. It now prints these values correctly. E.g., it now prints 16:46:40.000133 rather than 16:46:40.1.33e-4.

Bug19954 - Commit/rollback could sometimes result in crash

Fixed a programming error which could potentially cause a commit or rollback to result in a program crash.

Bug19950 - Handling of tranaction log files corrected

In this release the 'recovery' and 'archive' values for the TransactionLogRetain catalog parameter function correctly. Previously these two values were treated as if 'all' were specified.

Bug19933 - Made agload recognize "nq" file format (nquads).

Previously, agload would reject files with extension .nq. Now they are correctly identified as nquad files and loaded.

Bug19916 - Improve :terse output

Modified the heuristic for the :terse output so that it will not emit empty strings. Previously, a URI like 'http://www.foo.com/bar/' would have a terse representation of ''; now it will use 'bar/'.

Bug19836 - Date and datetime with negative year

XSD requires BCE dates, for example, these are now accepted:

(value->upi "-0002-04-12" :date)  
(value->upi "-0002-04-12T12:23:00Z" :date-time) 

Bug19682 - Federated-triple-stores error for type-mappings

Type mappings can now be used with federated stores where all component stores have the same type mapping, for example in methods such as collect-all-type-mappings and get-triples. Function collect-all-type-mappings returns the intersection of mappings for all component stores.

Mappings can still only be set on component stores, not a federated store.

Bug19116 - Datatype-mappings are not applied to graph

When adding a triple, if the graph is a literal with a datatype that can be mapped like object, it will be. Subjects and predicates are not mapped at all.

At most 3 triples can be added, which is the same as before:

  1. with object and graph mapped by datatype
  2. with object mapped by predicate-mapping and graph by datatype if this results in a different triple than #1
  3. with no mappings if :preserve-strings is t

HTTP Client

Rfe10360 - Better HTTP error messages for encoded-id errors.

Stack traces are no longer logged for too-wide encoded-id formats or bad re-registration of such ids.

Bug20124 - Make number of initial HTTP workers configurable.

The number of initial HTTP workers started by the AllegroGraph server used to be exactly the number of backends. However, that doesn't take into account any frontend sessions like WebView. This would cause long-running requests on startup (like opening a triple store) to cause shorter-lived requests to wait until the longer-running request was finished.

AllegroGraph now has a new top-level configuration parameter HTTPWorkers (defaulting to 50), which lets administrators specify how many HTTP worker threads to launch on startup.

Bug20022 - Add location-rewriting to the AG reverse-proxy.

The HTTP reverse-proxy that dispatched requests to server back-ends did not handle redirect responses with Location headers in any special ways. This could sometimes lead to clients being redirected to a backend URL, which did not work. The proxy now properly rewrites the Location header to point at the front-end URL.

Bug20013 - Requests to back-ends with Expect: headers fail

Due to a bug in the HTTP system, sending a request with a body an Expect: header (which CURL does by default) would cause a deadlock in the intra-server communication, and thus a timeout. This has been fixed.

Bug19948 - Allow full HTTP delete-duplicates control

The HTTP API used to expose deleteDuplicates as a boolean value, but there are actually three states: off, delete by SPO, and delete by SPOG. The strings "spo" and "spog" (as well as the existing booleans) can now be used in the HTTP API when inspecting or setting duplicate deletion.

SPARQL

Rfe10486 - SPARQL query planning should not require the PSOGI index

When a query involves filtering on potentially encoded triples (e.g., numeric or date ranges), then the SPARQL algebra query engine checks to see if certain ranges of triples exist during its planning phase. The checks that SPARQL makes requires the POSGI index and are inordinately slow if that index is not available. Previously, AllegroGraph would make the checks regardless. Now, it will now skip these checks if the index is not there.

Rfe10374 - Improve efficiency of some SPARQL queries

SPARQL's notion of equality requires AllegroGraph to examine several variants of some IRIs. For example, the literal "4" should be equivalent to the literal "4"^^xsd:string.

This change improves the efficiency of AllegroGraph's SPARQL equality computations. The performance improvement will depend on the query.

Rfe10371 - Improve SPARQL's handling of DISTINCT LIMIT queries

Improve SPARQL's performance for queries with a LIMIT whose solutions must be DISTINCT.

Rfe10348 - Iri= for subject and predicates (generalize SPARQL constraints)

Extended IRI= constraints for subjects and predicates. This allows queries like:

select * where { ?s ?p ?o . filter( ?s = ex:foo ) } 

to operate as if they were written:

select * where { ex:foo ?p ?o . } 

If multiple constraints are simultaneously operative, then only one will be used for a given pattern. Currently, these will be selected in the order: subject, object, predicate.

Queries for which this optimization is relevant will see greatly increased speed.

Rfe10344 - Improve SPARQL statistical BGP reordering logic

The statistical BGP reordering function could produce sub-optimal results if a SPARQL query had multiple levels of nested UNIONs or OPTIONALs. This could cause queries to take much longer than necessary.

Rfe10312 - Optimize IRI= for object fields in SPARQL queries

This optimization converts IRI= constraints for object fields in SPARQL queries into the equivalent storage level get-triples calls. Previously, a query like

SELECT ?article  
WHERE { ?article rdf:type ?type .  
        ?article swrc:pages ?value  
        FILTER (?type = bench:Article ) } 

would fetch all of the rdf:type triples and then filter for those whose object field was bench:Article. Now the query will be executed as if it had been written as

SELECT ?article  
WHERE { ?article rdf:type bench:Article .  
        ?article swrc:pages ?value  
 }  

Rfe10118 - Added interface to SPARQL query analysis to Python client

After calling RepositoryConnection.prepareTupleQuery() on a SPARQL select query, one can call analyze() on the resulting query object to get a string returned with information about which indices would be used to satisfy the query.

Currently, the query object's method is:

def analyze(self, analysisTechnique=None, analysisTimeout=None):

Analysis is only available for SPARQL queries.  
 
analysisTechnique defaults to "executed", which executes  
the query to perform dynamic analysis. "static" analysis is  
the other option.  
 
For analysisTimeout, pass a float for the maximum number of  
seconds to run the query if executed. 

Rfe9507 - Add initial support for SPARQL subqueries

SPARQL subquery support includes ORDER BY, LIMIT, and DISTINCT. An example query would be:

PREFIX fn:    <http://www.w3.org/2005/xpath-functions#>  
PREFIX boat:  <http://example/boats/vocab/>  
PREFIX boats: <http://example/boats/>  
SELECT ?boat ?boatName  
FROM boats:boats1  
FROM boats:boats2  
WHERE {  
  # Select the 2 boats with the shortest names.  
  { SELECT ?boat WHERE {  
    ?boat boat:placed ?position ;  
          boat:name ?name .  
    } ORDER BY asc(fn:string-length(?name)) LIMIT 3 }  
 
  # Select their crew and the boat's name.  
  ?boat boat:name ?boatName .  
}  
ORDER BY desc(fn:string-length(?boatName)) desc(?boatName) 

Note that subqueries are a pre-release SPARQL 1.1 feature; documentation and conformance tests are not yet available from the W3C, semantics are subject to change, and the implementation might be incomplete in a manner that is visible to user code. Please contact Franz support if you suspect that you have found a discrepancy.

Bug20070 - fix problem with geospatial encoded UPIs and some output formats

If a SPARQL query returned geospatial encoded UPIs and used the SPARQL-XML result format, then an error would occur. The same error could occur when serializing geospatial UPIs in the turtle format.

There was also a problem correctly serializing geospatial output using the SPARQL-JSON format: incorrect latitude and longitude values were sometimes generated.

All of these problems are corrected.

Bug20047 - Can't download query results with missing values as CSV

Taking a SPARQL query that produced unbound variables (say, with OPTIONAL) and asking for a comma-separated value return format would return an error. This has been fixed.

Bug19989 - Sparql CONSTRUCT could generate invalid blank nodes

SPARQL CONSTRUCT using the N-Triples format could generate invalid blank nodes.

Bug19832 - Correct bug in low level SPARQL equality code

Correct a bug in some rarely executed SPARQL equality code. The bug would have caused some queries in some datasets to fail to run.

AGWebView

Rfe10370 - Use only relative URLs in WebView

This makes it possible to set up a reverse-proxy that proxies, for example, host:80/allegrograph/ to host:10035/, without breaking WebView.

Bug20051 - Downloading query results from WebView was broken

After recent patch (bug20022), trying to download query results from WebView always resulted in a 404 response. This fixes that.

Bug20016 - Fix horizontal scrolling in WebView for Firefox

A bad CSS rule was preventing Firefox from ever displaying a horizontal scrollbar for WebView. This made it hard to inspect, for example, query result tables that were wider than the window. This is now fixed.

Bug20009 - 4.2 WebView can't handle some data saved by 4.1

Saved queries saved in 4.1 would cause an error when used in 4.2, because of a change in the way the client-side code is handling JSON data. This change fixes that, by falling back to the old parsing method if the new one fails.

Bug19972 - Webview breaks when it doesn't find JSON support

On older browsers without a JavaScript JSON object, WebView's detection of that object was failing, causing the whole script to abort. It now properly detects, the functionality, replacing it when not present.

Changes to the Lisp API

Rfe10420 - Add lisp client function optimize-indices

AllegroGraph's Lisp client now has a function optimize-indices that will cause the server to rearrange the indices for optimal query performance. This optimization pass can take a long time and cause a lot of I/O operations to be performed on large stores.

Rfe10066 - Add option to use rapper for load-turtle

The lisp API load-turtle now has a :use-rapper-p keyword parameter. See load-rdf/xml documentation for argument usage details.

Rfe9715 - Lisp client package now includes Lisp tutorial examples

The subdirectory tutorial now holds the Lisp and data example files discussed in the documentation.

Bug19672 - Triple-store-exists-p leaks server information

The Lisp client's successful return value from triple-store-exists-p sometimes provided the store's path on the server's filesystem.

The API has been fixed to return only the values nil or t.

Bug19133 - get-triple now applies geospatial mappings

get-triple now correctly applies geospatial mappings to the triple returned. Also, the get-triple now returns a fresh triple (rather than one which might be reused by internal mechanisms).

Documentation

New JavaScript document for programming the server

See JavaScript.

New Security document describing data security

Rfe10341 - Improve documentation search

Added basic documentation searching to the AllegroGraph documentation.

Python Client

Rfe10505 - use session to get new blankNodeIds

AGValueFactory was using the main repository url to get blank node ids. Use connection session instead if available. Using the session can reduce load on the primary server.

If the ValueFactory was created by Repository instead of RepositoryConnection, the old behavior remains.

Rfe10418 - Call indices/optimize from Python

Added an optimizeIndices(level, wait) method to RepositoryConnection for exposing the http indices/optimize POST request to Python client.

Rfe10326 - Support deleteDuplicates setting from Python client

The Catalog.createRepository function now has a deleteDuplicates keyword argument for setting the store's deleteDuplicates behavior on creation. If unspecified, the behavior is the server's default setting.

The store's deleteDuplicates setting is also available from a Repository object instance either as a property variable (delete_duplicates) or as setDeleteDuplicates and getDeleteDuplicates methods. This is a per-store setting.

See the HTTP Protocol - SPARQL Endpoint documention for setting values and behaviors (text search for deleteDuplicates).

Rfe10325 - Added evalJavaScript to Python client

Added an evalJavaScript function to the RepositoryConnection class for evaluating server-side JavaScript code.

See the server's JavaScript documentation for details on programming the server in JavaScript.

Java Client

Rfe10389 - Upgrade to sesame 2.3.2

With this change, the java client distribution has been upgraded to use openrdf sesame 2.3.2, a minor upgrade from 2.3.1 (see Sesame).

Rfe10373 - Support deleteDuplicates mode in java

With this change, the Java Client supports setting a repository's policy for deleting duplicates using methods AGRepository#setDeleteDuplicatesMode and AGRepository#getDeleteDuplicatesMode.

Rfe10372 - Support periodic commit during add/load

With this change, the java client now supports the /statements commit parameter for periodic commits during add/load via the AGRepositoryConnection's setUploadCommitPeriod and getUploadCommitPeriod methods.

Rfe10257 - Support script loading in sessions

Added method to AGRepositoryConnection: addSessionLoadScript(String scriptName)

When a dedicated session is created, the named scripts on the AllegroGraph Server will be loaded.

Rfe10256 - Support loadInitFile switch in java

Added method to AGRepositoryConnection: setSessionLoadInitFile(boolean loadInitFile)

When a dedicated session is created, the initFile on the AllegroGraph Server will be loaded if this param is true.

Javadocs on AGRepositoryConnection reorganized to describe sessions.

Rfe10201 - Support adding N-Quads from java

With this change, the java client now supports adding N-Quads data from Sesame and Jena using the RDFFormat AGRDFFormat.NQUADS and the language "NQUADS", respectively.

Rfe10177 - Support getting statements by id

With this change, the java client now supports getting statements by triple id using the AGRepositoryConnection getStatements(String... ids) method. Note that there is currently no method for obtaining a statement's id in the java client, so this api is for experimental use by developers and subject to change.

Rfe10174 - Support adding Turtle from Sesame and Jena

With this change, Turtle formatted data can now be added to a repository using the Sesame and Jena APIs.

Rfe10059 - SPOGI triple cache management

With this change, the SPOGI-cache can now be managed from the java client using AGRepositoryConnection methods enableTripleCache(size), getTripleCacheSize(), and disableTripleCache().

Rfe9436 - Session lifetime management

With this change the lifetime (maximum idle time) of a session can be set using the AGRepositoryConnection#setSessionLifetime method. The default session lifetime (in effect when not set with the method above) can be set with the AGHttpRepoClient.setDefaultSessionLifetime method. When none of these are set, the default session lifetime is 1 hour.

Rfe8362 - Allow geospatial queries against graph/context field

A boolean useContext parameter can be given to /geo/box, /geo/circle, /geo/haversine, and /geo/polygon, to make them query the context field, rather than the object field of the triples.

Bug19906 - Writing a JavaScript NaN value as JSON loops forever

When running, for example, the JavaScript program 'NaN', and asking for a JSON response, the server would never respond, because the float-printing code didn't handle 'special' float values. We now convert NaN and Infinity to strings in JSON output.

Bug19342 - Java LUBM Prolog and SPARQL queries

With this change, Java LUBM Prolog and SPARQL queries are updated to use distinct in appropriate places and located in the test.lubm package as AGLubmProlog and AGLubmSparql; ant targets lubm-prolog and lubm-sparql have been added to facilitate running the queries from the command line.

Welcome to AllegroGraph 4.2

The new (in 4.2) document Release Notes has information about the 4.2 release. The document you are reading simply lists, for the current and for previous releases, user-visible changes (new features, bug fixes, and other modifications) in an abbreviated format. All the 4.2 changes listed here are also listed in the Release Notes but that document does not list changes in earlier releases. We suggest that users start with that document and refer to this one only if they need information about earlier releases.

Welcome to AllegroGraph 4.2. See Release Notes for general release information. The following, organized by category, are user-visible changes in AllegroGraph 4.2.

AllegroGraph Server: General Changes

Rfe10293 - Warn against using ReiserFS.

As ReiserFS causes long delays on checkpoints, AllegroGraph now warns (in the server log file) if a store resides on ReiserFS. To ease debugging, it also logs the file system types of every store directory now.

Rfe10267 - Do not request duplicate removal if there are no duplicates detected

Internal handling of duplicate removal process is slightly improved. This fix also eliminates excessive "Dupes deleted: 0" messages from the agraph.log file.

Rfe10255 - Allow minimum text-index word size of 1

It is now possible to create freetext indices with a minimum word size of 1 character (the minimum used to be 2).

Rfe10216 - Support alternative tlog synchronization methods.

AllegroGraph now supports a new per-catalog configuration option, TlogSyncMethod, which allows users to specify one of three synchronized writing methods for transaction logs: ODIRECT, SYNC, and fsync.

As before, ODIRECT is the default and the recommended choice on ext3 file systems. For catalogs residing on non-ext3 file systems, the other choices may yield performance benefits.

Rfe10191 - Have install-agraph pass its arguments to configure-agraph

You can now do './install-agraph /dir/ --non-interactive ...' to install and configure agraph in a single command.

Rfe10182 - Improve efficiency when waiting for an in-memory chunk

Previously, when waiting for an in-memory chunk to become available, a commit operation would poll once per second. The polling has been replaced with a wait on a semaphore which will result in immediate notification when an in-memory chunk becomes available. This results in lower average times for commits that must wait for in-memory chunks.

Rfe10173 - Upgrade plugins to support TBC 3.4.0-Beta1

Previously the AllegroGraph plugins were only supported on TBC 3.3.x; with this change, the plugins can also be used with TBC 3.4.0-Beta1.

Rfe10167 - Log euid and runas information at daemon startup.

AllegroGraph now logs which user started the AllegroGraph service daemon, which user it was meant to switch to once running, and the currently effective user ID.

Rfe10162 - Perform a clean shutdown on agraph-control stop

Previously, running the agraph script with the stop argument would not verify that open databases were shut down correctly, leading to a forced recovery from transaction logs upon next startup.

Now, running the agraph script with the stop argument will wait for up to 60 seconds for open triple-stores to shut down cleanly.

To get the quicker shutdown behavior, use force-stop argument or kill the Service Daemon process with the QUIT signal.

Rfe10156, Rfe10157 - Improve lock acquisition and store creation on NFS.

Previously, creating a triple-store left open a very narrow window in which another process could clobber the store parameters. Now, the triple-store is locked immediately on creation. Additionally, newly-created triple-stores can now be cleaned up correctly when created on NFS.

Rfe10149 - Support setting a graph for inferred triples in HTTP sessions

Using the syntax <store>[rdfs++# ] it is now possible to specify which graph should be used for the inferred statements in a reasoning store created as an HTTP session.

Rfe10147 - Export 'custom-service' macro from the db.agraph package

This macro was crudely inserted directly into db.agraph.user before. It is now imported there from db.agraph, like all other AllegroGraph functionality.

Rfe10143 - support bulkMode in connections

With this change, connections can now be put into bulkMode. When in bulkMode, data can be added/loaded more quickly, but there is no guarantee of durability in the event of a crash. Added method AGRepositoryConnection#setBulkMode to set the connection's bulkMode (defaults to false), and added method AGRepositoryConnection#isBulkMode to check the mode.

Rfe10141, Rfe10142 - Use UPI-maps more often inProlog queries

Improved the Prolog query optimizer to use UPI-maps in more cases. Also added a new function, qm, that will use a UPI-map whenever possible. The optimizer changes will help existing queries run more quickly; the new functor will allow more hand-tuned query optimizations.

Rfe10117 - implement run-time (dynamic) query index analysis

Query index analysis now comes in two flavors: the existing :static flavor (from rfe10075) and the new :executed flavor. The later provides an accurate account of the index usage because it actually runs the query while monitoring which indices are touched. Because the query is run, this flavor of analysis can be much more time consuming.

Note that the default analysis technique is to use the new :executed flavor. You can use a different technique by using the queryAnalysisTechnique query parameter. You can also specify a timeout using the queryAnalysisTimeout parameter. See Query Analysis.

Rfe10069 - AllegroGraph v4.0.6 introduced a dependency on a patch to update message digest functions. Now, loading the AllegroGraph client will fail early if this patch is missing, and will report an actionable error.

In addition to the check for the SHA1 patch, AG will now report missing patch names to the user, and offer to install these patches specifically.

Rfe10021 - Support Soundex freetext indices

This adds a supported freetext wordfilter 'soundex', which will reduce words to their Soundex equivalent when indexing or searching. Soundex is described in this Wikipedia article.

Rfe9877 - Lazier collection of FTI results

The triple-ids matching a free-text query will no longer all be accumulated at the start of the query, but (as far as possible) lazily, while triples are being fetched. This should improve performance of limited queries with lots of results.

Rfe9861 - Improve method of sending an x-rdftransaction

Previously, sending an application/x-rdftransaction request required escaping to the client's http layer. With this change, the method AGRepositoryConnection#sendRDFTransaction can now be used; this method is useful for bundling add/remove operations into a single request and minimizing round trips to the server. Changes are committed iff the connection is in autoCommit mode. For increased throughput when sending multiple rdftransaction requests, consider using autoCommit false and committing less frequently.

Rfe9746 - Various improvements to agload

Rfe9657 - Unattended execution of configuration script

The configure-agraph script now has --non-interactive mode, where all its parameters are specified on the command line, so that it can be called from a script without human intervention.

Rfe9631, Rfe10028 - db-get-triples improvements, UPI map usage

Improved the performance of AllegroGraph's internal get-triples functors and extended them to use secondary indices (UPI maps) when they are available.

This greatly improves the performance of many standard queries for both SPARQL and Prolog select. For example, the SP2 benchmark for 10,000 authors goes from a total time of 112.58 seconds to 37.91 seconds.

Rfe9622 - support getting the server's version information

With this change, the client can now get the server's version, build date, and revision information using the AGServer methods getVersion(), getBuildDate(), and getRevision(), respectively.

Rfe9517 - Unify AServe and AGraph messages in the AGraph log

Our HTTP server used to output log messages in its own format. We now route these through the AGraph logging system, causing them to follow a unified format.

Rfe9515 - Drop deleted triples from text index during merge.

The freetext index used to never un-index anything it had indexed. It now leaves out triples that have been deleted when merging index chunks, which will prevent unbounded index growth in stores where lots of triples are being deleted.

Rfe9470 - Reduce madvise calls during compressed index lookup

Small performance improvement during load and query.

Rfe9296 - Transactionally safe duplicate triple elimination

There are improvements in the functionality to remove duplicate triples. Removal is done more eagerly and more robustly. When opening a database, you can pass one of the following values as the :delete-duplicates-during-merge argument:

Rfe8414 - Add encoded-ids to AllegroGraph

Adds support to register and query encoded-ids and to use these to generate sequences of UPIs in a triple-store.

Bug19910: agraph-recover now returns 0 when successful, 1 otherwise

Agraph-recover now correctly returns 0 to shell when successful. (see Point-in-Time Recovery).

Bug19897 - SHM hangs if socket creation fails

Previously, if a socket creation failure occured in the SHM server (which is a critical subprocesses associated with every database instance), database startup would hang. Now the error is immediately reported, allowing AllegroGraph server to otherwise continue operating.

Bug19891 - Fix log level test for log-backtrace and when-debugging.

Error logging no longer ignores the preferred log level for certain messages.

Bug19888 - Prevent logging gratuitous errors on forced shutdown.

AllegroGraph no longer leaves error messages in the server log file when force-terminated with the killall command. (Trying to write such log messages could result in the service daemon trying to communicate with dead processes.

Bug19864 - Some cursors could fail to correctly free resources

There were situations in SPARQL querying where a cursor could fail to correctly release its resources even when it was correctly discarded. This could lead to resource starvation. This is now fixed.

Bug19848 -- Improve behavior of qm when UPI-maps are not available

The qm functor now gracefully drops back to the behavior of the q functor when UPI-maps are not available. Previously, it would signal an error.

Bug19842 - Error message server-pid-and-client-pid-must-match-error

Fix error message when server is running as agraph and client is running as root.

Bug19813 - Fix store parameter merging for configured values.

Opening a triple-store by passing configuration parameters to open-triple-store now correctly overrides those values that are already present in the store's parameters configuration file.

Bug19816 - FTI Merger sometimes produces corrupt chunks

In a database with deleted triples, it was possible for the text-index merger to output a chunk that could not be read, which would bring down the whole FTI subsystem.

Bug19800 - support empty enumerations in owl:oneOf

AllegroGraph's RDFS++ reasoner signaled an error when an owl:oneOf claimed that some class X was one of the empty set. Now it works correctly for this case too.

Bug19791 - Fix checkdb's --port argument.

Previously, the checkdb utility could check triple-stores only if they were served on the default port, 10035. Now, it uses the value from its --port argument to determine the port to connect to.

Bug19786 - Checkdb now exits with 0 on success.

The checkdb utility now follows the unix convention of returning 0 if no problems were detected.

Bug19785 - Convertdb now exits with 0 on success.

The convertdb program now follows the unix convention of returning the status code 0 if there were no errors.

Bug19777 - Replication to multiple secondaries improved

The speed of replication will no longer be determined by the slowest secondary. Each replication job will now proceed as fast as each secondary database can process the data

Bug19766 - Sub-optimal index selection

Previously, in certain circumstances (such as when a range query was being used and one or more of the standard index flavors were not available) the index flavor selection code would exclude some index flavors that it could have used to satisfy a query more efficiently. This has been fixed.

Bug19773 - Dropping an index is not crash recovery proof

Fixed a bug where dropping an index followed by uncleanly shutting down a database would result in an error during restart recovery.

Bug19765 - Detect more cases of string table corruption.

The checkdb program can now detect many more inconsistencies in string tables.

Bug19764 - N-Quad output was incorrect

The output of AllegroGraph's N-Quad serializer when printing single triples was incorrect.

This is now fixed.

Bug19697 - improve performance of part printing

Increased the speed and memory efficiency of part printing.

Bug19657 - Fix race-condition in text-index merger

The merger could sometimes pick the chunks it was to merge from an outdated set of chunks, resulting in chunks being merged multiple times. This would cause duplicate triples to show up in FTI search results.

Bug19600 - direct-upi wire format does not handle nil values

The wire format used to communicate query results to a remote-triple-store broke when given a nil value, though the result of SPARQL queries with an OPTIONAL part may return them. This is now fixed.

Bug19587 - Suboptimal encoding of some characters in shm-queue.

Some characters were being encoded with an extra unnecessary byte. This problem has been corrected.

Bug19578 - Clearnamespaces XMLTransaction tag is broken

There was a bug in the XMLTransaction parser that causes requests containing a <clearNamespaces/> tag to fail. This has been rectified.

Bug19550 - Fix reification in rdf/xml parser

Fixed a bug in which certain statements imported from RDF/XML files would not be reified correctly.

Bug19549 - RDF/XML parser loses the trailing /

When importing from RDF/XML sources, AllegroGraph used to rewrite URIs that came without a path, but with a trailing slash and import them without the trailing slash (e.g., "http://franz.com/" would be imported as "http://franz.com"). Now, it imports all URIs verbatim.

Bug19530 - Fix off-by-one error in text-index code.

Adding a 255-character (the maximum) word to a text index would cause searching and merging of index chunks to break, because a too-small buffer was allocated when traversing the index. This is now fixed.

Bug19522 - Prevent spurious warning in some Prolog select queries

If both :limit and :count-only were specified in a Prolog select query, AllegroGraph would signal a spurious warning. This is now corrected.

Bug19517 - improve Lisp API for remote-triple-stores

Previously, if a :url parameter was used, AllegroGraph would ignore other parameters to create-triple-store like :user and :port that may be inconsistent with :url value. AllegroGraph now ensures that the :url and any other parameters agree and merges parameters into the :url if necessary. This means that you can now use constructs like:

(create-triple-store "foo"  
                     :triple-store-class 'remote-triple-store  
                     :url "http://blade4.franz.com"  
                     :catalog "java-tutorial"  
                     :user "test" :password "xyzzy"  
                     :port 11101) 

Bug19515 - Correct precision loss in printing of RDF time and dateTimes

Certain RDF time and dateTime values with fractional seconds could fail to print all of the digits stored for the value. This patch corrects this problem.

Bug19508 - Prevent text-index from leaking files on crashes

When processes crashed 'hard' (hang, or kill -9), the text index would sometimes leave files around that it never cleaned up. It now takes care to (at some point) delete every file it creates.

HTTP Client

Rfe10278 - Alphabetize store and catalog lists in HTTP responses

The lists of repositories and catalogs returned through the HTTP API were ordered more or less randomly. This made finding a repository in a long list in WebView cumbersome. They are now sorted by name.

Rfe10234 - Inefficient limited Prolog queries over HTTP

The HTTP server used to fetch all results, then throw away the extra ones, when executing a Prolog query with a limit argument. It now only fetches needed results.

Rfe10230 - Accept application/x-turtle as a synonym for text/turtle

Some systems are still using the pre-standard MIME type for Turtle documents. The AllegroGraph HTTP server now accepts that as a content-type when uploading triples.

Rfe10220 - Provide an HTTP interface for controlling bulk mode

It used to be only possible to turn on bulk-load mode per load request. This was slow when using many small requests. In a session, it is now possible to persistently turn bulk mode on and off through requests to /session/bulkMode.

Rfe10210 - Don't backtrace on unsupported-db-version error

The HTTP server no longer logs a backtrace and returns a server-error code when trying to open a store with the wrong version. Instead, it returns a PRECONDITION FAILED error.

Rfe10165 - Expose turtle loading over HTTP.

The /statements HTTP service now accepts request bodies in Turtle format, using a Content-Type header of text/turtle. Such files can now also be loaded from WebView.

Rfe10078 - Make HTTP protocol no longer hide inaccessible repositories.

It used to be that HTTP services /repositories and /catalogs hid all repositories and catalogs not accessible by the current user. This was unnecessarily confusing. Now all present objects are returned, with readable and writable flags indicating whether the current user has access to them.

Rfe10009 - Implement Turtle loading

Loading of files in the Turtle (Terse RDF Triple Language) format is now supported.

Rfe9986 - Support Sesame 2.2-style XML transactions

The Sesame project, which is the origin of the XML transaction format our HTTP server supports, have slightly changed the format to be simpler and more flexible. We now support this update, while remaining compatible with the old-style transactions.

Rfe9912 - Remove SharedMemSize parameter from HTTP protocol and HTTP docs

The SharedMemSize parameter was still mentioned in the HTTP protocol docs, and understood by some HTTP services, even though it is no longer used.

Rfe9910 - Reuse sockets for backend-frontend communication

Use HTTP keep-alive to reuse the sockets that the front-end uses to talk to the back-end.

Rfe9838 - Use HTTP keep-alive in the remote Lisp client.

Lisp remote-triple-stores now re-use their sockets when making HTTP requests.

Bug19909 - Inter-server HTTP communication should use local addresses

When the server tried to address itself over HTTP, it would often use the URL provided in the HTTP request that trigerred this communication. If that came from outside the local network, the address in this URL might not be useable. It now uses a local address.

Bug19871 - HTTP server does not unescape \u in resources

In resources passed to the HTTP server either through HTTP parameters, or through JSON data, N-Triples backslash escape sequences were left in the literals, rather than being unescaped. The server now properly handles these.

Bug19846 - Ensure the HTTP client explicitly frees SPARQL cursors

SPARQL queries from the HTTP client did not explicitly discard the cursor they created. This could cause resource exhaustion in situations where many, many queries were operating simultaneously. This is now fixed.

Bug19801 - HTTP character encoding should not depend on system locale

The system's default character encoding was accidentally being used when writing HTTP responses. This worked fine on systems with a UTF-8 locale, but caused the server to return inconsistently encoded responses on other locales.

Bug19661 - HTTP JSON output for query results truncates row on first unbound result.

The code that wrote out SELECT query results in application/json format would stop writing a row at the first null it encountered, leaving that null and any values after it out of the result.

Bug19616 - Allow multiple contexts to be specified in HTTP /size

Contrary to the Sesame docs, we only accepted a single context argument to our /size HTTP service. This has now been fixed -- you can pass multiple context parameters, and the returned value will be the sum of their sizes.

SPARQL

Rfe10207 - Efficiency improvement for some SPARQL graph queries

Improve the way the SPARQL engine handles simple GRAPH clauses with a variable graph argument that does not appear within the rest of the clause. For example:

select ?someGraph ?o  
where {  
 ex:a ex:inGraph ?someGraph .  
 graph ?g { ex:a a ?o . }  
} 

SPARQL semantics dictate that the query act as if the graph clause is checked for every graph in the dataset. In this case, however, it is clear that the value of ?g cannot alter the behavior of the inner query and that the intent is to determine the graph slots of any of the selected triples.

Note that this optimization is currently very conservative and only occurs when the graph clause is a simple BGP. Future work will extend the behavior to more complex queries.

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing

Nested instances of the SPARQL negation-as-failure idiom were not being parsed correctly leading to incorrect query results. This fix corrects the problem.

Rfe10030 - Share statistical planner between Prolog and SPARQL

Use the foundations of the Prolog select statistical planner in SPARQL queries to reorder BGPs. This improves the speed of many SPARQL queries. Note that this planner requires the POSGI index. If this index is not available, then the coverage planner is used instead.

Because the planner is statistical in nature, it will not always choose the best plan. In general, performance is improved (for example, the 10,000-triple SP2 benchmark queries almost double in speed (from 56-seconds down to 33-seconds) but some queries go slightly slower using the new planner. We will continue to enhance the statistics used and the planner's capabilities in future releases.

If your queries are running more slowly than expected, you can tell SPARQL to use the older coverage-based heuristic planner or tne identity planner (i.e., the planner that does no clause rearranging. Planner selection varies by client. See the client SPARQL documentation for more details.

Bug19809 - fti:matchExpression is sometimes treated like fti:match

There was a bug in the SPARQL code that sometimes caused S-espression-style text queries (as per fti:matchExpression) to be parsed as textual queries (fti:match).

Bug19681 - fix SPARQL problem in queries using both FROM and FROM NAMED

A SPARQL query that used FROM and FROM NAMED together to create the dataset could incorrectly omit some of the the graphs mentioned in the FROM clause. This could cause the query to produce the wrong results.

This problem has been corrected.

Rfe10061, Bug19663 - Improve SPARQL negation-as-failure parsing (repeated from Rfe10061 listing)

Nested instances of the SPARQL negation-as-failure idiom were not being parsed correctly leading to incorrect query results. This fix corrects the problem.

AGWebView

Rfe10300 - Allow optimizing of stores from WebView

There is now an optimize store control in the store overview in WebView. This can be used to optimize a store for querying after a bunch of data has been added.

Rfe10280 - Expose scripting in WebView

There is now a 'Scripts' link in the navigation bar, where one can run JavaScript and (with the right permissions) Lisp scripts interactively, as well as store and edit server-side user script files.

Rfe10277 - Improve help text for WebView part input

The examples given now demonstrate that both backslash-escapes (\n, \uHHHH) and unicode characters are allowed.

Rfe10244 - Add JavaScript server-scripting support

It is now possible to send an /eval request with content-type "text/javascript", and have the body be evaluated as JavaScript. Custom services can also be written in JavaScript.

Rfe10236 - Disable query planner selector in WebView for Prolog queries

To make it clear that the query planner dropdown doesn't apply to Prolog queries, it is now greyed out when selecting a non-SPARQL language.

Rfe10185 - Allow the user to specify a limit for queries in WebView

For heavy queries, the default limit (200) imposed by WebView might be too big. There is now a dropdown above the query input that allows one to choose the amount of results that should be retrieved.

Rfe10181 - Leave WebView widgets in the default browser style

This removes the custom CSS styles for WebView inputboxes and buttons, making the the application look a little more native in modern browsers.

Rfe10155 - Allow a query planner to be selected in WebView

Adds a drop-down menu to the WebView query input page where one can select which planner to use.

Rfe10128 - Rework WebView namespace management

Both the store's default namespaces and the user's namespaces are now managed in the same place. This to reduce confusion---people assumed they could use the namespaces they defined for the store in their queries, without first selecting them.

Rfe10126 - Show PIDs in WebView's process list

Under #processes, the Unix PID of each of the processes is now listed after the process name. The /processes HTTP service now returns tuples instead of just names.

Rfe10116 - Show running sessions in WebView

Once you navigated away, the browser history used to be the only way to get back to a session through WebView. There is now a list of active sessions (if any) in the catalog overview page.

Rfe10039 - Support backup and restore through WebView.

There is now (for superusers) the option to create stores from backups, and to back up existing stores, through the WebView interface. (This is not recommended for large stores.)

Bug19880: Resizing the query input field in WebView is broken

You could drag the resize corner widget, but when you released it nothing happened. This is now fixed.

Bug19868 - Webview tries to write to the installation dir

The server would try to cache gzipped JavaScript files in its installation directory, which fails when it is running under a user that doesn't have write access to that directory. It now uses the settings directory (called the Directory to store data and settings by the configuration program, with default name .../data/) for the cache.

Bug19858 - AGWebView accidentally shows Turtle as export format

We don't currently support exporting to Turtle, so this shouldn't be selectable.

Bug19762 - Fix problem in process page in WebView

A bug had snuck into the WebView code that prevented the #processes page from coming up when in a catalog view.

Bug19741 - Add example to context input field help in AGWebView import

It used to say only that it accepted 'N-Triples' syntax.

Bug19523 - Fix several bugs in WebView's query result display.

Prolog queries returning the default graph now display properly. Prolog queries returning non-flat-lists (such as (?x (?y ?z))) now produce an error message rather than no feedback at all. Literals ending in a backslash are now parsed properly, rather than treated as invalid.

Changes to the Lisp API

Rfe10296 - Check minimum build id on client startup.

The AllegroGraph lisp client will now check if the lisp has all the required runtime patches installed, and will present an error and resolution options if that is not the case.

Rfe7148 - Optimize away keyword scanning in get-triples

db.agraph:get-triples now uses a compiler macro to optimize the calls so no runtime CPU time is spent keyword scanning.

In Lisp if you want to trace get-triples calls, you should declare the function notinline. Alternatively, you can trace db.agraph::get-triples-positional, the non-exported internal function, which the compiler macro uses.

Bug19890 - load-rdf/xml-from-string (Lisp API) could cause memory error

If called with a very long string and no explicit base-uri, load-rdf/xml-from-string tried to create a base-uri by querying the file system with the string. In some situations, this could cause a buffer overflow which would lead to abnormal client termination. This commit fixes the base-uri resolution algorithm so that it handles strings without querying the file system. The underlying buffer problem has also been corrected.

Bug19678 - serialize-rdf-n3 fails with encoded UPIs

The Lisp API function serialize-rdf-n3 would fail if it was asked to serialize triples with any encoded UPIs. This is now corrected.

Bug19069 - improve error handling of N-Triples parser

The continue-on-errorp function to load-ntriples can now specify how to behave when an error occurs more precisely. It can return one of

:continue (the default)
to ignore an error and continue
:stop-parsing
to ignore the erorr and stop parsing the rest of the current source
:error
to signal the error in any case

Documentation

Rfe9657 - server-installation.html document changed to describe non-interactive running of configure-allegrograph.

Python Client

Rfe10221 - BulkMode support added to Python client

Bulk Mode is a repository-wide setting that affects all connected clients.

Use the bulk_mode property on a repository object to set and get the bulk_mode state. Here are some examples.

repository.bulk_mode = True print "The bulk mode state is ", repository.bulk_mode

If you only have a connection object, you can access the property with conn.repository.bulk_mode.

Any commits while repository.bulk_mode is true are done so using BulkMode on the server.

Java Client

Rfe10303 - Support revised bulk-load mode interface

With this change, the Java client now supports the revised bulk-load mode interface. Bulk mode is now persistent (sticks even after closing and reopening the store), and is controlled with the AGRepository#setBulkMode and AGRepository#isBulkMode methods, rather than per session.

Bulk-load mode can now only be controlled through a store-wide toggle (/bulkMode over HTTP), since it is a store-wide setting, and turning it on and off a lot is very slow.

Rfe10197 - Support for encodable-ids and generated URIs

With this change, a namespace (viewed as a set of URIs) can now be registered as encodable. URIs in the namespace have local parts that follow a declared format, allowing for efficient encoding and in some cases generation of unique URIs in the namespace. Once registered, a conformant URI in the namespace is auto-encoded when added to the store, resulting in a smaller string table and faster query speed. (This is a Java client version of the encoded-ids feature, see Encoded IDs for more information.)

Rfe10189 - Java Interface for Stored Procedures

The stored proc feature and API are experimental, and subject to change in a future release. For this reason, the new methods are marked as deprecated.

The AllegroGraph server defines a new API for defining Stored Procedures and they are installed like Custom Services.

The primary API is:

AGRepositoryConnection.callStoredProc(functionName, moduleName, args) 

A low-level API is also exposed:

AGHttpRepoClient.callStoredProcEncoded()  
AGSerializer.serializeAndEncode()  
AGDeserializer.decodeAndDeserialize()  

Rfe10119 - Add analyze() method to provide query analysis

Sesame AGQuery subclasses can now call the analyze() method to obtain a query analysis for the query rather than evaluating it.

Rfe9971 - JAVA API for HTTP protocol checkVariables

Related to bug19157: SPARQL error if ORDER variable isn't in query

Add method setCheckVariables(boolean) to both query classes: com.franz.agraph.jena.AGQuery com.franz.agraph.repository.AGQuery

Rfe9858 - Support ResultSet nextSolution() method

Previously, the legacy ResultSet API method nextSolution() would throw an UnsupportedOperationException; with this change the method is now supported.

Rfe9846 - JENA graph unions and graph-scoped reasoning

With this change, the method AGGraphMaker#createUnion can be used to create a union of named graphs. AGInfModels can now be created over models backed by a union graph, and reasoning will be properly scoped to just the union graph; this change allows sound reasoning over a proper subset of data in the store.

Rfe9534 - Add source jar to the client download

With this change, an agraph-[version]-src.jar file is now included in agraph-[version]-client-java.tar.gz downloads. The download's Eclipse project's .classpath is modified to attach the source jar to the agraph-[version].jar to facilitate browsing and debugging.

Welcome to AllegroGraph 4.1.1

Welcome to AllegroGraph 4.1.1. This release of AllegroGraph provides overall product stability and efficiency improvements, with special attention to improvements in AGWebView and documentation.

AllegroGraph Server: General Changes

Rfe9796 - Make tlog-write-checkpoint-indicator use :allocate-temporary

The checkpoint indicator file, which records the location of the last checkpoint record that has been written to the transaction log file, is now written using the new internal API for creating temporary files. This ensures atomic updates on disk.

Rfe8994 - Change behavior when a filesystem is full

A change has been made to the AllegroGraph server that makes the database server block if one of the file systems is found to be full during a write operation. Operations automatically continue once space has been made available by the system administrator.

Rfe9319 - Freetext index configuration and search support

With this change a free text index can be created by specifying its configuration, and free text performed can be performed w.r.t particular freetext indices.

Rfe9534 - Add sources for jena, arq, and json jars

Previously the source code for these 3rd party libs had to be obtained separately; this change includes them as a debugging convenience in the git repository, and sets up the .classpath to reference them in Eclipse.

Rfe9969 - Update clojure api

AGraph Clojure API updated for agraph v4, clojure 1.2. Build support for leiningen 1.3.1, as well as ant.

Clojure and Java source are in the Git repository: http://github.com/franzinc/agraph-java-client Added README.md for use and development instructions.

Clojure and Java jars are available in the Clojars Maven repository: http://clojars.org/groups/com.franz agraph-java-client and openrdf-sesame-onejar When AGraph v4.1 is released, these will be updated and agraph-clj-4.1.jar added.

Changed license to EPL, same as the java code.

Removed agclj.sh in preference for 'lein repl' or 'lein swank'.

Rfe9999 - Auto-adjust MinMergeChunks

If MergeThreshold is configured to a value lower than MinMergeChunks, the value of latter is now automatically adjusted to the value of the former. This ensures that a merge will actually be done when the threshold is reached.

Rfe10119 - Add analyze() method to provide query analysis

Sesame AGQuery subclasses can now call the analyze() method to obtain a query analysis for the query rather than evaluating it.

Bug19658 - Agload needs bin directory symlink

The symbolic link for the agload program was not being created in the 'bin' directory by install-agraph and the RPM installation. This has been fixed.

Bug19674 - After-the-fact index creation should mind the amount of files it opens

Fixed a bug that caused "too many open files" error and terminating the server when large number of chunks had to be merged.

Bug19698 - Changes to prepareGraphQuery and prepareBooleanQuery

prepareGraphQuery and prepareBooleanQuery now also accept the string "SPARQL" or "PROLOG" as a query language parameter. Previously, only prepareTupleQuery allowed the string values as a convenience in place of the QueryLanguage.SPARQL or QueryLanguage.PROLOG objects.

Bug19720 - Fix "don't attach as root" check

The check which ensures that you don't attach to a triple store as "root" was checking the real uid, not the effective uid, preventing it from working properly under su/sudo. This has been fixed.

Bug19739 - Compressed index files not unmapped when closed

Triple index files which were mapped during queries were not being properly unmapped when they became obsolete. This problem has been fixed.

HTTP Client

Rfe9756 - Return 400, not 500, when trying to delete an open store over HTTP.

The HTTP server was returning (and logging) an internal server error when it tried to delete a store that was still being held open (through a DELETE or PUT request to the store's URL). It now returns a 400 (bad request) response.

Rfe9971 - Java api for http protocol checkVariables

Related to bug19157: SPARQL error if ORDER variable isn't in query

Added method setCheckVariables(boolean) to both query classes: com.franz.agraph.jena.AGQuery and com.franz.agraph.repository.AGQuery.

Rfe10120 - Make sure custom-service exists in agraph4.fasl

This adds the HTTP backend code to the agraph fasl, so that it is possible to compile custom services in an image that has this fasl loaded.

SPARQL

Bug19157 - SPARQL error if ORDER variable isn't in query

When the order by clause has a var not known from the query, ignore it (the default), or identify it in an error message (when sparql-check-variable-errors-p or checkVariables in the http protocol is true).

Bug19618 - Properly escape strings in SPARQL/JSON output.

Newlines and other special characters are now escaped according to JSON rules in SPARQL/JSON results returned by the SPARQL engine.

Bug19619 - Fix typed literal writing in SPARQL/JSON output.

The SPARQL engine was not writing out typed literals in SPARQL/JSON documents properly. Those in UPI form got a wrong type property ("literal" rather than "literal-typed"), those in future-part form were written out as XML.

AGWebView

Rfe9492 - Add export to AGWebview

Adds the possibility to download repositories and query result sets through AGWebView. Several output formats are supported, depending on whether triple data (n-triples, n-quads, rdf/xml, trix) or select results (sparql+xml, csv) is downloaded.

Rfe9587 - Show progress indication in WebView uploads

Rather than just seeing a spinner animation, users doing an upload in WebView now see how much data has been uploaded already.

Rfe9898 - Add drop-down menus to WebView's navigation bar.

To make it possible to link more functionality from the navigation bar without running out of space, groups of links are now put into drop-down menus.

Rfe9900 - Optimize loading of WebView scripts and style-sheets.

Serves the JavaScript and CSS files used by WebView in concatenated form, allowing gzipped responses. This causes the base webview page to go from 25 files totalling over a quarter megabyte to 2 files and 68 kilobyte.

Rfe9950 - Provide a way to start exploring a store in WebView

This adds a paragraph to a repository overview containing links to some simple queries that can help the user explore the store.

Rfe9951 - In WebView user management, show drop-downs for stores.

When granting permissions on a store, the user no longer has to type the names, but can select them from a drop-down menu.

Rfe9964 - Ask confirmation before letting user un-superuser himself in WebView

Once your user loses its superuser flag, it can no longer manage users, so one wrong click here could lock yourself out. There is now a confirmation dialog to ensure you intended to do this.

Rfe9990 - Do not display \uHHHH character escapes in WebView

For non-ASCII characters, WebView showed the N-triples escape sequence instead of the character itself, even when 'short' part display was turned on. This is now fixed.

Rfe10025 - More robust recent-query saving in WebView

On modern browsers, you'll now be able to list your recent queries even after navigating away from WebView and back. On top of that, the list of recent queries is no longer tied to a specific repository.

Rfe10026 - WebView now shows a spinning animation when loading a file

Before, there was no proper feedback indicating that something was happening during a dataset upload.

Rfe10027 - In WebView's graph view, fix nodes after dragging

Nodes can be rearranged by dragging in the graph view. A node now stays fixed after having been dragged, making it much easier to lay out the graph.

Rfe10050 - Make specifying a graph when adding from WebView easier.

The graph input field in the add-triple dialog is now always visible. WebView will try to retrieve a list of graphs that are used in this store, and allow you to choose them from a drop-down.

Rfe10051 - Allow a graph/context to be specified when importing in WebView.

This extends the import file and import URL dialogs with a context field, which is used to specify a graph to import into.

Rfe10052 - Add a general delete-triples UI to WebView

This adds a 'delete statments' control to the repository overview page, from which you can delete triples, by wildcard. Also includes the existing-graph select control from rfe10050, to make it easy to drop a graph.

Rfe10127 - Allow init files to be loaded in sessions created from WebView

There is now a checkbox 'load initfile' in the widget that creates new sessions from a catalog view in WebView.

Bug19581 - Newly registered users in WebView don't get enough access

When user-registration is turned on, new users were supposed to get the same access permissions as the anonymous user, but due to a bug they did not get any access. This is now fixed.

Bug19536 - In WebView's graph view, clicking on a node was broken.

Clicking on a node in the graph did not actually take you to that node's overview page, but raised a script error instead. This is now fixed.

Bug19552 - Can't start a session from catalog page in WebView

There was a bug that would prevent users from starting a session from a (non-root) catalog page in WebView. This has been fixed.

Documentation

Rfe9574 - Lisp client documentation

The AllegroGraph Lisp Client Quick-Start guide has been expanded to include explicit Lisp Client set-up instructions and an explanation of local vs. remote triple-store connections.

Rfe9750 - Improve document navigation

The navigation panel (to the right side of most documentation pages) and the document map page have been revised and extended to provide more organized and thorough access to the many documents and web pages that describe AllegroGraph.

Rfe9883 - Javadoc included in the distribution

Javadocs for com.franz.agraph are in the javadoc folder in the distribution file, and they link to external javadocs for java, jena, and sesame.

Rfe9929 - User manual for 4.1.1 AGWebView

The AllegroGraph documentation has been extended to include a User's Guide for WebView. This document presents task-oriented guidance on using the WebView interface for managing users and repositories.

Rfe9936 - Document TopBraid Composer plugin

In this release the TopBraid Composer (TBC) plugin installation notes got an upgrade with numerous screen-shot illustrations and further notes on how to integrate TBC with the AllegroGraph Virtual Machine.

Rfe10046 - AllegroGraph Text Indexes

There is a new, high-level document describing AllegroGraph's free-text indexing features.

Rfe10077 - Add user-selected triple indices to tutorials

The Python, Java, and Jena tutorials have an augmented example1() showing how to add and remove triple indices for greater efficiency.

Rfe10108 - AllegroGraph Triple Indexes

There is a new, high-level document describing AllegroGraph's system of triple indexes and how to manage them.

Rfe10139 - Emphasize "agraph" user vs. AG superuser

The server-installation document now emphasizes the differences between the "agraph" operating system account and the AllegroGraph superuser account. Users sometimes confused the two accounts, which caused connection issues.

Rfe10159 - Rapper documentation

There is a new document describing the use of the Rapper command-line utility for pre-processing incoming data files to make them load more quickly in AllegroGraph.

Bug19620 - Fix typos in documentation of create-freetext-index

The description of the index-fields argument was a jumble of a sentence.

Welcome to AllegroGraph 4.1

Welcome to AllegroGraph 4.1. This release of AllegroGraph provides overall product stability and efficiency improvements, with special attention to backup/restore, point-in-time recovery, and warm standby features.

Rfe9535 - Use managed files throughout the storage layer

The storage layer has been changed so that all files are allocated through the central resource manager mechanism. This makes all files eligible for placement through FilePlacementRule parameters in the database configuration. Previously, some files could not be placed using such rules.

Rfe9772 - More advanced server scripting

This introduces server-side script files, which can be loaded into sessions, and a convenient way to define custom HTTP services from such files. See the HTTP protocol documentation for details.

Rfe9784 - Keep track of which pages of deleted bitmap have been touched

In order to speed up backup operations, the AllegroGraph server now keeps track of which pages in the deleted bitmap file are used.

Rfe9785 - Revise backup and restore

AllegroGraph now uses a new, single-file archive format for storing backups. The new backup and restore functionality is described in the server documentation.

Rfe9789 - Support callback parameter when writing JSON output

Requests returning JSON data can now specify a callback=[x] HTTP parameter, which will cause the response to be wrapped in [x]([response]), so that the resulting document can be loaded as a JavaScript program (which is useful for working around cross-domain restrictions in browsers).

Rfe9798 - Allow relative directories in config files

In the AllegroGraph server configuration file, relative pathnames can now be specified for file and directory parameters. Such pathnames are considered relative to the directory where the server configuration file is located.

Rfe9805 - Point in time recovery

The point in time recovery facilty allows the a database backup to be restored to its state at a particular point in time.

Rfe9835 - Use stream API to allocate blocks in sparse deleted bitmap

The database server code has been updated to explicitly allocate blocks in the deleted bitmap file using file system calls. That way, full filesystem conditions will be handled in a graceful manner.

Rfe9840 - Improve "disk full" behavior in agraph-backup

When agraph-backup detects a "disk full" condition, it now performs an interactive recovery if it was started from a terminal. When started non-interatively (i.e. from cron), no recovery is performed and an error message is generated.

The exit code of agraph-backup is now always 0 if the operation succeeded, 1 if it failed.

Progress reporting in agraph-backup has been improved.

Rfe9842 - Agraph-backup: merge parameters.dat while restoring

When restoring a database from a backup, the parameters established in the parameters.dat file stored in the backup database are copied to the restored database. File placement related parameters are skipped in this process.

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

When a back-end crashed, this often caused two back-traces to be logged, one for the back-end and one for the front-end. The second one was just noise, so it is now suppressed.

Rfe9889 - Allow resource manager to forget a file

A new internal API function has been added to the resource manager library to support the archival of transaction log files.

Rfe10055 - Properly expose the no-commit flag

The no-commit mode, used primarily by the warm standby replication process, is now something that can be explicitly turned on and off.

Rfe10056 - Require authentication for warm-standby connections.

The socket on which a warm-standby server listens used to be accessible by anyone. It now requires a valid username and password when initiating a connection.

Bug19391 - Agraph-install no longer outputs paths with double slashes

The helper script agraph-install would output paths with double slashes when given a path with a trailing slash as argument. This was mostly harmless, and is now fixed.

Bug19420 - Chunk writers leak disk space when db crashes

A problem has been corrected that could lead to stray files being left on disk when a database was brought down uncleanly.

Bug19446 - Catalog parsing problem - confusing error message

The error message that the configuration file parser generates when overlapping directories are specified in the configuration file has been improved.

Bug19487 - Solve ':void external-format' errors in logs.

The AG log files occasionally showed unexpected errors of 'Attempt to translate characters using a :void external-format'. These were harmless, but they are now handled properly, and no longer logged.

Bug19526 - Back up tlog file only up to backup checkpoint

The backup code has been changed so that transactions which executed while the backup was running are not included in the backup.

Bug19684 - Include the triple-id index in the standard indices

If specific indices are not requested when a store is created, then it will have the standard set of seven: :ospgi, :posgi, :spogi, :gospi, :gposi, :gspoi, and :i.

Prior to this change, the :i index was not included in this standard set.

Lisp Client

Bug19442 - Make sure remote-triple-store accepts :server argument.

The :server argument to the remote-triple-store class was being incorrectly rejected, making it impossible to open stores on other machines.

SPARQL

Bug19474 - Improve SPARQL describe queries

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

AGWebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

This fixes a problem where Internet Explorer 7 would display "Internet Explorer cannot display the webpage" when opening WebView (or any other page served by AG). Internet Explorer fails to handle deflated content, so we now serve it gzipped responses instead.

Rfe9899 - Fix typo and wrong word choice in WebView.

When adding a notable node, 'Desription' was corrected to 'Description'. When looking at a text index that indexes all predicates, it used to say 'None' under predicates, now it says 'All'.

Bug19500 - Fix problem with displaying geospatial results in WebView

When a query result contained geospatial data, a bug in WebView was triggered that caused the result to fail to display. This bug has been fixed.

Bug19525 - Fix failure when opening a federated session in WebView.

Trying to create a federated session in WebView resulted in a PRECONDITION FAILED error. This was caused by a bug in the way session data was being returned by the server, causing WebView to believe it was not on a federated store. The problem has been fixed.

Documentation

Spr37148 - Question about connecting using java

Two lines of code were inconsistent between the Java tutorial HTML file and the tutorial examples code. The tutorial has been edited to correct this issue.

Rfe9783 - AllegroGraph's isolation model

The AllegroGraph Overview document has been augmented with a new section on AllegroGraph's ACID-compliant isolation model.

Rfe9806 - Enhance SPARQL examples in tutorials

In the Python, Java and Jena tutorials, example13() now uses the SPARQL "contruct" to create has-grandchild triples, and then "describes" a grandparent and a grandchild.

Rfe9821 - Make SPARQL examples more robust

Example10() of the Python/Java tutorials has been extended to show examples of SPARQL's FROM, FROM DEFAULT, and FROM NAMED operators, and an example of the UNION operator.

Rfe9831 - Python/Java tutorials, illustrate use of conn.size() with contexts.

Example10() of the Python and Java tutorials has been extended to show examples of the conn.size() method returning the number of triples found in specific subgraphs.

Rfe9915 - Java and Python tutorials demonstrate SPARQL and RDFS++

Example19() of the Java and Python tutorials has been augmented to demonstrate how to use a SPARQL query with RDFS++ reasoning.

Rfe10104 - Point-in-time recovery document

There is a new document describing AllegroGraph's point-in-time recovery feature.

Rfe10105 - Replication and warm-standby document

A new document on triple store replication and warm stand-by has been added to the document set.

Rfe10106 - Update backup/restore document

The backup/restore document has been updated to reflect the new features available in AllegroGraph 4.1.

Rfe10107 - Transaction Log Archiving document

Transaction logs increase without limit unless archived. A new document describes the transaction log archiving feature of AllegroGraph.

Bug19454 - Python tutorial explains how "int" becomes "long"

In example15() of the Python tutorial we show registerDatatypeMapping() requesting a nativetype "int" and AllegroGraph responding with XMLSchema.LONG. This is correct, and the text now explains why.

Document MinimumCheckpointInterval and MaxRecoveryTime parameters

The meaning of these parameters have been documented in the daemon configuration documentation.

Welcome to AllegroGraph 4.0.6a

Welcome to AllegroGraph 4.0.6a. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9837 - Avoid log warning when opening an existing repo

Added method AGCatalog.hasRepository(id) which tests for existance. Added AGCatalog.openRepository(id) which will open but not create. Added AGCatalog.createRepository(id,strict) where the strict option will throw an exception if the repository already exists.

Rfe9980 - getCatalog now works with the root catalog

Added AGCatalog.isRootId(id) to identify id's for the root catalog. Enhanced AGServer.getCatalog(id) to accept root catalog identifiers. Also added AGServer.getCatalog() to return the root catalog.

Rfe9983 - Support the AllegroGraph Lisp Client in ANSI Lisp

Add support to the AllegroGraph Lisp client so that it works in both mlisp and alisp.

Rfe10041 - Implement owl:oneOf

AllegroGraph's RDFS++ reasoner now includes the following inference:

if the class X is owl:oneOf A, B and C  
then each of A, B and C are of type X. 

Rfe10075 - Implement query index use analyzer, version 1

If analyzeIndicesUsed=true is added to a SPARQL HTTP query string, then AllegroGraph will analyze the query to determine which indices it will use. The results are returned as plain text and written to the AllegroGraph log. They will look like

(desired spogi optimal 1 actual spogi optimal 1)  
(desired posgi optimal 6 actual ospgi suboptimal 4) 

Where each line refers to one of the indices required by the query. The line indicates which index flavor the query really wants and which index the query used based on what is available to the store. The additional information indicates whether or not the index choice was optimal, suboptimal or required a full scan. The numbers indicate whether or not additional filtering was required and can be ignored for now.

Bug19650 - Fix problem federating repositories from the root catalog

Previously an exception was thrown when attempting to federate repos from the root catalog; this change addresses that.

Welcome to AllegroGraph 4.0.6

Welcome to AllegroGraph 4.0.6. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9535 - Use managed files throughout the storage layer

The storage layer has been changed so that all files are allocated through the central resource manager mechanism. This makes all files eligible for placement through FilePlacementRule parameters in the database configuration. Previously, some files could not be placed using such rules.

Rfe9620 - Increase limit on number of index chunks

Previously, the maximum number of index chunks was based on the limit on the number of file descriptors (RLIMITNOFILE) which, in a default Linux configuration, resulted in a limit of 100 chunks. Now the maximum is based on the limit on the number of memory mappings (determined by reading /proc/sys/vm/maxmap_count), which usually results in a much higher limit.

If the limit on the number of index chunks is reached, further commit activity will be blocked. Therefore this higher chunk limit will reduce the likelihood of blocked commits.

Rfe9731 - Change transaction log aligment calculation

An internal change was made to the way the alignment for transaction log records is calculated so that opening transaction log files can be defered until records are written.

Rfe9762 - Add someValuesFrom and allValuesFrom restriction reasoning

AllegroGraph's hasValue reasoning has been renamed restriction-reasoning and now covers owl:hasValue, owl:someValuesFrom and owl:allValuesFrom. In particular, AllegroGraph will now infer:

C owl:onProperty P  
C owl:hasValue v  
a rdf:type C  
==> a P v  
 
C owl:onProperty P  
C owl:hasValue c  
a P c  
==> a rdf:type C  
 
C owl:onProperty P  
C owl:someValuesFrom D  
a P b  
a P x  
b rdf:type D  
(and will not infer x rdf:type D)  
==> a rdf:type C  
 
C owl:onProperty P  
C owl:allValuesFrom D  
a type C  
b P x  
==> x type D 

Because the reasoner now implements more than owl:hasValue, it has been renamed the restriction-reasoner. The accessor function has-value-reasoning-enabled has been deprecated and the new accessor is restriction-reasoning-enabled.

The HTTP protocol changes to use restriction instead of hasvalue as the name of the reasoner that supports hasValue reasoning (and the other restrictions).

Rfe9769 - Log merge statistics

In addition to logging how long a merge took, the following additional information is logged: Number of triples read, number of triples written, number of deleted triples expunged, and number of duplicate triples dropped.

Rfe9772 - More advanced server scripting

This introduces server-side script files, which can be loaded into sessions, and a convenient way to define custom HTTP services from such files. See the HTTP protocol documentation for details.

Rfe9776 - Standardize blank node serialization

Use the same blank node syntax in all of AllegroGraph's serializers (N-Triples, RDF/XML, TriX, etc.).

Rfe9789 - Support callback parameter when writing JSON output

Requests returning JSON data can now specify a callback=[x] HTTP parameter, which will cause the response to be wrapped in [x]([response]), so that the resulting document can be loaded as a JavaScript program (which is useful for working around cross-domain restrictions in browsers).

Rfe9803 - Bulk loading support

AllegroGraph now supports a bulk loading mode that allows changing the triple-store without updating the transaction log. Both the HTTP statements service (used to load data into a store) and the agload bulk loader have been enhanced to use this new feature.

Use the bulkMode parameter to specify that you want bulk loading on in the HTTP service

Use the --bulk option to specify that you want bulk loading on when using agload.

Depending on hardware and the actual data, loading with the transaction log turned off is between 10 and 33% faster than loading with it on.

Rfe9812 - Add client API for triple-store index manipulation

Add the following new functions to the Lisp API:

Note that changes made by 'add-index' and 'drop-index' are not made until 'commit-triple-store' is called.

Rfe9889 - Allow resource manager to forget a file

A new internal API function has been added to the resource manager library to support the archival of transaction log files.

Rfe9907 - Add with-indices support to agload

agload now supports custom index selection using the --with-indices command-line option.

Rfe9913 - Improve single-float printing

AllegroGraph could lose precision when printing single float values. For example, 0.0018333651 would print as 1.833365E-3 rather that 1.8333651E-3. This has been corrected.

Rfe9930 - Support adding, dropping, listing repository indices

Added methods addIndex, dropIndex, listIndices, and listValidIndices to AGRepositoryConnection to support managing repository indices.

Rfe9953 - Share transaction log buffer across attachees

Each process (backend) that attached to a store used to allocate its own in-memory buffer for writing to the transaction log.

A single buffer is now allocated in shared memory and used across all backends, saving valuable RAM.

Rfe9966 - Front-end server-socket backlog parameter is too low

The kernel will only save a certain amount of pending connection attempts. If the server doesn't accept them quickly enough, they get discarded. The default backlog size was too small. It is now set to use the maximum the operating system allows, making the chance of discarded connections much, much lower.

Rfe9970 - Upgrade path for configurable indices

Code has been added to convert pre-v4.0.6 databases to be useable with v4.0.6 in terms of configurable indices.

Rfe9972 - Updates to database upgrade code

The internal API for database upgrades has been improved to make more information about the upgraded database available to the code that performs (part of) the upgrade.

Rfe9992 - Backup/restore improvements

agraph-backup now accepts a --catalog command line parameter. Also, agraph-backup now only backs up the used portions of the string table files, resulting in faster backup and restore times.

Rfe10012 - Consistency checker for sstab

A utility has been added to the AllegroGraph distribution that validates the contents of the on-disk string table data structures.

To validate the integrity of a database, the new "checkdb" utility should be used.

Rfe10019 - Setup automatic datatype-mapping for triple-ids

There is now an automatic datatype mapping between triple-id UPIs and the datatype <http://franz.com/ns/allegrograph/4.0/triple-id>. This means that these UPIs will be be read and written in this format. For example, the triple

ex:test rdf:refersTo "18"^^<http://franz.com/ns/allegrograph/4.0/triple-id> . 

will be imported such that its object is an encoded UPI that refers to triple ID 18 using the AllegroGraph specific triple-ID reification.

Bug19171 - Importing RDF files with long text literals is much faster.

AllegroGraph now uses a string output stream to collect characters in text literals instead of string+'ing them together. This speeds up importing long literals with many XML escapes. Files that previously took 4 hours to import now take ~2 seconds.

Bug19181 - Correct newline parsing for n-triple imports

Previously, the AllegroGraph n-triple format parser would count newline and carriage return characters as white space, allowing incorrectly-formatted triples to be imported. Now AllegroGraph allows a triple to be specified on one line only.

Due to this change, the AllegroGraph n-triple parser now reports the correct line number when a parse error occurs. This line number can now be used in a text editor to jump to the location of the problem.

Bug19253 - Accept resources for :base-uri in load-rdf/xml.

Previously, the RDF parser disregarded resources used as :base-uri in the RDF import. Now, it accepts all resources and other future-parts and uses their value to determine the base URI for the import.

Bug19266 - Correct cursor re-use problem

AllegroGraph was aggressively re-using storage layer cursors which could lead clients to point at cursors that were no longer valid. This could lead to incorrect query results.

Bug19391 - Agraph-install no longer outputs paths with double slashes

The helper script agraph-install would output paths with double slashes when given a path with a trailing slash as argument. This was mostly harmless, and is now fixed.

Bug19407 - Allow the direct Lisp client to log to the server log file

The direct client (which is used internally in the HTTP server) can now write to the server log file directly.

Bug19420 - Chunk writers leak disk space when db crashes

A problem has been corrected that could lead to stray files being left on disk when a database was brought down uncleanly.

Bug19442 - Make sure remote-triple-store accepts :server argument.

The :server argument to the remote-triple-store class was being incorrectly rejected, making it impossible to open stores on other machines.

Bug19446 - Catalog parsing problem - confusing error message

The error message that the configuration file parser generates when overlapping directories are specified in the configuration file has been improved.

Bug19460 - Issue installing agraph 4.0.5 over 4.0.4

A bug has been fixed that resulted in errors when trying to open a database that was created with AllegroGraph v4.0.4. This problem has been resolved.

Bug19569 - Java client now helps delete orphaned saved queries

Preparing a Sesame AGQuery currently has the side effect of saving the prepared query on the server in memory for the duration of the session. A client app that prepares many queries during a session thus uses up server memory. If an AGQuery instance goes out of scope, the saved query is no longer needed on the server and is currently only deleted when the session expires. With this change, the client library no longer uses the saved query service during a prepare query, pending further cost-benefit analysis of the saved query service. Client code can explicitly request that a query be saved using an experimental AGQuery.setSaveName() method. The client library also tries to help delete orphaned saved queries sooner by tracking when AGQuery instances are garbage collected on the client and issuing delete requests for the orphaned saved queries in subsequent query requests.

Bug19474 - Improve SPARQL describe queries

Bug19487 - Solve ':void external-format' errors in logs.

The AG log files occasionally showed unexpected errors of 'Attempt to translate characters using a :void external-format'. These were harmless, but they are now handled properly, and no longer logged.

Bug19489 - Directory where pidfile resides is chown'd to agraph user

In prior releases, when running as root, the agraph-control script would chown the directory where the pidfile resides to the agraph user. This was okay when the recommended pidfile path of /var/run/agraph/agraph.pid was used, but if another path that was shared with non-agraph files was used, the chown was undesirable. This problem has been fixed.

Bug19497 - Fix convertdb

As AllegroGraph v4.0.6 changes the on-disk database format, existing databases need to be converted. The conversion process is performed by the "convertdb" utility, which has been brought up to date with respect to recent storage layer changes.

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

Bug19509 - Agload could load inconsistently on virtual machines

Some virtual machines could reveal a fault in agload that caused inconsistent triple loading. This could result in agload reporting a negative triples-per-second load rate and in different amounts of triples being loaded on different runs. This patch corrects the problem.

Bug19539 - Support dynamic catalogs

Support for dynamic catalogs as documented in the AllegroGraph manual has been restored.

Bug19548 - Correctly import default namespaces in rdf/xml literals.

Previously, when importing XML literals that had a default namespace set, the RDF/XML import in AllegroGraph would silently drop the default namespace. Now, the correct xmlns clause is set for these xml literals.

Bug19551 - Fix handling of blank nodes in RDF/XML parser.

Previously, the RDF/XML parser failed to import empty xml elements like <foo x="y"/> by creating references to empty string literals instead of blank nodes like the RDF/XML syntax specification mandates. Now, the RDF/XML parser correctly creates blank nodes.

Bug19560 - Provide sstab upsizing upgrade path

v4.0.5b increased the sizes of the string table files without changing the database version number. Databases created with v4.0.5a or earlier must be upgraded using the convertdb utility to have this problem corrected.

Bug19562 - Improve error message when AllegroGraph loaded on an unsupported platform

AllegroGraph is currently only supported on linux-64 but the error message produced when trying to load agraph4.fasl on a non-supported platform was nonsensical. This patch improves the error message to make it clear what has gone wrong and what remedies to take.

Bug19565 - Crash can occur when file descriptor limit is large

When running as a user with a large file descriptor limit (larger than 1024), it was possible for AllegroGraph to crash under certain circumstances. This problem has been corrected.

Bug19567 - Log handles remain open after SIGHUP

Due to a mistake in the way the daemon detached from the terminal, the initial log file would never be closed, even when a HUP signal was sent to the daemon. This is now fixed.

Bug19595 - Repair createGraph() for anonymous graphs

Previously createGraph() in the no-arg case was trying to use an illegal blank node id based on a Jena-internal AnonID; this fixes that problem.

Bug19602 - String table fails to recover short strings from transaction log

An error has been corrected that could have resulted in the loss of string table data if a database was uncleanly shut down.

Bug19614 - Bus error during prolog query

Some queries that returned AllegroGraph specific UPI types (e.g., for reified triple-ids or default-graphs) could cause a bus error rather than signaling the correct message that these types cannot be serialized using RDF/XML. Corrected.

Bug19632 - N-Quads serializer was emitting the default-graph incorrectly

The N-Quads serializer was needlessly including the graph even for triples in the default-graph of the triple-store. Furthermore, it was emitting these graphs using invalid syntax. Both of these issues are fixed.

HTTP Interface

Rfe9885 - Fix HTTP connection keep-alive for 204 responses.

HTTP requests that caused a 204 (no content) response incorrectly had their connection closed by the server, even when they requested keep-alive. This meant the client had to create a new socket for their next request, causing unnecessary overhead.

Rfe9888 - Do not log backtrace for HTTP proxy pipe errors.

When a back-end crashed, this often caused two back-traces to be logged, one for the back-end and one for the front-end. The second one was just noise, so it is now suppressed.

Rfe9906 - Support index inspection and manipulation over HTTP.

Under /indices, a triple store's indices can now be inspected, created, and dropped over HTTP. Adds an index argument to store-creation requests, to set the indices at creation time.

Bug19506 - Make HTTP server more cautious about closing stores

It was possible for the HTTP server to close a triple store that it was still reading from. This would cause the reading request to fail. The server now waits until pending requests have finished when closing one of its stores.

Documentation

Rfe9936 - User document for Top Braid Composer and AG

Added an installation page for the AllegroGraph TopBraid Composer plugin.

Lisp Client

Bug19452 - Race condition in lisp client startup

While connecting to a database, the Lisp client could sometimes crash with the following error message:

The value of db.agraph.spawn::hub is nil,  
which is not of type db.agraph.spawn::hub 

A fix has been developed to prevent that problem from occuring.

Python Client

Rfe9906 - Add support for index changes to the Python client

A connection object in the Python client now has listIndices, addIndex, and dropIndex methods for inspecting and manipulating the indices on a triple store.

Rfe9965 - Python miniclient now supports retry on ECONNRESET

The python miniclient now retries on connection failures with errno ECONNRESET. Previously it would just fail the request.

Java Client

Rfe9845 - Support owl:Restriction reasoning in the Java client

The Java client now supports hasValue, someValuesFrom, and allValuesFrom reasoning. In the Sesame AGQuery API one can now specify which entailment regime to use when including inferences in query results; the new Restriction regime is now available as an option, while RDFS++ remains the default. Similarly, using the Jena AGReasoner API, one can set up AGInfModels with either a Restriction reasoner or the default RDFS++ reasoner.

SPARQL

Bug19474 - Improve SPARQL describe queries

Bug19499 - Fix SPARQL XML and JSON output of geospatial values

Retrieving SPARQL SELECT results containing geospatial values as SPARQL+XML or SPARQL+JSON was broken because an internals change hadn't been propagated to the SPARQL engine. This is now fixed.

WebView

Rfe9815 - Fix accept-encoding issue for WebView in Internet Explorer 7

This fixes a problem where Internet Explorer 7 would display "Internet Explorer cannot display the webpage" when opening WebView (or any other page served by AG). Internet Explorer fails to handle deflated content, so we now serve it gzipped responses instead.

Bug19500 - Fix problem with displaying geospatial results in WebView

When a query result contained geospatial data, a bug in WebView was triggered that caused the result to fail to display. This bug has been fixed.

Bug19525 - Fix failure when opening a federated session in WebView.

Trying to create a federated session in WebView resulted in a PRECONDITION FAILED error. This was caused by a bug in the way session data was being returned by the server, causing WebView to believe it was not on a federated store. The problem has been fixed.

Welcome to AllegroGraph 4.0.5d

Welcome to AllegroGraph 4.0.5d. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9966 - Front-end server-socket backlog parameter is too low

The kernel will only save a certain amount of pending connection attempts. If the server doesn't accept them quickly enough, they get discarded. The default backlog size was too small. It is now set to use the maximum the operating system allows, making the chance of discarded connections much, much lower.

Welcome to AllegroGraph 4.0.5c

Welcome to AllegroGraph 4.0.5c. This release of AllegroGraph v4.0 provides efficiency improvements in the area of reasoning.

AllegroGraph Server: General Changes

Rfe9887 - Exclude inferences from Sesame queries by default

Sesame queries (Tuple, Graph, and Boolean) will now be set to exclude inferences by default (this applies when the setIncludeInferred method has not been called on the query instance).

Bug19484 - Jena always uses reasoning

Fixed so reasoning is off by default and is enabled when InfModel is used.

Bug19485 - setAutoCommit(true) causes exception

True is the default, but calling this would raise an exception, so this has been fixed.

Bug19491 - Jena reasoning not on full graph

In the Jena API, AGInfModel, reasoning was previously done on graphs (default or named), and is now done on the full repository. Reasoning on scoped graphs will be included in a future release.

Welcome to AllegroGraph 4.0.5b

Welcome to AllegroGraph 4.0.5b. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9866 - Increase limit on string file sizes

Previously, the files that held string data were limited to 32GB each. This release expands that limit to 512GB each.

Bug19503 - Reclaim cursor resources more aggressively

Certain SPARQL operations could acquire cursor resources and not release them immediately. Depending on garbage collection and server load, this could eventaully hang the server.

This patch ensures that SPARQL releases cursors immediately during triple-existence checks which prevents the resource issues from arising.

Welcome to AllegroGraph 4.0.5

Welcome to AllegroGraph 4.0.5. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9334 - License support for agraph with triple-limit and expiration

AG now requires a license code which includes a limit on the number of triples and an expiration date, both of which may be unlimited.

If the license code is missing, invalid, or expired, it defaults to Free version, 5 million triples with no expiration.

Rfe9348 - Added offset keyword argument to getStatements/evalFreeTextSearch

The getStatements and evalFreeTextSearch methods on RepositoryConnection objects now support an optional offset keyword argument for skipping a number of results before returning values from the result set.

Rfe9471 - Limit maximum checkpoint frequency by configuration variable

A new configuration variable, MinimumCheckpointInterval, has been added that can be set to modify the frequency at which checkpoints are written by AllegroGraph. It defaults to the the CheckpointInterval period or five minutes if interval based checkpointing is not used.

The SkipPostMergeCheckpoint configuration variable has been removed.

Rfe9564 - Alternate mode for checkpoint triggering

In addition to triggering checkpoints in regular time interval, AllegroGraph now offers an alternate mode for triggering checkpoints based on the expected recovery time. It is enabled by the MaxRecoveryTime catalog/database parameters and needs to be set to the maximum number of milliseconds that AllegroGraph should spend recovering transactions from the transaction logfile during recovery. When AllegroGraph detects that the established maximum recovery time would be exceeded, a checkpoint is written. Note that the maximum recovery time period does not include the flat startup fee for a database that is independent of the number of transaction to recover from the transaction log file.

Rfe9604 - Suppress unnecessary checkpoints

In previous releases, AllegroGraph automatically created checkpoints when a database was started, stopped or a certain time interval had elapsed. This has been changed so that checkpoints are only written in these situations if there had been any commits since the last checkpoint has been created.

Rfe9608 - Have HTTP /eval service handle multiple returned values

Extra values returned by evaluated forms used to be discarded. The server now attempts to return them as a list of values, if the Accept format supports such a thing.

Rfe9691 - Remove race condition in internal API

In order to support a new use case, a race condition has been removed from the internal resource manager API.

Rfe9709 - Improved automated test for database recovery

Automated test coverage for database recovery has been improved.

Rfe9713 - Document load-nquads

Added the documentation for load-nquads and load-nquads-from-string to the Lisp Reference Guide.

Rfe9726 - Minor agload improvements

Rfe9729: Make it possible to disable interval based checkpointing

Interval based checkpointing can now be disabled by setting the CheckpointInterval database parameter to zero.

Rfe9732 - Disable interval based checkpointing when recovery time based checkpointing is enabled

When recovery time based checkpointing is enabled, AG now switches off interval based checkpointing by default.

Rfe9766 - Ensure that the AllegroGraph server and the Lisp client use the same UID

Because the Lisp client must have full read/write access to a database's files and shared memory data structures, it must run under the same UID as the AllegroGraph server. This change enforces this requirement to avoid potentially confusing errors.

Bug19101 - with-variables->bindings-array is too strict

AllegroGraph's SPARQL engine raises an error when imposed bindings (the :with-variables keyword argument to run-sparql) are not a subset of the variables mentioned in the body of the query.

This error checking failed to include variables mentioned in a CONSTRUCT template or DESCRIBE variable list, which meant that query invocations which imposed bindings for variables mentioned in the CONSTRUCT template alone would raise an error.

This is now corrected.

Bug19168, Bug19332 - Ensure that user functions are compiled correctly

This patch improves the way that functions generated at run-time are compiled. This includes SNA generators and select query plans. These should all perform slightly better with this change.

Bug19209 - Fix bug with reasoner and rollback

In certain situations, the RDFS reasoner could become confused and fail to update its data caches after the triple-store was rolled-back. This is corrected.

Bug19339 - Update UPI serialization

There was a printer escape problem in the N-Triple serializer such that only one backslash was output when two should have been. I.e., \a instead of \a. This prevented serialized triples from being re-imported into AllegroGraph. This problem has been corrected.

Bug19429 - Problems with catalog option FilePlacementRule in agraph.cfg

A defect has been fixed that resulted in an error when a database created from a catalog that specified a FilePlacementRule was opened.

Bug19431 - Wrong parameters logged during database startup

A defect has been fixed that made the AllegroGraph server potentially log wrong parameters to the server log file during database startup.

Changes to the Lisp API

Rfe9643 - Remove synchronize-automatically from the Lisp API

In AllegroGraph 3, synchronize-automatically controlled whether or not a triple-store was immediately synchronized after the addition of new triples. Because this level of control is unnecessary in AllegroGraph 4, the variable is being removed.

Bug19188 - [Lisp API] create-triple-store :if-exists :error problem

The Lisp API function create-triple-store behaved incorrectly if the :if-exists keyword argument was :error. An error would occur whether or not the triple-store already existed. Fixed.

Documentation

Rfe9475 - Document catalog parameters vs database parameters

The AllegroGraph server configuration documentation has been updated to include information on how existing databases can be reconfigured.

Rfe9721 - Restore Lisp freetext tutorial

Adds a tutorial that describes the way freetext indexing works in version 4 of AllegroGraph.

Bug19411 - Configuration and performance documentation updated

The server documentation has been updated to reflect the removal of the 'Aux' catalog directive and the addition of the StringTableDir and FilePlacementRule directives.

Java Client

Rfe9743 - Upgrade to Sesame 2.3.1

Previously the Java client was using Sesame 2.2.4; this upgrades the client to the latest version of Sesame 2.

Jena Client

Rfe9744 - Upgrade to Jena 2.6.2

Previously the Java client was using Jena 2.6.0; this change upgrades the client to the latest version of Jena.

Welcome to AllegroGraph 4.0.4

Welcome to AllegroGraph 4.0.4. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe8792 - Clean up partially created triple stores

When creating a triple store fails, the AllegroGraph server now deletes any files and directories that it may have created up to the point where the error occured. This makes it easier to retry creating the triple store once the condition that made creation fail has been removed.

Rfe9123 - Extend test functions to start and stop db server

The AllegroGraph server can now be stopped by providing the --stop-server switch to the agraph executable. The --config or --pid-file parameters must be present on the command line.

Rfe9292 - AllegroGraph Loader

Added the AllegroGraph Loader tool for parallel loading of large files.

Rfe9367 - AllegroGraph log should include the config file

A change has been made to AllegroGraph so that configuration file directives are written to the server log file at "info" log level during configuration file parsing.

Rfe9451 - Pause text-index merges during checkpoints

We make sure the text-index merger is paused when a checkpoint is taking place, to prevent resource contention from making the checkpoint take long, which would make other subsystems wait.

Rfe9497 - Ensure that select queries release temporary indices

Previously, select queries that used temporary indices (UPI maps) would not free them when the query completed. This could result in poor memory management and eventual poor system responsiveness. This is now corrected because queries release their temporary indices when they complete.

Rfe9508 - Add configuration options for manual control of file placement

Two new configuration options, StringTableDir and FilePlacementRule, have been added to the catalog and database configuration. They can be used to control the placement of individual database files.

Rfe9518 - SHM server handles client failures

Increased robustness of the background shared memory server so that it won't crash when a client shuts down prematurely while communicating with the server.

Rfe9579 - Include catalog name in conditions signaled by the storage layer

Error messages generated by the storage layer now include the catalog name in addition to the name of the database that was being acted upon when the error occured.

Rfe9580 - Use less strict permissions on UPI map and transaction log files

In prior releases, UPI map and transaction log files were created with mode 0600, which prevented them from being read by other users. This made is difficult to share databases. Now, UPI map and transaction log files are created with mode 0666 and the umask is used to determine the final file permissions. This keeps the permissions on these two types of files in line with the other database files.

Rfe9602 - Optimize various UPI coercion functions

Optimized some of the code paths that AllegroGraph uses to convert strings and future-parts into UPIs. This will provide a small performance benefit (1-2%) in some aspects of triple-store loading and querying.

Rfe9642 - Transaction logging adjustments

The recovery code has been adjusted so that automatic recovery after a machine crash is possible if the crash happened while a record was written to the transaction log.

Rfe9642 - Transaction logging adjustments for warm standby

The transaction log now includes marker records for the beginning of a commit that is used by the warm standby mechanism. Existing databases must be upgraded to accomodate for the new transaction log format.

Rfe9654 - Remove Aux configuration parameter

The Aux configuration parameter that was used in previous versions of Allegrograph to make a database use multiple disks for its files has been removed. In order to make use of multiple disks, the TransactionLogDir, StringTableDir and FilePlacementRule should now be used.

Rfe9710 - Make warning about uploading big files through WebView clearer.

Since browsers aren't very good at uploading multi-gigabyte files, WebView makes it clear that there are better alternatives for doing that.

Bug19199 - Lock database directories

When an Allegrograph server opens a database, it will now lock the database directory so that if a second server is configured to use the same directory, it cannot open the database. This protects the database from corruption which would be the result of two servers opening it at the same time.

Bug19203 - Restore the :use-planner option of the select query planner

The :use-planner option of the select query planner was being ignored. It now functions correctly.

Bug19221 - Durability issue related to file deletion

A problem has been identified that could result in database files to be wrongly deleted during checkpoint processing, resulting in database corruption. The problem has been fixed.

Bug19251 - Spurious warning message about nonexistent file ctl0

During database creation, the server logged a spurious warning message about a file "ctl0" that could not be deleted. This has been fixed.

Bug19256 - Database recovery fails with partially written checkpoint

A problem was identified that could cause database recovery to fail when the transaction log contained an incomplete checkpoint record. This problem has been corrected.

Bug19263 - Server log contains cut-off lines

Due to a programming error, it could happen that lines in the server log file were garbled or cut off. This program has been corrected.

Bug19264 - Opening databases is slower than it needs to be

A bug has been found that caused opening databases to be much slower than needed. The reason was that the "df" command was invoked by the database server for every file instead of once per directory. This has been corrected.

Bug19279 - Deleted triple data structure maintenance only happens once

After each merge (but only if 15 minutes has elapsed), a task runs which performs maintenance on the data structures which track deleted triples. Due to a logic bug, this operation was only happening once, even if subsequent merges occurred. This has been corrected.

Bug19282 - Improve parsing of XSD single and double floats

AllegroGraph was parsing XSD doubles using a single float parser which led to some values being incorrectly converted to infinity. AllegroGraph was also treating numbers outside the range of IEEE floating point representations inconsistently. It now always converts numbers smaller than the minimum representable value for a float to zero and numbers larger than the representable value for a float to positive or negative infinity (as appropriate).

Bug19303 - Improve algorithm used to find sort-upi-map program

In some cases, AllegroGraph was unable to find an external program needed for UPI map creation. This patch fixes the problem.

Bug19308 - Make namespaces available to code evaluated server-side.

Code run through the eval-in-server service is now given access to the namespaces that the user defined. It used to run in an environment without namespaces.

Bug19328 - Make remote stores raise an error when created while open

It is now an error to call create-triple-store on a store that is still being held open as a remote-triple-store (in Lisp).

Changes to the Lisp API

Bug19298 - Improve the printing of future-parts

(Lisp only) Future-parts were not printing properly when print-circle was bound to true. In addition, future-parts like !<foo> were not printing correctly (the angle brackets were left off). This patch corrects both of these problems.

Documentation

Rfe9339 - Improve 3-to-4 code conversion document

The Java and Jena sections of the 3.x to 4.x code conversion document have been greatly expanded.

Rfe9549 - Many improvements in the Lisp documentation

Improved and clarified many minor issues in the Lisp reference guide.

Rfe9690 - Documentation for AllegroGraph Loader

A documentation page for the AllegroGraph Loader has been added to AllegroGraph 4.0.4.

Rfe9711 - More detail in Java client installation instructions

The instructions for importing the AllegroGraph Java client into Eclipse have been augmented with more detail to make the process easier for new users.

WWW495 - Add LUBM Benchmarks to documentation navigation list

A link to the web page on AllegroGraph's performance on the Lehigh University benchmark (LUBM) has been added to the navigation bar of the Franz documentation pages.

Text Indexing

Rfe8932 - Allow sorting of freetext-searches by relevance.

freetext-get-triples and the HTTP /freetext service now support an argument that will cause the results to come back in sorted order of relevance, using a simple weighting scheme that takes into account word frequencies in the document and commonness of words across documents.

Rfe9356 - Add fuzzy text-index searching.

Freetext search expressions can now contain fuzzy terms, written as (fuzzy "term" [max-distance]) in s-expression queries, and term~ or term~2 in textual queries, which will match anything with a Levenshtein distance less than max-distance (or 1/5 word length when no explicit distance is given).

WebView

Rfe9504 - Improve handling of WebView file uploads to use less memory.

File uploads through WebView went through a few too many memory buffers, causing them to fail on systems without a lot of free memory. They are now directly streamed, and never stored in memory in their entirety.

Welcome to AllegroGraph 4.0.3

Welcome to AllegroGraph 4.0.3. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe8999 - Use full blank-node IDs in RDF/XML output.

The nodeID properties found in RDF/XML documents created by AllegroGraph can now be sent back to the server to identify the blank node they refer to. They used to be shorter IDs, which were meaningless outside of the document.

Rfe9363 - Revise use of fsync and fdatasync and move to checkpoint time

In previous versions, AllegroGraph used the fsync() system call in various places to synchronize the on-disk state of database files. This could result in unexpected I/O activity when databases resided on ext3 file systems, as fsync() on ext3 always synchronizes all files, not just the file that has been requested to be synchronized. To improve this, AllegroGraph now syncs files only at checkpoint time.

Rfe9490 - increase application's initial heap size

Increase AllegroGraph's initial heap size to reduce garbage collection frequency and thereby improve application performance.

Rfe9505 - Improve efficiency of string dictionary lookup

Increased the efficiency of string lookup by roughly 20%.

Rfe9541 - Server installation script avoids writing double-slashes to agraph.cfg

Previously in configure-agraph, if the user gave a directory name with a trailing slash, the cfg file would be written with double-slashes in various dir names.

Now, the trailing slash is removed to avoid double-slashes. A double-slash is legal in Unix, but is visually unappealing.

Rfe9560 - Truncated cursors free resources more aggressively

Truncated cursors now clean up their environment more quickly which improves general AllegroGraph resource management.

Rfe9567 - Don't print a backtrace on socket timeout errors.

The server no longer logs a backtrace when a socket timeout is raised.

Rfe9586 - Prevent unneeded file reads in metadata-rollback

This provides an increase in efficiency when rolling back a triple store. The metadata system was re-reading its status file on rollback, even when it didn't have to.

Rfe9593 - Optimize string-dictionary

Optimized and stream-lined AllegroGraph's string-dictionary code. This provides a 5 to 6-percent speed increase in many string related operations.

Rfe9598 - Improve efficiency of string table intern operations.

Previously when strings were interned, more work than necessary was being performed if an entry already existed for a string. This extra work is now avoided.

Rfe9665 - Adjusted merge invocation algorithm

Changes have been made to the algorithm which determines when index merges should occur. These changes should make merge behavior more predictable and efficient.

Rfe9673 - Make sort-upi-map operation more verbose

In order to aid in tracking down a hard-to-reproduce bug, the sort-upi-map program now logs information about which files it is manipulating.

Bug19041 - Delete-triple-store only deletes files from the main directory (not the aux)

Previously, if a database used Main and Aux directories and was deleted using delete-triple-store, the files in the Aux directories were not deleted. This has been corrected.

Bug19085 - Fixed race condition in delete-list.dat handling

A file named "delete-list.dat" is used to maintain a list of which files should deleted at the next checkpoint. There was a race condition in the handling of this file which could result in it being deleted at an inopportune time. This problem has been corrected.

Bug19178 - Fix race condition in metadata system

In some circumstances, opening a store or rolling back could cause the system to try to read a non-existent file. This is now solved.

Bug19182 - Performance regression with concurrent writes.

A performance problem has been identified in the mechanism that Allegrograph uses to translate from internally-used logical file names to real on-disk path names. Previously, such lookups were made by a centralized process using a remote procedure call. Now, lookups are solely made within the requesting process without requiring an inter-process exchange.

Bug19192 - Merger may not notice that the database is shutting down

Previously, under certain circumstances, the merger may fail to shut down in an orderly manner during an otherwise normal database shutdown. This problem has been fixed.

Bug19189 - Prevent HTTP back-ends from holding their lock too long

This bug was most visible when using cursors in remote-triple-stores -- the client had to read out the whole cursor before it could make other requests. This is now fixed.

Bug19201 - Catch and log errors raised in the process-services thread.

Errors raised within the per-process thread that managed, for example, back-trace requests were not properly logged, but caused the debugger to be entered. This is now fixed.

Bug19206 - Unrecoverable database due to checkpoint file id reuse

A problem has been identified that could result in an unrecoverable database in situations where there have been unclean shutdowns during recovery. This problem has been fixed.

Bug19222 - Do not fsync temporary files during checkpoint processing

Previously, AllegroGraph called fsync for all files, including temporary files, that make up a database. This has been changed so that temporary files are not fsynced.

Bug19223 - Recovery hangs

A problem has been identified that could cause AllegroGraph to hang during database recovery afer an unclean shutdown. This problem has been fixed by starting the merger processes earlier so that they are available during the recovery process.

Bug19236, Bug19237 - improve geospatial subtype mapping support

The geospatial sub-type mapping code was occasionally being called incorrectly which could lead to spurious errors during query. In addition, the error message reported when a geospaital subtype was not found was being displayed incorrectly. Both of these problems are now corrected.

Bug19244 - Delete old deleted-shmem-files

It has been discovered that the deleted-shmem-files, which are used by the triple deletion functionality, have never been deleted when they became obsolete. This wasted disk space with stale data. The problem has been fixed.

Bug19257 - Deleted hash file deleted at the wrong point in time

A problem has been identified that could lead to the deletion of a deleted hash table file while it was still being referenced. This has been fixed.

Bug19276 - Possible unnecessary post-merge full scan of indexes

After an index merge, a task sometimes runs to perform maintenance of the data structures used to track deleted triples. This operation requires a full scan of indexes. Previously, this full scan was performed even if it was unnecessary. This has been fixed.

Changes to the Lisp API

Rfe9595 - Reduce unnecessary type-checking from Lisp API functions

The exported Lisp API functions (like get-triples or add-triple) were performing unnecessary type checks. This commit removes these checks and thereby increases the speed of code that uses this API.

This change also means that the Lisp API no longer supports using triple-store names to designate stores. That is, functions now require a triple-store instance and will no longer accept a triple-store name.

Bug19173 - Corrected bug in Lisp API for remote-triple-stores

Previously, cursors for remote-triple-stores could produce an error when a query returned too many unique strings. This error will no longer occur.

Bug19177 - Fix broken interpretation of namespaces in Sesame transaction documents

The 'name' and 'prefix' attributes of the addNamespace and removeNamespace tags were interpreted the wrong way. ('name' should hold the URL, and 'prefix' the short name).

Documentation

Rfe9339 - 3.x to 4.x code conversion document added

At the request of users, we have provided a document that lists the changes in the AllegroGraph Lisp, Python, and Java APIs in the upgrade from AllegroGraph 3.x to AllegroGraph 4.x versions.

Rfe9432 - Updates to Python API document

There were several updates to the Python API documnent to better delineate the distinction between public and private methods in 4.0.3.

Rfe9504 - Document TempDir configuration directive

The location where the server creates temporary files can now be controlled by specifying a TempDir option in a configuration file. This option is now described in the Daemon Configuration web page.

Rfe9540 - Audited the Python API doc for currency

The Python API document has been updated to reflect recent changes in the Python API.

Rfe9599 - Update Java section of 3-to-4 conversion document.

The v3-to-v4 code conversion document, convert3to4.md, has been update to enlarge on the information about the new AllegroGraph Java Client.

Bug19320 - Include links to upgrading information in documentation

Added links from the navigation section of each document to the "converting from AllegroGraph 3 to 4" documentation.

SPARQL

Rfe9561 - Improvement in SPARQL recycling of storage-layer cursors.

The SPARQL query engine now more aggressively discards and recycles cursors over on-disk indices. This improves the behavior of the system under load.

Gruff

New overview/scroller panes

The View menu has three new commands for using an overview/scroller pane.

New submenu File | Export Displayed Data As ...

A new "Export" child menu on the File menu allows exporting the currently-displayed triples or nodes to a file in one of several standard formats.

Child menu names are now included in the documentation.

Until now, a menu command that's on a cascading child menu, such as "Drawing Options | Derived Node Labels | Capitalize First Word", appeared in the docs as simply "Drawing Options | Capitalize First Word", which could make it hard to find on the menus. The docs now include the full path.

New freetext command "Add | Edit the Set of Registered Predicates".

There are new commands for registering some or all predicates for freetext queries at one time, and for unregistering some in agraph4.

Rfe9090 - Initial port to the agraph4 freetext API

The freetext commands now work on agraph 4.

Export commands failed when bridge lines were present

The new File | Export commands failed when bridge lines are present. This is now fixed.

Welcome to AllegroGraph 4.0.2

Welcome to AllegroGraph 4.0.2. This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9084 - Return Prolog parse errors as bad requests, not internal server errors.

When issuing a Prolog with a syntax error, the response no longer indicates an internal server error, but is returned as a bad request error, tagged "MALFORMED QUERY".

Rfe9313, Bug18592 - Improved clause reordering in select queries

Modified the clause size estimation code to make it work better with AllegroGraph 4.0. The change greatly increases the speed of some Prolog-based select queries.

Rfe9345 - Ensure that distribution does not contain extra debug scaffolding

We now ensure and test that certain developer tools and debugging aids are not part of the distributed AllegoGraph. Removing these from the released code will have a small performance gain in some parts of the system.

Rfe9351 - Don't write deleted hash table if no triples have been deleted

Previously, the shared memory hash table that is used to track deleted triples was written to a file at every checkpoint. With this change, the table is only written when deletion of triple actually has happened. This speeds up trivial checkpoints as well as database open and close operations by about 35%.

Rfe9404 - Improve error message when bulk loading N-Triple data

The error signaled when loading invalid N-Triple data has been made clearer.

Rfe9435 - Support offset argument to geospatial query services

For consistency, all services that support a limit argument now also have an offset argument, which allows results to be skipped.

Rfe9440 - Improve cursor code efficiency in some situations

Improved some low-level cursor implementation details that provide a small but measurable speed and space boost for some queries.

Rfe9447 - Improve detection of SPARQL Negation-As-Failure idiom.

This change allows for SPARQL's idiom for Negation-As-Failure to be detected more reliably, and thus executed more optimally.

In particular, this allows for the OPTIONAL component to contain FILTERs, so long as they refer only to variables bound within the OPTIONAL itself.

One such query might be:

PREFIX foo: <http://www.example.com/foo/foo_model#>  
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>  
SELECT ?foo ?type  
WHERE {  
   ?foo a foo:Bar .  
   ?foo a ?type .  
   OPTIONAL {  
    ?subType rdfs:subClassOf ?type .  
    FILTER(?subType != ?type)  
   }  
   FILTER(!BOUND(?subType))  
} 

Rfe9448 - Add upi< and upi<= functors to prolog select

Added additional functors to the Prolog-based select language to make some operations easier to use.

Rfe9462 - Improved database shutdown process

The shutdown process that AllegroGraph uses when a database is closed has been improved to be more orderly and reliable.

Rfe9464 - Support adding and removing of namespaces from Sesame XML transactions

A client program that uses the <setNamespace> and <removeNamespace> tags has been shown to exist (TopBraid), so they have been implemented.

Rfe9493 - Added mixed workload to events test

A -m parameter was added to the events test to run a Phase 6 mixed workload after the normal phases complete. This runs a mixture of queries, adds, and deletes until it is interrupted.

Bug19070 - /etc/init.d/agraph 'status' doesn't work

Previously, running "/etc/init.d/agraph" resulted in an error message instead of reporting the status on the agraph service. This has been fixed.

Bug19078 - Improve error message for invalid use of create or open-triple-store

The error message signaled when creating (or opening) a triple-store with incorrect parameters has been improved. The documentation of both functions is also clearer.

Bug19089 - Make sure daemon doesn't hang when it encounters an error

Turns off a debugging switch that was, unintentially, switched on by default, causing the service daemon to enter the debugger when encountering an error.

Bug19090 - Improve handling of graph-filtered-stores

Graph filtered triple-stores are used in various places in AllegroGraph's query engine. Prior to this fix, they did not implement some of the necessary API and could generate an error.

Bug19091 - Prevent server shutdown from locking up when a session is open.

The server would lock up when sent a sigterm if an HTTP session was still open. This is now fixed.

Bug19101 - Error detection for with-variables and CONSTRUCT queries.

AllegroGraph's SPARQL engine raises an error when imposed bindings (the :with-variables keyword argument to run-sparql) are not a subset of the variables mentioned in the body of the query.

This error checking failed to include variables mentioned in a CONSTRUCT template or DESCRIBE variable list, which meant that query invocations which imposed bindings for variables mentioned in the CONSTRUCT template alone would raise an error.

The resolution for this bug corrects the issue.

Bug19121 - Fix bug in namespace caching for HTTP back-ends.

Occasionally (depending on the system clock rolling over to the next second at the wrong moment) a newly defined (or deleted) namespace wasn't being seen by a back-end, causing 'unknown namespace' errors. This is now fixed.

Bug19123 - Allow reasoning-triple-stores to create new-blank-nodes

Reasoning-triple-stores (and other encapsulated-triple-store classes) were unable to mint new blank IDs. This prevented them from loading any RDF data that contained blank nodes. This fixes the problem.

Bug19145 - sendfd library does not handle "out of file descriptors" condition gracefully

When a client process experienced file descriptor shortage while connected to an Allegrograph store, memory corruption could occur. This has been corrected.

Bug19151 - Refcounts on open indexes not updated aggressively

Previously, if regular query operations were performed against a database that was continually growing, file handle exhaustion could occur due to incorrect reference counts on indexes. This problem has been fixed.

Bug19155 - Correct printing of invalid-characters-in-name condition

This fixes a syntax error in the formatting of the condition used to signal an invalid triple-store name or catalog. Previously, the error message produced was difficult to interpret; now it is clearer.

AllegroGraph Server: HTTP Interface

Rfe9227: Allow an HTTP proxy to be configured for the server

The HTTPProxy configuration directive can now be used when the server has to use an HTTP proxy to connect to the internet (for example, to load data from a URL).

Rfe9318 - Remove persistent federations from the HTTP protocol

Accessing federations of stores is now done by opening a session for such a federation, not by creating a persistent federated store and opening that.

Rfe9450 - Allow committing and rolling back through HTTP headers

As a performance optimization, it is now possible to have commit or rollback commands piggyback on other HTTP requests, using the x-commit and x-rollback HTTP headers.

Rfe9458 - Expose a full-featured textual free-text query language over HTTP

The 'pattern' argument to the /freetext service now understands strings like 'foo | bar' (foo OR bar), 'foo*' (everything starting with foo), '"foo bar"' (matching the exact string "foo bar") and even 'quux (foo | bar)' (quux AND (foo OR bar)).

Rfe9458 - Tweak free-text query expression format

When searching from the Lisp API, a phrase search is now done with '(phrase "foo bar") rather than "\"foo bar\"", and a wildcard match is done with '(match "wild") rather than just "wild".

Rfe9483: Add continueOnError parameter to HTTP statement loading

For N-Triples and N-Quads data, this argument can be used to make the load continue when malformed data is encountered, skipping the broken triples.

Rfe9496 - Support clearNamespaces tag in Sesame XML transaction documents

This tag was overlooked in the original implementation. It is now supported.

Rfe9499 - Support prepared queries in the HTTP protocol

It is now possible to 'prepare' SPARQL queries, either as a side-effect of executing them, or as a separate action. The requests for executing the prepared query will be more lightweight, since they don't have to pass in the query itself and most of its arguments, and skip the query-parsing stage.

Bug19088 - Allow a default namespace to be defined over HTTP

Due to the server stripping off trailing slashes from URLs and redirecting, the PUT request to ../namespaces/, which is required to create a namespace without a name, was redirected to ../namespaces, and didn't work. The redirecting behaviour is now turned off for this URL.

Changes to the Lisp API

close-triple-store has new 'commit' keyword

close-triple-store has a new 'commit' keyword argument.If true, then the commit-triple-store will be called before the store is closed.

user-reasoner keyword removed

The deprecated use-reasoner keyword argument has been removed from make-tutorial-store; use apply-reasoner-p instead.

sync-triple-store has been removed

The deprecated sync-triple-store has been removed; use commit-triple-store instead.

Rfe9576 - :expected-size argument to open-triple-store and create-triple-store no longer supported

To specify the expected size of a new database, edit the catalog configuration file before creating the database. To change the expected size of an existing database, change the ExpectedStoreSize parameter in the parameters.dat file of the database.

Bug19046 - Fix error generated by future-part printer

If a namespace mapping did not exist, then the future-part printer could signal an error instead of correcty printing the part. [Lisp API]

Bug19081 - Change return type of ego-group-layers from vector to list

The ego-group-layers function was incorrectly returning a vector of results instead of a list. Its behavior now matches the documentation: it returns a list of lists where each sublist contains nodes discovered at that level. [Lisp API]

Documentation

Improved Lisp reference manual documentation

Made several minor formatting changes and added documentation for the upi<= function and functor.

Document new free-text index HTTP API.

The creation and configuration of multiple text indices over HTTP is now described in the protocol document.

Rfe9169 - Add links to Javadocs from AllegroGraph documentation

Added links from the navigation bar of each AllegroGraph documentation webpage to the Javadocs.

Rfe9314 - Update Lisp API docs to reflect changes in freetext-indexing API

Indexing is no longer managed by keeping a list of predicates, but by creating one or more indices, each with multiple configurable parameters.

Rfe9318 - Update the HTTP protocol documentation for changed federation API

The services for creating and deleting persistent federations are gone, a more flexible way to open sessions, which also allows federation of stores, has been added.

Rfe9377 - Update free-text indexing example in Python tutorial

The 12th example in the Python tutorial now shows how to create multiple indexes and search for both text literals and URIs. The example introduces these new methods:

createFreeTextIndex  
getFreeTextIndexConfiguration()  
evalFreeTextSearch() 

Rfe9378 - Update Python documentation to reflect new federation API

The Python tutorial has been updated to reflect recent changes in the Python API for creating federated sessions.

Rfe9383 - Document new-style federation in the java client

The 16th examples in the Java tutorial has been updated to reflect the new API for creating a federated repository.

Rfe9500 - Update free-text-index entries in Python API html doc.

The connection object methods for free-text indexing have been updated to reflect recent changes in the API.

Java Client

Rfe9318 - Support session-based (rather than persistent) federation

The methods in AGServer for creating and deleting federations have been replaced by a single method, federate, which takes any number of repository objects, and returns a 'virtual' repository that federates all of them.

Rfe9392 - Use accept-encoding gzip to get back compressed results

Get and Post requests now use the Accept-Encoding: gzip header to request that responses be compressed, to improve performance when results are large.

Modified both tutorial examples files to call closeAll prior to deleteRepository in exampele1, as not doing so can result in orphaned sessions on the toClose list.

Rfe9436 - Make session lifetime configurable

Session lifetimes can now be configured for each connection, and should be set prior to making a method call that initiates a dedicated session. The default session lifetime is 1 hour. Tutorial example 22 sets the session lifetime to 120 seconds to demonstrate.

Rfe9442 - Add basic information to the jar manifest

The Java client jar manifest now includes information about the current implementation, including name, version, vendor name, vender URL, time of creation, etc.

Rfe9499 - Support query preparation

Calls to prepareTupleQuery, prepareGraphQuery, and prepareBooleanQuery now return query objects that will, during the next call to evaluate(), request that the server save the query for future reuse (in dedicated sessions only), eliminating some parsing and other overhead in subsequent calls to evaluate(). Tutorial example 14 has been modified to use non-autocommit mode so that it demonstrates query preparation in a dedicated session and multiple calls to evaluate() with different variable bindings.

Bug19086 - JenaTutorialExamples now uses Graph#getName()

The tutorial examples now use Graph#getName() to get the graph's name, rather than using Graph#toString(), which now prints out additional human readable information about the graph.

Bug19092 - Jena ResultSet and PrefixMapping compliance

AGPrefixMapping now supports methods qnameFor, removeNsPrefix, samePrefixMappingAs, setNsPrefixes, shortForm, expandPrefix, lock, and withDefaultMappings. The toString method now returns human readable information about the prefix mapping.

AGResultSet now supports methods getResourceModel, getResultVars, and remove.

AGQuerySolution now supports methods contains, get, getLiteral, getResource, and varNames.

Python Client

Rfe8610 - Help finding pairs of UPIs

The 23rd example in the Python tutorial has been extended to demonstrate how to use inequalities to reduce the number of permutation matches in SPARQL and Prolog queries.

Rfe9311 - Expose revised FTI API in the Python client

Text-indexing is no longer configured by adding predicates, but by creating (and updating or deleting) multiple separate indices.

Rfe9318 - Move to session-based federation API.

To federate stores, one now calls AllegroGraphServer.openFederated, which opens a session that federates the given stores, rather than creating a persistent federated store and opening that.

Rfe9357 - Support wordFilter option to text-indices in Python client

The createFreeTextIndex and modifyFreeTextIndex methods of a connection now allow a wordFilters argument to be specified, which allows accent removal and word stemming to be configured.

Text Indexing

Rfe8775 - Support multiple text-indices per repository, add configuration parameters.

Multiple free-text indices can now be active in a repository, and there is an API to create, query, modify, and delete them. New parameters supported are: stop words, minimum word length, whether to index resources, which literals to index, indexing non-object parts, and the possibility to index all triples, regardless of predicate.

Rfe8775 - Add triples to free-text index as soon as they are added.

A commit used to be necessary to be able to find newly added triples through the free-text index. They can now be queried (by the current transaction) as soon as they are added.

Rfe9357 - Add word-filters to text-index configuration (accent removal, stemming)

It is now possible (both in the Lisp API and over HTTP) to specify a list of word filters when defining a freetext index. These are operations that are performed on every word that is indexed, and every word that is queried, allowing various 'normalisations' of words. Currently allows two filters, "drop-accents" to remove accents from letters, and "stem.english" to apply basic, English-language word stemming.

Changes to WebView

Rfe8775 - Allow free-text reconfiguration through WebView.

WebView now allows the creation, modification, and deletion of free-text indices, exposing a UI for all the new configuration parameters.

Rfe9408 - Expose free-form sessions in WebView

There is now, for users with the "session" permission flag, a control on the main WebView page for starting and entering a session over a federated, reasoning, filtered, and/or remote store.

Rfe9438 - Show server version in WebView

WebView now shows the exact version of the server in its header.

Rfe9453 - Support bulk-adding of namespaces in WebView

Clicking 'bulk input' when adding a namespace now pops up a new input dialog, where a set of namespaces can be typed or pasted all at once.

Bug19105 - Restore Safari-compatibility for WebView

WebView was broken on recent versions of the Safari browser. This has now been remedied.

Welcome to AllegroGraph 4.0.1

Welcome to AllegroGraph 4.0.1. This version of AllegroGraph contains updates to the AllegroGraph 4.0 release, including:

This release of AllegroGraph v4.0 provides overall product stability and efficiency improvements.

AllegroGraph Server: General Changes

Rfe9137: Heuristic optimization for negation as failure in SPARQL queries.

SPARQL queries which use the common OPTIONAL/FILTER (!bound(?foo)) idiom to express negation as failure are now handled more efficiently, computing fewer unnecessary intermediate results.

This optimization can be disabled by setting the non-exported variable sparql.algebra::rewrite-naf-p to nil.

Rfe9320: Improved caching of RDFS++ reasoning results

Improved RDFS++ reasoner efficiency by selectively caching intermediate results. This makes many queries involving the reasoner between 2 and 10 times faster.

Rfe9352, Bug19067, Bug19068: Improve internal cursor operations

Optimizing how AllegroGraph treats empty cursors (i.e., cursors known to return no results) so that operations with them are faster.

Rfe9374: Improved SPARQL range-query efficiency in some cases

Improved the detection of range queries that will return no results which allows SPARQL to optimize other cursor operations.

Bug18978: Optimizing cursor creation and filtering

Improved the way AllegroGraph internally manages cursor creation and triple-filtering to increase performance.

Documentation

Hardware Profiles added to Performance-Tuning Guide.

The performance-tuning guide has a new section describing the kind of hardware Franz recommends for users who will be deploying applications on the scale of 3-4 billion triples.

Rfe9374: SPARQL query performance tuning documentation added.

The performance tuning guide has been extended to describe possible approaches to handling typed literals to improve SPARQL query performance for numeric and date FILTERs.

Welcome to AllegroGraph 4.0

Welcome to the new AllegroGraph v4.0. This latest version of AllegroGraph is designed take full advantage of symmetric multi-processing hardware and brings enterprise database features to the world of Semantics and RDF. AllegroGraph, now incorporates:

This release of AllegroGraph v4.0 is focused on enhancements to database backup and restore, Java Jena client library compliance, and overall product stability improvements.

AllegroGraph Server: General Changes

Add a Lisp function to list the catalogs on a server.

The function #'catalogs, exported from the db.agraph package, can now be used to find out which catalogs a server provides.

Added ego-group-layers function

ego-group-layers returns a nodes ego-group organized by the depth at which each node was found. [Lisp API]

Documentation: ego-group-layers (node depth generator)

Return a list of lists of the nodes in nodes ego group.

Each element of the result is a list of the nodes discovered at that list's depth.

These are the nodes in the graph that can be reached by following paths of at most length depth starting at node. The paths are determined using the generator to select neighbors of each node in the ego-group. See the description of SNA generators for additional details.

Added support for auto-commit to the bulk load functions

The commit parameter to load-ntriples, load-rdf/xml and load-trix can be one of

Changed behavior of breadth-first-path-search when maximum-depth is given.

Previously, when the breadth-first-path-search functions were given a maximum depth, AllegroGraph would return the shortest path or paths found that were no longer than that depth. Now, AllegroGraph returns all paths found that are no longer than maximum depth.

This can be very expensive for graphs with high connectivity. To return only the shortest path, use a maximum depth of nil. [Lisp API].

Since the Prolog SNA functors use the underlying SNA functions, their behavior has also changed.

Change the keyword value used to identify typed literals

Previously, AllegroGraph used :typed-literal to specify a typed literal. To make things more consistent with :literal :literal-language, this has been changed to :literal-typed. [Lisp API]

Define an error type to signal failures to connect to the AllegroGraph server.

Any operation (opening, creating, deleting, probing) that has to talk to the server will now raise a condition of type db.agraph:could-not-connect-to-server-error if it fails to find a server on the specified port.

Fix mistake in instance-subprocess communication protocol.

This would cause the instance to go into an infinite loop and become unresponsive when one of its children died.

Make delete-duplicates-on-merge setting persistent.

The flag was not properly restored when opening a triple store.

Rfe7381 - Add a db keyword argument to several API commands Added the db argument to the following commands

[Lisp API]

Rfe8765 - Improved Prolog query performance

Many Prolog queries execute more than 30% more quickly.

Rfe8860 - Geospatial functions now accept strings

The geospatial functions in AllegroGraph previously required a subtype object. They now accept strings which name a subtype object. This makes them easier to use. [Lisp API]

Rfe9146 - Allow 'merging' of store metadata on concurrent change.

Before, two sessions both adding a type-mapping and then committing would cause a conflict error. The metadata system is now clever enough to merge such changes.

Rfe9209 - Don't log process termination messages before cleanup.

Both instance processes and their children used to output '... terminated' before they cleaned up, which in some cases led to them hanging or crashing after they claimed to have terminated.

Rfe9212 - Periodically clean up deleted triple data structures

Previously, the data structures which track deleted triples could grow without bounds. Now, they are periodically examined and obsolete portions are purged.

Rfe9291 - Use XSD instead of internal type-codes for /mappings

The /mapping protocol now returns lists of

mapping-kind datatype/predicate XSD-type 

rather than

mapping-kind datatype/predicate numeric-type-code 

This makes both debugging and general use of the mappings much simpler for clients.

Rfe9294 - Added SNA path functors to bind only the first matching path

Added breadth-first-search-path and bidirectional-search-path functors to correspond with the existing depth-first-search-path functor. These all run SNA path finding algorithms and stop after finding the first matching path.

Rfe9315 - Do early probe to see if file system supports >4GB file offsets

Allegrograph requires that the underlying file systems support offsets longer than 32 bits. Some file systems (i.e. NFSv2) do not support long offsets, which caused file system related failures in varying situations. Allegrograph now probes each of the file systems specified in a configuration to make sure that long file offsets are supported.

Rfe9322 - Changes in encoded date and time UPIs.

Corrected a problem where the fractional seconds stored in an encoded UPI could be lost. The update fixes the stored representation and invalidates existing triple-stores that have any time, date or date-time encoded UPIs.

Rfe9329 - Improved loading of a required shared library.

Previously, AllegroGraph required loading of a particular shared library (librt) that did not always exist by the same name on all Linux distributions. This problem has been corrected.

Rfe9330: Set up selinux security context on shared library during installation

Previously, on systems with SELinux enabled, the agraph server would not start up properly unless the operator issued the following command first:

chcon -t textrel_shlib_t <agraph lib dir>/libacl*.so 

Now, this command is automatically performed during RPM package install or when using the install-agraph script that comes with the tarball package.

Rfe9340 - Disallow overlapping catalog directories.

Having, for example, one catalog in /tmp/foo, and another in /tmp/foo/bar, or two catalogs in the exact same directory, is no longer allowed, since this is bound to cause stores to be corrupted.

Rfe9362: Output a log message when deleting a triple-store.

Deleting a store wasn't visible in the log before. It now notes which store was deleted.

Spr36556 - Reduce number of duplicate rdf:type inferences

AllegroGraph's type inference was returning a triple for each possible inference path rather than filtering out the duplicates. I.e., if a rdf:type B could be inferred two ways, then the reasoner would return two triples. This no longer happens.

Spr 36565 - Standardize the printing of encoded types

AllegroGraph would sometimes print typed literals non-syntactically. This no longer occurs.

Bug18679 - Made part->value function more consistent

The part->value function now returns type-codes for both UPIs and future-parts. Previously, it returned a type-code for UPIs and a keyword (like :literal or :anon) for future-parts. [Lisp API].

Bug18679 - Removed the deprecated part-value

The deprecated part-value function has been removed. Use part->value in its place. [Lisp API]

Bug18738 - Make query-planner work on remote-triple-stores

Previously, select queries that needed to reorder their clauses would instead signal an error on remote-triple-stores. This patch corrects this problem and lets remote-triple-stores work correctly with the query-planner. [Lisp API]

Bug18840, Bug19003 - Inconsistent state after duplicate purging

Total triple counts were wrong after duplicate purging. By default, AllegroGraph purges duplicate triples from the index during merge operations. Previously, when duplicate triples were purged, the total count of triples in the store was not updated. Also, in most cases, one of the seven triple indexes would not be purged of duplicates, resulting in wasted storage and potentially inconsistent query results. These problems have been corrected.

Bug18879 - Corrected add-triple so that it could accept large strings

Unlike the bulk loaders, the add-triple function was unable to accept strings longer than 8092 characters and would signal an error. Now add-triple can accept strings of arbitrary size.

Bug18800 - Unbound prolog variables could pollute select query results

Queries like

(select (?s ?t)  
 (q ?s !<http://www.franz.com/sna#marriage>  
!<http://www.franz.com/sna#peruzzi>)  
 (optional (q ?s !rdf:type ?t))) 

could return unbound expessions like `{Unbound 14a170c9}` instead of `nil`. Now these queries correctly return `nil`.

Bug18880 - Corrected serialization of RDF/XML

The serializer sometimes added extra angle brackets around typed literals. Now it correctly outputs them as, e.g.,

"42"^^<http://www.w3.org/2001/XMLSchema#int> 

Bug18947 - Make service daemon responsible for deleting stores

delete-triple-store used to only check whether the store was open in the current image, and thus deleted stores even if another process had them open. This moves the responsibility for deleting stores to the service-daemon, which actually knows whether anyone has the store open.

Bug18949 - Fix geospatial type mapping in federated stores

AllegoGraph was not handling geospatial type-mappings in federated-triple-stores and could fail to return all of the triples from every leaf store. This is corrected.

Bug18972 - Correct printing of times with fractional seconds

When a time or dateTime with fractional seconds had leading zeros in the fractional seconds part, AllegroGraph could fail to print it. I.e., a time like 10:31:55.028192 might print as 10:31:55.281900. This printing problem has been corrected.

Bug19010 - Fix 'store in use' bug when accessing store over HTTP.

Fixes a bug where a store accessed over HTTP could not be deleted or replaced through the direct Lisp API until its handle had timed out.

Bug19018 - Make sure instance processes are cleaned up when breaking

Fixes a few cases where unexpected errors would cause the daemon to get confused about the stores it had open (instance crash, client dying while opening or creating).

Bug19024 - Show catalog name in store-status error messages

Errors like 'store exists' and 'store not found' used to only contain the store name, now they also have the catalog. Trying to perform an operation on a non-existent catalog now raises an error that clearly indicates the catalog doesn't exist.

Bug19045 - Use . as the fractional seconds separator in times and dateTimes

Previously AllegroGraph used the comma (,) as the separator in its printed representation of XSD times and dateTimes. This is acceptable ISO 8601 notation but not acceptable XSD notation. This bug fixes the problem so that fractional seconds are separated from whole seconds by a period (.).

Bug19055: Fix potential hang when committing

Under certain circumstances, commits could hang indefinitely while waiting for chunk merging to complete, but chunk merging would never commence due to a counter having an invalid value. This problem has been corrected.

Bug19061: upi->String table race condition fix

Previously, under high load conditions, errors could occur during string table operations. This problem has been fixed.

AllegroGraph Server: HTTP Interface

Allow nested lists in JSON-shaped Prolog-query output.

application/json results for Prolog queries can now contain nested arrays of results, to make it easier to use some of our SNA functionality.

Allow the creation of sessions with federated, graph-filtered, or reasoning stores.

It is now possible to spawn HTTP sessions on a store built up by applying an arbitrary combination reasoning, federation, and graph-filtering to concrete triples.

Cause output of --http-trace switch to actually get flushed to the file.

Before, it was buffered, and would only be visible after a large amount was written, or the server was shut down.

Clean up namespaces when a repository is deleted.

Before, deleting and re-creating a repository would leave the old namespaces intact.

Enable streaming of XML bodies when loading statements.

A problem in our SAX parser made this difficult before, but that has since been fixed. This makes it possible to load large files with less overhead.

Fix a bug that prevented session permissions to show up in WebView user-contol UI.

(No further text.)

Fix a bug with namespace caching.

Sometimes, back-ends wouldn't notice a user's namespaces had changed when they were updated twice within the same second.

Fix bug in remote geospatial queries.

There was a problem that broke remote-triple-stores when more than one geospatial subtype occurred in a query response. This is now fixed.

Fix user-access bug in HTTP protocol.

Fix a bug that prevented users with access to a store in the root catalog, but not the whole catalog, from accessing this store over HTTP.

Make user-access-configuration interface in WebView more obvious

It was easy to mistake the widget for adding access for an indication that access had already been granted.

Support changing user passwords through the HTTP interface

There is now /users/[NAME]/password, to which one can POST a new password.

Support resetting namespaces to the defaults.

There is now a reset argument to the namespace clearing interface, which, when true, will cause the default namespaces to be restored.

Rfe9131 - Add defaultGraphName parameter to SPARQL query interface

This is used to provide a way to refer to the default graph inside a SPARQL query using a URI.

Rfe8999 - Use a unified format for serializing blank nodes.

Before, each component had its own format. This moves the HTTP server to the unified format.

Rfe9183 - Silence AServe's socket-reset warnings

The log is no longer polluted with spurious connection-reset messages.

Rfe9185 - Support delete-duplicates-during-merge remotely

Remote-triple-stores now support the :delete-duplicates-during-merge initarg, and the db-delete-duplicates-during-merge accessor.

Rfe9217 - Add more help text to the WebView interface.

Potentially confusing input fields in WebView (such as those where resources or literals are entered) now have a question-mark icon that pops up an explanation.

Rfe9236 - Support SPARQL/Update over HTTP.

POST requests to a repository can now issue a SPARQL/Update query, if the user has write access to the store.

Rfe9247 - Expose server processes in HTTP interface

It is now possible to list, inspect, and kill the server's processes through the HTTP interface. Additionally, one can start up telnet servers in the processes, in order to directly debug them.

Rfe9247(2) - Allow process control from WebView

Superusers can now view the server processes, and open telnet servers in them, through the WebView UI. (See the 'Processes' link in the navigation bar at the top-level page.)

Rfe9295 - Add a commit argument to PUT/POST /statements.

This supports auto-committing every X triples, to make uploading huge files viable.

Rfe9310 - The PUT /repository/[name] now overwrites by default.

The PUT /repository/[name] service in the HTTP protocol now overwrites existing repositories by that name unless override=false is passed.

Bug19002 - Fix bug in HTTP protocol namespace caching.

Fix a bug that caused problems with namespaces after deleting and re-creating (over HTTP) a store that had namespaces defined.

Bug19005 - Corrected problem in client lookups of individual datatype mappings.

Previously, a request for an individual datatype mapping would return nil regardless of how the mapping was defined. Now it returns the correct value.

Documentation

Server Installation Page Updated to be Ubuntu-Friendly

The server-installation instructions have been updated to make them more friendly to people who don't know how to install Python, python-pycurl, and python-cjson packages. This included some specific suggestions for Ubuntu users for RPM installation and Python client installation.

Rfe9322 - Added client tutorials to the documentation

The documentation now contains tutorials on how to use the Python and Java clients. Please see the links off of the main web pages.

Java Client

Expanded Coverage of Literal Values in Java Tutorial.

The Java tutorial has been revised and expanded to document the behavior of literal typed values (in example5()). This includes the default behavior of untyped literals, as well as their use in getStatements queries as well as filtered and direct SPARQL queries. The section covers strings, floats, ints, untyped values, dates and times.

Java Tutorial Describes Managing User Accounts

The Java tutorial has a new section on "Creating AllegroGraph Users with WebView." It describes how to create a manage new AllegroGraph user accounts.

Rfe8812 - Jena compliance improvements

Added compliance tests for Jena GraphMaker and Model interfaces and corrected several non-compliances.

Properly handle the case of getting a namespace uri for a prefix that does not exist, and add support for expanding a QName.

Add GraphMaker support for creating anonymous graphs, listing graphs, checking existence of a graph, and returning existing graph objects rather than creating new ones when possible.

Throw AlreadyExistsException when attempting to create an existing graph in strict mode, and throw a DoesNotExist exception when attempting to remove or open a non-existent graph in strict mode.

Rfe8812 - Jena Graph compliance improvements

Added compliance tests for the Jena Graph interface and corrected several non-compliances.

AGBulkUpdateHandler's removeAll method properly clears a graph.

Added a custom AGCapabilities class declaring handlesLiteralTyping to be false, as AGGraph does not currently support D-entailment.

Add support for the executeInTransaction method in AGTransactionHandler, though using begin/commit/rollback explicitly is preferred.

AGTripleIterators are now tied to AGGraphs, calling remove on the iterator will delete from the graph.

Rfe9359 - Jena Prolog support.

Added support for Prolog select queries over a Jena model. Queries must be written to select for variables that are bound to RDF parts, rather than to arbitrary Lisp objects.

Python Client

Removed Common Logic

Removed the experimental and undocumented client side Common Logic query language. Please contact support@franz.com for information if you were using this language.

Removed JDBC result sets

Removed the ability to get JDBC result sets. Please contact support@franz.com if you need ways to get the result data without instantiating the corresponding Literal, URI, or BNode objects.

Namespace implementation is now server-side

Removed the client-side namespace implementation in favor of a server-side namespace management.

"Duplicate Triples" added to Python Tutorial.

The Python Tutorial has a new section on "Duplicate Triples", based on example23(). It describes the sources of duplicate triples and repetitive query results, and prescribes techniques for bringing them under control.

Python Tutorial Describes Managing User Accounts.

The Python tutorial has a new section on "Creating AllegroGraph Users with WebView." It describes how to create a manage new AllegroGraph user accounts.

Expanded Coverage of Literal Values in Python Tutorial.

The Python tutorial has been revised and expanded to document the behavior of literal typed values (in example5()). This includes the default behavior of untyped literals, as well as their use in getStatements queries as well as filtered and direct SPARQL queries. The section covers strings, floats, ints, untyped values, dates and times.

Python Examples from Command Line

The Python tutorial example file (tutorialexamples40.py) can now be run from the command line.

$ python tutorial_examples_40.py  runs all tests.  
$ python tutorial_examples_40.py all  runs all tests.  
$ python tutorial_examples_40.py 1 5 22   runs tests 1, 5, and 22 

Rfe9295 - Changes addStatements and addFile

addStatements and addFile can now auto-commit triples based on a count for large loads of triples. Set the add commit size to a positive integer by calling setAddCommitSize or setting the addcommitsize property on the RepositoryConnection for size-based commits, or set to None or 0 for regular commit semantic on the add methods.

Using this feature can decrease the server's memory usage during loading.

Rfe9316 - RepositoryConnection.setRuleLanguage removed

RepositoryConnection.setRuleLanguage has been removed. Both addRules and loadRules already take an language parameter which defaults to QueryLanguage.PROLOG as the only language currently supported for rules.

Bug19001 - Changes to RepositoryConnection.clearNamespaces API

RepositoryConnect.clearNamespaces now supports a reset parameter (which defaults to True). clearNamespaces deletes all namespaces in the repository for the current user. If the reset argument is True, the user's namespaces are reset to the default set of namespaces, otherwise all namespaces are cleared.

Bug19004 - Make sure Python clients ping their session.

Previously, the thread to keep a dedicated session alive was created but not started, making it possible for Python clients to have their session time out on them.

Welcome to AllegroGraph 4.0m2

General

The M2a release includes the following major functionality:

There were many minor enhancements and bug fixes.

Note that AllegroGraph databases created with release m1 or m2 will have to be converted to a new format before use with version m2a. See the Upgrade Guide.

Performance and Robustness

Other Enhancements

Fixed bugs and other corrections

SPARQL

Java Client

Python Client