AllegroGraph Java Connection Pooling introduction
The relevant package in the Javadocs is com.franz.agraph.pool.
The agraph-java-client library now has a feature for pooling connections to the AllegroGraph server. This is useful in webserver and appserver applications where many threads need to access the same AllegroGraph database. By using a pool of connections, AllegroGraph server resources can be minimized while supporting concurrent access.
The API to the pool is the AGConnPool class in the com.franz.agraph.pool package. AGConnPool may be configured in code, or through JNDI with the AGConnPoolJndiFactory class. The AGConnPool implements ObjectPool from the Apache commons-pool library, and the objects it manages are AGRepositoryConnection. More details can be found in the javadocs.
Features
The features of the pool are exposed through properties so it may be configured in JNDI or java code.
Related to the connection itself are properties: serverUrl, catalog, repository, username, password, session, and sessionLifetime. The session configures whether connections are shared, dedicated (autoCommit), or dedicated/transactional - it is recommended that applications set this appropriately on the pool, then not change the setAutoCommit property of the connection within the application code. The sessionLifetime helps manage server resources and is discussed more below.
Note that users of the pool will share the same connections and sessions (though not concurrently). For example, all connections will have the same repository and username. If different properties are needed, then multiple pools must be configured.
Not all features of the agraph-java-client library are supported yet in the pool properties. This includes managing catalogs and repositories, server scripts, rules, SNA, mappings, bulkMode, duplicates, and federation. These features must be managed in application code, and care must be taken since the connections will be reused after returning to the pool.
Related to the pooling mechanism are more properties: initialSize, shutdownHook (discussed below), maxActive, maxWait, testOnBorrow, and testOnReturn. MaxActive is important to limit server resources. The correct number is dependent on server resources and how many concurrent requests need to be handled. The server should have some number of SessionPorts configured based on machine resources, and to avoid running out of sessions, all possible pools (usually different JVMs) should have maxActive be a fraction of that number so that they can't request more than the server has. MaxWait is important so that borrowing a connection does not block your application request forever - it should be set to approximately the longest tolerable time a request should last before failing. TestOnBorrow is important so the application does not try to use stale connections, usually resulting in a "connection refused" error because the server session has been reclaimed due to inactivity (see sessionLifetime).
Also, commons-pool will run a background "eviction thread" based on these properties: maxIdle, minIdle, testWhileIdle, timeBetweenEvictionRunsMillis, minEvictableIdleTimeMillis, softMinEvictableIdleTimeMillis, and numTestsPerEvictionRun. This may reduce time that a request waits for a connection if testOnBorrow were called, and a new connection is established.
Lifetime of the Pool
Care must be taken with the lifetime of the pool. Like all resource objects in Java, it must be closed/shutdown properly. Note that AllegroGraph server sessions are OS processes that must be closed explicitly - there is not a persistent socket connection for it to know when the client disappears, so close must be called. The server sessions should have a lifetime set so they will end after inactivity; the javadoc has more details as well as the server http protocol.
AGConnPool supports a property called shutdownHook to register itself with the JVM to shutdown resources before the JVM exits - this is a good last resort, but is not guaranteed and may not be as timely as other means. The lifetime of the pool may be controlled by the web/appserver based on JNDI configuration. A shutdownHook is not always appropriate, as in a webapp if the pool library exists within the application instead of configured with the web/appserver JNDI. In particular, if the shutdownHook is used and the webserver undeploys the pool without closing it, the shutdownHook will become the source of a leak, keeping the pool object, connections, and classes alive in the JVM. Closing a pool will remove its shutdownHook if there is one.
For webapps directly controlling the pool (not by JNDI), an easy way to manage the lifecycle of the pool is by implementing a javax.servlet.ServletContextListener to create and shutdown the pool, keeping the pool object where it can be accessed during requests.
When configuring the pool in JNDI with Tomcat, a Lifecycle Listener can be implemented and configured.
Lifetime of Connections
Connections (class AGRepositoryConnection) must not be used by concurrent threads. The agraph-java library, and supporting libraries (Jena and Sesame), are not built to be thread-safe.
Connections are borrowed from the pool just before use, and returned to the pool as soon as the thread is done with it. The return of the connection (method pool.returnObject(connection) or connection.close()) must be done in a finally block to ensure the connection is not leaked.
If a connection is leaked, the sessionLifetime (if used) will cause the server process to be closed eventually, and the only leak will be the connection objects in JVM memory, until the pool is shutdown. If sessionLifetime is not set, the AllegroGraph server may run out of resources or session ports, new connections will fail, and the server process must be killed by an admin.
For webapps, an easy way to manage the lifecycle of connections is by implementing a javax.servlet.ServletRequestListener to borrow and return a connection from the pool and keep it in a ThreadLocal. This may not be the most efficient use of connections if requests last significantly longer than the connection is really used.
Libraries
This library uses the Apache commons-pool library for the underlying pool functionality.
If the pool is to be configured in a webserver with JNDI, the webserver will need access to the agraph jar and all of it's dependencies (sesame, jena, commons-httpclient, commons-pool, logging, etc).
Alternatively, if the application is configuring AGConnPool directly, the agraph jar and its dependencies may be packaged with the application.
The agraph-java-client library uses the SLF4J API for logging, so you can choose the implementation see www.slf4j.org. Connections whose server session has expired (see sessionLifetime) will log exceptions as DEBUG, but automatically recover. Apache commons-httpclient will log http errors and exceptions to INFO even though agraph-java ends up recovering.
