http://franz.com/ns/allegrograph/8.0.0/llm/askMyDocuments

Collect background information from a vector database to build a prompt and return the response to that prompt, along with matching URI citations and matching scores.

Namespace: 

PREFIX llm: <http://franz.com/ns/allegrograph/8.0.0/llm/>

General forms: 

(?response ?score ?citation) llm:askMyDocuments (?query ?vectorDatabaseName ?topN ?minScore)

 (?response ?score ?citation) llm:askMyDocuments (?query ?vectorDatabaseName ?topN)

 (?response ?score ?citation) llm:askMyDocuments (?query ?vectorDatabaseName)

 (?response ?score ?citation ?originalText) llm:askMyDocuments (?query ?vectorDatabaseName ?topN ?minScore)

 (?response ?score ?citation ?originalText) llm:askMyDocuments (?query ?vectorDatabaseName ?topN)

 (?response ?score ?citation ?originalText) llm:askMyDocuments (?query ?vectorDatabaseName)

This predicate implements Retrieval Augmented Generation (RAG) by collecting background information through embedding based matching. Beginning with a search of ?vectorDatabaseName for the ?topN best matches to ?query, above a minimum matching score of ?minScore. it then combines this question, the matching citations and background info into a big prompt for the LLM. This helps ensure that the LLM has a source of truth to answer the question, and reduces the chance of hallucination.

The 'big prompt' also asks the LLM to return only those citations whose content contributed to the final resposne.

The optional object parameter ?topN, if not specified, has a default value of 5,

The optional object parameter ?minScore, if not specified, has a default value of 0.8,

The predicate returns a response as well as the matching score, citation URI, and source content

from the vector database.

You can use agtool to build a vector database from text literals stored in an Allegrograph repository (see the documentation on using agtool for LLM embedding).

There is a fully worked out example in the LLM Examples using data about Noam Chomsky.

API Key

You need an API key to utilize this predicate. You need an OpenAI API key to utilize this predicate. See https://platform.openai.com/overview for instructions on obtaining a key (start with the Quickstart Tutorial and follow the links there to get a key). There are three ways to configure your API key, as a query option prefix or in a couple of places in the Allegrograph configuration.

As a query option prefix, write: 

PREFIX franzOption_openaiApiKey: <franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno>

Syntax for config file: 

QueryOption openaiApiKey=<franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno>

In the file data/settings/default-query-options: 

(("franzOption_openaiApiKey" "<franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno>"))

API Options

The proprietary OpenAI API exposes many options and parameters for interaction with their LLM models. Currently the AllegroGraph magic predicates and functions take an opinionated approach and hide most of these options behind the scenes. Specifically, we set 

API endpoint: https://api.openai.com/v1/chat/completions

Endpoint parameters: 

best-of: gpt:openai-default-best-of

echo: gpt:openai-default-echo

frequency-penalty: gpt:openai-default-frequency-penalty

function-call: gpt:openai-default-function-call

functions: gpt:openai-default-functions

logit-bias: gpt:openai-default-logit-bias

logprobs: gpt:openai-default-logprobs

max-tokens: gpt:openai-default-max-tokens

min-score: gpt:openai-default-min-score

model: gpt:openai-default-ask-chat-model

n: gpt:openai-default-n

output-format: gpt:openai-default-output-format

presence-penalty: gpt:openai-default-presence-penalty

stop: gpt:openai-default-stop

suffix: gpt:openai-default-suffix

temperature: gpt:openai-default-temperature

timeout: gpt:openai-default-timeout

top-n: gpt:openai-default-top-n

top-p: gpt:openai-default-top-p

user : gpt:openai-default-user

vector-database-name: ?vectorDatabaseName

 verbose nil

Additionally we impose an API timeout of 10 seconds.

Finally, when the OpenAI API times out, returns an error, or the magic predicate implementation fails to parse the response, or any other error occurs, the magic predicate displays an informative message in Webview. Please contact AllegroGraph Support if you require different API options or customization.

Notes

The following namespace abbreviations are used:

The SPARQL magic properties reference has additional information on using AllegroGraph magic properties and functions.