http://franz.com/ns/allegrograph/8.0.0/llm/askMyDocuments

Collect background information from a vector database to build a prompt and return the response to that prompt, along with matching URI citationIds and matching scores.

Namespace:

PREFIX llm: <http://franz.com/ns/allegrograph/8.0.0/llm/> 

General forms:

(?response ?score ?citationId) llm:askMyDocuments (?text ?vectorRepoSpec ?topN ?minScore)
(?response ?score ?citationId) llm:askMyDocuments (?text ?vectorRepoSpec ?topN)
(?response ?score ?citationId) llm:askMyDocuments (?text ?vectorRepoSpec)
(?response ?score ?citationId ?originalText) llm:askMyDocuments (?text ?vectorRepoSpec ?topN ?minScore)
(?response ?score ?citationId ?originalText) llm:askMyDocuments (?text ?vectorRepoSpec ?topN)
(?response ?score ?citationId ?originalText) llm:askMyDocuments (?text ?vectorRepoSpec)

This predicate implements Retrieval Augmented Generation (RAG) by collecting background information through embedding based matching. Beginning with a search of ?vectorRepoSpec for the ?topN best matches to ?text, above a minimum matching score of ?minScore. it then combines this question, the matching citationIds and background info into a big prompt for the LLM. This helps ensure that the LLM has a source of truth to answer the question, and reduces the chance of hallucination.

The 'big prompt' also asks the LLM to return only those citationIds whose content contributed to the final resposne.

The optional object parameter ?topN, if not specified, has a default value of 5,

The optional object parameter ?minScore, if not specified, has a default value of 0.8,

The predicate returns a response as well as the matching score, citationId URI, and source content from the vector database.

Note that llm:askMyDocuments may utilize keyword syntax.

You can use agtool to build a vector database from text literals stored in an Allegrograph repository (see the documentation on using agtool for LLM embedding).

There is a fully worked out example in the LLM Examples using data about Noam Chomsky.

API Key

You need an API key to utilize this predicate. You need an OpenAI API key to utilize this predicate. See https://platform.openai.com/overview for instructions on obtaining a key (start with the Quickstart Tutorial and follow the links there to get a key). There are three ways to configure your API key, as a query option prefix or in a couple of places in the Allegrograph configuration.

As a query option prefix, write:

PREFIX franzOption_openaiApiKey: <franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno> 

Syntax for config file:

QueryOption openaiApiKey=<franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno> 

In the file data/settings/default-query-options:

(("franzOption_openaiApiKey" "<franz:sk-U01ABc2defGHIJKlmnOpQ3RstvVWxyZABcD4eFG5jiJKlmno>")) 

API Options

The proprietary OpenAI API exposes many options and parameters for interaction with their LLM models. Currently the AllegroGraph magic predicates and functions take an opinionated approach and hide most of these options behind the scenes. Specifically, we set

API endpoint: https://api.openai.com/v1/chat/completions 

Endpoint parameters:

best-of: gpt:openai-default-best-of
echo: gpt:openai-default-echo
frequency-penalty: gpt:openai-default-frequency-penalty
function-call: gpt:openai-default-function-call
functions: gpt:openai-default-functions
logit-bias: gpt:openai-default-logit-bias
logprobs: gpt:openai-default-logprobs
max-tokens: gpt:openai-default-max-tokens
minScore: gpt:openai-default-min-score
model: gpt:openai-default-ask-chat-model
n: gpt:openai-default-n
output-format: gpt:openai-default-output-format
presence-penalty: gpt:openai-default-presence-penalty
stop: gpt:openai-default-stop
suffix: gpt:openai-default-suffix
temperature: gpt:openai-default-temperature
timeout: gpt:openai-default-timeout
topN: gpt:openai-default-top-n
top-p: gpt:openai-default-top-p
user : gpt:openai-default-user
vectorRepoSpec: ?vectorRepoSpec
verbose nil

Additionally we impose an API timeout of 10 seconds.

Finally, when the OpenAI API times out, returns an error, or the magic predicate implementation fails to parse the response, or any other error occurs, the magic predicate displays an informative message in Webview. Please contact AllegroGraph Support if you require different API options or customization.

Notes

The following namespace abbreviations are used:

The SPARQL magic properties reference has additional information on using AllegroGraph magic properties and functions.