GenAI integrations

Neo4j’s Vector indexes and Vector functions allow you to calculate the similarity between node and relationship properties in a graph. A prerequisite for using these features is that vector embeddings have been set as properties of these entities. The GenAI plugin enables the creation of such embeddings using GenAI providers.

To use the GenAI plugin you need an account and API credentials from any of the following GenAI providers: Vertex AI, OpenAI, Azure OpenAI, and Amazon Bedrock.

To learn more about using embeddings in Neo4j, see Vector indexes → Vectors and embeddings in Neo4j.

For a hands-on guide on how to use the GenAI plugin, see GenAI documentation - Embeddings & Vector Indexes Tutorial → Create embeddings with cloud AI providers.

Installation

The GenAI plugin is enabled by default in Neo4j Aura.

The plugin needs to be installed on self-managed instances. This is done by moving the neo4j-genai.jar file from /products to /plugins in the Neo4j home directory, or, if you are using Docker, by starting the Docker container with the extra parameter --env NEO4J_PLUGINS='["genai"]'. For more information, see Operations Manual → Configure plugins.

Prior to Neo4j 5.23, the GenAI plugin was only available on Neo4j Enterprise Edition.

Example graph

The examples on this page use the Neo4j movie recommendations dataset, focusing on the plot and title properties of Movie nodes.

genai graph

The graph contains 28863 nodes and 332522 relationships. There are 9083 Movie nodes with a plot and title property.

To recreate the graph, download and import this dump file to an empty Neo4j database (running version 5.17 or later). Dump files can be imported for both Aura and on-prem instances.

The embeddings on this are generated using OpenAI (model text-embedding-ada-002), producing 1536-dimensional vectors.

Generate a single embedding and store it

Use the genai.vector.encode() function to generate a vector embedding for a single value.

Signature for genai.vector.encode() Function
genai.vector.encode(resource :: STRING, provider :: STRING, configuration :: MAP = {}) :: LIST<FLOAT>

  • The resource (a STRING) is the object to transform into an embedding, such as a chunk text or a node/relationship property.

  • The provider (a STRING) is the case-insensitive identifier of the provider to use. See identifiers under GenAI providers for supported options.

  • The configuration (a MAP) contains provider-specific settings, such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider. Note that because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.

This function sends one API request every time it is called, which may result in a lot of overhead in terms of both network traffic and latency. If you want to generate many embeddings at once, use Generating a batch of embeddings and store them.

Use the db.create.setNodeVectorProperty procedure to store an embedding to a node property.

Signature for db.create.setNodeVectorProperty Procedure
db.create.setNodeVectorProperty(node :: NODE, key :: STRING, vector :: ANY)

Use the db.create.setRelationshipVectorProperty procedure to store an embedding to a relationship property.

Signature for db.create.setRelationshipVectorProperty Procedure Introduced in 5.18
db.create.setRelationshipVectorProperty(relationship :: RELATIONSHIP, key :: STRING, vector :: ANY)

  • node or relationship is the entity in which the new property will be stored.

  • key (a STRING) is the name of the new property containing the embedding.

  • vector is the object containing the embedding.

The embeddings are stored as properties on nodes or relationships with the type LIST<INTEGER | FLOAT>.

Example 1. Create an embedding from a single property and store it
Create an embedding property for the Godfather
MATCH (m:Movie {title:'Godfather, The'})
WHERE m.plot IS NOT NULL AND m.title IS NOT NULL
WITH m, m.title || ' ' || m.plot AS titleAndPlot (1)
WITH m, genai.vector.encode(titleAndPlot, 'OpenAI', { token: $token }) AS propertyVector (2)
CALL db.create.setNodeVectorProperty(m, 'embedding', propertyVector) (3)
RETURN m.embedding AS embedding
1 Concatenate the title and plot of the Movie into a single STRING.
2 Create a 1536 dimensional embedding from the titleAndPlot.
3 Store the propertyVector as a new embedding property on The Godfather node.
Result
+----------------------------------------------------------------------------------------------------+
| embedding                                                                                          |
+----------------------------------------------------------------------------------------------------+
| [0.005239539314061403, -0.039358530193567276, -0.0005175105179660022, -0.038706034421920776, ... ] |
+----------------------------------------------------------------------------------------------------+
This result only shows the first 4 of the 1536 numbers in the embedding.

Generating a batch of embeddings and store them

Use the genai.vector.encodeBatch procedure to generate many vector embeddings with a single API request. This procedure takes a list of resources as an input, and returns the same number of result rows, instead of a single one.

This procedure attempts to generate embeddings for all supplied resources in a single API request. Therefore, it is recommended to see the respective provider’s documentation for details on, for example, the maximum number of embeddings that can be generated per request.
Signature for genai.vector.encodeBatch Procedure
genai.vector.encodeBatch(resources :: LIST<STRING>, provider :: STRING, configuration :: MAP = {}) :: (index :: INTEGER, resource :: STRING, vector :: LIST<FLOAT>)

  • The resources (a LIST<STRING>) parameter is the list of objects to transform into embeddings, such as chunks of text.

  • The provider (a STRING) is the case-insensitive identifier of the provider to use. See GenAI providers for supported options.

  • The configuration (a MAP) specifies provider-specific settings such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider. Note that because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.

Each returned row contains the following columns:

  • The index (an INTEGER) is the index of the corresponding element in the input list, to aid in correlating results back to inputs.

  • The resource (a STRING) is the name of the input resource.

  • The vector (a LIST<FLOAT>) is the generated vector embedding for this resource.

Example 2. Create embeddings from a limited number of properties and store them
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH m
LIMIT 20
WITH collect(m) AS moviesList (1)
WITH moviesList, [movie IN moviesList | movie.title || ': ' || movie.plot] AS batch (2)
CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $token }) YIELD index, vector
WITH moviesList, index, vector
CALL db.create.setNodeVectorProperty(moviesList[index], 'embedding', vector) (3)
1 Collect all 20 Movie nodes into a LIST<NODE>.
2 Use a list comprehension ([]) to extract the title and plot properties of the movies in moviesList into a new LIST<STRING>.
3 db.create.setNodeVectorProperty is run for each vector returned by genai.vector.encodeBatch, and stores that vector as a property named embedding on the corresponding node.
Example 3. Create embeddings from a large number of properties and store them
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH collect(m) AS moviesList (1)
     count(*) AS total,
     100 AS batchSize (2)
UNWIND range(0, total, batchSize) AS batchStart (3)
CALL (moviesList, batchStart, batchSize) { (4)
    WITH [movie IN moviesList[batchStart .. batchStart + batchSize] | movie.title || ': ' || movie.plot] AS resources (5)
    CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $token }) YIELD index, vector
    CALL db.create.setNodeVectorProperty(moviesList[batchStart + index], 'embedding', vector) (6)
} IN TRANSACTIONS OF 1 ROW (7)
1 Collect all returned Movie nodes into a LIST<NODE>.
2 batchSize defines the number of nodes in moviesList to be processed at once. Because vector embeddings can be very large, a larger batch size may require significantly more memory on the Neo4j server. Too large a batch size may also exceed the provider’s threshold.
3 Process Movie nodes in increments of batchSize.
4 A CALL subquery executes a separate transaction for each batch. Note that this CALL subquery uses a variable scope clause (introduced in Neo4j 5.23) to import variables. If you are using an older version of Neo4j, use an importing WITH clause instead.
5 resources is a list of strings, each being the concatenation of title and plot of one movie.
6 The procedure sets vector as value for the property named embedding for the node at position batchStart + index in the moviesList.
7 Set to 1 the amount of batches to be processed at once.
This example may not scale to larger datasets, as collect(m) requires the whole result set to be loaded in memory. For an alternative method more suitable to processing large amounts of data, see GenAI documentation - Embeddings & Vector Indexes Tutorial → Create embeddings with cloud AI providers.

GenAI providers

The following GenAI providers are supported for generating vector embeddings. Each provider has its own configuration map that can be passed to genai.vector.encode or genai.vector.encodeBatch.

Vertex AI

Vertex AI provider details
Table 1. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

projectId

STRING

GCP project ID.

Required

model

STRING

The name of the model you want to invoke.

Supported values:

  • "textembedding-gecko@001" Introduced in 5.17

  • "textembedding-gecko@002" Introduced in 5.19

  • "textembedding-gecko@003" Introduced in 5.19

  • "textembedding-gecko-multilingual@001" Introduced in 5.19

"textembedding-gecko@001"

region

STRING

GCP region where to send the API requests.

Supported values:

  • "us-west1"

  • "us-west2"

  • "us-west3"

  • "us-west4"

  • "us-central1"

  • "us-east1"

  • "us-east4"

  • "us-south1"

  • "northamerica-northeast1"

  • "northamerica-northeast2"

  • "southamerica-east1"

  • "southamerica-west1"

  • "europe-west2"

  • "europe-west1"

  • "europe-west4"

  • "europe-west6"

  • "europe-west3"

  • "europe-north1"

  • "europe-central2"

  • "europe-west8"

  • "europe-west9"

  • "europe-southwest1"

  • "asia-south1"

  • "asia-southeast1"

  • "asia-southeast2"

  • "asia-east2"

  • "asia-east1"

  • "asia-northeast1"

  • "asia-northeast2"

  • "australia-southeast1"

  • "australia-southeast2"

  • "asia-northeast3"

  • "me-west1"

"us-central1"

taskType

STRING

The intended downstream application (see provider documentation). The specified taskType will apply to all resources in a batch. Introduced in 5.19

title

STRING

The title of the document that is being encoded (see provider documentation). The specified title will apply to all resources in a batch. Introduced in 5.19

OpenAI

OpenAI provider details
Table 2. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

model

STRING

The name of the model you want to invoke.

"text-embedding-ada-002"

dimensions

INTEGER

The number of dimensions you want to reduce the vector to. Only supported for certain models.

Model-dependent.

Azure OpenAI

Unlike the other providers, the model is configured when creating the deployment on Azure, and is thus not part of the configuration map.
Azure OpenAI provider details
Table 3. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

resource

STRING

The name of the resource to which the model has been deployed.

Required

deployment

STRING

The name of the model deployment.

Required

dimensions

INTEGER

The number of dimensions you want to reduce the vector to. Only supported for certain models.

Model-dependent.

Amazon Bedrock

Amazon Bedrock provider details
Table 4. Configuration map
Key Type Description Default

accessKeyId

STRING

AWS access key ID.

Required

secretAccessKey

STRING

AWS secret key.

Required

model

STRING

The name of the model you want to invoke.

Supported values:

  • "amazon.titan-embed-text-v1"

"amazon.titan-embed-text-v1"

region

STRING

AWS region where to send the API requests.

Supported values:

  • "us-east-1"

  • "us-west-2"

  • "ap-southeast-1"

  • "ap-northeast-1"

  • "eu-central-1"

"us-east-1"