AWS Bedrock procedures
These procedures leverage the Amazon Bedrock API.
Here is a list of all available Aws Bedrock procedures:
name | description |
---|---|
apoc.ml.bedrock.custom(body, $config) |
To create a customizable Bedrock API call |
apoc.ml.bedrock.list($config) |
To get the list of foundation or custom models |
apoc.ml.bedrock.embedding(texts, $config) |
To create an API call to generate embedding |
apoc.ml.bedrock.chat(messages, $config) |
To create a Chat Completion API call |
apoc.ml.bedrock.completion(prompt, $config) |
To create a Text Completion API call |
apoc.ml.bedrock.image(body, $config) |
To create an API call to get an image |
All the procedures, leverage the apoc.ml.bedrock.custom
procedures,
and support the same config parameter, but unlike the custom
one,
they have some different default parameters and model id.
Moreover, the return data is consistent with the called API,
instead of returning a generic Object
as a result
Config
name | type | default | description |
---|---|---|---|
keyId |
String |
null |
The AWS key ID. We can also evaluate it via |
secretKey |
String |
null |
The AWS secret access key. We can also evaluate it via |
region |
String |
us-east-1 |
The AWS region |
endpoint |
String |
see below |
The AWS endpoint. |
method |
String |
|
The HTTP Method |
headers |
Map<String, Object> |
|
The HTTP Header |
model |
String |
see below |
(This config is ignored with the |
path |
String |
"foundation-models" |
(Valid only with the |
openAICompatible |
String |
false |
To pass a body request compatible with OpenAI Chat Completions API, using the apoc.ml.bedrock.chat,
for example: |
The endpoint
config takes precedence over the model
one.
In case of all procedures, except the bedrock.list
, the default endpoint
is "https://bedrock-runtime.<regionConfigValue>.amazonaws.com/model/<modelConfigValue>/invoke"
.
So, with the default region
config, i.e. "us-east-1", the default endpoint is "https://bedrock-runtime.us-east-1.amazonaws.com/model/<modelConfigValue>/invoke"
.
The <modelConfigValue>
part must be configured if we use the ml.bedrock.custom
procedure,
while with the bedrock.chat
, bedrock.completion
, bedrock.embedding
, bedrock.image
ones,
has a default value of "anthropic.claude-v2", "ai21.j2-ultra-v1", "anthropic.claude-v2" and "stability.stable-diffusion-xl-v0" respectively.
Authentication settings
To authenticate to bedrock services, we can set in the apoc.conf
file the following entries.
apoc.aws.key.id=<AWS Key ID>
apoc.aws.secret.key=<AWS Secret Access Key>
Alternatively we can set them as $config
parameters, i.e.: {keyId: '<AWS Key ID>', secretKey:'<AWS Secret Access Key>'}
.
Or also, we can put an Authorization header, by using the header
parameter,
i.e. {header: {Authorization: 'AWS4-HMAC-SHA256 <CredentialAndSignature..>', …other entries…} }
.
Note that the default Content-Type: application/json
and the Accept: */*
header entries,
are always passed to the http request, unless overridden via the config header
.
In the following examples,
we assume that we set Key id and Secret Access Key via apoc.conf
.
Usage Examples
Chat Completion API
This procedure apoc.ml.bedrock.chat
takes a list of maps of chat exchanges between assistant and user (with optional system context), and will return the next message in the flow.
Additional configuration is passed to the API, the default model used is anthropic.claude-v2
.
CALL apoc.ml.bedrock.chat([
{
prompt: "\n\nHuman: Hello world\n\nAssistant:",
max_tokens_to_sample: 50,
top_k: 250,
top_p: 1,
stop_sequences: ["\\n\\nHuman:"]
}
])
value |
---|
{"stop_reason": "stop_sequence","completion": " Hello!"} |
We can use the config openAICompatible: true
, to use a message
body request consistent with the apoc.ml.openai.chat procedure.
With this config, the prompt
request will be placed in an entry {content: '<promts>'}
and will have a default "\n\nHuman:"` prefix and \n\nAssistant:
suffix, if not present.
For example, instead of:
CALL apoc.ml.bedrock.chat(
[ {prompt: "\n\nHuman: Hello world\n\nAssistant:",max_tokens_to_sample: 200} ]
)
we can execute this query (note that the role:"system"
entry is optional, it is just to be consistent with the OpenAI body):
CALL apoc.ml.bedrock.chat([
{role:"system", content:"Hello world"}
])
Text Completion API
This procedure apoc.ml.bedrock.completion
can continue/complete a given text.
Additional configuration is passed to the API, the default model used is ai21.j2-ultra-v1
.
CALL apoc.ml.bedrock.completion('What color is the sky? Answer in one word: ')
value |
---|
|
Image API
This procedure apoc.ml.bedrock.completion
can get a base64 image.
Additional configuration is passed to the API, the default model used is stability.stable-diffusion-xl-v0
.
CALL apoc.ml.bedrock.image({
text_prompts: [{text: "picture of a bird", weight: 1.0}],
cfg_scale: 5,
seed: 123,
steps: 70,
style_preset: "photographic"
})
base64Image |
---|
"iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAABjmVYSWZNTQAqAAAACAAGAQAABAAAAAEAAAIAAQEABAAA…." |
List of models
CALL apoc.ml.bedrock.list()
modelId | modelArn | modelName | providerName | responseStreamingSupported | customizationsSupported | inferenceTypesSupported | inputModalities | outputModalities |
---|---|---|---|---|---|---|---|---|
"amazon.titan-tg1-large" |
"arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-tg1-large" |
"Titan Text Large" |
"Amazon" |
true |
["FINE_TUNING"] |
["ON_DEMAND"] |
["TEXT"] |
["TEXT"] |
"amazon.titan-e1t-medium" |
"arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-e1t-medium" |
"Titan Text Embeddings" |
"Amazon" |
null |
[] |
["ON_DEMAND"] |
["TEXT"] |
["EMBEDDING"] |
… |
… |
… |
… |
null |
[] |
… |
… |
… |
Custom AWS API Call
Via the apoc.ml.bedrock.custom
we can create a customizable Bedrock API Request, by choosing the HTTP Method, the endpoint, the region and the additional headers.
Useful both for invoke a model,
in the case the response is incompatible with the previous procedures, and to use any other Bedrock API.
For example, we can call the GetModelInvocationLoggingConfiguration API
by executing the following query (note that the body
parameter is null, since the API does not have a request body.):
CALL apoc.ml.bedrock.custom(null,{
endpoint: "https://bedrock.us-east-1.amazonaws.com/logging/modelinvocations",
method: "GET"
})
value |
---|
|