AWS Bedrock - Portkey Docs

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including models hosted on AWS Bedrock. With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a Provider system.

Provider Slug. bedrock

Portkey SDK Integration with AWS Bedrock

Portkey provides a consistent API to interact with models from various providers. To integrate Bedrock with Portkey:

1. Install the Portkey SDK

Add the Portkey SDK to your application to interact with Anthropic’s API through Portkey’s gateway.

NodeJS
Python

npm install --save portkey-ai

pip install portkey-ai

2. Initialize Portkey with the Bedrock Provider

There are two ways to integrate AWS Bedrock with Portkey:

AWS Access Key

Use your AWS Secret Access Key, AWS Access Key Id, and AWS Region to create your AI Provider on Portkey’s app.

Integration Guide

AWS Assumed Role

Take your AWS Assumed Role ARN and AWS Region to create the virtaul key.

Integration Guide

NodeJS SDK
Python SDK

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    provider:"@PROVIDER" // Your Bedrock Provider Slug
})

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER"   # Replace with Your Bedrock Provider Slug
)

Using Bedrock Provider with AWS STS

If you’re using AWS Security Token Service, you can pass your aws_session_token along with the AI Provider slug:

NodeJS
Python

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    provider:"@PROVIDER" // Your Bedrock Provider Slug,
    aws_session_token: ""
})

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER"   # Replace with your Provider Slug for Bedrock,
    aws_session_token=""
)

Not using Bedrock Provider from Model Catalog?

Check out this example on how you can directly use your AWS details to make a Bedrock request through Portkey.

3. Invoke Chat Completions with AWS bedrock

Use the Portkey instance to send requests to Anthropic. You can also override the provider slug directly in the API call if needed.

NodeJS SDK
Python SDK

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens: 250 // Required field for Anthropic
});

console.log(chatCompletion.choices);

completion = portkey.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Say this is a test' }],
    model= 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens=250 # Required field for Anthropic
)

print(completion.choices)

Using the /messages Route with Bedrock Models

Access Bedrock’s Claude models through Anthropic’s native/messages endpoint using Portkey’s SDK or Anthropic’s SDK.

This route only works with Claude models on Bedrock. For other models, use the standard OpenAI compliant endpoint.

cURL
Python SDK
NodeJS SDK
Anthropic Python SDK
Anthropic TypeScript SDK

curl --location 'https://api.portkey.ai/v1/messages' \
--header 'x-portkey-provider: @your-bedrock-provider' \
--header 'Content-Type: application/json' \
--header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \
--data '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 250,
    "messages": [
        {
            "role": "user",
            "content": "Hello, Claude"
        }
    ]
}'

    Coming Soon!

    Coming Soon!

Anthropic Python SDK

  import anthropic

  client = anthropic.Anthropic(
      api_key="dummy", # we will use portkey's provider slug
      default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
      base_url="https://api.portkey.ai/v1"
  )
  message = client.messages.create(
      model="@your-provider-slug/your-model-name",
      max_tokens=250,
      messages=[
          {"role": "user", "content": "Hello, Claude"}
      ],
  )
  print(message.content)

Anthropic TS SDK

   import Anthropic from '@anthropic-ai/sdk';

   const anthropic = new Anthropic({
     apiKey: 'dummy', // we will use portkey's provider slug
     baseURL: "https://api.portkey.ai/v1",
     defaultHeaders: { "x-portkey-api-key": "YOUR_PORTKEY_API_KEY" }
   });

   const msg = await anthropic.messages.create({
     model: "@your-provider-slug/your-model-name",
     max_tokens: 1024,
     messages: [{ role: "user", content: "Hello, Claude" }],
   });
   console.log(msg);

Counting Tokens

Portkey also supports the token counting endpoint for bedrock. Checkout the example in this link for more details.

Using Vision Models

Portkey’s multimodal Gateway fully supports Bedrock’s vision models anthropic.claude-3-sonnet, anthropic.claude-3-haiku, and anthropic.claude-3-opus For more info, check out this guide: Vision

Extended Thinking (Reasoning Models) (Beta)

The assistants thinking response is returned in the response_chunk.choices[0].delta.content_blocks array, not the response.choices[0].message.content string.

Models like us.anthropic.claude-3-7-sonnet-20250219-v1:0 support extended thinking. This is similar to openai thinking, but you get the model’s reasoning as it processes the request as well. Note that you will have to set strict_open_ai_compliance=False in the headers to use this feature.

Single turn conversation

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",
    strict_openai_compliance=False
)

# Create the request
response = portkey.chat.completions.create(
  model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  max_tokens=3000,
  thinking={
      "type": "enabled",
      "budget_tokens": 2030
  },
  stream=True,
  messages=[
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
              }
          ]
      }
  ]
)
print(response)
# in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array
# response = portkey.chat.completions.create(
#   ...same config as above but with stream: true
# )
# for chunk in response:
#     if chunk.choices[0].delta:
#         content_blocks = chunk.choices[0].delta.get("content_blocks")
#         if content_blocks is not None:
#             for content_block in content_blocks:
#                 print(content_block)

Multi turn conversation

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",
    strict_openai_compliance=False
)

# Create the request
response = portkey.chat.completions.create(
  model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  max_tokens=3000,
  thinking={
      "type": "enabled",
      "budget_tokens": 2030
  },
  stream=True,
  messages=[
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
              }
          ]
      },
      {
          "role": "assistant",
          "content": [
                  {
                      "type": "thinking",
                      "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                      "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                  }
          ]
      },
      {
          "role": "user",
          "content": "thanks that's good to know, how about to chennai?"
      }
  ]
)
print(response)

Inference Profiles

Inference profiles are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests. To use inference profiles, your IAM role needs to additionally have the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:GetInferenceProfile"
            ],
            "Resource": [
                "arn:aws:bedrock:*:*:inference-profile/*",
                "arn:aws:bedrock:*:*:application-inference-profile/*"
            ]
        }
    ]
}

This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request. For reference, see the following documentation: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html

Bedrock Guardrails

You can use Bedrock guardrails directly in your chat completions requests to add content filtering and safety measures. Guardrails help ensure that model responses adhere to your specific safety and content policies.

We recommend using guardrails through the Portkey UI for easier management and configuration. You can learn more about guardrails here.

Using Guardrails in Chat Completions

To enable guardrails, include the guardrailConfig parameter in your request:

NodeJS SDK
Python SDK
cURL

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens: 250,
    guardrailConfig: {
        guardrailIdentifier: "your-guardrail-id",
        guardrailVersion: "DRAFT", // or specific version number
        trace: "enabled" // optional: "enabled" or "disabled"
    }
});

completion = portkey.chat.completions.create(
    messages=[{ "role": 'user', "content": 'Say this is a test' }],
    model='us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens=250,
    guardrail_config={  # Note: snake_case also supported
        "guardrailIdentifier": "your-guardrail-id",
        "guardrailVersion": "DRAFT",
        "trace": "enabled"
    }
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @your-bedrock-provider" \
  -d '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 250,
    "messages": [{"role": "user","content": "Say this is a test"}],
    "guardrailConfig": {
      "guardrailIdentifier": "your-guardrail-id",
      "guardrailVersion": "DRAFT",
      "trace": "enabled"
    }
  }'

Guardrail Configuration Parameters

Parameter	Type	Required	Description
`guardrailIdentifier`	string	Yes	The unique identifier of your Bedrock guardrail
`guardrailVersion`	string	Yes	Version of the guardrail (`"DRAFT"` for the latest draft version, or a specific version number)
`trace`	string	No	Controls trace generation (`"enabled"` or `"disabled"`)

Both guardrailConfig (camelCase) and guardrail_config (snake_case) parameter names are supported for compatibility.

When a guardrail is triggered, the response will include a guardrail_intervened stop reason. You can access detailed trace information if tracing is enabled.

Bedrock Converse API

Portkey uses the AWS Converse API internally for making chat completions requests. If you need to pass additional input fields or parameters like anthropic_beta, top_k, frequency_penalty etc. that are specific to a model, you can pass it with this key:

"additionalModelRequestFields": {
    "frequency_penalty": 0.4
}

If you require the model to respond with certain fields that are specific to a model, you need to pass this key:

"additionalModelResponseFieldPaths": [ "/stop_sequence" ]

Managing AWS Bedrock Prompts

You can manage all prompts to AWS bedrock in the Prompt Library. All the current models of Anthropic are supported and you can easily start testing different prompts. Once you’re ready with your prompt, you can use the portkey.prompts.completions.create interface to use the prompt in your application.

Making Requests without using Portkey’s Model Catalog

If you do not want to add your AWS details to Portkey vault, you can also directly pass them while instantiating the Portkey client.

Mapping the Bedrock Details

Node SDK	Python SDK	REST Headers
awsAccessKeyId	aws_access_key_id	x-portkey-aws-access-key-id
awsSecretAccessKey	aws_secret_access_key	x-portkey-aws-secret-access-key
awsRegion	aws_region	x-portkey-aws-region
awsSessionToken	aws_session_token	x-portkey-aws-session-token

Example

NodeJS
Python
cURL

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "bedrock",
    awsAccessKeyId: "AWS_ACCESS_KEY_ID",
    awsSecretAccessKey: "AWS_SECRET_ACCESS_KEY",
    awsRegion: "us-east-1",
    awsSessionToken: "AWS_SESSION_TOKEN"
})

from portkey_ai import Portkey

client = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="bedrock",
    aws_access_key_id="",
    aws_secret_access_key="",
    aws_region="us-east-1",
    aws_session_token=""
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: bedrock" \
  -H "x-portkey-aws-access-key-id: $AWS_ACCESS_KEY_ID" \
  -H "x-portkey-aws-secret-access-key: $AWS_SECRET_ACCESS_KEY" \
  -H "x-portkey-aws-region: $AWS_REGION" \
  -H "x-portkey-aws-session-token: $AWS_TOKEN" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

AWS GovCloud Support

AWS GovCloud provides isolated cloud infrastructure for US government agencies and regulated workloads. Portkey supports Bedrock in GovCloud regions with custom endpoint configuration.

GovCloud regions

us-gov-west-1 (US West)
us-gov-east-1 (US East)

Configuration

When creating your Bedrock AI Provider in Portkey’s Model Catalog, configure the custom host with the GovCloud endpoint: Custom host format:

bedrock-runtime.{region}.amazonaws.com

Examples:

bedrock-runtime.us-gov-west-1.amazonaws.com
bedrock-runtime.us-gov-east-1.amazonaws.com

Ensure your AWS credentials have appropriate permissions for the GovCloud region you’re targeting.

Usage example

Python
NodeJS

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@GOVCLOUD_PROVIDER"  # Your GovCloud Bedrock Provider
)

completion = portkey.chat.completions.create(
    messages=[{"role": "user", "content": "Hello from GovCloud"}],
    model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    max_tokens=250
)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "@GOVCLOUD_PROVIDER"  // Your GovCloud Bedrock Provider
});

const completion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Hello from GovCloud' }],
    model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens: 250
});

Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

Though using assumed role is in itself enough for enterprise security. You can additional configure AWS PrivateLink for Bedrock to ensure that your requests are not traversed outside your VPC.

Create a private link between the VPC you’ve deployed Portkey and AWS Bedrock (the endpoint is in most cases https://bedrock.{your_region}.amazonaws.com).
When configuring your integration on portkey, simply configure the custom host option to point to your VPC endpoint for the private link.

Supported Models

List of supported Amazon Bedrock model IDs

How to Find Your AWS Credentials

Navigate here in the AWS Management Console to obtain your AWS Access Key ID and AWS Secret Access Key.

In the console, you’ll find the ‘Access keys’ section. Click on ‘Create access key’.
Copy the Secret Access Key once it is generated, and you can view the Access Key ID along with it.

On the same page under the ‘Access keys’ section, where you created your Secret Access key, you will also find your Access Key ID.

And lastly, get Your AWS Region from the Home Page of AWS Bedrock as shown in the image below.

Next Steps

The complete list of features supported in the SDK are available on the link below.

SDK

You’ll find more information in the relevant sections:

Ecosystem

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

MCP Clients

MCP Servers

​Portkey SDK Integration with AWS Bedrock

​1. Install the Portkey SDK

​2. Initialize Portkey with the Bedrock Provider

AWS Access Key

AWS Assumed Role

​Using Bedrock Provider with AWS STS

​Not using Bedrock Provider from Model Catalog?

​3. Invoke Chat Completions with AWS bedrock

​Using the /messages Route with Bedrock Models

Counting Tokens

​Using Vision Models

​Extended Thinking (Reasoning Models) (Beta)

​Single turn conversation

​Multi turn conversation

​Inference Profiles

​Bedrock Guardrails

​Using Guardrails in Chat Completions

​Guardrail Configuration Parameters

​Bedrock Converse API

​Managing AWS Bedrock Prompts

​Making Requests without using Portkey’s Model Catalog

​Mapping the Bedrock Details

​Example

​AWS GovCloud Support

​GovCloud regions

​Configuration

​Usage example

​Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

​Supported Models

List of supported Amazon Bedrock model IDs

​How to Find Your AWS Credentials

​Next Steps

SDK

Portkey SDK Integration with AWS Bedrock

1. Install the Portkey SDK

2. Initialize Portkey with the Bedrock Provider

Using Bedrock Provider with AWS STS

Not using Bedrock Provider from Model Catalog?

3. Invoke Chat Completions with AWS bedrock

Using the /messages Route with Bedrock Models

Using Vision Models

Extended Thinking (Reasoning Models) (Beta)

Single turn conversation

Multi turn conversation

Inference Profiles

Bedrock Guardrails

Using Guardrails in Chat Completions

Guardrail Configuration Parameters

Bedrock Converse API

Managing AWS Bedrock Prompts

Making Requests without using Portkey’s Model Catalog

Mapping the Bedrock Details

Example

AWS GovCloud Support

GovCloud regions

Configuration

Usage example

Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

Supported Models

How to Find Your AWS Credentials

Next Steps