🔀 Bedrock Proxy Endpoint

Spin up your own custom OpenAI API server endpoint for easy AWS Bedrock LLM text inference (using standard baseUrl, and apiKey params)

Maintained by

Why Bedrock Proxy Endpoint?

Are you are stuck with using AWS Bedrock for all LLM text inference, but you want to keep your application platform agnostic? Are you tired of figuring out how to format your LLM inference calls to work with the Bedrock SDK? Are you going crazy with all the differences between configuration from model to model?

Bedrock Proxy Endpoint makes it easy to continue using the OpenAI API client that you are use to by standing up your own OpenAI text compatible endpoint that will proxy all your calls to Bedrock in a compatible way.
Great for getting existing OpenAI API compatible applications working with AWS Bedrock.

Prerequisites

Before getting started, make sure you have the following installed:

Node.js (version 12 or higher)
npm (Node Package Manager)

Installation

Clone the repository:

git clone https://github.com/jparkerweb/bedrock-proxy-endpoint.git

Navigate to the project directory:
```
cd bedrock-proxy-endpoint
```
Install the dependencies:
```
npm ci
```

Docker Usage

You can run Bedrock Proxy Endpoint using Docker in two ways:

Option 1: Using Pre-built Image from GitHub Container Registry

Pull and run the latest image from GHCR:

# Pull the latest image
docker pull ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest

# Run with environment file
docker run -d \
  --name bedrock-proxy-endpoint \
  -p 88:88 \
  --env-file .env \
  ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest

# Or run with inline environment variables
docker run -d \
  --name bedrock-proxy-endpoint \
  -p 88:88 \
  -e HTTP_ENABLED=true \
  -e HTTP_PORT=88 \
  -e CONSOLE_LOGGING=true \
  ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest

Option 2: Build and Run Locally

Build the Docker image locally and run it:

# Build the image locally
docker build -t bedrock-proxy-endpoint .

# Run with environment file
docker run -d \
  --name bedrock-proxy-endpoint \
  -p 88:88 \
  --env-file .env \
  bedrock-proxy-endpoint

# Or run with custom port mapping
docker run -d \
  --name bedrock-proxy-endpoint \
  -p 8080:8080 \
  -e HTTP_PORT=8080 \
  --env-file .env \
  bedrock-proxy-endpoint

Docker Compose Example

Create a docker-compose.yml file for easier management:

version: '3.8'
services:
  bedrock-proxy-endpoint:
    image: ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest
    # Or for local build: build: .
    ports:
      - "88:88"
    env_file:
      - .env
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "node", "-e", "const http = require('http'); const req = http.request({host: 'localhost', port: process.env.HTTP_PORT || 3000, timeout: 2000}, (res) => process.exit(res.statusCode === 200 ? 0 : 1)); req.on('error', () => process.exit(1)); req.end();"]
      interval: 30s
      timeout: 10s
      retries: 3

Then run with:

docker-compose up -d

Configuration

Update the .env file in the root directory of the project with the following
environment variables based on your desired configuration:

key	value type	example	notes
CONSOLE_LOGGING	boolean	false	Show realtime logs
HTTP_ENABLED	boolean	true	Start a HTTP server
HTTP_PORT	integer	80	HTTP server port
MAX_REQUEST_BODY_SIZE	string	50mb	Maximum size for request body
HTTPS_ENABLED	boolean	false	Start a HTTPS server
HTTPS_PORT	integer	443	HTTPS server port
HTTPS_KEY_PATH	string	./path/mykey.key	Path to key file for HTTPS
HTTPS_CERT_PATH	string	./path/mycert.pem	Path to cert file for HTTPS
IP_RATE_LIMIT_ENABLED	boolean	true	Enable rate limiting by IP
IP_RATE_LIMIT_WINDOW_MS	integer	60000	Window in milliseconds
IP_RATE_LIMIT_MAX_REQUESTS	integer	100	Max requests per IP per window

Authentication

Bedrock Proxy authenticates with AWS via IAM. Since the OpenAI API intance accpets an API Key we will utilize this value to hold your credentials. Construct your apiKey for inference in the next step following this format:

${AWS_REGION}.${AWS_ACCESS_KEY_ID}.${AWS_SECRET_ACCESS_KEY}
example apiKey value:
us-west-2.AKIAWSXXXXXXXXXXX.YYYYYYYYYYYYYYYYYYYYYYYYY

Usage

Start the server via: node server

You are now ready to make a standard chat completions to the server.
Important values
- baseUrl: Root address of server based on your .env configuration.
- apiKey: Descibed in the Authentication section above.
- messages: Array of objects in role / content format.
- model: This is the modelName from the list of supported models found on the Bedrock Wrapper README file here; The /models enpoint of this server will also return a list of supported models.
- include_thinking_data: Optional boolean parameter that when set to true will include the model's thinking process in the response (only used with thinking models such as Claude-3-7-Sonnet-Thinking).
- use_converse_api: Optional boolean parameter that when set to true will use AWS Bedrock's Converse API instead of the Invoke API. The Converse API provides a unified request/response format across all models, better conversation management, and cleaner multimodal handling. Defaults to false for backward compatibility.
- stop or stop_sequences: Optional parameter to specify text sequences that will cause the model to stop generating (e.g., ["STOP", "END"]).

Example OpenAI API Call

Look at the example folder for complete examples of how to use the server:

example.js - Basic text completion example
example-vision.js - Vision model example with image processing (image can be passed as a base64 string or a URL)
example-stop-sequences.js - Stop sequences example demonstrating how to control where generation stops

import OpenAI from 'openai';

const messages = [
    {
        role: "system",
        content: "You are a helpful AI assistant that follows instructions extremely well. Answer the user questions accurately.",
    },
    {
        role: "user",
        content: "Describe why the OpenAI API standard is so great. Limit your response to five sentences.",
    },
    {
        role: "assistant",
        content: "",
    },
];

const baseURL = "http://localhost"; // URL of the Bedrock Proxy Endpoint
const apiKey = `${AWS_REGION}.${AWS_ACCESS_KEY_ID}.${AWS_SECRET_ACCESS_KEY}` // Your AWS Creds / API Key

const openai = new OpenAI({
    baseURL: baseURL,
    apiKey: apiKey,
});

async function main() {
    try {
        const chatCompletion = await openai.chat.completions.create({
            messages: messages,
            model: "Claude-4-5-Sonnet",
            max_tokens: 2048,
            temperature: 0.4,
            top_p: 0.7,
            stream: true,
            include_thinking_data: false,  // Set to true for thinking models like "Claude-4-5-Sonnet-Thinking"
            use_converse_api: true,  // Set to true to use the unified Converse API instead of Invoke API (default: false)
        });

        if (chatCompletion) {
            for await (const chunk of chatCompletion) {
                const response = chunk.choices[0]?.delta?.content || "";
                process.stdout.write(response);
            }
        }
    } catch (error) {
        console.error('Error:', error);
    } finally {
        process.exit(0);
    }
}

main();

Root Info Page

Point your browser to the root of your endpoint server to view the info page: (example: http://localhost)

Note

Alternativly you can incorporate 🪨 Bedrock Wrapper core directly into your code base. If you would like to explore that option checkout the npm package here: https://www.npmjs.com/package/bedrock-wrapper

Please consider sending me a tip to support my work 😀

jparkerweb/bedrock-proxy-endpoint