What is Portkey?

Teams use Portkey to monitor and improve the cost, performance, and accuracy of their Gen AI apps.

It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, and more accurate at the same time.

Here's a product walkthrough (3 mins):

Integrate in 3 Lines of Code

# pip install portkey-ai

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai", 
        api_key="PORTKEY_API_KEY"
    )
)

chat_complete = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

print(chat_complete.choices[0].message.content)

// npm i portkey-ai

import OpenAI from 'openai';
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "openai", 
    apiKey: "PORTKEY_API_KEY"
    })
});

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
  });

  console.log(chatCompletion.choices);
}

main();

🚀 Quickstart

Setting up Portkey takes less than 2 mintues.

🤝 Integrations

Find the best integration for you with 200+ models across LLM providers and multiple frameworks.

✨ Product Features

Jump to the product section to learn more about the Portkey modules and the use cases they solve.

📔 API Reference

Head to the API reference and code samples for all Portkey functionality available through REST APIs and SDKs

While you're here, why not give us a star? It helps us a lot!

Languages Supported

Language

Supported Library

Javascript

portkey-node-sdk openai-node

Python

portkey-python-sdk openai-python

Java

Rust

Ruby

AI Providers Supported

Portkey is multimodal by default - along with chat and text models, we also support audio, vision, and image generation models.

AI Provider

Status

OpenAI

fully supportedpublic

Anthropic

fully supportedpublic

Azure OpenAI

fully supportedpublic

Cohere

fully supportedpublic

Anyscale

fully supportedpublic

Google Palm

fully supportedpublic

Google Gemini

fully supportedpublic

Together AI

fully supportedpublic

Perplexity

fully supportedpublic

Mistral

fully supportedpublic

Stability

fully supportedpublic

Nomic

fully supportedpublic

AWS Bedrock

fully supportedpublic

Ollama

fully supportedpublic

AzureML

partially supported

BYOLLM

fully supportedpublic

Jina AI

fully supportedpublic

Fireworks AI

fully supportedpublic

LocalAI

partially supportedpublic

Predibase

fully supportedpublic

ZhipuAI (ChatGLM)

fully supportedpublic

Deepinfra

fully supportedpublic

Lingyi (01.ai)

fully supportedpublic

Openrouter

fully supportedpublic

Moonshot

fully supportedpublic

AI21

fully supportedpublic

Reka AI

fully supportedpublic

Workers AI

fully supportedpublic

Deepbricks

fully supportedpublic

SiliconFlow

fully supportedpublic

View all the supported integration guides.

Frameworks Supported

Framework

Status

Langchain

nativepythontypescript

Llamaindex

nativepythontypescript

Autogen

nativepython

Vercel

nativetypescript

Instructor

nativepythontypescript

Promptfoo

nativetypescript

CrewAI

nativepython

NextMake Your First Request

Last updated 8 months ago

Was this helpful?

# pip install portkey-ai from openai import OpenAI from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders client = OpenAI( base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( provider="openai", api_key="PORTKEY_API_KEY" ) ) chat_complete = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Say this is a test"}], ) print(chat_complete.choices[0].message.content)

// npm i portkey-ai import OpenAI from 'openai'; import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai' const openai = new OpenAI({ baseURL: PORTKEY_GATEWAY_URL, defaultHeaders: createHeaders({ provider: "openai", apiKey: "PORTKEY_API_KEY" }) }); async function main() { const chatCompletion = await openai.chat.completions.create({ messages: [{ role: 'user', content: 'Say this is a test' }], model: 'gpt-3.5-turbo', }); console.log(chatCompletion.choices); } main();