SDK

The Portkey SDK client enables various features of Portkey in an easy to use config-as-code paradigm.

Install the Portkey SDK

Add the Portkey SDK to your application to interact with Portkey's gateway.

NodeJS

npm install --save portkey-ai

Python

pip install portkey-ai

Export Portkey API Key

export PORTKEY_API_KEY=""

Basic Client Setup

The basic Portkey SDK client needs 2 required parameters

1. The Portkey Account's API key to authenticate all your requests

2. The virtual key of the AI provider you want to use OR The config being used

This is achieved through headers when you're using the REST API.

For example,

NodeJS

import Portkey from 'portkey-ai';

// Construct a client with a virtual key
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "VIRTUAL_KEY"
})

// Construct a client with a config id
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    config: "cf-***" // Supports a string config slug or a config object
})

Find more info on what's available through configs here.

Python

from portkey_ai import Portkey

# Construct a client with a virtual key
portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="VIRTUAL_KEY"
)

# Construct a client with provider and provider API key
portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config="cf-***" # Supports a string config slug or a config object
)

Find more info on what's available through configs here.

cURL

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-virtual-key: $VIRTUAL_KEY" \ 
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

curl https://api.portkey.ai/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'x-portkey-api-key: $PORTKEY_API_KEY' \
  -H 'x-portkey-config: cf-***' \ 
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

Find more info on what's available through configs here.

Making a Request

You can then use the client to make completion and other calls like this

NodeJS

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
});

console.log(chatCompletion.choices);

Python

completion = portkey.chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
)

Passing Trace ID or Metadata

You can choose to override the configuration in individual requests as well and send trace id or metadata along with each request.

NodeJS

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
}, {
    traceId: "39e2a60c-b47c-45d8", 
    metadata: {"_user": "432erf6"}
});

console.log(chatCompletion.choices);

Python

completion = portkey.with_options(
    trace_id = "TRACE_ID", 
    metadata = {"_user": "USER_IDENTIFIER"}
).chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'gpt-3.5-turbo'
)

---

Async Usage

Portkey's Python SDK supports Async usage - just use AsyncPortkey instead of Portkey with await:

Python

import asyncio
from portkey_ai import AsyncPortkey

portkey = AsyncPortkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="VIRTUAL_KEY"
)

async def main():
    chat_completion = await portkey.chat.completions.create(
        messages=[{'role': 'user', 'content': 'Say this is a test'}],
        model='gpt-4'
    )

    print(chat_completion)

asyncio.run(main())

---

Parameters

Following are the parameter keys that you can add while creating the Portkey client.

Keeping in tune with the most popular language conventions, we use:

• camelCase for Javascript keys

• snake_case for Python keys

• hyphenated-keys for the headers

NodeJS

Parameter	Type	Key
API Key Your Portkey account's API Key.	stringrequired	apiKey
Virtual Key The virtual key created from Portkey's vault for a specific provider	string	virtualKey
Config The slug or config object to use	stringobject	config
Provider The AI provider to use for your calls. (supported providers).	string	provider
Base URL You can edit the URL of the gateway to use. Needed if you're self-hosting the AI gateway	string	baseURL
Trace ID An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.	string	traceID
Metadata Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey. You can also send any other keys as part of this object.	object	metadata
Cache Force Refresh Force refresh the cache for your request by making a new call and storing that value.	boolean	cacheForceRefresh
Cache Namespace Partition your cache based on custom strings, ignoring metadata and other headers.	string	cacheNamespace
Custom Host Route to locally or privately hosted model by configuring the API URL with custom host	string	customHost
Forward Headers Forward sensitive headers directly to your model's API without any processing from Portkey.	array of string	forwardHeaders
Azure OpenAI Headers Configuration headers for Azure OpenAI that you can send separately	string	azureResourceName azureDeploymentId azureApiVersion azureModelName
Google Vertex AI Headers Configuration headers for Vertex AI that you can send separately	string	vertexProjectId vertexRegion
AWS Bedrock Headers Configuration headers for Bedrock that you can send separately	string	awsAccessKeyId awsSecretAccessKey awsRegion awsSessionToken

Python

Parameter	Type	Key
API Key Your Portkey account's API Key.	stringrequired	api_key
Virtual Key The virtual key created from Portkey's vault for a specific provider	string	virtual_key
Config The slug or config object to use	stringobject	config
Provider The AI provider to use for your calls. (supported providers).	string	provider
Base URL You can edit the URL of the gateway to use. Needed if you're self-hosting the AI gateway	string	base_url
Trace ID An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.	string	trace_id
Metadata Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey. You can also send any other keys as part of this object.	object	metadata
Cache Force Refresh Force refresh the cache for your request by making a new call and storing that value.	boolean	cache_force_refresh
Cache Namespace Partition your cache based on custom strings, ignoring metadata and other headers.	string	cache_namespace
Custom Host Route to locally or privately hosted model by configuring the API URL with custom host	string	custom_host
Forward Headers Forward sensitive headers directly to your model's API without any processing from Portkey.	array of string	forward_headers
Azure OpenAI Headers Configuration headers for Azure OpenAI that you can send separately	string	azure_resource_name azure_deployment_id azure_api_version azure_model_name
Google Vertex AI Headers Configuration headers for Vertex AI that you can send separately	string	vertex_project_id vertex_region
AWS Bedrock Headers Configuration headers for Bedrock that you can send separately	string	aws_access_key_id aws_secret_access_key aws_region aws_session_token

REST Headers

Parameter	Type	Header Key
API Key Your Portkey account's API Key.	stringrequired	x-portkey-api-key
Virtual Key The virtual key created from Portkey's vault for a specific provider	string	x-portkey-virtual-key
Config The slug or config object to use	string	x-portkey-config
Provider The AI provider to use for your calls. (supported providers).	string	x-portkey-provider
Base URL You can edit the URL of the gateway to use. Needed if you're self-hosting the AI gateway	string	Change the request URL
Trace ID An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.	string	x-portkey-trace-id
Metadata Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey. You can also send any other keys as part of this object.	string	x-portkey-metadata
Cache Force Refresh Force refresh the cache for your request by making a new call and storing that value.	boolean	x-portkey-cache-force-refresh
Cache Namespace Partition your cache based on custom strings, ignoring metadata and other headers	string	x-portkey-cache-namespace
Custom Host Route to locally or privately hosted model by configuring the API URL with custom host	string	x-portkey-custom-host
Forward Headers Forward sensitive headers directly to your model's API without any processing from Portkey.	array of string	x-portkey-forward-headers
Azure OpenAI Headers Configuration headers for Azure OpenAI that you can send separately	string	x-portkey-azure-resource-name x-portkey-azure-deployment-id x-portkey-azure-api-version api-key x-portkey-azure-model-name
Google Vertex AI Headers Configuration headers for Vertex AI that you can send separately	string	x-portkey-vertex-project-id x-portkey-vertex-region
AWS Bedrock Headers Configuration headers for Bedrock that you can send separately	string	x-portkey-aws-session-token x-portkey-aws-secret-access-key x-portkey-aws-region x-portkey-aws-session-token