Portkey provides a robust and secure gateway to facilitate the integration of various models into your apps, including , , , and models hosted on the .
With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a system.
Provider Slug: fireworks-ai
Portkey SDK Integration with Fireworks Models
Portkey provides a consistent API to interact with models from various providers. To integrate Fireworks with Portkey:
1. Install the Portkey SDK
npm install --save portkey-ai
pip install portkey-ai
2. Initialize Portkey with the Virtual Key
To use Fireworks with Portkey, , then add it to Portkey to create the virtual key.
import Portkey from 'portkey-ai'
const portkey = new Portkey({
apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
})
from portkey_ai import Portkey
portkey = Portkey(
api_key="PORTKEY_API_KEY", # Defaults to os.env("PORTKEY_API_KEY")
virtual_key="FIREWORKS_VIRTUAL_KEY" # Your Virtual Key
)
3. Invoke Chat Completions with Fireworks
You can use the Portkey instance now to send requests to Fireworks API.
const chatCompletion = await portkey.chat.completions.create({
messages: [{ role: 'user', content: 'Say this is a test' }],
model: 'accounts/fireworks/models/llama-v3-70b-instruct',
});
console.log(chatCompletion.choices);
completion = portkey.chat.completions.create(
messages= [{ "role": 'user', "content": 'Say this is a test' }],
model= 'accounts/fireworks/models/llama-v3-70b-instruct'
)
print(completion)
Now, let's explore how you can use Portkey to call other models (vision, embedding, image) on the Fireworks API:
Using Embeddings Models
const embeddings = await portkey.embeddings.create({
input: "create vector representation on this sentence",
model: "thenlper/gte-large",
});
console.log(embeddings);
embeddings = portkey.embeddings.create(
input='create vector representation on this sentence',
model='thenlper/gte-large'
)
print(embeddings)
import Portkey from 'portkey-ai';
import fs from 'fs';
const portkey = new Portkey({
apiKey: "PORTKEY_API_KEY",
virtualKey: "FIREWORKS_VIRTUAL_KEY"
});
async function main(){
const image = await portkey.images.generate({
model: "accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
prompt: "An orange elephant in a purple pond"
});
const imageData = image.data[0].b64_json as string;
fs.writeFileSync("fireworks-image-gen.png", Buffer.from(imageData, 'base64'));
}
main()
from portkey_ai import Portkey
import base64
from io import BytesIO
from PIL import Image
portkey = Portkey(
api_key="PORTKEY_API_KEY",
virtual_key="FIREWORKS_VIRTUAL_KEY"
)
image = portkey.images.generate(
model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
prompt="An orange elephant in a purple pond"
)
Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("fireworks-image-gen.png")
Fireworks Grammar Mode
Grammar mode is set with the response_format param. Just pass your grammar definition with {"type": "grammar", "grammar": grammar_definition}
Let's say you want to classify patient requests into 3 pre-defined classes:
from portkey_ai import Portkey
portkey = Portkey(
api_key="PORTKEY_API_KEY", # Defaults to os.env("PORTKEY_API_KEY")
virtual_key="FIREWORKS_VIRTUAL_KEY" # Your Virtual Key
)
patient_classification = """
root ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
"""
completion = portkey.chat.completions.create(
messages= [{ "role": 'user', "content": 'Say this is a test' }],
response_format={"type": "grammar", "grammar": patient_classification},
model= 'accounts/fireworks/models/llama-v3-70b-instruct'
)
print(completion)
import Portkey from 'portkey-ai'
const portkey = new Portkey({
apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
})
const patient_classification = `
root ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
`;
const chatCompletion = await portkey.chat.completions.create({
messages: [{ role: 'user', content: 'Say this is a test' }],
response_format: {"type": "grammar", "grammar": patient_classification},
model: 'accounts/fireworks/models/llama-v3-70b-instruct',
});
console.log(chatCompletion.choices);
NOTE: Fireworks Grammer Mode is not supported on Portkey prompts playground
Fireworks JSON Mode
You can force the model to return (1) An arbitrary JSON, or (2) JSON with given schema with Fireworks' JSON mode.
from portkey_ai import Portkey
portkey = Portkey(
api_key="PORTKEY_API_KEY", # Defaults to os.env("PORTKEY_API_KEY")
virtual_key="FIREWORKS_VIRTUAL_KEY" # Your Virtual Key
)
class Recipe(BaseModel):
title: str
description: str
steps: List[str]
json_response = portkey.chat.completions.create(
messages = [{ "role": 'user', "content": 'Give me a recipe for making Ramen, in JSON format' }],
model = 'accounts/fireworks/models/llama-v3-70b-instruct',
response_format = {
"type":"json_object",
"schema": Recipe.schema_json()
}
)
print(json_response.choices[0].message.content)
import Portkey from 'portkey-ai'
const portkey = new Portkey({
apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
})
asyn function main(){
const json_response = await portkey.chat.completions.create({
messages: [{role: "user",content: `Give me a recipe for making Ramen, in JSON format`}],
model: "accounts/fireworks/models/llama-v3-70b-instruct",
response_format: {
type: "json_object",
schema: {
type: "object",
properties: {
title: { type: "string" },
description: { type: "string" },
steps: { type: "array" }
}
}
}
});
}
console.log(json_response.choices[0].message.content);
main()
Fireworks Function Calling
Managing Fireworks Prompts
Once you're ready with your prompt, you can use the portkey.prompts.completions.create interface to use the prompt in your application.
Next Steps
The complete list of features supported in the SDK are available on the link below.
You'll find more information in the relevant sections:
Call any with the familiar OpenAI embeddings signature:
Portkey natively supports :
Portkey also supports calling in the familiar OpenAI signature:
Fireworks lets you define to constrain model outputs. You can use it to force the model to generate valid JSON, speak only in emojis, or anything else. ()
.
.
Portkey also supports function calling mode on Fireworks. .
You can manage all Fireworks prompts in the . All the current 49+ language models available on Fireworks are supported and you can easily start testing different prompts.
