OpenInference OpenAI Instrumentation
Project description
OpenInference OpenAI Instrumentation
Python auto-instrumentation library for OpenAI's python SDK.
The traces emitted by this instrumentation are fully OpenTelemetry compatible and can be sent to an OpenTelemetry collector for viewing, such as arize-phoenix
Installation
pip install openinference-instrumentation-openai
Quickstart
In this example we will instrument a small program that uses OpenAI and observe the traces via arize-phoenix.
Install packages.
pip install openinference-instrumentation-openai "openai>=1.26" arize-phoenix opentelemetry-sdk opentelemetry-exporter-otlp
Start the phoenix server so that it is ready to collect traces. The Phoenix server runs entirely on your machine and does not send data over the internet.
python -m phoenix.server.main serve
In a python file, setup the OpenAIInstrumentor and configure the tracer to send traces to Phoenix.
import openai
from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
# Optionally, you can also print the spans to the console.
tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
if __name__ == "__main__":
client = openai.OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Write a haiku."}],
max_tokens=20,
stream=True,
stream_options={"include_usage": True},
)
for chunk in response:
if chunk.choices and (content := chunk.choices[0].delta.content):
print(content, end="")
Since we are using OpenAI, we must set the OPENAI_API_KEY environment variable to authenticate with the OpenAI API.
export OPENAI_API_KEY=your-api-key
Now simply run the python file and observe the traces in Phoenix.
python your_file.py
FAQ
Q: How to get token counts when streaming?
A: To get token counts when streaming, install openai>=1.26 and set stream_options={"include_usage": True} when calling create. See the example shown above. For more info, see here.
More Info
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file openinference_instrumentation_openai-0.1.27.tar.gz.
File metadata
- Download URL: openinference_instrumentation_openai-0.1.27.tar.gz
- Upload date:
- Size: 21.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.13.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 26a4036bbe4745f90b3eab6a596c361301d86b98a01fb13f9f7f1883404a4856 |
|
| MD5 | 139582b085d47e1235058c09dd784f12 |
|
| BLAKE2b-256 | fa3b1fb2ad6c9317fd72bccb1e1c2dff1d26abaa6ebdb6fcf63cba9b1d5db7a5 |
File details
Details for the file openinference_instrumentation_openai-0.1.27-py3-none-any.whl.
File metadata
- Download URL: openinference_instrumentation_openai-0.1.27-py3-none-any.whl
- Upload date:
- Size: 28.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.13.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 1486a3db22b0ccb31216725baf65a8f2c913da138dd8e3ac3683132bfe55529d |
|
| MD5 | bcce98caf69f3273d99609fb1a371f49 |
|
| BLAKE2b-256 | 2b82c232040d7fa6d6d60ebd4adffc120e76a3a65706dd704eca5eca07e13975 |
