Structured Outputs

Structured outputs allow you to control and define the format of a model’s response using a schema. This makes it easy to enforce consistency, guarantee valid JSON, and parse results programmatically. Below we show two approaches: simple JSON output using json_object and fully validated structured output using Pydantic classes.

---

Simple JSON Example

JSON output is ideal for simple structured responses where you only need the model to return well-formed JSON.

To enable JSON Output, users should:

1. Set the response_format parameter to {'type': 'json_object'}.

2. Include the word "json" in the system or user prompt, and provide an example of the desired JSON format to guide the model in outputting valid JSON.

3. Set the max_tokens parameter reasonably to prevent the JSON string from being truncated midway.

import json
from openai import OpenAI

client = OpenAI(
    api_key="afm-13cf46d35fd48a6aa2da4c8d62424de8", 
    base_url="https://api.arcee.ai/api/v1"
)

system_prompt = """
The user will provide some exam text. Please parse the "question" and "answer" and output them in JSON format. 

EXAMPLE INPUT: 
Which is the highest mountain in the world? Mount Everest.

EXAMPLE JSON OUTPUT:
{
    "question": "What is the capital of France?",
    "answer": "Paris"
}
"""

user_prompt = "What is the capital of Italy?"

messages = [{"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}]

response = client.chat.completions.create(
    model="arcee-ai/trinity-mini-thinking",
    messages=messages,
    response_format={
        'type': 'json_object'
    }
)

print(json.loads(response.choices[0].message.content))

Pydantic Example

For strict validation and predictable structure, you can define a schema using a Pydantic class. The class is converted to a JSON Schema that the model must follow.

from openai import OpenAI
from pydantic import BaseModel
import json

# Define your structured output model
class QAResponse(BaseModel):
    question: str
    answer: str

# Convert the Pydantic class to a JSON schema
qa_schema = QAResponse.model_json_schema()

client = OpenAI(
    api_key="afm-13cf46d35fd48a6aa2da4c8d62424de8",
    base_url="https://api.arcee.ai/api/v1"
)

system_prompt = "Extract the question and answer. Only return valid JSON."

user_prompt = "Who discovered penicillin? Alexander Fleming."

response = client.chat.completions.create(
    model="arcee-ai/trinity-mini-thinking",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt}
    ],
    temperature=0,
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "QAResponse",
            "schema": qa_schema
        }
    }
)

# Validate and parse the model output into a Pydantic object
parsed = QAResponse(**json.loads(response.choices[0].message.content))

print(parsed)