LangSmith

LangSmith is an end-to-end platform for debugging, evaluating, and monitoring language model applications. It integrates seamlessly with LangChain and other LLM frameworks, allowing developers to visualize traces, inspect prompts, measure performance, and manage production-grade evaluation workflows. LangSmith helps you build more reliable AI systems by offering deep observability and experiment tracking—turning opaque model calls into structured, analyzable data.

This tutorial will guide you through integrating Arcee AI language models into LangSmith using an OpenAI-compatible endpoint, enabling full tracing, monitoring, and evaluation of Arcee-powered agents and workflows. This tutorial focuses on tracing; however, the same logic applies for other LangSmith features.

Prerequisites

• Python: >=3.10 and <3.14

• LangSmith Account and API Key:

  • If you do not have one, you can create one here

• Arcee Model Endpoint: Local or remote OpenAI-compatible Arcee API (e.g., http://127.0.0.1:8080/v1)

Integration Steps

1. Create a folder for your project

mkdir arceeai_langsmith && cd arceeai_langsmith

1. Setup Python Virtual Environment and Install LangSmith tooling

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install langsmith langchain-openai

If you run into any errors installing langsmith, follow their documentation.

1. LangSmith uses environment variables for configuration. We'll also include the Arcee AI variables here. Create a .env file and include the following

# .env
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT=https://api.smith.langchain.com
LANGSMITH_API_KEY=<your-api-key>
LANGSMITH_PROJECT=arcee-langsmith-project

ARCEE_API_BASE=http://127.0.0.1:8080/v1
ARCEE_API_KEY=<your-api-key>
ARCEE_MODEL_NAME="afm-4.5b"

1. Create a new python file called arceeai_langsmith.py and copy in the following contents

import os
from typing_extensions import Annotated, TypedDict

from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langchain_core.messages import AnyMessage, SystemMessage, HumanMessage
from langchain_openai import ChatOpenAI

class State(TypedDict):
    messages: Annotated[list[AnyMessage], add_messages]

# Configure Arcee AI Model
ARCEE_BASE = os.getenv("ARCEE_API_BASE", "http://127.0.0.1:8080/v1")
ARCEE_KEY  = os.getenv("ARCEE_API_KEY", "your-arcee-api-key")
ARCEE_MODEL = os.getenv("ARCEE_MODEL_NAME", "afm-4.5b")

# Initialize Arcee AI model with OpenAI-compatible configuration
arcee_llm = ChatOpenAI(
    model=ARCEE_MODEL,
    api_key=ARCEE_KEY,
    base_url=ARCEE_BASE,
)

# Define a simple graph node that uses the Arcee AI model
def summarize(state: State) -> State:
    # Add a system instruction once, if not present
    msgs = state["messages"]
    if not any(getattr(m, "role", "") == "system" for m in msgs):
        msgs = [SystemMessage(content="You are a concise technical writer.")] + msgs
    # Ask Arcee to respond
    ai = arcee_llm.invoke(msgs)
    return {"messages": [ai]}

# Build the graph that LangGraph will execute
builder = StateGraph(State)
builder.add_node("summarize", summarize)
builder.add_edge(START, "summarize")
builder.add_edge("summarize", END)
graph = builder.compile()

if __name__ == "__main__":
    text = """Arcee AI is a foundation model provider with a focus on building the highest performing models per parameter. 
They offer a range of models from on-device and edge optimized models to large language models. Their suite of models 
provides customers with the flexibility to choose the right model for the right task. All models are released Apache 2.0 
enabling the community to use safe, built-in-the-US models in their own environment or via the Arcee AI API platform."""
    inputs: State = {"messages": [HumanMessage(content=f"Summarize the following in three bullets:\n\n{text}")]}

    # Execute the LangGraph agent
    result = graph.invoke(inputs)
    
    # Print the results
    print("\n=== RESULT ===\n")
    print(result["messages"][-1].content)

This works out-of-the-box if you have an Arcee AI model running locally on your laptop. If you do not, change ARCEE_BASE , ARCEE_KEY , and ARCEE_MODEL .

1. Run your Arcee AI powered LangGraph Agent with LangSmith Tracing

python arceeai_langsmith.py