{"slug": "boxagnts-introduction-7-openai-api-and-anthropic-api", "title": "BoxAgnts Introduction (7) — OpenAI API and Anthropic API", "summary": "BoxAgnts has built a model-agnostic abstraction layer that lets users switch between AI providers—including OpenAI, Anthropic, and Google Gemini—by changing a single parameter. The system normalizes each provider's unique API format, authentication, and streaming protocol into unified `ProviderRequest` and `ProviderResponse` types, supporting over 30 providers through native and OpenAI-compatible implementations. The `LlmProvider` trait and `ProviderRegistry` handle all internal conversion logic, enabling seamless model swapping without altering upper-layer code.", "body_md": "The 2025 AI model market is in full bloom. But each provider has its own API format, authentication method, and streaming protocol. BoxAgnts' design goal: **users switch models by changing just one parameter, with all internal logic remaining unchanged**.\n\nThis article dissects this abstraction across four levels:\n\n`LlmProvider`\n\ntrait defines a \"model provider\"Everything starts with the interface definition:\n\n```\n// boxagnts-api/src/provider.rs\n#[async_trait]\npub trait LlmProvider: Send + Sync {\n    fn id(&self) -> &ProviderId;                              // Unique identifier\n    fn name(&self) -> &str;                                   // Human-readable name\n\n    async fn create_message(                                  // Non-streaming request\n        &self,\n        request: ProviderRequest,\n    ) -> Result<ProviderResponse, ProviderError>;\n\n    async fn create_message_stream(                           // Streaming request\n        &self,\n        request: ProviderRequest,\n    ) -> Result<\n        Pin<Box<dyn Stream<Item = Result<StreamEvent, ProviderError>> + Send>>,\n        ProviderError,\n    >;\n\n    async fn list_models(&self) -> Result<Vec<ModelInfo>, ProviderError>;  // Model list\n    async fn check_connectivity(&self) -> Result<ProviderStatus, ProviderError>; // Health check\n    fn capabilities(&self) -> ProviderCapabilities;           // Capability declaration\n}\n```\n\nBoth input and output use provider-agnostic unified types:\n\n```\npub struct ProviderRequest {\n    pub model: String,\n    pub messages: Vec<Message>,          // Unified conversation format\n    pub system_prompt: Option<SystemPrompt>,\n    pub tools: Vec<ToolDefinition>,      // Unified tool definitions\n    pub max_tokens: u32,\n    pub temperature: Option<f64>,\n    pub thinking: Option<ThinkingConfig>, // Deep thinking configuration\n    pub provider_options: Value,          // Provider-specific parameters\n}\n\npub struct ProviderResponse {\n    pub id: String,\n    pub content: Vec<ContentBlock>,      // Unified content blocks\n    pub stop_reason: StopReason,         // Unified stop reason\n    pub usage: UsageInfo,                // Token usage\n    pub model: String,\n}\n```\n\nThe core value of the normalization layer: **whether the underlying is Claude, GPT, or Gemini, upper-layer code only sees ProviderRequest and ProviderResponse**.\n\n```\n// boxagnts-api/src/registry.rs\npub struct ProviderRegistry {\n    providers: HashMap<ProviderId, Arc<dyn LlmProvider>>,\n    default_provider_id: ProviderId,\n}\n\nfn provider_from_key(provider_id: &str, key: String) -> Option<Arc<dyn LlmProvider>> {\n    match provider_id {\n        // Native implementations — each with its own API format\n        \"anthropic\" => Some(Arc::new(AnthropicProvider::from_config(...))),\n        \"openai\"    => Some(Arc::new(OpenAiProvider::new(key))),\n        \"google\"    => Some(Arc::new(GoogleProvider::new(key))),\n        \"github-copilot\" => Some(Arc::new(CopilotProvider::new(key))),\n        \"cohere\"    => Some(Arc::new(CohereProvider::new(key))),\n\n        // OpenAI-compatible providers — share the same conversion logic, only change base_url\n        \"deepseek\", \"groq\", \"ollama\", \"mistral\", \"xai\",\n        \"perplexity\", \"openrouter\", \"siliconflow\", \"moonshot\",\n        \"zhipu\", \"stepfun\", \"fireworks\", \"llamacpp\",\n        \"sambanova\", \"huggingface\", \"nvidia\", \"cerebras\",\n        // ... 30+ OpenAI-compatible providers in total\n        _ => None,\n    }\n}\n```\n\nThree implementation strategies:\n\n| Type | Representative | Conversion Strategy | Count |\n|---|---|---|---|\nNative Anthropic |\nclaude-sonnet-4-5 | Near-zero conversion (internal format = Anthropic format) | 1 |\nNative OpenAI |\ngpt-4o, o3 | ProviderRequest → Chat Completions | 1 |\nNative Google |\ngemini-2.5-flash | ProviderRequest → generateContent | 1 |\nOpenAI Compatible |\ndeepseek, groq, ollama, etc. | Same logic as OpenAI, only URL changes | 30+ |\nOther Native |\ngithub-copilot, cohere | Independent format conversion | 3+ |\n\nAnthropic, OpenAI, Google Gemini — three APIs with vast differences in message format. Understanding these differences is essential to understanding the value of the conversion layer.\n\n| Feature | Anthropic | OpenAI | Google Gemini |\n|---|---|---|---|\n| Location | Top-level `\"system\"` field |\nmessages[0], `role:\"system\"`\n|\nTop-level `\"systemInstruction\"` field |\n| Type | string or ContentBlock array | string only | content parts array only |\n\n```\n// Anthropic — top-level standalone field\n{\"model\": \"claude-sonnet-4-5\", \"system\": \"You are helpful.\", \"messages\": [...]}\n\n// OpenAI — embedded in messages array\n{\"model\": \"gpt-4o\", \"messages\": [{\"role\":\"system\",\"content\":\"You are helpful.\"}, ...]}\n\n// Google — uses systemInstruction field, structure differs from messages\n{\n  \"systemInstruction\": {\"parts\": [{\"text\": \"You are helpful.\"}]},\n  \"contents\": [{\"role\": \"user\", \"parts\": [{\"text\": \"Hello\"}]}]\n}\n```\n\n| Feature | Anthropic | OpenAI | |\n|---|---|---|---|\n| Field | `\"tools\": [{name, description, input_schema}]` |\n`\"tools\": [{type:\"function\", function:{...}}]` |\n`\"tools\": [{functionDeclarations: [{name, description, parameters}]}]` |\n| Wrapping Layers | 0 | 1 | 1, with different nesting names |\n\n```\n// Anthropic — native block in content array\n{\"content\": [{\"type\":\"tool_use\", \"id\":\"toolu_01A\", \"name\":\"read\", \"input\": {...}}]}\n\n// OpenAI — standalone tool_calls array, arguments is JSON string\n{\"tool_calls\": [{\"id\":\"call_abc\", \"function\": {\"name\":\"read\", \"arguments\": \"{\\\"path\\\":\\\"...\\\"}\"}}]}\n\n// Google — functionCall embedded in parts, args is JSON object\n{\"candidates\": [{\"content\": {\"parts\": [{\"functionCall\": {\"name\":\"read\", \"args\": {...}}}]}}]}\n// Anthropic — tool_result is a block in the user message content array\n{\"role\":\"user\", \"content\": [{\"type\":\"tool_result\", \"tool_use_id\":\"toolu_01A\", \"content\":\"...\"}]}\n\n// OpenAI — requires a separate role: \"tool\" message\n{\"role\":\"tool\", \"tool_call_id\":\"call_abc\", \"content\":\"...\"}\n\n// Google — functionResponse embedded in user content parts\n{\"role\":\"user\", \"parts\": [{\"functionResponse\": {\"name\":\"read\", \"response\": {...}}}]}\n```\n\n| Anthropic | OpenAI | |\n|---|---|---|\n`user` |\n`user` |\n`user` |\n`assistant` |\n`assistant` |\n`model` |\n\nGoogle uses `model`\n\ninstead of `assistant`\n\n— this is the most easily overlooked but most error-prone difference.\n\n`OpenAiProvider`\n\nis the most complete example of the conversion layer:\n\n```\n// boxagnts-api/src/providers/openai.rs\nimpl OpenAiProvider {\n    fn to_openai_messages(\n        messages: &[Message],\n        system_prompt: Option<&SystemPrompt>,\n    ) -> Vec<Value> {\n        let mut result: Vec<Value> = Vec::new();\n\n        // Step 1: system prompt → role: \"system\" message\n        if let Some(sys) = system_prompt {\n            result.push(json!({\"role\": \"system\", \"content\": sys_text}));\n        }\n\n        for msg in messages {\n            match msg.role {\n                Role::User => {\n                    // User messages may mix text and tool_result blocks\n                    // tool_result needs to be split into separate role: \"tool\" messages\n                    Self::append_user_messages(&mut result, &msg.content);\n                }\n                Role::Assistant => {\n                    let (text, tool_calls) = Self::assistant_content_to_openai(&msg.content);\n                    result.push(json!({\n                        \"role\": \"assistant\",\n                        \"content\": text,\n                        \"tool_calls\": tool_calls\n                    }));\n                }\n            }\n        }\n        result\n    }\n\n    fn to_openai_tools(tools: &[ToolDefinition]) -> Vec<Value> {\n        tools.iter().map(|td| {\n            json!({\n                \"type\": \"function\",\n                \"function\": {\n                    \"name\": td.name,\n                    \"description\": td.description,\n                    \"parameters\": td.input_schema\n                }\n            })\n        }).collect()\n    }\n}\n```\n\nThe most complex part is tool_use_id sanitization — Anthropic's tool IDs (e.g., `toolu_01Bx...`\n\n) may contain characters that OpenAI does not accept.\n\n`GoogleProvider`\n\nshows how to handle an API format that is different from both Anthropic and OpenAI:\n\n```\n// boxagnts-api/src/providers/google.rs\n// URL pattern completely different from OpenAI's /v1/chat/completions\nfn generate_url(&self, model: &str) -> String {\n    format!(\n        \"{}/v1beta/models/{}:generateContent?key={}\",\n        self.base_url, model, self.api_key  // API Key in URL query parameters!\n    )\n}\n```\n\nKey differences from OpenAI:\n\n| Difference | Google Gemini | OpenAI |\n|---|---|---|\n| API Key Location | URL query parameter `?key=`\n|\nHTTP Header `Authorization: Bearer`\n|\n| Endpoint Format | `/v1beta/models/{model}:generateContent` |\n`/v1/chat/completions` |\n| Streaming Endpoint | `/v1beta/models/{model}:streamGenerateContent?alt=sse` |\n`/v1/chat/completions` + `stream:true`\n|\n| Message Roles |\n`user` / `model` (not assistant) |\n`user` / `assistant`\n|\n| Tool Results |\n`functionResponse` in parts |\nSeparate `role: tool` message |\n| Image Input |\n`inlineData` base64 |\n`image_url` or content parts |\n\n`ThinkingConfig`\n\nis the normalized deep thinking configuration — but different providers handle it completely differently:\n\n```\n// Normalized configuration\npub struct ThinkingConfig {\n    pub budget_tokens: u32,   // Thinking token budget\n}\n\n// When building ProviderRequest, decides whether to pass based on provider capabilities\nlet provider_request = ProviderRequest {\n    // ...\n    thinking: if caps.thinking {\n        effective_thinking_budget\n            .map(|b| ThinkingConfig::enabled(b))\n    } else {\n        None  // This provider doesn't support thinking, don't pass\n    },\n};\n```\n\n| Provider | Thinking Support | How It's Passed |\n|---|---|---|\n| Anthropic (Claude 3.5+) | ✓ | `\"thinking\": {\"type\": \"enabled\", \"budget_tokens\": N}` |\n| Google (Gemini 2.5+) | ✓ | `\"thinkingConfig\": {\"thinkingBudget\": N}` |\n| OpenAI (o1/o3 series) | Partial | Via `reasoning_effort` parameter |\n| Other OpenAI Compatible | Mostly unsupported | Not passed |\n\nAt request construction time, `ProviderCapabilities`\n\ndeclares each provider's capabilities:\n\n```\npub struct ProviderCapabilities {\n    pub thinking: bool,              // Whether deep thinking is supported\n    pub prompt_caching: bool,        // Whether prompt caching is supported\n    pub image_input: bool,           // Whether image input is supported\n    pub native_tool_use: bool,       // Whether native tool calling exists\n    pub supports_streaming: bool,    // Whether streaming responses are supported\n    // ...\n}\n```\n\nOpenAI-compatible providers' APIs are roughly compatible, but all have subtle differences. `ProviderQuirks`\n\nhandles these:\n\n```\npub struct ProviderQuirks {\n    /// Specific error message patterns for context overflow\n    pub overflow_patterns: Vec<String>,\n    /// Local services that don't require API Keys (e.g., Ollama, LM Studio)\n    pub no_api_key_required: bool,\n    /// Whether streaming responses include usage info\n    pub include_usage_in_stream: bool,\n    /// Providers like DeepSeek need the reasoning_content field\n    pub reasoning_field: Option<String>,\n}\n```\n\nFor example, DeepSeek's streaming response returns reasoning content with a field name different from OpenAI's — adapted via `reasoning_field`\n\n. Ollama's context overflow error message is `\"exceeds the available context size\"`\n\n, while LM Studio's is `\"greater than the context length\"`\n\n— adapted via `overflow_patterns`\n\n.\n\nStreaming responses are also completely different across the three APIs:\n\n| Feature | Anthropic (SSE) | OpenAI (SSE) | Google (SSE) |\n|---|---|---|---|\n| Event Granularity | High: 6 event types (start/delta/stop × 2) | Low: each chunk is a complete delta | Medium: pushed by chunk, but structure is flat |\n| Tool call Increment | Fragmented send of `input_json_delta`\n|\nSingle send of complete `arguments` string |\nSingle send of complete `functionCall`\n|\n| Termination Signal |\n`message_stop` event |\n`data: [DONE]` marker |\nStream ends naturally |\n| Need to Reassemble by index | Yes (reassemble by index for multiple tool_use) | Yes | Yes |\n\nAll three formats are normalized to the same `StreamEvent`\n\nenum:\n\n```\npub enum StreamEvent {\n    MessageStart { id, model, usage },\n    ContentBlockStart { index, content_block },\n    TextDelta { text },\n    ThinkingDelta { thinking },\n    InputJsonDelta { index, partial_json },\n    ContentBlockStop { index },\n    MessageDelta { stop_reason, usage },\n    MessageStop,\n}\n```\n\nEach provider's error format is also different:\n\n```\n// Unified error types\npub enum ProviderError {\n    Auth { ... },             // Authentication failure\n    RateLimited { ... },      // Rate limiting\n    ContextOverflow { ... },  // Context exceeds window (matched via ProviderQuirks)\n    InvalidRequest { ... },   // Invalid request parameters\n    ServerError { ... },      // Server error\n    StreamError { ... },      // Stream interruption\n    Other { ... },            // Unknown error\n}\n```\n\nIn the query loop, specific errors trigger specific recovery strategies:\n\n```\nRateLimited / Overloaded → Switch to fallback_model\nContextOverflow → Trigger auto_compact\nStreamError (stall) → Retry (max 2 times, 45s timeout)\nAuth → Unrecoverable, return error\n```\n\nBoxAgnts defines environment variable name mappings for each provider:\n\n```\n// boxagnts-workspace/src/config.rs\npub fn api_key_env_vars_for_provider(provider_id: &str) -> &'static [&'static str] {\n    match provider_id {\n        \"anthropic\" => &[\"ANTHROPIC_API_KEY\"],\n        \"openai\" => &[\"OPENAI_API_KEY\"],\n        \"google\" => &[\"GOOGLE_API_KEY\", \"GOOGLE_GENERATIVE_AI_API_KEY\"],\n        \"deepseek\" => &[\"DEEPSEEK_API_KEY\"],\n        \"mistral\" => &[\"MISTRAL_API_KEY\"],\n        \"xai\" => &[\"XAI_API_KEY\"],\n        \"zhipu\" => &[\"ZHIPU_API_KEY\"],\n        // ... 40+ provider environment variables\n    }\n}\n```\n\nThree-tier priority: `Environment Variables > User Config JSON > No Default`\n\n. This design supports different scenarios such as multi-tenancy, CI/CD, and local development.\n\nBoxAgnts' model abstraction layer solves the essential problem of \"one set of code adapting to all APIs\":\n\n```\n┌──────────────────────────────────────────────┐\n│  boxagnts-query (Agent reasoning loop)        │\n│  Only uses ProviderRequest / ProviderResponse │\n└────────────────────┬─────────────────────────┘\n                     │\n┌────────────────────▼─────────────────────────┐\n│  LlmProvider trait                            │\n│  + ProviderRegistry (40+ providers)           │\n├──────────┬──────────┬──────────┬─────────────┤\n│Anthropic │ OpenAI   │ Google   │ OpenAiCompat │\n│Provider  │ Provider │ Provider │ (30+ vendors)│\n│(Near-zero│ (Full    │ (Independent│ (Shares    │\n│ conversion)│ format  │ format    │ OpenAI      │\n│          │ conversion)│ conversion)│ conversion  │\n│          │          │          │ +Quirks)     │\n└──────────┴──────────┴──────────┴─────────────┘\n```\n\nThree key capabilities:\n\n`--model`\n\nparameter`run_query_loop()`\n\nhas no idea what's underneathThis is not a simple \"adapter pattern\" — it's a production-grade abstraction validated against 40+ real-world APIs.", "url": "https://wpnews.pro/news/boxagnts-introduction-7-openai-api-and-anthropic-api", "canonical_source": "https://dev.to/guyoung/boxagnts-introduction-7-openai-api-and-anthropic-api-11o7", "published_at": "2026-05-31 08:03:58+00:00", "updated_at": "2026-05-31 08:11:17.826371+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "large-language-models", "artificial-intelligence"], "entities": ["BoxAgnts", "OpenAI", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/boxagnts-introduction-7-openai-api-and-anthropic-api", "markdown": "https://wpnews.pro/news/boxagnts-introduction-7-openai-api-and-anthropic-api.md", "text": "https://wpnews.pro/news/boxagnts-introduction-7-openai-api-and-anthropic-api.txt", "jsonld": "https://wpnews.pro/news/boxagnts-introduction-7-openai-api-and-anthropic-api.jsonld"}}