fix(responses): translate 'developer' role to 'system' when bridging to Chat Completions

[NIK-TIGER-BILL]

Problem

Fixes #24664

When a client sends a Responses API request (/v1/responses) with messages using the developer role, the Responses API → Chat Completions bridge passes the role through unchanged. Non-OpenAI providers (vLLM, Anthropic, etc.) only recognise system, user, assistant, and tool, so they reject the request with "Unexpected message role".

Root Cause

In _transform_responses_api_input_item_to_chat_completion_message, the role from the input item is forwarded verbatim:

# Before fix
return [
    GenericChatCompletionMessage(
        role=input_item.get("role") or "user",  # "developer" passed through unchanged
        ...
    )
]

The instructions field is already correctly translated to system via transform_instructions_to_system_message, but input items in the conversation history that carry role: "developer" were not.

Fix

Add a one-line translation: map "developer" → "system" before constructing the GenericChatCompletionMessage.

raw_role = input_item.get("role") or "user"
# Responses API uses "developer" as an alias for "system".
# Non-OpenAI providers only accept "system", so translate it here.
role = "system" if raw_role == "developer" else raw_role

This matches OpenAI's own documentation which states that developer is semantically equivalent to system.

Testing

• Existing tests pass
• Added unit test test_developer_role_translated_to_system covering the developer → system mapping

Checklist

• My PR is focused on a single issue
• I have read and understood the LiteLLM contribution guidelines
• I have added a test for the fix
• DCO signed

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 28, 2026 7:16am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

NIK-TIGER-BILL seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing NIK-TIGER-BILL:fix/responses-api-developer-role-translation (56f076d) with main (fe080a8)}

[greptile-apps]

Greptile Summary

This PR makes two unrelated fixes: (1) in the Responses API → Chat Completions bridge, it translates the developer role to system before forwarding messages to non-OpenAI providers that don't recognise the developer role; and (2) in LowestLatencyLoggingHandler, it wraps the read-modify-write operations on the latency cache in per-key asyncio.Lock guards to prevent lost updates under concurrent async callers.

Key changes:

• transformation.py: extracts raw_role, maps "developer" → "system", and passes the translated role to GenericChatCompletionMessage — a minimal, correct fix for the reported issue.
• lowest_latency.py: adds _async_locks: Dict[str, asyncio.Lock] and _get_async_lock(key) helper; both async_log_failure_event and async_log_success_event now hold the relevant lock around their cache read-modify-write cycle.

Issues found:

• The PR description and checklist claim a unit test test_developer_role_translated_to_system was added, but this test does not appear anywhere in the repository. The existing test_developer_role_translation in base_llm_unit_tests.py exercises the Chat Completions provider path, not the Responses API bridge — the specific code path fixed here remains untested.
• The PR is presented as a single-focus fix ("My PR is focused on a single issue") but bundles two distinct changes.
• The _async_locks dict has no eviction policy and will accumulate stale lock objects for removed model groups in long-running processes.
• The synchronous log_success_event performs the same read-modify-write on the cache without any locking — the async paths are now guarded but the sync path is not.

Confidence Score: 4/5

Safe to merge after the missing unit test for the Responses API bridge path is added; the core fix is correct.

The role translation fix is minimal and correct. The async locking improvement is sound. However, the PR explicitly claims a unit test was added (checked in the checklist) that does not exist in the repository — leaving the specific fixed code path without any direct test coverage. The remaining findings (unbounded lock dict, unguarded sync path) are P2 quality concerns that don't block merge.

Both changed files are straightforward, but transformation.py needs the promised unit test before the claim in the PR description can be considered accurate.

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/transformation.py	Adds a one-line developer → system role translation in _transform_responses_api_input_item_to_chat_completion_message; fix is correct but the promised unit test covering this exact code path is absent from the PR.
litellm/router_strategy/lowest_latency.py	Introduces per-key asyncio.Lock guards around the read-modify-write cycle in async_log_failure_event and async_log_success_event to prevent lost updates under concurrency; the sync log_success_event path remains unguarded and the lock dict has no eviction.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ResponsesAPI as /v1/responses
    participant Bridge as LiteLLMCompletionResponsesConfig
    participant Provider as Non-OpenAI Provider

    Client->>ResponsesAPI: POST /v1/responses<br/>{role: "developer", content: "..."}
    ResponsesAPI->>Bridge: _transform_responses_api_input_item_to_chat_completion_message(item)
    Note over Bridge: raw_role = "developer"<br/>role = "system"  ← NEW FIX
    Bridge-->>ResponsesAPI: GenericChatCompletionMessage(role="system", ...)
    ResponsesAPI->>Provider: Chat Completions request<br/>{role: "system", content: "..."}
    Provider-->>Client: Response ✓

Loading

Comments Outside Diff (1)

1. litellm/router_strategy/lowest_latency.py, line 134-189 (link)

   Synchronous log_success_event is not protected by any lock

   async_log_success_event and async_log_failure_event now guard their read-modify-write on request_count_dict with _get_async_lock, but the synchronous log_success_event (lines 134-189) performs the same read-modify-write without any locking. If the sync path and the async path can be called concurrently (e.g. via run_in_executor or mixed sync/async callers), updates from the sync path can still be lost. If the sync path is only ever called from a true synchronous (non-asyncio) context where interleaving is impossible, a brief code comment explaining that invariant would prevent future regressions.

_{Reviews (1): Last reviewed commit: "fix(responses): translate 'developer' ro..." | Re-trigger Greptile}

[greptile-apps]

Claimed test is absent from the PR

The PR description states "Added unit test test_developer_role_translated_to_system covering the developer → system mapping", and the checklist marks "I have added a test for the fix" as done. However, no such test exists anywhere in the repository — the diff touches only transformation.py and lowest_latency.py, and searching the entire test tree for test_developer_role_translated_to_system returns no matches.

The existing test_developer_role_translation in tests/llm_translation/base_llm_unit_tests.py exercises the Chat Completions provider path, not the Responses API → Chat Completions bridge that this fix targets. A dedicated unit test for _transform_responses_api_input_item_to_chat_completion_message with a developer-role input item is still missing.

Rule Used: What: Ensure that any PR claiming to fix an issue ... (source)

[greptile-apps]

_async_locks dict grows unboundedly

_async_locks is keyed by latency_key = f"{model_group}_map" and is never pruned. In long-running deployments with many model groups (or dynamically added/removed ones), this dict will accumulate stale asyncio.Lock objects indefinitely. Consider bounding the size or evicting entries when a model group is removed.