E3Rollouts
DEVELOPER RESOURCES / OVERVIEW

Every rollout.
One place.

Configure centrally. Consume in your service.
Change the split when you’re ready.

01 Save a configuration02 Refresh in your service03 Apply to new calls

E3 Rollouts — integration guide

#Overview

E3 Rollouts is a shared home for experiment percentages and JSON configuration. Save a value in the dashboard, read it in your service, and apply it to new calls or requests. You can make the same edits through Cosmos MCP.

Saving configuration does not route traffic by itself. Your service owns refreshing the value, selecting a variant, and keeping a fallback.

#Quickstart

Read the existing Ender configuration. This request does not change anything.

sh
curl --fail --silent --show-error \
  https://rollouts.e3g.ai/v1/entries/ender.stt.production

The response includes the configuration and its revision. These are example values; read the endpoint for the current revision.

json
{
  "key": "ender.stt.production",
  "kind": "rollout",
  "value": {"flux": 85, "qwen": 15},
  "revision": 4,
  "updated_at": "2026-09-24T06:43:00+00:00",
  "note": "Snapshot; not connected to live traffic"
}

Use value as your configuration and revision to identify which version your service used. Continue with code examples or the consumer contract.

Canonical domain: use https://rollouts.e3g.ai for the dashboard, API, and direct MCP. Python's default User-Agent is blocked by the current Cloudflare rule; the Python example identifies itself as E3Rollouts/1.0, verified to work. See operations and limits if a client cannot set headers.

#Code examples

Python

Uses only the standard library. Run this from startup or a background refresh, outside the audio processing path.

python
import json
from urllib.request import Request, urlopen

url = (
    'https://rollouts.e3g.ai'
    '/v1/entries/ender.stt.production'
)
request = Request(url, headers={'User-Agent': 'E3Rollouts/1.0'})
with urlopen(request, timeout=1) as response:
    record = json.load(response)
assert record['kind'] == 'rollout'
weights = record['value']
revision = record['revision']

This minimal snippet raises on failure. The reference implementation includes caching, validation, timeout, and fallback behavior.

JavaScript · Node.js

javascript
const response = await fetch(
  'https://rollouts.e3g.ai/v1/entries/ender.stt.production',
  { signal: AbortSignal.timeout(1000) }
);
if (!response.ok) throw new Error(`Rollouts HTTP ${response.status}`);
const record = await response.json();
console.log(record.value, record.revision);

#Create and update

Writes persist immediately. Only run this when a configuration change is intended. This creates an illustrative key, not an active negotiation experiment:

sh
curl --fail-with-body -X PUT \
  https://rollouts.e3g.ai/v1/entries/example.my-experiment \
  -H 'Content-Type: application/json' \
  --data '{"kind":"rollout","value":{"control":85,"experiment":15},"expected_revision":0,"note":"Initial example"}'

Use expected_revision 0 only for a new key. For an existing key, GET it first and use its exact revision in the PUT body. A successful write increments revision and records history atomically. HTTP 409 means another edit won: reread and review instead of blindly retrying. HTTP 422 means invalid key/value/shape; 413 means request too large; 404 means key missing; 503 means storage unavailable. Revert by saving a historical value against the current revision. No delete endpoint exists.

#API reference

Base URL: rollouts.e3g.ai. Reads and writes currently require no service key. Keep secrets and caller data out of this store.

An entry has key, kind (json or rollout), value, revision, updated_at, and note.

Method Path Purpose
GET /healthz Database readiness
GET /v1/entries List entries
GET /v1/entries/{key} Read value/revision; link opens the Ender example
PUT /v1/entries/{key} Create/update
GET /v1/entries/{key}/history Last 100 revisions; link opens the Ender example
GET /docs This guide
GET /docs.md Markdown guide
POST /mcp Streamable HTTP setup

Validation and errors

Keys and variant names allow letters, digits, dots, underscores and dashes, begin with a letter or digit, and are at most 128 characters. Rollout values contain 1–32 integer percentages from 0 to 100 totaling exactly 100. Generic JSON must be finite and at most 48KB; request bodies are limited to 64KB.

Status Meaning Action
404 Missing key Check the key; use your fallback
409 Stale revision Reread and review before retrying
413 Body too large Keep the request below 64KB
422 Invalid value Check kind, key, and percentage totals
503 Storage unavailable Keep last-known-good configuration

#Cosmos and MCP

Connect through Cosmos, using your normal Cosmos authentication. Start with documentation, locate E3 Rollouts, and call rollouts__get_docs for this guide.

Cosmos tool Arguments
rollouts__get_docs {}
rollouts__list_entries {}
rollouts__read_entry {"key":"ender.stt.production"}
rollouts__read_history {"key":"ender.stt.production"}
rollouts__update_entry {"key":"example.my-experiment","kind":"rollout","value":{"control":85,"experiment":15},"expected_revision":0,"note":"Initial example"}

The update example is a persistent mutation, not a read-only test.

Direct connection

Use the direct MCP endpoint with a Streamable HTTP MCP client. It is a protocol endpoint, not a webpage. Direct tools use the same names without rollouts__; no service key is needed. Refresh your client's tool list after connecting.

text
Transport: Streamable HTTP
URL: https://rollouts.e3g.ai/mcp
Authentication: none
Read tool: read_entry
Arguments: {"key": "ender.stt.production"}

#Consumer contract

Fetch outside the audio/turn critical path. Validate the expected key, kind, variants and revision. Cache in each service; refresh periodically with a bounded timeout. Keep last-known-good on failure and use the existing environment default on a cold start. Freeze a decision for the lifetime of a call/request and record the key/revision/source with telemetry. A process-local cache is lost on restart; durable cache and fleet propagation policies require explicit consumer implementation.

Percentage weights alone do not assign traffic. Consumers own stable hashing, subject IDs, variant ordering, fallback and experiment isolation. Changing these can reshuffle assignments. Verify actual traffic independently of stored weights.

#Reference implementation

Don't touch — example only. Negotiation draft PR #1059 is a reference. Do not merge, deploy, or enable it.

The draft includes a cached Python consumer, tests, and integration notes. It is not wired into negotiation's runtime and does not change Railway variables or call routing. GitHub access to the private repository is required.

Negotiation versus Ender

Negotiation's VOICE_AGENT_STT_PROVIDER supports deepgram or proxy. Ender independently owns upstream transcriber/turn-detector selection behind that proxy. ender.stt.production (flux/qwen) is a snapshot and must not be interpreted as negotiation's provider selector. The draft proposes a separate JSON key negotiation.stt.provider.production with {"provider":"proxy"}; that key is not created or activated. Keep proxy secrets, endpoints and timeouts in the deployment secret store.

#Scheduled ramps

Select an existing key in the dashboard. Its allocation and Schedule appear together. Edit Every, the service and direction, Points / step, and Stop when. The paired service automatically gets the opposite change; the preview shows both services now, after the next step, and at the final limit. With three or more services, the preview lists the shares that stay unchanged. A target cannot exceed the selected pair's combined share.

One coordinated rule per key is intentional for migrating traffic between an incumbent and a new service. For example: Qwen +5 points each hour, Flux −5, stopping at Qwen 25%. It moves 15/85 → 20/80 → 25/75, then stops. Review the canary results, raise the limit to 50% or 100%, and save again. To drain the new service instead, choose Qwen −5 until 0%; Flux returns to 100%. Other variants under the key stay unchanged.

To change an existing schedule, select its key, edit the loaded fields, and choose Save schedule changes. Saving resets the next interval from now. Pause preserves the current allocation; Save & resume starts from the current percentages. Completed schedules can be extended by raising the limit and saving again.

Allocation percentages are directly editable beside the pie chart. With exactly two services, editing either percentage balances the other to 100%. With more services, set the percentages to total 100%. The chart previews unsaved values. Save changes at the top right saves the allocation and pauses any active schedule; it is disabled until the configuration differs from the saved version and is valid. Restoring the original values disables it again. Discard changes restores the saved configuration. Save or discard allocation edits before configuring a schedule. Key metadata and the change note are under Key settings & change note.

  • Common intervals: 5, 15, 30 minutes; 1 hour; 24 hours. Enter the amount and choose minutes or hours.
  • Custom intervals use minutes or whole hours, with a minimum of 5 minutes and increments of 5 minutes. Maximum: 365 days.
  • The first step happens after the interval, on a five-minute scheduler check. Railway may run late. This is elapsed-time scheduling, not a local-time daily appointment.
  • Upward and downward ramps both stop exactly at the target. Other variants are unchanged; the sum remains 100%.
  • Pause keeps the current percentages. Save & resume starts a new interval from now.
  • Manual configuration edits automatically pause an active schedule. Refresh before replacing a schedule that another person or cron has changed.
  • Schedules persist across deployments. After downtime, at most one step runs; missed steps are not applied in a burst.
  • Each executed step writes a new configuration revision and history note in the same transaction. Duplicate cron requests cannot apply the same due step twice.

This is a time-based ramp, not a performance feedback controller. It does not read success rates, latency, or other evaluation metrics. Consumer refresh latency still applies.

REST and MCP

Action HTTP Cosmos tool
List and check scheduler heartbeat GET /v1/schedules rollouts__get_schedules
Read a schedule GET /v1/entries/{key}/schedule rollouts__get_schedule
Start, replace, or resume PUT /v1/entries/{key}/schedule rollouts__set_schedule
Pause POST /v1/entries/{key}/schedule/pause rollouts__stop_schedule

Read the entry and schedule first. expected_revision is the schedule revision (0 if none); expected_entry_revision is the current configuration revision. Start/replace body:

json
{
  "variant": "qwen",
  "counterpart": "flux",
  "step": 5,
  "target": 50,
  "interval_minutes": 15,
  "expected_revision": 0,
  "expected_entry_revision": 4
}

These revisions are illustrative. Saving this body starts an automatic ramp: use fresh revisions and an explicitly intended target. MCP takes the same fields plus key. Pause takes expected_revision (and key for MCP). Invalid intervals return 422; stale revisions return 409. No existing production ramp is enabled by deployment alone.

#Operations and limits

One Railway instance uses SQLite on /data. Values survive redeploys; history shares the same database. Brief redeploy downtime is possible. Backups are not configured, and history is not an independent backup. A separate Railway cron worker checks due schedules every five minutes; the UI shows its last check-in. The private tick endpoint requires a machine token, while the dashboard and configuration API remain open. No per-user audit attribution, routing engine, automatic consumer integration, SDK or automatic fallback exists in the server. Production adoption must explicitly address the currently public write access.

Resource Link
Rollout editor Open dashboard
Shareable documentation Integration guide
Source for agents Markdown guide
Cosmos MCP Cosmos endpoint
Direct MCP Rollouts endpoint
Example integration Draft PR #1059 — Don't touch
Implementation tracking COR-4421
E3 design language Doctrine

Client troubleshooting

The Railway hostname https://rollouts-production.up.railway.app remains a fallback for clients whose User-Agent is blocked by Cloudflare and cannot be configured. Cosmos currently uses that transport internally. All public links and examples use the canonical rollouts.e3g.ai domain. MCP endpoints require an MCP client, not a browser GET.