> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trygravity.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference API

> Serve free LLM inference to your users with Gravity's OpenAI-compatible endpoint. Covered by ad payouts during the pilot.

Gravity Inference is an OpenAI-compatible chat API for AI platforms (publishers). You call it the same way you call OpenAI Chat Completions. Gravity covers model cost during the pilot from ad payouts, so you can offer a free tier without BYOK.

Base URL: `https://inference.trygravity.ai/v1`

## Before you start

* Your Gravity AI platform account is approved.
* Gravity has enabled Inference for your account (ask your Gravity contact if the **Inference** page is missing in the dashboard).
* You will create a **dedicated inference key** (`gr_inf_...`). Do not reuse your publisher ads API key.

## 1. Create an inference key

1. Open [app.trygravity.com/platform/inference](https://app.trygravity.com/platform/inference).
2. Create a key. Copy it once. Gravity stores only a hash.
3. Save it as `GRAVITY_INFERENCE_KEY` on your server.

> [!WARNING]
>
> Keep the inference key on your server. Do not put it in client code or a public repository.

## 2. Models

| Model ID | Label |
| - | - |
| `@cf/deepseek-ai/deepseek-v4.1-flash-1017` | DeepSeek V4.1 Flash |
| `@cf/zai-org/glm-5.3-flash` | GLM 5.3 Flash |

List models with `GET /v1/models` using your inference key. Context window sizes are not claimed in the API.

## 3. Call chat completions

Use any OpenAI SDK. Point `base_url` at Gravity and pass your inference key.

Every chat request **must** include a session ID (1 to 256 characters) so Gravity can tie inference to ads in the same session. Send it in any one of:

1. `session_id` in the JSON body (preferred)
2. `X-Gravity-Session-Id` header
3. OpenAI `user` field

When more than one is present, `session_id` wins, then the header, then `user`. A request with none of them returns `400` with code `session_id_required` and never reaches the model.

Optional:

* `email_hash`: SHA-256 hex of the lowercased, trimmed email (raw emails are rejected). Ties the request to the same person as ads `hashed_email`.
* OpenAI `user` (1 to 128 characters): also recorded as the end user for rate limits and usage.

None of these identity fields are sent to the model provider.

```python Python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.trygravity.ai/v1",
    api_key="YOUR_DEDICATED_GRAVITY_INFERENCE_KEY",
    max_retries=0,
)

result = client.chat.completions.create(
    model="@cf/deepseek-ai/deepseek-v4.1-flash-1017",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=1024,
    stream=False,
    extra_body={"session_id": "YOUR_SESSION_ID"},
)
print(result.choices[0].message.content)
```

```ts TypeScript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.trygravity.ai/v1",
  apiKey: process.env.GRAVITY_INFERENCE_KEY!,
  maxRetries: 0,
});

const result = await client.chat.completions.create({
  model: "@cf/deepseek-ai/deepseek-v4.1-flash-1017",
  messages: [{ role: "user", content: "Hello" }],
  max_tokens: 1024,
  // @ts-expect-error Gravity extension
  session_id: "YOUR_SESSION_ID",
});
```

```bash curl theme={null}
curl https://inference.trygravity.ai/v1/chat/completions \
  -H "Authorization: Bearer $GRAVITY_INFERENCE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "@cf/deepseek-ai/deepseek-v4.1-flash-1017",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 1024,
    "session_id": "YOUR_SESSION_ID"
  }'
```

### Supported features

* Streaming (`stream: true`, SSE ending in `data: [DONE]`) and non-streaming
* Tool calling (`tools`, `tool_choice`, `parallel_tool_calls`, assistant `tool_calls`, `tool` messages)
* String or content-part array message content
* `temperature`, `top_p`, `stop`, `seed`, `presence_penalty`, `frequency_penalty`, `response_format`, `reasoning_effort`
* `developer` messages are sent to the provider as `system`
* Unknown top-level parameters are ignored (not rejected), so standard harness adapters work
* Usage in the stream only when the client sets `stream_options.include_usage`

### Not supported

* `n` (multiple completions)
* Audio
* Browser CORS (use server-side clients)

Request bodies are capped at 8 MiB. `max_tokens` / `max_completion_tokens` (not both) is optional, 1 to 65,536; omit to use the provider default.

## 4. Rate limits (pilot)

Limits protect against abuse while you roll out a free tier. You do **not** need to implement these yourself for Gravity to enforce them.

### Live today

| Scope | Limit | Response when exceeded |
| - | - | - |
| Account | **600 requests / minute** (overrideable by Gravity) | OpenAI-shaped `429` (`rate_limit_error`) |

There is **no per-key** rate limit today. Keys are protected by hashing, revocation, and account entitlement.

Provider-side rate limits return OpenAI-shaped `429` with code `provider_rate_limited`. No provider tokens were consumed; retries are safe.

### Documented for the pilot (enforcement landing soon)

These are the rational per-user caps for the pilot. Treat them as the contract even if your account does not see them enforced yet. Gravity will enforce them in the API without a change on your side.

| Scope | Limit | Notes |
| - | - | - |
| Per `end_user` / `user` | **60 requests / minute** | Use a stable user id. Do not send a shared constant such as `"anonymous"`. |
| Per `end_user` / `user` | **500,000 tokens / UTC day** | Sum of input + output tokens recorded for that user. |
| Per `session_id` | **120 requests / minute** | Soft guard when `user` is missing. Prefer sending both. |

Until enforcement ships, you may optionally apply the same caps in your product for a smoother free-tier UX. After enforcement lands, Gravity returns OpenAI-shaped `429` with a clear code (for example `end_user_rate_limited` or `end_user_daily_token_limit`).

## 5. Errors to know

| HTTP | Meaning |
| - | - |
| `400` `session_id_required` | No session id in body, header, or `user` |
| `401` | Invalid or revoked inference key, or Inference not enabled |
| `403` / `423` | Account traffic suspended (operator action) |
| `424` | Non-streaming completion succeeded upstream, but Gravity could not finalize usage. **Do not retry.** Contact support. |
| `429` | Account, user, or provider rate limit |

Set `max_retries=0` (or equivalent) on the OpenAI client. Gravity intentionally uses statuses outside the OpenAI SDK's default retry set for bookkeeping failures so a failed ack does not spend tokens again.

## 6. Ads and sessions

Inference responses do **not** embed ads. Request ads separately with the Gravity ads SDK or API, using the **same** `session_id` (and optional hashed email) you send on inference. That is how Gravity attributes free inference to ad revenue for the pilot.

> [card] [Request ads](https://docs.trygravity.com/ai-platforms/request-ads.md)
>
> Request ads with the same session id you send on inference.

> [card] [Show ads](https://docs.trygravity.com/ai-platforms/show-ads.md)
>
> Show the ad in your chat UI.

## 7. Dashboard

On [app.trygravity.com/platform/inference](https://app.trygravity.com/platform/inference) you can:

* Create and revoke inference keys
* See request volume, errors, tokens, and latency
* Try models in the playground (non-streaming)

## Pilot checklist

1. Confirm Inference appears in your dashboard.
2. Create a key and hit `GET /v1/models`.
3. Send a non-streaming chat with `session_id`.
4. Wire streaming if your product streams.
5. Pass a stable per-user id in `user` / `end_user` so per-user limits apply correctly.
6. Launch your free-tier campaign with the same session ids you use for ads.
7. Sync with Gravity after \~5 days of token data to tune limits and models.

Questions: ping your Gravity Slack channel or [support@trygravity.com](mailto:support@trygravity.com).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.