Open Source, Optimized: Wafer on the Agentuity AI Gateway

In the ever-changing AI industry, it feels like one thing that's consistent is the cycle of open-source models making waves every so often: "the field" catches up more and more to the frontier labs, enough that these models become genuinely helpful in everyday workflows, often as the cheaper but capable models developers route simpler requests to.

Recent complications in model releases from Anthropic and OpenAI have only amplified that conversation. It's clear why open-source is having its moment once again. Meanwhile, new open-source models like GLM 5.2 have picked up real developer support.

But beyond pure model advancements, another important factor is the inference layer, and how this helps make open-source models more viable for actual, real-world use cases.

Wafer is doing the systems work that makes running these models fast and cost-effective to serve:

• GPU kernel code
• memory layout underneath a model
• request scheduling
• the parts that decide how much you get out of each chip

Their goal is to maximize intelligence per watt. At a glance, that's a measure of how much useful work you can get out of a watt of compute, not how a model scores on benchmarks (which don't necessarily reflect what models are actually good at, anyway).

That work is live on the Agentuity AI Gateway, and as of today, it includes GLM 5.2 Fast.

What Wafer is building

Wafer works close to the hardware, where serving details decide whether a GPU is spending most of its time generating tokens or waiting on memory. In practice, that shows up in batching, token generation, and how much throughput you get from the same chips.

The result is more tokens per second from the same hardware. In their work on AMD accelerators, Wafer took Kimi 2.5 from 22.5 tok/s to 255.2 tok/s on an 8x MI350X/MI355X setup. More recently, they served GLM 5.2 on MI355X at 2,626 tok/s per node aggregate and 213 tok/s single-stream, at over 2x lower cost than Blackwell-based serving.

Wafer puts all of this behind an OpenAI-compatible API, so you can call its models with the same Chat Completions request format.

Two GLM 5.2 routes, one Gateway call

The Gateway now has two Wafer-backed GLM 5.2 routes: oss/glm-5.2 for the regular tier, and oss/glm5.2-fast for Wafer's faster serving tier. The Fast tier grew out of Wafer's work on GLM 5.2 serving performance as inference demand outpaced supply. At roughly 100-250 tokens per second, the Fast tier is the fastest serverless GLM 5.2 on the market.

Both routes keep the 1M-token context window, and both run with zero data retention (more on that below). The Gateway call stays the same; only the model ID changes:

 agentuity cloud aigateway request /v1/chat/completions \
     --body '{
-        "model": "oss/glm-5.2",
+        "model": "oss/glm5.2-fast",
         "messages": [{"role": "user", "content": "Hello!"}],
         "max_tokens": 64,
         "thinking": {"type": "disabled"}
     }'

What the Gateway adds

That API is what the Agentuity AI Gateway wires in for you. Catalog and routing details are in the AI Gateway docs. You don't manage a separate Wafer key or hard-code Wafer's direct model names as the catalog changes, which keeps API key management out of your app code.

Your app uses a Gateway catalog ID, like oss/glm-5.2, and the Gateway handles which provider serves the request.

Without the AI Gateway	With the Agentuity AI Gateway
Manage a Wafer key alongside project credentials	Use your Agentuity project credential
Pick a provider endpoint yourself	Pass the Gateway model ID
Track provider-specific model names and routing changes in app code	Keep provider-specific names out of your code

The open-source catalog

The live Agentuity catalog is the source of truth for Gateway calls: use those model IDs in your code, and treat display names as labels for humans. GLM 5.2 is a good place to start; the catalog is 12 open-source models (at the time of writing):

Model	Gateway ID	Notes
GLM 5.2	oss/glm-5.2	1M-token context; long-horizon tasks
GLM 5.2 Fast	oss/glm5.2-fast	Same model, faster serving tier; Wafer reports 100-250 tok/s
Kimi K2.6	oss/kimi-k2p6	256K-token context; text and image input

An earlier draft of this post had two more rows here: DeepSeek V4 Flash and DeepSeek V4 Pro. Between that draft and this one, Wafer decommissioned both to put more of its fleet behind GLM 5.2. Here's the part that matters for your app: oss/deepseek-v4-flash and oss/deepseek-v4-pro still answer. The Gateway routes them to a different provider now, and nothing in the calling code changed. A provider retired two models and, from the app's side, it was a non-event. That's the pitch for catalog IDs.

Pull the same list from the CLI:

agentuity cloud aigateway models --provider oss --refresh-models

See the model catalog docs for the full list. Usage is available in the web app.

What changes with ZDR

ZDR, at a glance

Zero Data Retention (ZDR) means the provider processes the request but does not retain the request body after returning the response. That distinction matters more for agents than for simple chat prompts. A single agent request can include customer context, retrieved documents, tool results, and internal notes.

ZDR doesn't need to change the answer you get back. It changes the route the request is allowed to take.

Because of this, ZDR is a little different than something like a no-training policy. The latter says your prompts and outputs shouldn't train future models (with some room for short-term retention, for things like abuse checks or debugging).

ZDR is stricter for the request itself: either the route can meet the retention requirement, or the request fails instead of quietly using another route (when calling Wafer directly, that rejection uses model_zdr_not_supported).

So the mental model is:

Without required ZDR
request -> standard provider route -> response

With required ZDR
request -> ZDR-supported route -> response

With required ZDR, but no supported route
request -> rejected before upstream

What changes: The prompt stays the same. Direct Wafer calls use Wafer's endpoint, Wafer's model name, and the Wafer-ZDR header when you need it. Gateway calls use a Gateway model ID, and the Gateway handles the provider-specific routing details.

For your code, the split is:

Call path	Header you need to set	What happens
Direct Wafer, no ZDR requirement	None	Wafer routes the request normally
Direct Wafer, required ZDR	Wafer-ZDR: required	Wafer routes supported models, or rejects unsupported ones with model_zdr_not_supported
Agentuity Gateway	None required	The Gateway sets Wafer-ZDR: required on the outgoing Wafer request for ZDR-supported routes, even if your request already included a different value

Calling Wafer directly

Without (requiring) ZDR, a direct Wafer request is a normal OpenAI-compatible call:

curl -sS "https://pass.wafer.ai/v1/chat/completions" \
    -H "Authorization: Bearer <YOUR_WAFER_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "GLM-5.2",
        "messages": [
            {
                "role": "system",
                "content": "Return only the final answer. No reasoning, no preface. Keep it to exactly two sentences. Do not mention retention, redaction, email addresses, or escalation aliases in the answer."
            },
            {
                "role": "user",
                "content": "Turn this incident-room note into a two-sentence executive handoff: Acme Inc. saw elevated 401s after an SSO certificate rollover at 02:13 UTC. The rollback restored login at 02:41 UTC. The raw thread contains customer email addresses and an internal escalation alias. Owner: identity platform."
            }
        ],
        "max_tokens": 120,
        "thinking": {"type": "disabled"}
    }'

When the request must require ZDR, the JSON body stays the same. Add the Wafer header:

+# Same JSON body as above (the prompt does not change).
 curl -sS "https://pass.wafer.ai/v1/chat/completions" \
     -H "Authorization: Bearer <YOUR_WAFER_API_KEY>" \
+    -H "Wafer-ZDR: required" \
     -H "Content-Type: application/json" \
     -d '{ ... }'

From the Agentuity CLI

Your app, or agent via the Agentuity CLI, calls the Agentuity AI Gateway:

agentuity cloud aigateway request /v1/chat/completions \
    --body '{
        "model": "oss/glm-5.2",
        "messages": [
            {
                "role": "system",
                "content": "Return only the final answer. No reasoning, no preface. Keep it to exactly two sentences. Do not mention retention, redaction, email addresses, or escalation aliases in the answer."
            },
            {
                "role": "user",
                "content": "Turn this incident-room note into a two-sentence executive handoff: Acme Inc. saw elevated 401s after an SSO certificate rollover at 02:13 UTC. The rollback restored login at 02:41 UTC. The raw thread contains customer email addresses and an internal escalation alias. Owner: identity platform."
            }
        ],
        "max_tokens": 120,
        "thinking": {"type": "disabled"}
    }'

To try another ZDR-supported model, change the Gateway model ID:

- "model": "oss/glm-5.2"
+ "model": "oss/glm5.2-fast"

Some Gateway OSS models route through Wafer, including GLM 5.2 and GLM 5.2 Fast. When the selected Gateway model takes a ZDR-supported Wafer path, the Gateway rewrites the request body to Wafer's upstream model name and sends this to Wafer:

POST https://pass.wafer.ai/v1/chat/completions
Wafer-ZDR: required

{"model":"<Wafer upstream model name>", ...}

This means you don't need to set Wafer-ZDR in your app code. If you send one anyway, the Gateway still sets the outgoing Wafer request to Wafer-ZDR: required when the selected route is ZDR-supported. With default Gateway routing, models that Wafer doesn't mark as ZDR-supported stay off the Wafer route.

Example output, trimmed to the assistant message:

Acme Inc. experienced elevated 401 errors following an SSO certificate rollover at 02:13 UTC, resolved by a rollback that restored login at 02:41 UTC. The identity platform team owns this incident.

These examples pin reasoning off, which is Wafer's serverless default anyway. To let the model think first, enable it and raise the budget. Reasoning tokens count against max_tokens, and if they eat the whole budget, Wafer returns a truncation notice instead of an answer:

-        "max_tokens": 120,
-        "thinking": {"type": "disabled"}
+        "max_tokens": 1600,
+        "thinking": {"type": "enabled"}

From your app

Keep the model ID in config so you can swap open-source models without editing request code.

Note that the example below leaves out sampling controls. Add them only after checking the model docs or catalog entry.

A minimal Gateway call looks like this:

import { AIGatewayClient } from '@agentuity/aigateway';

const gateway = new AIGatewayClient();
const OPEN_MODEL = process.env.AGENTUITY_AIGATEWAY_MODEL ?? 'oss/glm-5.2';

export async function summarizeSupportNote(note: string): Promise<string> {
    const result = await gateway.completeText({
        model: OPEN_MODEL,
        max_tokens: 300,
        messages: [
            {
                role: 'system',
                content:
                    'Summarize support notes for an internal handoff. Keep unresolved questions.',
            },
            {
                role: 'user',
                content: `Support note:\n\n${note}`,
            },
        ],
    });

    if (!result.hasText) {
        throw new Error(
            `AI Gateway returned no text. finish_reason=${result.finishReason ?? 'unknown'}`
        );
    }

    return result.text;
}

AIGatewayClient can use an explicit apiKey, AGENTUITY_AIGATEWAY_KEY, or the same SDK credentials used by the rest of your Agentuity project. The API reference lists every request field. If you're already using the Gateway, these model IDs work without any new setup.

Get started

1. Sign up at app.agentuity.com if you don't have an account.
2. Run agentuity cloud aigateway models --provider oss, or browse the catalog docs.
3. Pass the Gateway model ID to a Gateway call.
4. Keep the model ID in an env var so you can swap without touching your code.

Wafer's Serverless API docs cover the full request format, and the zero data retention docs cover what gets dropped and when.

As the catalog changes, you can keep the Gateway call stable and change only the model ID. The DeepSeek re-route above is what that looks like in practice, and it's the same bet behind our ongoing path to being agnostic: your app code should not have to care which provider, runtime, or upstream model name sits underneath.

If you're already on Agentuity, swap in one of the catalog IDs and you're set. If you're new, sign up first. Tell us how it goes in Discord!

Resources

Helpful Wafer links

• Wafer platform
• Wafer Serverless docs
• Wafer ZDR docs

More from Agentuity

• Agentuity AI Gateway
• AI Gateway docs
• Web App
• Discord Community
• SDK repo