Cut AI coding costs 5× with self-hosted GLM-5.2

For the past year, Anthropic has defined this category. Claude Opus 4.7 and 4.8 are exceptional models — among the best available for agentic coding tasks. We used them exclusively at modulaa. Our entire development workflow, from code generation to multi-step pipeline orchestration, was built around Claude. But the economics became impossible to ignore.

The trigger was a single sprint. Our team ran a multi-agent refactoring pass on the modulaa platform’s auth module — three sub-agents, each making dozens of LLM calls. The invoice at the end of the week read $1,847. That wasn’t an outlier; it was the new normal. Claude’s per-token pricing compounds fast when you’re running multi-agent systems that make hundreds of LLM calls per task. A single complex coding session could burn through $50–100 in API costs. Multiply that across a team, across a sprint, and you’re looking at figures that make CFOs nervous.

And it’s not just Anthropic. Cursor charges you per request to use your own model through their IDE. Claude Code Desktop has its own subscription costs. The entire ecosystem locks you in, and every layer adds another fee. We started talking to other teams in the community and heard the same story everywhere: the quality is great, but the cost is brutal, and there’s no clear exit ramp. That exit ramp is what this post is about.

Enter GLM-5.2: Open Source, Benchmark-Competitive

Zhipu AI's GLM-5.2 is a 753B parameter mixture-of-experts model released under the MIT license — no regional restrictions, no access gates, no per-seat pricing. Only ~37B parameters are active per token, which keeps inference tractable. It's the top-ranked open-source model on the coding benchmarks we track:

At modulaa, GLM-5.2 is very competitive with Claude Opus 4.8 across the coding benchmarks we track — close enough that the difference isn’t worth paying 5× more for. Combined with the cost differential (roughly 5× cheaper when self-hosted), the choice is straightforward.

Beyond the benchmarks, the specs matter for deployment planning:

• Parameters: 753B (MoE — only ~37B active per token)
• Context length: Up to 1M tokens (we serve ~196K; see the deployment config below)
• License: MIT (commercial use, no restrictions)
• Inference frameworks: vLLM, SGLang, Transformers, KTransformers
• Hugging Face: zai-org/GLM-5.2 (282K downloads in the first month after release)

The Deployment Stack: RunPod + vLLM + GLM-5.2

For GPU hosting, we chose RunPod Pods. Four reasons drove the decision:

• Pay-per-second billing — no idle costs when your pod is stopped (just stop it when you’re done; billing is per-second while running)
• Global GPU availability — H200s, A100s, and H100s across 30+ datacenters
• vLLM worker template — pre-built Docker images with vLLM, ready to deploy in minutes
• OpenAI-compatible API — drop-in replacement for any tool that talks to Claude or GPT

The full stack looks like this:

1Step 1: Deploy GLM-5.2 on RunPod

Option A: Deploy via the RunPod Console (Fastest)

1. 1.Create a RunPod account at runpod.io and generate an API key.
2. 2.Find the vLLM template in the RunPod Hub, or use the pinned image directly: vllm/vllm-openai@sha256:251eba5cc7c12fed0b75da22a9240e582b1c9e39f6fbc064f86781b963bd814f (vLLM 0.24.0 — pinned, not :latest). The native Anthropic Messages API support at /v1/messages that Step 3 relies on is version-specific to 0.24.0, so pinning matters. Click Deploy.
3. 3.Configure the model:
   • Model: cyankiwi/GLM-5.2-AWQ-INT4 (4-bit quantized — fits on 4× H200 with room for KV cache; see the memory budget note below)
   • Served Model Name: modulaa/glm-5.2 (or whatever you prefer)
   • Max Model Length: 196608 (~196K tokens)
   • GPU Memory Utilization: 0.90
   • Tensor Parallel Size: 4 (requires 4× NVIDIA H200)
   • Container Disk: 60 GB
   • Network Volume: 500 GB, mounted at /runpod/volume, in a region that has H200s (e.g. US-GA-2). This stores the ~410 GB model so restarts are fast. Volumes are region-locked — a volume in US-GA-2 can only attach to pods in US-GA-2.
   • Expose HTTP port: 8000
   • Expose TCP port: 22

Add environment variables (Advanced → Environment Variables):

{
  "HF_HOME": "/runpod/volume/huggingface-cache",
  "HUGGINGFACE_HUB_CACHE": "/runpod/volume/huggingface-cache/hub",
  "HF_HUB_OFFLINE": "0",
  "PYTORCH_CUDA_ALLOC_CONF": "expandable_segments:True",
  "VLLM_API_KEY": "your-secret-bearer-key"
}

Add container arguments:

--model cyankiwi/GLM-5.2-AWQ-INT4 \
--served-model-name modulaa/glm-5.2 \
--tensor-parallel-size 4 \
--block-size 64 \
--max-model-len 196608 \
--gpu-memory-utilization 0.90 \
--kv-cache-dtype fp8 \
--safetensors-load-strategy prefetch \
--trust-remote-code \
--enable-expert-parallel \
--max-num-seqs 8 \
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--chat-template-content-format string \
--default-chat-template-kwargs.enable_thinking false \
--host 0.0.0.0 \
--port 8000

Why `--default-chat-template-kwargs.enable_thinking false`? GLM-5.2’s thinking mode emits ~2.5× more tokens for identical answers and caused KV-cache OOMs under heavy concurrent fan-out (multiple sub-agents hitting the endpoint at once). We disable it by default; re-enable with --thinking if your workload benefits from extended reasoning.

Memory budget sanity check: A 753B AWQ-INT4 model is ~376 GB of weights (753B × 0.5 bytes/param). On 4× H200 (141 GB each = 564 GB total), at 0.90 utilization (~508 GB usable), weights (~376 GB) leave ~132 GB for KV cache and activations across 4 GPUs.

With FP8 KV cache and 196K context, this is tight but feasible. If you have 8× H200, consider the FP8 checkpoint for higher quality at the cost of more VRAM.

Supply-chain note: --trust-remote-code executes Python from the model repo (cyankiwi/GLM-5.2-AWQ-INT4). For a community-uploaded quant, audit the repo’s modeling_*.py before trusting it, or pin to a vetted commit digest.

Click Deploy. RunPod will provision GPUs and download the model (~15–20 minutes for first boot, ~2–3 minutes with a pre-staged volume). Once the pod is running, your endpoint is https://<POD_ID>-8000.proxy.runpod.net/v1 — find the POD_ID in the RunPod console (Pods → your pod).

Option B: Deploy via API (Reproducible)

For teams that want version-controlled, reproducible deployments, create the pod via the RunPod REST API. (You’ll need a templateId from a saved vLLM template — Option A creates one you can reuse.)

# Set your credentials
export RUNPOD_API_KEY="your-runpod-api-key"

# Create the pod (4× H200, TP=4)
curl --request POST \
  --url https://rest.runpod.io/v1/pods \
  --header "Authorization: Bearer $RUNPOD_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "glm-5.2",
    "imageId": "<your-saved-vllm-template-id>",
    "gpuTypeId": "NVIDIA_H200",
    "podType": "RUN",
    "containerDiskInGb": 60,
    "minVcpuCount": 4,
    "minMemoryInGb": 30,
    "gpuCount": 4,
    "volumeInGb": 500,
    "volumeMountPath": "/runpod/volume",
    "ports": "8000/http,22/tcp",
    "env": {
      "HF_HOME": "/runpod/volume/huggingface-cache",
      "HUGGINGFACE_HUB_CACHE": "/runpod/volume/huggingface-cache/hub",
      "HF_HUB_OFFLINE": "0",
      "PYTORCH_CUDA_ALLOC_CONF": "expandable_segments:True",
      "VLLM_API_KEY": "your-secret-bearer-key"
    }
  }'

Once the pod is live, test it with a single inference call (replace <POD_ID> with the id returned by the create call, or find it in the RunPod console):

curl -X POST "https://<POD_ID>-8000.proxy.runpod.net/v1/chat/completions" \
  -H "Authorization: Bearer $VLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "modulaa/glm-5.2",
    "messages": [{"role": "user", "content": "Hello, GLM-5.2!"}]
  }'

Option C: Write a Small Watch Script (Fully Automated)

The "GPUs only show up sporadically in random regions" problem is the real friction with H200s. The clean fix is a small script that polls RunPod’s API for 4× H200 availability across regions and deploys the instant capacity appears anywhere. You can write one in a few dozen lines against RunPod’s REST API:

# Pseudocode for a watch-and-deploy loop
while true; do
  # 1. Query GET /v1/pods for available 4× H200 capacity across regions
  # 2. If capacity found in a region with a pre-staged volume, POST /v1/pods to deploy
  # 3. Else sleep 30s and retry
  sleep 30
done

A watch script like this also lets you stage a pre-staged volume in-region to cut cold starts from 15–20 minutes down to 2–3 minutes. Note that pre-staged volumes are region-locked: a volume in US-GA-2 can’t serve a pod in EU-RO-1, so your script should track which regions have a staged volume and only deploy there.

2Step 2: Connect VS Code to Your Self-Hosted Endpoint

Once your pod is running, you get an OpenAI-compatible API URL: https://<POD_ID>-8000.proxy.runpod.net/v1. Three common options for connecting VS Code:

GitHub Copilot in VS Code (Bring Your Own Key)

VS Code supports custom OpenAI-compatible endpoints via the Language Models editor (BYOK), not a settings field:

1. 1.Open the chat model picker → Manage Language Models (gear icon), or run Chat: Manage Language Models from the Command Palette.
2. 2.Click Add Models → Custom Endpoint.
3. 3.Enter a group name (e.g. "GLM-5.2 (RunPod)"), a display name, and your VLLM bearer key.
4. 4.Select API type: Chat Completions.
5. 5.VS Code opens chatLanguageModels.json. Add your model:

[{
  "name": "GLM-5.2 (RunPod)",
  "vendor": "customendpoint",
  "apiKey": "your-vllm-api-key",
  "apiType": "chat-completions",
  "models": [{
    "id": "modulaa/glm-5.2",
    "name": "GLM-5.2",
    "url": "https://<POD_ID>-8000.proxy.runpod.net/v1/chat/completions",
    "toolCalling": true,
    "maxInputTokens": 196608,
    "maxOutputTokens": 8192
  }]
}]

Select the model from the chat model picker.

Continue.dev (OpenAI-Compatible Extension)

// ~/.continue/config.json
{
  "models": [{
    "title": "GLM-5.2 (RunPod)",
    "provider": "openai",
    "model": "modulaa/glm-5.2",
    "apiBase": "https://<POD_ID>-8000.proxy.runpod.net/v1",
    "apiKey": "your-vllm-api-key"
  }]
}

Cline (VS Code Extension)

1. 1.Install the Cline extension
2. 2.Open Cline settings
3. 3.Set API Provider to "OpenAI Compatible"
4. 4.Set Base URL to https://<POD_ID>-8000.proxy.runpod.net/v1
5. 5.Set API Key to your VLLM bearer key
6. 6.Set Model ID to modulaa/glm-5.2

3Step 3: Connect Claude Code to Your Self-Hosted Endpoint

Claude Code speaks the Anthropic Messages API. The good news: your pod runs vLLM 0.24.0, which serves that API natively at /v1/messages — so you can point Claude Code straight at the pod with a few env vars. No proxy required. (LiteLLM is only a fallback — see Path B below.)

Claude Code  ──(Anthropic Messages API)──►  vLLM 0.24.0 on RunPod (4× H200)

Path A — Direct (recommended)

1. Smoke-test the Anthropic endpoint first. Confirm vLLM is answering in Anthropic format before wiring up Claude Code:

POD=https://<POD_ID>-8000.proxy.runpod.net
KEY=<YOUR_VLLM_API_KEY>

curl -s "$POD/v1/messages" \
  -H "Authorization: Bearer $KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "modulaa/glm-5.2",
    "max_tokens": 64,
    "messages": [{"role": "user", "content": "Say hello in 5 words."}]
  }'

A JSON reply with "type":"message" and content means you’re ready. A 404 on /v1/messages means native Anthropic support isn’t active on your image → use Path B.

2. Launch Claude Code pointed at the pod:

export ANTHROPIC_BASE_URL="https://<POD_ID>-8000.proxy.runpod.net"
export ANTHROPIC_AUTH_TOKEN="<YOUR_VLLM_API_KEY>"   # sent as the bearer
export ANTHROPIC_API_KEY="<YOUR_VLLM_API_KEY>"      # some versions read this too
export ANTHROPIC_DEFAULT_OPUS_MODEL="modulaa/glm-5.2"
export ANTHROPIC_DEFAULT_SONNET_MODEL="modulaa/glm-5.2"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="modulaa/glm-5.2"

claude

All three model tiers (Opus/Sonnet/Haiku) map to modulaa/glm-5.2 so every Claude Code request routes to your one served model.

3. (Optional) Make it permanent. Instead of exporting each shell, put it in Claude Code’s settings so claude always uses it:

~/.claude/settings.json

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://<POD_ID>-8000.proxy.runpod.net",
    "ANTHROPIC_AUTH_TOKEN": "<YOUR_VLLM_API_KEY>",
    "ANTHROPIC_API_KEY": "<YOUR_VLLM_API_KEY>",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "modulaa/glm-5.2",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "modulaa/glm-5.2",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "modulaa/glm-5.2"
  }
}

Path B — LiteLLM proxy (fallback / advanced)

Use this if Path A’s /v1/messages 404s, if tool-calling acts up, or if you want a stable local URL, auth, retries, or model fallback in front of the pod.

1. Install + configure:

pip install "litellm[proxy]"

litellm-glm.yaml

model_list:
  - model_name: claude-*                       # catch every model Claude Code asks for
    litellm_params:
      model: openai/modulaa/glm-5.2            # openai/<served-model-name>
      api_base: https://<POD_ID>-8000.proxy.runpod.net/v1
      api_key: os.environ/VLLM_API_KEY
litellm_settings:
  drop_params: true                            # ignore Anthropic-only params vLLM rejects
  modify_params: true
  request_timeout: 600
general_settings:
  master_key: sk-litellm-local                 # the token Claude Code will present

2. Run it and point Claude Code at LiteLLM:

export VLLM_API_KEY=<YOUR_VLLM_API_KEY>
litellm --config litellm-glm.yaml --port 4000
# serving Anthropic Messages API on http://0.0.0.0:4000/v1/messages

export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_AUTH_TOKEN="sk-litellm-local"
claude

For persistent setup, add these to a .env file (with chmod 600, not committed and not in ~/.zshrc — long-lived bearer tokens in shell profiles end up in shell history and dotfile backups):

# .env (chmod 600, gitignored)
ANTHROPIC_BASE_URL="http://localhost:4000"
ANTHROPIC_AUTH_TOKEN="sk-litellm-local"

Troubleshooting

Escaping the Cost Trap: Why Self-Hosting Wins

The self-hosted stack has two cost components — GPU time and storage — and both are billed per-second (or per-month for storage), so you only pay for what you use:

1. GPU time (the pod): ~$17.56/hr while the pod is running, $0 while it’s stopped. Billing is per-second — stop the pod when you’re done.

2. Network volume storage (the model cache): the 500 GB network volume that holds the ~410 GB model costs ~$0.05–0.07/GB·mo, i.e. ~$30–35/month. This is a standing cost — you pay it whether the pod is running or not.

The reason to keep it: the volume persists so cold starts stay fast (2–3 min instead of 15–20 min re-downloading from HuggingFace). Volumes are region-locked, so if you pre-stage volumes in multiple regions for failover, multiply this by the number of regions.

So the real monthly bill looks like this:

For context, a team running Claude Opus 4.8 via the API at the same usage level typically sees $2,000–5,000/month in token costs — and that scales with every token, including thinking tokens, with no way to stop the meter. With self-hosted GLM-5.2, you pay for GPU hours and a small standing storage fee, not tokens.

The key tradeoff is cold-start time — a stopped pod takes 15–20 minutes to boot from scratch (downloading or loading the ~410 GB model), or 2–3 minutes with the pre-staged network volume. Your options:

• Stop when idle, boot on demand: cheapest GPU cost — you only pay for active hours plus the ~$30–35/mo volume. But every cold start costs 15–20 min (or 2–3 min with the volume). Best for batch/async workloads.
• Leave the pod running: no cold starts, but ~$12,800/month (1 pod × $17.56/hr × 730 hrs) + ~$30–35/mo volume. Best for interactive coding during a heavy sprint if budget allows.
• Pre-staged volume + stop when idle: the practical middle ground — 2–3 min cold starts, pay only for active GPU hours plus the standing volume fee. This is what a watch script (Option C) configures by default.

How modulaa actually runs it: a hybrid OpenRouter + RunPod strategy

We don't keep the pod running 24/7. For day-to-day, low-volume traffic — a single developer asking the occasional question, a quick code review, a one-off generation — we route to GLM-5.2 on [OpenRouter](https://openrouter.ai), which is priced per-request. No GPU to provision, no cold start to wait for, no standing storage fee. You pay only for the tokens you use, and the meter stops the instant you stop asking.

We only spin up the RunPod pod when we're running heavy traffic to the LLM — multi-agent pipeline sprints, batch refactoring passes, or any workload that would rack up a large OpenRouter bill. That's when the per-second GPU economics flip in favor of self-hosting: once you're spending enough on per-request tokens to cover even a few hours of $17.56/hr GPU time, the pod wins decisively, and there's no rate limit or queue to contend with.

The decision rule is simple:

This is the same model we use with our clients: keep the pod stopped by default, let OpenRouter handle the long tail of light requests, and bring the pod up for the bursts where self-hosting pays for itself. The pre-staged volume keeps the spin-up to 2–3 minutes, so the "heavy traffic" switch is near-instant when a sprint kicks off.

With Claude's API, every token — including thinking tokens — costs money. With self-hosted GLM-5.2, you pay for GPU hours and a small storage footprint, not tokens.

For Enterprise Teams

If you're evaluating this stack for a mid-market or enterprise rollout, a few considerations:

• Data residency: RunPod regions determine where your code is processed. Pin the region in your watch script (e.g. only deploy to US-GA-2) to meet EU or US data-residency obligations.
• Access control: The VLLM bearer key is a static token with no per-user auth or rate limiting. For team use, put the endpoint behind a gateway (e.g. LiteLLM’s proxy) that adds per-user authentication, rate limits, and audit logging.
• Key rotation: Treat the VLLM API key and the pod URL as secrets. Rotate regularly and never commit them to version control or store them in ~/.zshrc.
• Governance: The "your code never leaves your infrastructure" property is the core enterprise selling point. Pair it with your existing SOC 2 / ISO 27001 controls — the self-hosted model fits cleanly inside a VPC-style perimeter.

Where modulaa Comes In

We built this stack for ourselves first — it powers modulaa's own Agentic SDLC platform and our Agentic Consulting engagements. We're not just writing about it; we run it every day.

If you want the cost savings without standing up 4× H200s and tuning vLLM args yourself, that's exactly what we do with our clients:

• [Agentic Consulting](https://www.modulaa.ai) — We deploy, operate, and optimize this stack for your team. You get the cost savings and data privacy without the DevOps burden.
• [Agentic SDLC Tool](https://www.modulaa.ai) — Our platform orchestrates AI agents (on GLM-5.2, or routed to OpenRouter for light traffic) across your software development lifecycle, with human approval at every critical decision.

We succeed when you succeed. That’s the partnership model — not a vendor relationship.

Quick Reference: All the Links

Conclusion: You Have a Choice

The AI coding agent market has leaned on a single vendor for a while now. That’s starting to change. Claude is excellent — but GLM-5.2 is competitive with it on coding work, and it’s roughly 5× cheaper when self-hosted.

GLM-5.2 proves that open-source models can compete at the frontier. RunPod proves that self-hosting doesn’t require a DevOps team.

The stack we’ve described here — GLM-5.2 + RunPod + vLLM + your existing IDE — gives you:

• Coding performance competitive with Claude Opus 4.8
• 5× cost reduction
• Full data privacy (your code never leaves your infra)
• No vendor lock-in
• Your existing VS Code / Claude Code workflow, unchanged

The migration took us a weekend. We’ve been shipping on it ever since.

If you're feeling the cost squeeze from Claude, Cursor, or any other vendor-hosted model, the exit ramp is right here. You can deploy the stack yourself with the configs above — or talk to our team and we'll deploy and operate it with you.