Headroom: compress tool outputs and RAG content to reduce LLM tokens
A local component that compresses AI-agent outputs and RAG content to save tokens without sending the source text away.
GitHub headroomlabs-ai/headroom Updated 2026-10-20 Branch main Stars 76.2K Forks 5.9K
Python Rust TypeScript LLM context compression RAG ONNX Runtime Claude Code MCP Apple Silicon

🧭 Decision Guide

Why trending now: Cannot be determined from the provided material

Try it if you

  • You use Claude Code or another coding agent that processes large amounts of tool output, logs, or files.
    The README's “Get started” and “Agents” content provides `headroom wrap claude`; the core description explicitly includes tool outputs, logs, and files.
  • You want to connect existing LLM calls through a proxy without changing application code.
    The README's “Get started” provides `headroom proxy --port 8787` and labels it “drop-in proxy, zero code changes.”
  • Your Python application uses the OpenAI client and you want compression statistics directly.
    The README's Python example uses `from headroom import compress`, `model="gpt-4o"`, and `result.tokens_saved`.
  • Your environment is Apple Silicon or Linux/x86 with AVX2 support.
    The README's “Install” says native wheels cover Apple Silicon and Linux, and that ONNX features on x86/x86_64 require AVX2.

Skip it if you

  • You must use the default source installation on Intel macOS and do not want to configure Homebrew onnxruntime.
    The README's “Intel macOS: no prebuilt ONNX Runtime” says the `x86_64-apple-darwin` source build fails by default and requires `ORT_LIB_LOCATION` and other settings.
  • You install only the npm package but need the `headroom` command-line tool.
    The README's “Get started” explicitly says npm `headroom-ai` is TypeScript SDK only and provides no `headroom` command.
  • Your x86 host lacks AVX2 but you require the ONNX paths for Magika or embedding relevance.
    The README's “CPU requirement (x86/x86_64)” says that without AVX2 it falls back to BM25 relevance and heuristic detection.
  • Your network blocks both `cdn.pyke.io` and `huggingface.co`, and you cannot pre-download assets or set offline variables.
    The README's “Two download hosts can still need attention” requires pre-providing ONNX Runtime or pre-downloading models and using `HF_HUB_OFFLINE=1`.

Requirements

  • Python installation requires Python 3.10+; the CLI examples use Python 3.13. / Python安装路径要求Python 3.10+；CLI示例使用Python 3.13。
  • ONNX-backed features require AVX2 on x86/x86_64; without it, BM25 and heuristic fallbacks are used. / ONNX-backed功能在x86/x86_64上需要AVX2；无AVX2会使用BM25和启发式回退。
  • The CLI comes from the PyPI package `headroom-ai[all]`; npm `headroom-ai` provides only the TypeScript SDK. / CLI来自PyPI包`headroom-ai[all]`，npm `headroom-ai`仅提供TypeScript SDK。
  • When model or ONNX assets are needed, the README lists `cdn.pyke.io` and `huggingface.co`. / 需要模型或ONNX资源时，README列出`cdn.pyke.io`和`huggingface.co`。

First step (verbatim from README)

uv tool install --python 3.13 "headroom-ai[all]"

Watch out

  • Serena for Claude Code is registered by default only with local scope for the wrapped project.
    The README's `headroom wrap` section says Serena is registered in `~/.claude.json` as a local-scope server; use `--code-memory-scope user` for global scope.
  • MCP clients may not inherit PATH, so configure the absolute path returned by `command -v headroom`.
    The README's “uv, pipx, and MCP clients that don't inherit your PATH” section explicitly requires the absolute path.
  • On Intel macOS, `ORT_LIB_LOCATION` must point to the lib directory rather than the Homebrew prefix.
    The README's “Intel macOS” section says `ORT_LIB_LOCATION` must point to `lib/` and requires `ORT_PREFER_DYNAMIC_LINK=1`.
  • Enabling `[vector]` requires a C++ toolchain and is not included in `[all]`.
    The README's “Install” section labels `[vector]` as an optional HNSW backend that needs a C++ toolchain and is not in `[all]`.

Alternatives

  • BM25 relevance and heuristic detection：Better when the x86 host lacks AVX2 and the README's non-ONNX fallback paths are acceptable.
    Install
  • Pure gateway mode：Better when you only need gateway forwarding without compression assets, because the README says this mode needs neither asset.
    Install

Not stated in the README

  • The provided README material does not give latency, memory, or CPU overhead data for different models.
  • The provided README material does not include the complete compatibility matrix from the “Agent compatibility” section.
  • Apart from the 10,144→1,260 token example, the material does not provide separate compression results for logs, files, and RAG chunks.
  • The material does not describe systematic evaluation coverage for answer quality across models; it only gives an example with the same FATAL.
  • The material does not provide the changes in Release v0.41.0, so its specific capability changes cannot be determined.
  • The material does not describe the seat, permission, billing, or deployment boundaries in “Headroom for Teams.”

💡 Deep Analysis

6
Yes I use Headroom through MCP in Codex, but the client does not inherit my interactive shell PATH. How should I configure it to avoid the `headroom` command failing to start?
For: A Codex MCP client user whose environment does not inherit the interactive shell PATH

Yes, but the MCP client must use Headroom’s absolute executable path. The README explicitly warns that Codex and similar clients often do not inherit the interactive shell PATH.

  • Run command -v headroom, then place the returned absolute path in the MCP configuration’s command field.
  • The example uses args = ["mcp", "serve"]; it does not treat the npm package as a CLI.
  • The CLI is shipped only with the PyPI package. npm install headroom-ai installs the TypeScript SDK and provides no headroom command.
  • If installed with uv, the uv tool directory must also be available when resolving the executable; otherwise the client may still fail to launch it.

The README does not specify the exact Codex version, MCP configuration file location, or the client’s diagnostics for stdio startup failures, so those details require client-specific documentation.

  • uv, pipx, and MCP clients that don't inherit your PATH: “Codex and other MCP clients often cannot inherit an interactive shell `PATH`”
  • README example: `command = "/Users/you/.local/bin/headroom"` and `args = ["mcp", "serve"]`
  • Install: “The `headroom` CLI ships only in the PyPI package. The npm `headroom-ai` package is the TypeScript SDK”
  • Install: `uv tool update-shell`
command -v headroom
Not stated in the README：The README does not identify the MCP configuration file path for the current Codex version.；It does not describe how Codex handles MCP stdio logs, startup timeouts, or exit codes.；It does not specify port or process-resource requirements when multiple MCP servers run concurrently.
Yes I already use Python's OpenAI SDK with `gpt-4o`; my main issue is oversized tool and JSON outputs, and I do not want to rewrite the agent. Can I insert Headroom only before the model request?
For: An application developer with an existing OpenAI Python call path who wants low-change context compression

Yes, because Headroom provides both a Python inline API and an OpenAI-compatible proxy that can avoid major application changes.

  • The README calls compress(messages, model="gpt-4o") and passes result.messages to client.chat.completions.create.
  • If you do not want to alter the call path, run headroom proxy --port 8787; the README describes it as a “drop-in proxy, zero code changes” option.
  • It specifically handles JSON, tool output, logs, files, and RAG chunks. The project description claims 60–95% fewer tokens for JSON, although the actual result depends on the input.
  • Compression runs locally, so prompts and file content are not sent to a remote compression service.

The proxy adds a local routing layer, and the README does not state whether your existing OpenAI SDK settings, proxy environment variables, and retry logic require additional configuration.

  • Inline, in Python: `from headroom import compress` and `compress(messages, model="gpt-4o")`
  • Get started (60 seconds): `headroom proxy --port 8787 # drop-in proxy, zero code changes`
  • Project description: “60-95% fewer tokens for JSON”
  • Quickstart: “Compression runs on your machine; no prompt or file content is sent anywhere to be compressed.”
headroom proxy --port 8787
Not stated in the README：The README does not specify the required OpenAI SDK base URL, authentication variables, or streaming configuration.；It does not define full proxy compatibility for function calls, tool-call arguments, or multimodal messages.；It does not provide the average latency added by compression.
It depends I need to compress sensitive logs and JSON on an x86 cloud VM without AVX2, and I do not want the content to leave the machine. Is Headroom still usable?
For: An operations developer deploying Headroom on an x86 cloud VM without AVX2 and requiring local processing of sensitive logs

It depends, because Headroom remains usable without AVX2 on x86, but ONNX-backed features fall back to non-ONNX paths whose quality and performance may differ.

  • The README says Magika content detection and embedding relevance use a precompiled ONNX Runtime that requires AVX2.
  • Without AVX2, Headroom falls back to BM25 relevance and heuristic detection. This preserves availability but is not equivalent to the full ONNX path.
  • The README explicitly says compression runs locally and that prompts and file content are not sent to a remote compression service, which fits sensitive-log requirements.
  • Docker deployment is supported; the README specifically recommends Docker-native installation for Intel macOS, but does not extend that exact statement to every x86 cloud VM.

Therefore, it can be considered if BM25 and heuristic processing are acceptable. If embedding relevance or full Magika behavior is required, the README does not guarantee quality, latency, or resource usage without AVX2.

  • Install: “The ONNX-backed features — Magika content detection and embedding relevance — use a precompiled ONNX Runtime that needs AVX2”
  • Install: “Headroom falls back to its non-ONNX paths — BM25 relevance, heuristic detection”
  • Quickstart: “Compression runs on your machine; no prompt or file content is sent anywhere to be compressed.”
  • Install: `docker pull ghcr.io/headroomlabs-ai/headroom:latest`
docker pull ghcr.io/headroomlabs-ai/headroom:latest
Not stated in the README：The README does not compare compression ratio, relevance quality, or latency between no-AVX2 fallback and the full ONNX path.；It does not specify minimum memory, CPU count, or Docker resource limits for cloud VMs.；It does not state whether local persistent deployment needs additional configuration without AVX2.
Yes I use Claude Code every day on large code repositories and want to reduce tokens from tool output, files, and conversation history without manually changing the agent configuration. Is Headroom suitable?
For: A Claude Code user who wants to reduce coding-agent context tokens

Yes, because the README provides a dedicated wrapper for coding agents and performs compression locally.

  • headroom wrap claude starts a local proxy, configures Claude Code, and can install Serena for semantic code navigation.
  • It handles tool output, logs, files, and conversation history. The README shows a reduction from 10,144 tokens to 1,260 while finding the same FATAL.
  • The project describes “20% fewer tokens for coding agents,” but this is not a guarantee for every repository or task.

If the team wants Serena available across projects, use --code-memory-scope user; use --code-memory none to skip it. The README does not report systematic results across Claude Code versions, repository sizes, or post-compression code-change correctness.

  • Get started (60 seconds): `headroom wrap claude`
  • Quickstart: “Headroom compresses everything your AI agent reads — tool outputs, logs, RAG chunks, files, and conversation history”
  • Agents: “headroom wrap starts a local proxy, installs Serena for semantic code navigation”
  • Project description: “20% fewer tokens for coding agents”
headroom wrap claude
Not stated in the README：The README does not provide a compatibility matrix for different Claude Code versions.；It does not quantify the effect on code-edit correctness, test pass rate, or end-to-end latency.；It does not specify Serena and Headroom resource usage on large repositories.
Yes My corporate network performs SSL inspection, the platform standardizes on Python 3.13, and I want to install an isolated Headroom CLI with uv. How should I determine whether Headroom is suitable when installation fails?
For: A platform engineer using Python 3.13 behind corporate TLS inspection who wants to install the CLI with uv

Yes, because the README covers isolated Python 3.13 installation with uv, corporate CA problems, and alternatives that avoid a local Rust build.

  • The recommended command is uv tool install --python 3.13 "headroom-ai[all]", which installs the CLI in an isolated application environment.
  • For CERTIFICATE_VERIFY_FAILED, the README explains that corporate SSL inspection may introduce an untrusted company CA and provides a Rust-first workaround.
  • If a matching prebuilt wheel is available, pip install --only-binary headroom-ai headroom-ai avoids downloading Rust during the build; wheels are listed for Windows, Linux, and macOS.
  • At runtime, headroom doctor --network reports certificate signers, trust status, and whether a gateway block page is present.

However, the README does not provide complete configuration for proxy authentication, internal PyPI mirrors, or model-download allowlists, and [all] does not include every framework adapter.

  • Get started (60 seconds): `uv tool install --python 3.13 "headroom-ai[all]"`
  • Corporate networks and SSL inspection: explanation of `CERTIFICATE_VERIFY_FAILED` and corporate CAs
  • Corporate networks and SSL inspection: `pip install --only-binary headroom-ai headroom-ai`
  • Corporate networks and SSL inspection: `headroom doctor --network`
  • Install: “`[all]` covers the core stack but not the framework adapters.”
uv tool install --python 3.13 "headroom-ai[all]"
Not stated in the README：The README does not specify which authentication variables or proxy parameters are required for corporate proxies.；It does not describe configuration for an internal PyPI mirror or model-download allowlists.；It does not provide a complete pre-download checklist for `[all]` dependencies and model files in offline installations.
It depends I use LangChain to build a RAG application. Retrieved results contain many repetitive chunks, and I want to preserve relevant content while reducing tokens sent to the LLM. Can Headroom be integrated directly?
For: A Python application team using LangChain for RAG and needing to compress retrieved chunks

It depends, because Headroom provides a LangChain adapter and relevance processing, but compression changes retrieved context and is suitable only when filtering or restructuring the source is acceptable.

  • The LangChain adapter must be installed separately; the README says [all] does not include framework adapters and shows headroom-ai[langchain].
  • The project handles RAG chunks and offers BM25, embeddings, and optional vector backends for relevance processing.
  • headroom perf and headroom dashboard expose compression results; the dashboard requires the proxy to be running.
  • It is not suitable for tasks requiring verbatim source text, complete numeric detail, or every citation boundary, because compression can lose information.

The README does not provide a compatibility matrix for LangChain chains, retriever types, or embedding models, nor a unified benchmark for recall, citation accuracy, and answer consistency before and after compression.

  • Install: “`[all]` covers the core stack but not the framework adapters”
  • Install: `pip install "headroom-ai[langchain]"`
  • Project insight: compression covers “RAG chunks”
  • Project insight usage limitations: unsuitable for tasks requiring verbatim complete source text, auditing, forensics, or precise data processing
  • Get started (60 seconds): `headroom perf` / `headroom dashboard`
pip install "headroom-ai[langchain]"
Not stated in the README：The README does not specify supported LangChain versions, chain types, or retriever implementations.；It provides no quantitative results for RAG recall, citation accuracy, or answer consistency.；It does not state model download size or runtime latency when embeddings or vector backends are enabled.

✨ Highlights

  • Compresses tool outputs, logs, files, and RAG chunks before they reach the LLM
  • The example reduces 10,144 tokens to 1,260 and finds the same FATAL
  • Supports a library, proxy, MCP server, and Claude Code wrapping mode
  • Compression runs locally; the README says prompts and file contents are not sent away
  • ONNX features require AVX2 on x86 hosts, otherwise they fall back to BM25 and heuristics

🔧 Engineering

  • `headroom proxy --port 8787` provides a zero-code-change proxy mode
  • `headroom wrap claude` starts a local proxy and configures Serena
  • Python can use `compress` and return `tokens_saved` plus the compression ratio

⚠️ Risks

  • Intel macOS has no prebuilt ONNX Runtime, so source installation fails by default
  • Without AVX2 on x86, Magika and embedding relevance use fallback paths
  • `npm install headroom-ai` provides only the TypeScript SDK and no `headroom` command
  • Runtime may need downloads from `cdn.pyke.io` and `huggingface.co`

👥 For who?

  • Teams using Claude Code, MCP, or coding agents who care about context cost
  • Developers integrating with the OpenAI Python client or the TypeScript SDK
  • Users running on Apple Silicon, Linux, or x86 environments with AVX2