Building rustygrep

LLM coding agents run thousands of grep calls per session. Every token in the context window is money and attention. Standard grep output is verbose — file paths repeated, whitespace everywhere, ANSI codes the model can't use.

I built rustygrep to fix that.

What it does

rustygrep is a fast grep replacement with token-compressed output. Same search speed as ripgrep, 60-95% fewer tokens.

It has three modes: plain output (human-readable), --llm (compressed for agents), and --json (structured for piping). Plus an MCP server so agents can call it directly.

Why Rust

I needed it to be fast enough that agents don't wait. Ripgrep's crates — ignore for gitignore-aware walking, grep-regex for SIMD matching, memchr for byte search — are battle-tested. I built on top of them and added the output layer.

Parallel search across all CPU cores via rayon. Gitignore-aware by default. File type filters. Regex support. Everything grep and ripgrep do, plus the token compression.

The token trick

The --llm mode strips ANSI codes, removes redundant whitespace, and groups results by file with match counts. Instead of:

src/main.rs:42:    fn calculate_total(items: &[Item]) -> u64 {
src/main.rs:43:        items.iter().map(|i| i.price).sum()

You get:

--- src/main.rs (2 matches)
42:fn calculate_total(items: &[Item]) -> u64 {
43:  items.iter().map(|i| i.price).sum()

65% fewer tokens. Same information. The --llm-budget N flag hard-caps output to N tokens for agents with tight context windows.

MCP integration

rustygrep mcp starts a Model Context Protocol server. Claude Code, Cursor, and OpenCode can call rustygrep_search, rustygrep_files, and rustygrep_count directly without shelling out.

This means the agent gets token-compressed results without the overhead of parsing CLI output.

What's next

Published as v0.1.1 on crates.io. cargo install rustygrep works. Plans: streaming output for huge result sets, custom output templates, and maybe a --watch mode for file changes.

---

Built on ripgrep's crates. Binary size under 3MB. MIT licensed.