Earshot Explained: Giving Claude Code Ears with a CPU-Only Audio MCP Server

Claude Code can write the shader, wire up the render pipeline, and ship the launch video. It still can't tell you the explosion sound effect clips at -2dB or that the whoosh comes in two frames late. That's the gap Earshot is built to close. It's a weirdly specific problem to pick, which is exactly why it's interesting.

Earshot is an open-source, MIT-licensed toolkit from developer Leon Lin (@LexnLin) that gives coding agents something like hearing. Not speech recognition, not a voice assistant. It analyzes sound design: sound effects, mixes, foley, synthesized audio. It runs as a CLI, a Python API, or an MCP server that plugs straight into Claude Code, and the whole thing runs on CPU. No GPU, no API key, no cloud call per request.

The problem it's actually solving

Lin's framing on the announcement thread is blunt: Opus 5.5 can build a whole launch video in code, but the sound effects come out sloppy, and the agent has no way to know that. Vision-and-text agents can review their own screenshots. They have no equivalent for audio. They generate a sound, and the loop ends. Earshot adds the missing half: hear what you made, score it, and iterate.

This matters more than it sounds like it should, because agents are increasingly generating product demos, motion-design assets, and procedural game audio directly in code. Without a way to evaluate the output, "add a satisfying click sound" is a prompt an agent can execute but never verify.

What it actually does

Earshot ships a set of composable commands, exposed as MCP tools once you wire it into Claude Code: describe, match, find, compare, timeline, sheet, critique, measure, forge, fit, sfx, kit, and gallery.

In practice that breaks down into a few workflows:

• Describe and match: tag a clip with labels and measurements, or check how well a rendered sound matches a text brief, with a confidence verdict rather than a binary pass/fail.
• Find and timeline: locate a specific sound event inside a longer file, or get a second-by-second map of a whole edit synced to picture cuts.
• Critique: score a soundtrack against a reference profile built from professional launch films.
• Forge and fit: the two-way part. forge synthesizes sound effects procedurally in code (modal resonators, contact excitations, filtered noise, so no sampled audio and no licensing questions), and fit goes the other direction, turning an existing recording into an editable parameterized recipe.

As of v0.3, released October 19, 2026, Earshot also ships a bank of 201 pre-rendered, code-generated sound effects that agents can pull by ID and tweak (duration, pitch, timing) instead of synthesizing from scratch every time.

How it runs, and what it costs you

Two small open models do the heavy lifting: CLAP (laion/larger_clap_general) for text-audio matching, and an AudioSet-finetuned AST model for sound classification. Combined they're roughly 1.1–1.4 GB, downloaded once into a local cache, then everything runs offline. Setup is a standard Python virtualenv:

git clone https://github.com/Leonxlnx/earshot && cd earshot
python -m venv .venv
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install -e ".[mcp,llm,dev]"

Then point Claude Code at it:

claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /absolute/path/to/earshot/.venv/bin/python -m earshot mcp

Requirements are modest: Python 3.10+, ffmpeg/ffprobe on PATH, and literally no GPU. "Models load on the first call and stay warm" is the only latency concession. The first MCP call in a session is slower. After that it's not.

Does it actually work? Lin published the numbers

Credit where it's due: the repo ships a docs/VALIDATION.md with real benchmark numbers instead of just vibes, and the limitations section is more candid than most open-source tool READMEs bother to be.

The good: the "strong match" verdict correctly identifies sound type 98% of the time (exact variant is a rougher 61%), real-vs-synthesized detection sits at 0.92 ROC-AUC on held-out sound families, and event detection inside a mixed 60-second film track catches 93% of effects (89% for quiet ones buried under music).

The honest part: Lin explicitly lists what the models can't do. There's no judgment of "taste or emotion, provenance or licenses, speech content, or anything above 14 kHz," and genre/mood tags are hints, not something to trust. That 14 kHz ceiling matters if you're checking cymbal shimmer or high-frequency synth work; the models simply don't see that range.

Why the CPU-only framing is the real pitch

Plenty of audio ML tooling assumes you have a GPU box sitting around. Earshot's whole design (small models, local cache, no inference server) is aimed at the exact audience that doesn't: someone running Claude Code on a laptop who wants an agent that can self-check a sound effect without spinning up cloud infrastructure for it. It's the same instinct behind cpu-only forecasting tools showing up elsewhere this year: push capability down to hardware people already have instead of up to hardware they'd have to rent.

The project went from launch to 2.1k GitHub stars and 124 forks inside about a week, which for a narrowly-scoped audio-analysis MCP server is a genuinely fast pickup (reportedly over 1,100 stars in the first four days alone). Lin has also mentioned Earshot is already being used inside "cinetic," a separate motion-design skill, which is a decent early signal that this isn't just a demo toy.

Where this fits and what's missing

Earshot doesn't do speech, doesn't do music mastering, and doesn't replace a sound designer's ears for anything requiring taste. What it does is give an agent a self-check loop for the specific, narrow, previously-impossible task of verifying "does this sound effect match the brief and land on the right frame," which is a smaller claim than "AI can hear" but a much more useful one. If you're building agent pipelines that touch video or game audio, it's worth wiring in now while it's free, CPU-friendly, and still early enough that the limitations list is still short.