Audio8 ASR Infinite review: is Edge0's streaming speech-to-text model worth your GPU?
· 12 min read

Audio8 ASR Infinite review: is Edge0's streaming speech-to-text model worth your GPU?

Audio8 ASR Infinite promises flat-memory streaming transcription on your own hardware. We read the model card, benchmarks and license before you download it.

Audio8 ASR Infinite is a speech-to-text model that claims it can transcribe a live audio stream for as long as you keep it running, with delays under a second. It is also a model you have to host yourself. Our verdict: worth watching, but not yet worth installing for most beginners.

This review is based on the official model card, the project repository, vendor benchmarks, and third-party coverage — we did not run the model hands-on.

Start with the model card and the GitHub repo if you want the primary material. Everything below cites those sources or the vendor’s own pages.

What is Audio8 ASR Infinite?

Audio8 ASR Infinite is an open-weight speech recognition model from the vendor Edge0. It transcribes Chinese and English audio as it arrives, rather than waiting for a full recording. The checkpoint is a preview: it covers the core recognition model only, and the vendor’s fuller release is still in progress.

  • Vendor: Edge0, which describes itself as “On-Device AI” and promotes “cloud-level intelligence that runs locally” on its research page. That is marketing copy, not a measurement.
  • Size: 4 billion parameters in bfloat16, with an 8.17 GB main weights file plus a separate semantic VAD file.
  • Languages: Chinese and English only.
  • Status: Preview release, ASR base only.
  • Traction at the source snapshot (11 November 2026): 877 likes and 994 downloads last month on Hugging Face, with no inference provider hosting it, plus four finetunes, six quantizations and three Spaces. On GitHub the repo had 125 stars, 16 forks and three commits. This is an early, small community.

Verdict and score

Audio8 ASR Infinite scores 6/10. The streaming engineering is real and well documented, but you must bring your own CUDA GPU, the model handles only Chinese and English, and no vendor support channel is documented. Treat it as a self-host project for technical readers, and a model to watch for everyone else.

DimensionScore
Ease of use5/10
Features7/10
Performance7/10
Documentation6/10
Support2/10
Overall Score6/10
  • Ease of use (5): The Docker Compose path and bundled web demo are genuinely simple. But you need your own CUDA GPU, a patched vLLM build and fully merged weights, and there is no hosted API to fall back on. That GPU burden is the main penalty.
  • Features (7): Native streaming, selectable 80/120/160 ms clocks, configurable delays, a rolling cache and a semantic VAD design. It loses points for being bilingual only and for the frame-level semantic perception still being in progress.
  • Performance (7): Strong vendor figures on Chinese, but it trails Voxtral on both English sets. All numbers are vendor-published at its released operating point, and none has been independently replicated.
  • Documentation (6): The model card is detailed, with commands, operating points and examples. It offers no VRAM guidance, its own paper is still unpublished, and the vendor pricing page returns a 404.
  • Support (2): No vendor support desk, channel or SLA is documented in any source we checked, so this score rests on the visible GitHub issues on a three-commit repository alone. There is no hosted service and no published contact route for help.
  • Overall (6): This is an editorial call, not an average of the five scores.

See where it lands in our model arena.

What “Infinite” actually means — and what it doesn’t

“Infinite” describes a session that can run for a long time without memory or delay growing, not a model with no limits. Audio8 does this with a rolling 30-second window of audio context that drops old frames. The vendor makes this claim, and independent confirmation is not yet available.

The model keeps a rolling KV cache with a native context of 30 seconds. When old frames are evicted, it re-bases RoPE (the rotary position embeddings that track where each piece of audio sits in the sequence), which in plain terms means it renumbers its position markers so the window can slide indefinitely. The vendor says memory and latency stay flat across 24/7 operation.

  • Audio clock: 80, 120 or 160 ms. Each step emits one text token, which works out to 12.5, 8.3 or 6.25 decisions per second. A faster clock means more decisions and more compute.
  • Delay: target_delay_ms runs from 240 to 560 ms and must be an integer multiple of the clock. The vendor frames it as trading delay for accuracy.
Audio clockFrame lengthStreaming left-pad tokensSupported delays (ms)
80 ms418240, 320, 480, 560
120 ms612240, 480
160 ms89320, 480

The semantic VAD head sorts speech into 8 classes with horizons of 0.5, 1.0, 2.0 and 3.0 seconds. The vendor says it can tell a thinking pause or stutter from a genuine end of turn, a case where acoustic voice activity detection usually fails. That matters for voice agents. But frame-level semantic perception is the headline feature still listed as in progress, so treat it as a design claim until the formal release.

If you are building an always-listening assistant, NiteAgent writes about voice assistants and agents, where streaming recognition becomes the input layer.

Benchmarks: how to read them without being fooled

The model card reports error rates, where lower is better. On the vendor’s own tests, Audio8 leads on Chinese by a wide margin, while Voxtral wins both English sets. Every number is vendor-published at the released operating point, so treat them as claims to check, not verdicts.

CER (character error rate) counts wrong characters, and WER (word error rate) counts wrong words. Chinese is written without spaces between words, so CER is the fair metric there, while WER is standard for English. A figure of 1.750 means roughly 1.75 errors per 100 characters or words.

Test setMetricAudio8 ASR InfiniteVoxtral-Mini-4B-Realtime-2602nemotron-3.5-asr-streaming-0.6b
aishell1/testCER1.75016.79512.927 (@560 ms)
aishell4/testCER2.89316.45614.677 (@560 ms)
LibriSpeech test.cleanWER3.0422.2103.353 (@560 ms)
LibriSpeech test.otherWER6.8085.5527.140 (@560 ms)
Average3.62310.2539.524

The card’s settings are greedy decoding, EOS suppressed, an 80 ms clock and a 480 ms delay. The third-party explainer from MindStudio reads the results as heavily tuned for Chinese, with a small English trade-off against Voxtral.

Three caveats matter before you compare numbers:

  • The averages aren’t like-for-like. The card labels Voxtral’s 10.253 as covering two reported sets, yet that figure equals the mean of all four Voxtral values in the table. The averages also mix CER and WER. Compare the per-set figures instead.
  • Nemotron is shown at 560 ms, not the 480 ms used for Audio8, so its delay setting differs.
  • Nothing is independently replicated. The checkpoint’s own paper has not been published yet. The one paper cited in the sources, arXiv:2605.28139, belongs to the 0.1B sibling, not this checkpoint.

What you need to run it

You need your own CUDA-capable GPU, Docker, and the fully merged checkpoint on disk. The documented route is a patched vLLM build started with Docker Compose. No VRAM minimum is published, so judge your hardware against the 8.17 GB main file plus the VAD heads rather than trusting a guess.

  • Canonical path: cd docker and then AUDIO8_MODEL_DIR=/path/to/checkpoint docker compose up -d. The bundled web demo runs on localhost:8080 (plain) and localhost:8443 (self-signed TLS). The realtime service listens on 18190 inside compose and is published on host port 18191, with a WebSocket endpoint at /v1/realtime.
  • Terminal client: the vendor’s sample client accepts --target-delay-ms 480 and --pace, which lets you stream a WAV file.
  • Torch path: a simulated streaming decode that is slower and intended for testing only. The Python API uses Audio8ASRInfiniteForConditionalGeneration with trust_remote_code=True.
  • Weights: only fully merged weight directories work. Adapter-style or partially converted weights are rejected.
  • Serving background: the patched build sits on top of vLLM’s documentation, so read those docs first if the serving layer is new to you.
  • Community build: a compressed community build exists in mlx, onnx and gguf formats. It is experimental and publishes no quality numbers.
  • Cost: there is no hosted API and no per-minute price, and the vendor’s pricing page returns a 404. Your cost is your hardware.

Developers who want to read the patched serving code before running it can use codeintel.xyz as a starting point for code inspection. For hardware questions, our guide to running AI on your laptop covers what consumer machines can handle, and our Ollama vs LM Studio local-LLM comparison shows how local model tooling compares. You can also browse our tools directory.

The license trap most beginners will miss

The repository’s code and Edge0’s own contributions are Apache-2.0. But the decoder and LM head were initialized from Qwen2.5-3B-Instruct, and those weights follow the Qwen Research License. The model is therefore not simply Apache-2.0. Read the actual terms before making any commercial plan.

  • The repo includes a NOTICE file and a Qwen Research License file. Start there.
  • The sources do not spell out redistribution or commercial-use terms for the combined model, so do not assume either way.
  • Beginner takeaway: “open weights” does not mean “do whatever you want.”

Audio8 ASR Infinite vs alternatives: which should you pick?

Pick Audio8 ASR Infinite if you need Chinese and English streaming and can host it yourself. Pick Voxtral Realtime if English accuracy matters most, and a smaller streaming model if your GPU is limited. If you need more languages or no GPU at all, look at other Audio8 models or hosted services.

  • English-first streaming: Voxtral Realtime wins both LibriSpeech sets in the card’s own table. It is also a 4B model, so its hardware demands are similar. Its license and run options are not stated in the sources we checked.
  • Smaller streaming footprint: nemotron-3.5-asr-streaming-0.6b is a 0.6B model, with its vendor figures taken at 560 ms.
  • Offline, multilingual, no latency pressure: Whisper-style batch ASR fits better. Transcribing a finished recording is a different job, and it doesn’t need streaming.
  • No GPU, no setup: hosted cloud speech APIs. Infinite has no hosted API, so there is no price to compare, and this review cites no prices for those services.
  • More languages: the Audio8 ASR 3B and 0.6B models cover 19 languages according to the vendor. The 3B targets high-accuracy GPU use, and the 0.6B targets a balance of GPU, workstation and server.
  • CPU, edge or mobile: the Audio8 ASR 0.1B covers 7 languages, ships ONNX INT8 and INT4 variants and an iOS Neural Engine build, and is licensed cc-by-nc-4.0, which is non-commercial.
  • Speech output instead: the Audio8-TTS-Realistic-Air model is the family’s text-to-speech line.
ModelSizeStreamingLanguagesWeights and licenseHow you run itCost
Audio8 ASR Infinite4B (8.17 GB bf16)Yes, native; 80/120/160 ms clock, 240–560 ms delayChinese, EnglishApache-2.0 code plus Qwen Research License weightsYour own CUDA GPU via patched vLLM Docker; torch path for testing onlyNo hosted API or price published; you pay for hardware
Voxtral-Mini-4B-Realtime-26024BRealtime streaming (per its card)Not stated in the sources we checkedNot statedNot statedNot stated
nemotron-3.5-asr-streaming-0.6b0.6BYes (benchmarked at 560 ms)Not stated in the sources we checkedNot statedNot statedNot stated
Audio8 ASR 3B3BNot stated in the sources we checked19Not statedHigh-performance GPUNot stated
Audio8 ASR 0.1B0.1BNot stated in the sources we checked7 (Chinese, English, Cantonese, French, Japanese, German, Korean)cc-by-nc-4.0CPU, edge, mobile; ONNX INT8/INT4; iOS Neural EngineNot stated

“Not stated” means the sources we checked don’t say, not that the answer is no. No per-minute price exists for any Audio8 model in the sources. For more options, see our comparisons hub.

Who should wait for the formal release

Most beginners should wait. The checkpoint is a preview with the ASR base only, no independent replication exists, and the checkpoint’s paper has not been published. If you lack a CUDA GPU or are uneasy with Docker, watch the project rather than install it.

Edge0’s own limitations note is worth reading before any real use:

  • Transcripts can be affected by noise, accents, overlapping speech, specialist terms and source quality.
  • Transcripts are not verified records without review.
  • Medical, legal and financial uses need human oversight.
  • Obtain consent before processing anyone’s voice.

Install it now only if you are a developer with a CUDA GPU who wants to study streaming ASR and can accept preview-level stability.

Audio8 ASR Infinite FAQ

Short answers to the questions readers ask most about this checkpoint: what it costs, which languages it handles, whether the streaming claim holds, how much VRAM it needs, and whether it is ready for production. Each answer cites the model card or the vendor’s own pages rather than hands-on testing.

Is Audio8 ASR Infinite free to use? The weights are publicly downloadable, but the sources publish no free tier and no hosted API. Its code is Apache-2.0, but the Qwen-derived weights follow the Qwen Research License, so check the terms before commercial use. Your real cost is your own GPU.

Does it support languages besides Chinese and English? No. This checkpoint is bilingual. The 19-language support belongs to the Audio8 ASR 3B and 0.6B models.

Can it really transcribe forever? The vendor says a rolling 30-second KV cache with RoPE re-basing keeps memory and latency flat over 24/7 operation. “Infinite” means bounded, continuous sessions, not unlimited capability, and the claim has not been independently verified.

How much VRAM do I need? Not stated in the sources. The main weights file is 8.17 GB in bfloat16, plus a separate semantic VAD file, and the documented path is a patched vLLM Docker build on your own CUDA GPU.

Is it production-ready? No. It is a preview release with the ASR base only. Frame-level semantic perception, the checkpoint’s own paper and any independent replication are all still pending.


Audio8 ASR Infinite is a serious streaming research release with a narrow audience today: bilingual, self-hosted and still in preview. If that matches your setup, read the model card and license closely before you start. If it doesn’t, keep it on your watch list and check back when the formal release ships.

📊 See how Audio8 ASR Infinite compares →

  • ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
  • CodeIntel Log — code quality, debugging, and software engineering benchmarks

Cross-links automatically generated from None.

Back to all posts