🎙️ A voice browser that hears Chinese · เบราว์เซอร์สั่งด้วยเสียงที่ฟังภาษาจีนได้ · 一个听得懂中文的语音浏览器

node --version      # must be v20 or newer; the fork declares "node": ">=20"
npm --version
python --version    # 3.9+ for the bridge in Step 4
git --version

# Windows
winget install OpenJS.NodeJS.LTS

# macOS
brew install node

# Debian / Ubuntu — the distro package is usually too old, so use NodeSource
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# Keep the path SHORT. See the warning below.
python -m venv C:\asr\venv          # Windows
C:\asr\venv\Scripts\activate

python3 -m venv ~/asr/venv          # macOS / Linux
source ~/asr/venv/bin/activate

pip install --upgrade pip

# Create a key at https://console.typesafe.ai/keys, then either put it in .env:
#   TYPESAFE_API_KEY=sk-...
# or export it. The legacy name JEV_API_KEY is still accepted.

export TYPESAFE_API_KEY=sk-...             # macOS / Linux
$env:TYPESAFE_API_KEY = "sk-..."           # Windows PowerShell
setx TYPESAFE_API_KEY "sk-..."             # Windows, persistent (reopen the shell)

# Windows
"C:\Program Files\Google\Chrome\Application\chrome.exe" ^
  --remote-debugging-port=9229 --user-data-dir=C:\chrome-cdp

# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --remote-debugging-port=9229 --user-data-dir=/tmp/chrome-cdp

# Linux
google-chrome --remote-debugging-port=9229 --user-data-dir=/tmp/chrome-cdp

# Prove CDP is actually listening before you go any further:
curl http://127.0.0.1:9229/json/version

1 One line of English, hardcoded · อังกฤษบรรทัดเดียว ฝังตายไว้ · 一行写死的英文

🔎 UPSTREAM, src/public/index.html LINE 297

rec.continuous = true; rec.interimResults = true; rec.lang = "en-US"; rec.maxAlternatives = 1;

💻 SEE IT FOR YOURSELF

git clone https://github.com/moritzkremb/jev-voice-browser
cd jev-voice-browser
npm install
cp .env.example .env               # paste your key from Step 0

sed -i.bak 's/rec.lang = "en-US"/rec.lang = "zh-CN"/' src/public/index.html   # -i.bak works on GNU and BSD sed

./run.sh                           # Windows: set the key, then  npm start
# open http://localhost:8787, click Start Mic Stream, say 搜索成都美食

2 Chinese has no spaces to split on · จีนไม่มีช่องว่างให้ตัด · 中文没有空格可切

🔎 THE FORK, src/spans.js — CHINESE PAYLOAD VERBS

// Chinese payload verbs with destination prefix (must come before generic verbs)
/(?:在|到)?(?:搜索框|输入框|评论框|文本框)(?:中|里|内)?\s*(?:输入|键入|打入|填写|写)\s*/i,
/(?:在|到)?(?:搜索框|输入框|评论框|文本框)(?:中|里|内)?\s*(?:搜索|搜一下|搜|查找|查询)\s*(?:关于)?\s*/i,
/(?:在|去)?(?:谷歌|百度|必应|维基百科|youtube|b站|github|知乎)\s*(?:上|里)?(?:搜索|搜一下|搜|查找)\s*(?:关于)?\s*/i,
/(?:搜索|搜一下|搜|查找|查询)\s*(?:关于)?\s*/i,
/(?:输入|键入|打入|填写|写)\s*/i,

3 Path A — the Doubao bridge · ทาง A — สะพาน Doubao · 路线 A —— 豆包桥

💻 ON THE MAC

brew install blackhole-2ch          # the virtual audio device the trick needs

# Install the Doubao IME (豆包输入法) by hand, then in System Settings grant:
#   Privacy & Security -> Microphone     -> Doubao
#   Privacy & Security -> Accessibility  -> the bridge app (or your Terminal)

git clone https://github.com/winter-loo/doubao-voice-bridge
cd doubao-voice-bridge
swift build -c release
scripts/build-mac-app.sh

dist/DoubaoVoiceBridge.app/Contents/MacOS/doubao-bridge-mac
# now listening: TCP 4387 control, TCP 5004 audio

💻 ON THE MACHINE RUNNING THE FORK (SAME BOX, OR ANY BOX ON THE LAN)

git clone https://github.com/winter-loo/jev-voice-browser
cd jev-voice-browser
npm install
cp .env.example .env               # put your Jev key in it

# Chrome must already be running with CDP open:
#   chrome --remote-debugging-port=9229

node src/server.js --port 8787 \
  --cdp http://127.0.0.1:9229 \
  --doubao-host 192.168.1.50 \
  --doubao-port 4387 --doubao-audio-port 5004
# then open http://localhost:8787 and click Start Mic Stream

4 Path B — your own recogniser, same two sockets · ทาง B — ตัวถอดเสียงของคุณเอง สองซ็อกเก็ตเดิม · 路线 B —— 你自己的识别器，同样两个套接字

💻 INSTALL

pip install numpy                       # the stub engine needs only this
pip install funasr torch torchaudio     # and this, for the real one

# The model downloads itself on first run, about 900 MB:
#   paraformer-zh-streaming
# Use --device cpu if you have no CUDA.

📄 asr_bridge.py

"""Stand in for Doubao Voice Bridge with any Chinese ASR you can run yourself.

The fork only needs two sockets: line-delimited JSON out on the control port,
and raw 48kHz s16le PCM in on the audio port. Speak that and it cannot tell
the difference. --engine stub proves the wiring without downloading a model.
"""
import argparse, json, socket, threading, time
import numpy as np

CHUNK_MS = 200          # how often we look at the audio
SILENCE_DBFS = -42.0    # below this counts as silence
SILENCE_MS = 700        # this much silence ends the utterance


class Stub:
    """Emits a fixed Chinese command so you can test the socket contract alone."""
    def __init__(self): self.n = 0
    def feed(self, pcm16k, is_final):
        self.n += 1
        text = "在谷歌搜索成都美食"
        return text[: min(len(text), self.n * 3)] if not is_final else text
    def reset(self): self.n = 0


class FunASR:
    """paraformer-zh-streaming. Chinese-first, streams, runs on CPU or CUDA."""
    def __init__(self, model, device):
        from funasr import AutoModel
        self.model = AutoModel(model=model, device=device, disable_update=True)
        self.cache, self.text = {}, ""
    def feed(self, pcm16k, is_final):
        res = self.model.generate(input=pcm16k, cache=self.cache, is_final=is_final,
                                  chunk_size=[0, 10, 5], encoder_chunk_look_back=4,
                                  decoder_chunk_look_back=1)
        for r in res:
            self.text += r.get("text", "")
        return self.text
    def reset(self): self.cache, self.text = {}, ""


def dbfs(x):
    if not len(x): return -120.0
    rms = float(np.sqrt(np.mean(x.astype(np.float64) ** 2)))
    return 20 * np.log10(max(rms, 1e-9) / 32768.0)


class Bridge:
    def __init__(self, engine):
        self.engine = engine
        self.ctrl = None
        self.pcm = bytearray()
        self.running = False
        self.session = 0
        self.active = False
        self.lock = threading.Lock()

    def emit(self, obj):
        if not self.ctrl: return
        try:
            self.ctrl.sendall((json.dumps(obj, ensure_ascii=False) + "\n").encode())
        except OSError:
            self.ctrl = None

    def control(self, conn):
        self.ctrl = conn
        self.emit({"type": "hello", "bridge": "asr_bridge.py"})
        buf = b""
        while True:
            try:
                data = conn.recv(4096)
            except OSError:
                break
            if not data: break
            buf += data
            while b"\n" in buf:
                line, buf = buf.split(b"\n", 1)
                cmd = line.decode(errors="ignore").strip()
                if cmd == "start":
                    self.session += 1
                    self.running = True
                    self.active = False
                    self.engine.reset()
                    with self.lock: self.pcm.clear()
                    self.emit({"type": "status", "phase": "recording", "recording": True})
                elif cmd == "stop":
                    self.running = False
                    self.emit({"type": "status", "phase": "idle", "recording": False})
        self.ctrl = None

    def audio(self, conn):
        while True:
            try:
                data = conn.recv(16384)
            except OSError:
                break
            if not data: break
            if self.running:
                with self.lock: self.pcm.extend(data)

    def recognize(self):
        """48k -> 16k by averaging every 3 samples, then feed the engine."""
        need = int(48000 * 2 * CHUNK_MS / 1000)
        quiet = 0
        while True:
            time.sleep(CHUNK_MS / 1000)
            if not self.running: continue
            with self.lock:
                if len(self.pcm) < need: continue
                take = bytes(self.pcm[:need]); del self.pcm[:need]
            x = np.frombuffer(take, dtype="<i2")
            x16 = x[: len(x) // 3 * 3].reshape(-1, 3).mean(axis=1).astype(np.int16)
            level = dbfs(x16)
            self.emit({"type": "audio_level", "rmsDBFS": round(level, 1)})
            if level >= SILENCE_DBFS:
                self.active, quiet = True, 0
            else:
                quiet += CHUNK_MS
                if not self.active:
                    continue          # silence between utterances: nothing to do
            done = quiet >= SILENCE_MS
            text = self.engine.feed(x16.astype(np.float32) / 32768.0, done)
            if not text: continue
            if done:
                self.emit({"type": "final", "text": text, "session_id": self.session})
                self.engine.reset()
                self.active, quiet = False, 0
            else:
                self.emit({"type": "partial", "text": text, "session_id": self.session})


def serve(port, handler):
    s = socket.socket(); s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(("0.0.0.0", port)); s.listen(1)
    while True:
        conn, _ = s.accept()
        threading.Thread(target=handler, args=(conn,), daemon=True).start()


if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--port", type=int, default=4387)
    p.add_argument("--audio-port", type=int, default=5004)
    p.add_argument("--engine", choices=["stub", "funasr"], default="funasr")
    p.add_argument("--model", default="paraformer-zh-streaming")
    p.add_argument("--device", default="cuda")
    a = p.parse_args()

    engine = Stub() if a.engine == "stub" else FunASR(a.model, a.device)
    b = Bridge(engine)
    threading.Thread(target=b.recognize, daemon=True).start()
    threading.Thread(target=serve, args=(a.audio_port, b.audio), daemon=True).start()
    print(f"control :{a.port}  audio :{a.audio_port}  engine {a.engine}", flush=True)
    serve(a.port, b.control)

📄 probe.py — pretends to be the fork, so you can test without a browser

"""Pretend to be the fork: send 'start', stream PCM, print what comes back."""
import json, socket, sys, threading, time
sys.stdout.reconfigure(encoding='utf-8')
import numpy as np

events = []
ctrl = socket.create_connection(("127.0.0.1", 4387))
def reader():
    buf = b""
    while True:
        d = ctrl.recv(4096)
        if not d: return
        buf += d
        while b"\n" in buf:
            line, buf = buf.split(b"\n", 1)
            if line.strip():
                e = json.loads(line)
                events.append(e)
                if e["type"] != "audio_level":
                    print(json.dumps(e, ensure_ascii=False))
threading.Thread(target=reader, daemon=True).start()
time.sleep(0.3)
ctrl.sendall(b"start\n")
audio = socket.create_connection(("127.0.0.1", 5004))

t = np.arange(48000) / 48000.0                      # 1.0 s of speech-ish tone
loud = (np.sin(2 * np.pi * 220 * t) * 8000).astype("<i2")
quiet = np.zeros(48000, dtype="<i2")                # then 1.0 s of silence
for block in (loud, quiet):
    for i in range(0, len(block), 4800):            # 100 ms at a time
        audio.sendall(block[i:i + 4800].tobytes())
        time.sleep(0.05)
time.sleep(1.5)
ctrl.sendall(b"stop\n")
time.sleep(0.3)

kinds = [e["type"] for e in events]
print("---")
print("event types seen:", sorted(set(kinds)))
print("partials:", kinds.count("partial"), " finals:", kinds.count("final"))
fin = [e for e in events if e["type"] == "final"]
print("final text:", fin[-1]["text"] if fin else "NONE")

💻 RUN IT

# terminal 1 — the bridge, with the fake recogniser first
C:\asr\venv\Scripts\activate            # Windows
source ~/asr/venv/bin/activate          # macOS / Linux
python asr_bridge.py --engine stub

# terminal 2 — prove the contract without a browser
python probe.py

# then swap in the real one, and point the fork at this machine
python asr_bridge.py --engine funasr --device cuda    # or --device cpu
node src/server.js --port 8787 --cdp http://127.0.0.1:9229 \
  --doubao-host 127.0.0.1 --doubao-port 4387 --doubao-audio-port 5004

📋 WHAT CAME BACK — REAL OUTPUT, 26 SEPTEMBER 2026

{"type": "hello", "bridge": "asr_bridge.py"}
{"type": "status", "phase": "recording", "recording": true}
{"type": "partial", "text": "在谷歌", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "partial", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "final", "text": "在谷歌搜索成都美食", "session_id": 1}
{"type": "status", "phase": "idle", "recording": false}
---
event types seen: ['audio_level', 'final', 'hello', 'partial', 'status']
partials: 8  finals: 1
final text: 在谷歌搜索成都美食

5 Path C — the browser already has a Chinese recogniser · ทาง C — เบราว์เซอร์มีตัวถอดเสียงจีนอยู่แล้ว · 路线 C —— 浏览器本来就有中文识别器

💻 THE WHOLE PATCH

git clone https://github.com/moritzkremb/jev-voice-browser
cd jev-voice-browser && npm install

# 1. the language, in src/public/index.html around line 297
sed -i.bak 's/rec.lang = "en-US"/rec.lang = "zh-CN"/' src/public/index.html   # -i.bak works on GNU and BSD sed

# 2. the grammar. Copy the Chinese TEXT_VERBS block out of the fork's
#    src/spans.js and paste it into yours, ABOVE the generic verbs:
#    https://github.com/winter-loo/jev-voice-browser/blob/main/src/spans.js

cp .env.example .env               # the key from Step 0 is still required
./run.sh                           # Windows: npm start, with the key exported

# Chrome must already be listening on 9229 — see Step 0.
curl http://127.0.0.1:9229/json/version

6 Nothing here speaks. Thirty lines that do. · ไม่มีอะไรพูดได้ สามสิบบรรทัดที่พูดได้ · 这里没有会说话的。三十行让它会说。

📄 speak.js — drop it into src/public/index.html

// Say a short Mandarin confirmation when the fork acts. The page already
// knows what it did; this only gives that a voice.
let zhVoice = null;

function pickChineseVoice() {
  const zh = speechSynthesis.getVoices().filter(v => /^zh/i.test(v.lang));
  // Rank, never take the first one. Chrome's "Google ..." voices are far
  // better than the OS defaults, so they win even though they are remote.
  const score = v => (/^google/i.test(v.name) ? 100 : 0)
                   + (/natural|neural/i.test(v.name) ? 20 : 0)
                   + (v.lang === "zh-CN" ? 10 : 0);
  zhVoice = zh.sort((a, b) => score(b) - score(a))[0] || null;
  // No Chinese voice at all is a real outcome, not an error. Stay quiet
  // rather than reading 已点击 aloud in an English voice.
  if (!zhVoice) console.warn("[speak] no Chinese voice installed; staying silent");
}
pickChineseVoice();
speechSynthesis.onvoiceschanged = pickChineseVoice;  // voices load late

function say(text) {
  if (!zhVoice) return;
  speechSynthesis.cancel();          // never queue: the last action is the only one that matters
  const u = new SpeechSynthesisUtterance(text);
  u.voice = zhVoice;
  u.lang  = zhVoice.lang;
  u.rate  = 1.05;
  speechSynthesis.speak(u);
}

// Wire it to whatever the page already logs. Keep them short — a spoken
// sentence outlasts the action and you end up talking over yourself.
function onAction(kind, target) {
  if (kind === "click")  say("已点击 " + target);
  if (kind === "search") say("正在搜索");
  if (kind === "nav")    say("已打开");
  if (kind === "unsure") say("没听清");
}

📋 WHAT CHROME ACTUALLY HAS — MEASURED 26 SEPTEMBER 2026, WINDOWS 11

Chrome (normal window)          22 voices, 3 Chinese:
    Google 普通话（中国大陆）   [zh-CN]  localService = false
    Google 粤語（香港）         [zh-HK]  localService = false
    Google 國語（臺灣）         [zh-TW]  localService = false

Chromium bundled with Playwright     0 voices, 0 Chinese

💻 RENDER THE PHRASES ONCE, WITH PIPER

pip install piper-tts

mkdir -p voices src/public/audio
cd voices
curl -LO https://huggingface.co/rhasspy/piper-voices/resolve/main/zh/zh_CN/huayan/medium/zh_CN-huayan-medium.onnx
curl -LO https://huggingface.co/rhasspy/piper-voices/resolve/main/zh/zh_CN/huayan/medium/zh_CN-huayan-medium.onnx.json
cd ..

# ASCII filenames keep the JavaScript simple.
M=voices/zh_CN-huayan-medium.onnx
echo 已点击   | piper -m $M -f src/public/audio/clicked.wav
echo 正在搜索 | piper -m $M -f src/public/audio/searching.wav
echo 已打开   | piper -m $M -f src/public/audio/opened.wav
echo 没听清   | piper -m $M -f src/public/audio/unsure.wav
echo 已确认   | piper -m $M -f src/public/audio/confirmed.wav
echo 已取消   | piper -m $M -f src/public/audio/cancelled.wav

📄 the offline replacement for say()

// The Node server already serves src/public, so the wavs are just there.
// No voice list, no onvoiceschanged, and it works in Playwright's Chromium.
const CLIPS = { click: "clicked", search: "searching",
                nav: "opened",  unsure: "unsure" };

function say(kind) {
  const name = CLIPS[kind];
  if (!name) return;
  const a = new Audio("audio/" + name + ".wav");
  a.play().catch(() => {});   // autoplay can refuse before the first click
}

# Windows
netstat -ano | findstr "9229 8787 4387 5004"

# Linux
ss -ltnp | grep -E '9229|8787|4387|5004'

# macOS
lsof -nP -iTCP:9229 -iTCP:8787 -iTCP:4387 -iTCP:5004 -sTCP:LISTEN

# Chrome's CDP endpoint should answer with JSON, not a refusal:
curl http://127.0.0.1:9229/json/version

# And the control page should answer 200:
curl -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8787/

# Windows
taskkill /F /IM node.exe
taskkill /F /IM python.exe

# macOS / Linux
pkill -f "src/server.js"
pkill -f asr_bridge.py