Robust Bengali Speech Recognition Under Domain Shift

Architecture

How the system is shaped. Full implementation stays private.

1. Step 1

   Data

   OOD-Speech scale corpus with dialect, noise, and channel shift across speakers.

2. Step 2

   Baselines

   Whisper-small, IndicWav2Vec, and regional Bengali Whisper for fair comparison.

3. Step 3

   Adaptation

   GroupDRO fine-tuning with Optuna to reduce worst-group error under shift.

4. Step 4

   Serving

   Portfolio Cloudflare gallery for recruiters. Full Gradio/FastAPI with weights on request.

Mean WER (published eval)

Published overall WER table aligned with the ACM TALLIP writeup. Frozen regional Whisper drops WER by about 76% versus Whisper-small. GroupDRO drops WER by about 40% versus that frozen regional run.

Algorithm

GroupDRO domain-robust fine-tuning

Optimize for high-loss domains so the model does not overfit easy recording conditions.

for step in training:
  losses = [loss(batch_d) for d in domains]
  weights = soft_update(weights, losses)  # up-weight hard domains
  total = sum(w * L for w, L in zip(weights, losses))
  total.backward(); optimizer.step()

Key logic

Compact illustrative snippet (Worst-group emphasis (illustrative)). Not the full codebase.

def group_dro_loss(losses: list[float], weights: list[float], eta: float):
    # Raise weight on domains with higher loss
    weights = normalize([w * exp(eta * L) for w, L in zip(weights, losses)])
    return sum(w * L for w, L in zip(weights, losses)), weights

Stack