CodeWithYou

Skip Self-Hosting Whisper - Automate Meeting Transcription with a REST API

Published on
Authors
Skip Self-Hosting Whisper — Automate Meeting Transcription with an API

A while back I wrote about automating meeting notes from video using a self-hosted pipeline: Whisper for transcription, sumy for summarization, and MarianMT for translation. It works, and the code is still on GitHub if you want to run it yourself.

But after using it for a while, a few pain points kept coming back:

  • GPU cost. Whisper is fast on a GPU and painfully slow on CPU — for occasional use, renting GPU time just to transcribe a 40-minute meeting doesn't pencil out.
  • Environment drift. ffmpeg not found, sentencepiece build failures, CUDA version mismatches — the usual self-hosted ML tax.
  • Translation quality. MarianMT is fine for a side project, but it's noticeably behind what a modern LLM produces for Vietnamese, especially for domain-specific business/journalism vocabulary.
  • Speaker separation. Whisper alone doesn't tell you who said what — you need a separate diarization model bolted on, with its own dependency headaches.

So for anything beyond a personal experiment, I ended up building Amaniva — a hosted transcription + summarization API that does the same job as that pipeline, minus the infrastructure.

The API, in one request

No GPU, no ffmpeg install, no dependency conflicts — just an API key (grab one for free from your account settings after signing up) and a file:

curl -X POST https://transcribe-api.amaniva.com/v1/transcribe \
  -H "X-API-Key: YOUR_API_KEY" \
  -F "file=@meeting.mp3" \
  -F "enable_diarization=true"

{
  "task_id": "b7e1c9a0-...",
  "status": "PENDING",
  "message": "Transcription task submitted"
}

Transcription runs in the background (a 40-minute meeting isn't instant), so you poll for the result:

curl https://transcribe-api.amaniva.com/v1/transcribe/status/b7e1c9a0-... \
  -H "X-API-Key: YOUR_API_KEY"

{
  "task_id": "b7e1c9a0-...",
  "status": "SUCCESS",
  "result": {
    "text": "Good morning everyone, let's start with the sprint review...",
    "language": "en",
    "duration_sec": 2418.4,
    "speakers": [
      { "speaker": "Speaker 1", "text": "Good morning everyone...", "start_time": 0.0, "end_time": 4.2 },
      { "speaker": "Speaker 2", "text": "Sounds good, let's dive in.", "start_time": 4.5, "end_time": 6.1 }
    ]
  }
}

That's the whole integration. No model to download, no worker process to keep alive.

What you get beyond raw Whisper

  • Speaker diarization included — no separate model or dependency to wire up.
  • AI summarization (concise / bullet points / podcast-style write-up) as a separate endpoint, so you can keep the raw transcript and the summary independent.
  • Files are deleted right after processing — nothing is kept around or used to train anything, which matters if you're transcribing anything remotely sensitive (which, let's be honest, meeting recordings usually are).
  • A free tier (10 jobs/month) if you just want to try it against a real recording before deciding anything.

When self-hosting still makes more sense

If you're processing a large, steady volume of audio and already have GPU infrastructure sitting around, the self-hosted route in my original post is still worth it — you own the whole pipeline and there's no per-request cost. The API trades that control for "it just works," which is the right trade for anyone who wants meeting transcripts without becoming an ML-ops person on the side.

If that's you, amaniva.com is free to try, no credit card required.

Happy coding!

Advertisement