How to automatically transcribe audio/video

[Reverend Jim]

I recently completed, for the most part, a wxPython app to convert speech to text in files containing an audio stream. As someone who is hearing impaired I find I am relying more and more on captioning. I am also finding that a number of videos, even when they include captioning, have captions that are irritatingly out of sync with the video. For videos that have no captions, Windows Live Captions are often inadequate.

I found a package (Whisper-cli) on github that is free and open source, and has a large number of downloadable models. I found one model which does a pretty good job of converting speech to text. But, the software runs from the command line and I found it cumbersome so I wrote a wxPython wrapper.

The wrapper offers a few extras. For example, Whisper works only on mp3 and wav files. If you have a local copy of ffmpeg, my GUI will convert the mp4 to mp3 and then do the transcription. In addition, you can convert the output into subtitle srt format.

If there is interest here I will write this up as a project and post the code.

[Contesthelper44]

That sounds like a useful project, especially for people who rely on accurate captions. A simple GUI with MP4 conversion and SRT export would make Whisper much easier to use. I'd definitely be interested in seeing the code and a write-up.

[gdtraveller]

I would like to to see how this project develops.
I'm quite interested in any project that combines an llm with audio input.
Having a wxpython wrapper would definitely be a bonus!

Thanks

[Reverend Jim]

We have company at the moment. Once we have the house back I'll do some more clean up and post the code.

[waqasahmed567]

That sounds like a useful approach, especially for videos where the existing captions are badly synchronized.

Having FFmpeg handle the video-to-audio conversion and then passing the audio to Whisper makes the workflow much more practical than using the command line alone. SRT export is also a nice addition since it makes the transcription immediately useful for videos.

I’d definitely be interested in seeing the project and code when you write it up. It could be particularly helpful to include which Whisper model you found gives the best balance between accuracy and processing time, along with any limitations you’ve noticed with accents, background noise, or multiple speakers.

[markjafry]

That sounds like a really useful project, especially for videos where the built-in captions are out of sync. The MP4 to MP3 conversion and SRT export would make the workflow much easier than using Whisper from the command line.

I’d definitely be interested in seeing the project and the wxPython code when you’re ready to share it.

[Reverend Jim]

A few things happening here are going to keep me from doing a proper cleanup for a bit so I'm just going to post the "working but not completely tidied up" version. I'll post the cleaned up code when I can get to it. But I'll still answer questions.

"""
Name:

    Transcribe.pyw

Description:

    Front end GUI for Whisper command line text extractor. Input files can be
    mp3 or wav files.

Notes:

    The "Strip" button will remove the header and footer lines and strip off
    all the timecode information. It will then rewrap all the text into one
    long stream.

    The "Make srt" button will reformat the output into the form used by srt
    subtitle files. Because Whisper does not read mp4 files, if you select
    one, an mp3 file will the same base name will be created using ffmpeg.
    If you want this feature you must first download ffmpeg (free) from

        www.ffmpeg.org

    then unzip it into a folder and add the bin folder to either the USER or
    SYSTEM %PATH% environment variable.

    The Whisper package can be downloaded at

        github.com/ggml-org/whisper.cpp

    Go to the above page and click "Releases" in the right pane. For Windows
    you'll want

        whisper-bin-x64.zip

    And you'll need to download a trained model from

        huggingface.co/ggerganov/whisper.cpp/tree/bin

    The model you want is

        ggml-base.bin

    Save it in the folder "model" where you unzipped Whisper

    An odd note - Whisper can occasionally generate multiple duplicate lines.
    The make_srt code will only generate one subtitle block for duplicate
    lines.

Audit:

    2026-07-31  rj  set default save name based on input file and output format
    2026-07-30  rj  add progress bar for mp4->mp3
    2026-07-29  rj  added Make srt
    2026-07-28  rj  original code
"""

import wx
import subprocess
import threading
import os
import re

TITLE = "Whisper Transcriber"

# Modify to reflect where you unzipped whisper
WHISPER_EXE = r"D:\apps\Whisper\whisper-cli.exe"
MODEL_DIR   = r"D:\apps\Whisper\models"

time_pattern = re.compile(r"\[(\d+):(\d+):(\d+\.\d+)")

class WhisperGUI(wx.Frame):

    def __init__(self):

        super().__init__(None, title=TITLE, size=(1200,800))

        self.process = None

        panel = wx.Panel(self)
        vbox  = wx.BoxSizer(wx.VERTICAL)

        # File picker - allow mp3 mp4 wav
        self.file_picker = wx.FilePickerCtrl(
            panel,
            message="Select an audio file",
            wildcard="Audio files (*.mp3;*.wav;*.mp4)|*.mp3;*.wav;*.mp4"
        )
        vbox.Add(self.file_picker, 0, wx.EXPAND | wx.ALL, 5)

        # Model picker
        self.model_choice = wx.Choice(panel)
        self.load_models()
        vbox.Add(self.model_choice, 0, wx.EXPAND | wx.ALL, 5)

        # Buttons
        hbox = wx.BoxSizer(wx.HORIZONTAL)
        self.run_btn = wx.Button(panel, label="Transcribe")
        self.cancel_btn = wx.Button(panel, label="Cancel")
        self.strip_btn = wx.Button(panel, label="Strip")
        self.makesrt_btn = wx.Button(panel, label="Make srt")
        self.save_btn = wx.Button(panel, label="Save Output")

        self.run_btn.SetToolTip("Transcribe selected file")
        self.cancel_btn.SetToolTip("Cancel the current operation")
        self.strip_btn.SetToolTip("Strip header, footer and time codes")
        self.makesrt_btn.SetToolTip("Convert to subtitle (srt) format")
        self.save_btn.SetToolTip("Save displayed output to file")

        # Disable buttons until their actions are valid
        self.cancel_btn.Disable()
        self.strip_btn.Disable()
        self.makesrt_btn.Disable()
        self.save_btn.Disable()

        hbox.Add(self.run_btn, 0, wx.ALL, 5)
        hbox.Add(self.cancel_btn, 0, wx.ALL, 5)
        hbox.Add(self.strip_btn, 0, wx.ALL, 5)
        hbox.Add(self.makesrt_btn, 0, wx.ALL, 5)
        hbox.Add(self.save_btn, 0, wx.ALL, 5)

        vbox.Add(hbox)

        # Progress bar
        self.progress = wx.Gauge(panel, range=100)
        vbox.Add(self.progress, 0, wx.EXPAND | wx.ALL, 5)

        # Output
        self.output = wx.TextCtrl(panel, style=wx.TE_MULTILINE)
        font = self.output.GetFont()
        font.SetPointSize(14)
        self.output.SetFont(font)
        vbox.Add(self.output, 1, wx.EXPAND | wx.ALL, 5)

        panel.SetSizer(vbox)

        # Bindings
        self.run_btn.Bind(wx.EVT_BUTTON, self.on_run)
        self.cancel_btn.Bind(wx.EVT_BUTTON, self.on_cancel)
        self.strip_btn.Bind(wx.EVT_BUTTON, self.on_strip)
        self.makesrt_btn.Bind(wx.EVT_BUTTON, self.on_makesrt)
        self.save_btn.Bind(wx.EVT_BUTTON, self.on_save)

    def load_models(self):
        """
        Builds a list of all available (previously downloaded) models
        """

        if not os.path.exists(MODEL_DIR):
            return

        models = [f for f in os.listdir(MODEL_DIR) if f.endswith(".bin")]
        self.model_choice.Set(models)
        if models:
            self.model_choice.SetSelection(0)


    def convert_mp4_to_mp3(self, mp4_path):
        """
        Convert mp4 file in mp4_path to mp3 with same base name
        """

        # Get video duration for progress calculation
        duration_cmd = [
            "ffprobe",
            "-v", "error",
            "-show_entries", "format=duration",
            "-of", "default=noprint_wrappers=1:nokey=1",
            mp4_path
        ]

        try:
            duration_output = subprocess.check_output(duration_cmd, text=True)
            total_duration  = float(duration_output.strip())
        except Exception:
            total_duration = None

        cmd = [
            "ffmpeg",
            "-i", mp4_path,
            "-vn",
            "-acodec", "mp3",
            "-ab", "192k",
            "-y",
            self.tempfile
        ]

        # Start the conversion
        self.ffmpeg_process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
            bufsize=1
        )

        time_re = re.compile(r"time=(\d+):(\d+):(\d+\.\d+)")

        def reader():
            """
            Reads the lie output from ffmpeg in a separate thread
            to avoid blocking the main thread.
            """

            last_percent = -1

            for line in self.ffmpeg_process.stdout:
                match = time_re.search(line)
                if match:
                    h, m, s = match.groups()
                    current_time = int(h)*3600 + int(m)*60 + float(s)

                    if total_duration:
                        percent = int((current_time / total_duration) * 100)
                        percent = max(0, min(100, percent))

                        if percent != last_percent:
                            last_percent = percent
                            wx.CallAfter(self.progress.SetValue, percent)
                    else:
                        # fallback if duration unknown
                        wx.CallAfter(self.output.AppendText, f"{line}")

            self.ffmpeg_process.wait()

            if self.ffmpeg_process.returncode != 0:
                wx.CallAfter(self.output.AppendText, "ffmpeg failed\n")
            else:
                wx.CallAfter(self.output.AppendText, "Conversion complete\n")

        # Non-blocking thread
        thread = threading.Thread(target=reader, daemon=True)
        thread.start()

        # Wait for completion without freezing GUI
        while thread.is_alive():
            wx.Yield()

        return self.tempfile

    def on_run(self, event):
        """
        Perform audio to text transcription with conversion from
        mp4 to mp3 if required.
        """

        self.strip_btn.Disable()
        self.save_btn.Disable()
        self.makesrt_btn.Disable()
        self.run_btn.Disable()
        self.cancel_btn.Enable()
        self.made_srtfile = False

        audio = self.file_picker.GetPath()
        self.base,self.extn = os.path.splitext(audio)

        # Convert to mp3 if user selected an mp4 file
        if self.extn.lower() == '.mp4':
            self.tempfile = self.base + '.mp3'
            audio = self.convert_mp4_to_mp3(audio)
        else:
            self.tempfile = None

        if not audio:
            wx.MessageBox("Select an audio file first.")
            return

        # Get the selected model
        model_name = self.model_choice.GetStringSelection()
        if not model_name:
            wx.MessageBox("Select a model.")
            return

        model_path = os.path.join(MODEL_DIR, model_name)

        self.output.Clear()
        self.progress.SetValue(0)

        cmd = [
            WHISPER_EXE,
            "-m", model_path,
            audio
        ]

        self.output.AppendText(' '.join(cmd) + '\n\n')

        # Run the transcription with hidden window
        startupinfo = subprocess.STARTUPINFO()
        startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW

        self.process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
            bufsize=1,
            startupinfo=startupinfo,
            creationflags=subprocess.CREATE_NO_WINDOW
        )

        self.is_running = True
        self.run_btn.Disable()
        self.cancel_btn.Enable()

        threading.Thread(target=self.read_output, daemon=True).start()

    def read_output(self):
        """
        Runs on separate thread to capture and display whisper.exe
        output in real time.
        """

        max_time = 0

        for line in self.process.stdout:
            wx.CallAfter(self.output.AppendText, line)

            match = time_pattern.search(line)
            if match:
                h, m, s = match.groups()
                current_time = int(h)*3600 + int(m)*60 + float(s)

                if current_time > max_time:
                    max_time = current_time

                # crude scaling (just for visual feedback)
                progress = min(int((current_time / (max_time + 1)) * 100), 100)
                wx.CallAfter(self.progress.SetValue, progress)

        self.process.wait()

        wx.CallAfter(self.on_done)

    def on_done(self):
        """
        Transcription completed. Set button states and
        remove mp3f if one was generated.
        """

        self.is_running = False
        self.run_btn.Enable()
        self.strip_btn.Enable()
        self.cancel_btn.Disable()
        self.save_btn.Enable()
        self.makesrt_btn.Enable()
        self.run_btn.Enable()
        self.progress.SetValue(100)

        if self.tempfile:
            os.remove(self.tempfile)
            self.tempfile = None

    def on_cancel(self, event):
        """
        Cancel transcription or conversion in progress
        """

        # Cancel transcription if in progress
        if self.process:
            self.process.terminate()
            self.output.AppendText("\n--- Cancelled ---\n")
            self.run_btn.Enable()

        # Cancel mp4 -> mp3 conversion if in progress
        if hasattr(self, "ffmpeg_process") and self.ffmpeg_process:
            self.ffmpeg_process.terminate()
            if self.tempfile:
                os.remove(self.tempfile)
                self.tempfile = None

    def on_save(self, event):
        """
        Save the output as either a txt or srt file
        """

        if self.made_srtfile:
            # Reformatted as subtitle blocks.
            wildcard = "Subtitle files (*.srt)|*.srt"
            defaultFile = os.path.split(self.base)[1] + '.srt'
        else:
            # Text only
            wildcard = "Text files (*.txt)|*.txt"
            defaultFile = os.path.split(self.base)[1] + '.txt'

        with wx.FileDialog(self, "Save output", wildcard=wildcard,
                           style=wx.FD_SAVE | wx.FD_OVERWRITE_PROMPT,
                           defaultFile=defaultFile) as dlg:
            if dlg.ShowModal() == wx.ID_CANCEL:
                return

            path = dlg.GetPath()
            with open(path, "w", encoding="utf-8") as f:
                f.write(self.output.GetValue())

    def on_strip(self, event):
        """
        Strip off the process header and footer lines, remove the
        timecode data, then wrap the remaining text into one long
        stream.
        """

        lines = self.output.GetValue().splitlines()
        cleaned_lines = []

        for line in lines:

            if line.startswith('['):
                # Remove anything in square brackets at the start
                line = re.sub(r"^\[.*?\]\s*", "", line)

                # Skip empty lines
                if line.strip():
                    cleaned_lines.append(line)

        # Remove this line if you don't want to word-wrap the text
        self.output.SetValue(" ".join(cleaned_lines))
        self.strip_btn.Disable()

    def on_makesrt(self, event):
        """
        Reformat the output into subtitle (srt) format. Duplicate
        lines are ignored.
        """

        # Set this flag so that when we do a save we
        # get *.srt as a save option.
        self.made_srtfile = True

        lines = self.output.GetValue().splitlines()
        srt   = []
        index = 0
        skip  = '♪'
        prev  = ''

        for line in lines:

            if line.startswith('['):
                time = line[1:32].replace('.',',')
                time = time.replace(']','')
                time = time.strip()
                if (text := line[32:].strip()) != skip:
                    if text != prev:
                        index += 1
                        srt.append(f'{index}\n{time}\n{text}\n')
                        prev = text

        self.output.SetValue("\n".join(srt))
        self.makesrt_btn.Disable()


if __name__ == "__main__":
    app = wx.App()
    frame = WhisperGUI()
    frame.Show()
    app.MainLoop()

[gdtraveller]

I copied Jim's code to my ubuntu box and converted everything to work in linux.
As advertised, it seems like my wav file was almost 100% transcribed correctly.
Thanks a lot, Jim.

[gdtraveller]

I reviewed the transcription file.
The text of the transcription file was 92 words.
The llm missed 1 word and wasn't able to figure out 3 other words.
However, punctuation was perfect!
The length of the audio in the audio file was 43 seconds.
Total processing time was 4.7 seconds.
My computer does not have a GPU.
I'm impressed.
Thanks

[Reverend Jim]

Glad you found it useful. Still some cleanup to do but it shouldn't change the functionality.