subtitle

AWS 推出了 Live 時全自動上字幕的功能

AWS 推出了在直播時就自動上字幕的功能：「Introducing Live Streaming with Automated Multi-Language Subtitling」，其實就是把現有的服務兜出來：「Live Streaming with Automated Multi-Language Subtitling」。

The solution deploys Live Streaming on AWS which includes AWS Elemental MediaLive, MediaPackage, Amazon CloudFront. The solution also deploys AWS Lambda, Amazon Simple Storage Service, Amazon Transcribe, and Amazon Translate.

對於比較沒那麼要求翻譯品質的情況也許可以玩看看...？

自動校正字幕時間的軟體 subsync

看到用 Python 寫的「smacke/subsync」這個軟體，可以自動校正字幕時間：

Language-agnostic automatic synchronization of subtitles to video, so that subtitles are aligned to the correct starting point within the video.

他把上面這個變成下面這個：

因為不是用 machine learning 所以速度意外的快。演算法是對影片本身產生一個 array，然後對字幕也產生 array，最後對兩個 array 用個簡單的公式校準：

Discretize video and subtitles by time into 10ms windows.
For each 10ms window, determine whether that window contains speech. This is trivial to do for subtitles (we just determine whether any subtitle is "on" during each time window); for video, use an off-the-shelf voice activity detector (VAD) like the one built into webrtc.
Now we have two binary strings: one for the subtitles, and one for the video. Try to align these strings by matching 0's with 0's and 1's with 1's. We score these alignments as (# matching digits) - (# mismatched digits).

就這樣解決問題 XD