Whisper+剪映制作精听材料

Vim · 2024 年10 月 14 日 11:26

给 SubtitleEdit 提了一个issue，如果能够实现，那一切就完美理顺了。

当然使用其他工具，如python, lua 之类应该也能很容易实现，希望有编程经验的高人能够填平这个最后一公里的痛点。之所以在 SubtitleEdit 提交issue，仅仅是因为SubtitleEdit已经有很多同步字幕时间戳的功能，但独缺这一份，集成到一块就更方便了。但若有python或lua的实现，那么自定义就更加方便了。

github.com/SubtitleEdit/subtitleedit

Feature Request: Synchronize SRT with Whisper's Word-Level Timestamps JSON

opened 11:11AM - 14 Oct 24 UTC

VimWei

Audio to text (Whisper) is an important feature for generating subtitles using A…I. This is an example python script to use whisper: [whispertranscribe3.zip](https://github.com/user-attachments/files/17362981/whispertranscribe3.zip) However, it has a frustrating drawback regardless of how we adjust the parameters: 1. **Default Setting**: With the setting in the python script, `use_default_line_breaks=True` , Whisper can segment text by sentences or clauses (Advantage: complete content that's easy to understand). However, for long sentences, the output often results in too many characters per line, frequently requiring multiple lines to display the content (Disadvantage: poor visual experience). 2. **Parameter Control**: With the setting in the python script, `use_default_line_breaks=False` , by adjusting parameters like `max_line_width`, `max_line_count`, and `max_words_per_line`, we can control the number of characters or words per subtitle line (Advantage: improved visual experience). However, this segmentation can be quite abrupt, often not adhering to grammatical structures, leading to fragmented content in each subtitle line (Disadvantage: harder to understand). "I noticed that Whisper can generate word-level timestamps when the parameter is set to `word_timestamps=True`, and this information can be saved in a JSON file. For example: Input - mp4: https://github.com/user-attachments/assets/aab478eb-72d9-4d1c-939d-35cb727b6dca Output - json: [beekeeping is most difficult.json](https://github.com/user-attachments/files/17363028/beekeeping.is.most.difficult.json) Output - srt: ```srt 1 00:00:00,000 --> 00:00:05,160 Experts assert that the most difficult thing in agriculture is animal husbandry, 2 00:00:05,320 --> 00:00:08,740 and the most difficult thing in animal husbandry is beekeeping. ``` **To enhance the viewing experience, I may want to adjust the line breaks in the subtitles.** ```srt New 1 00:00:00,000 --> 00:00:01,900 Experts assert that 2 00:00:01,900 --> 00:00:05,160 the most difficult thing in agriculture is animal husbandry, 3 00:00:05,320 --> 00:00:08,740 and the most difficult thing in animal husbandry is beekeeping. ``` In this simple case, I know I can easily achieve this goal using SubtitleEdit. However, when dealing with a large number of subtitles, it becomes quite time-consuming and labor-intensive. While the JSON file output by Whisper contains timestamps for each word, **is it technically possible to automate the synchronization between the new SRT and the JSON file to obtain the precise timestamps?**

PS：这个issue里边，我放了一个自己编写的 WhisperTranscribe.py，大家拿来即可使用，相对原汁原味的Whisper，增加了两个特性：

实现对srt输出的参数控制，从而可以实现逐字srt
- max_line_width，max_line_count，max_words_per_line
实现对srt断行控制的自由切换：人工还是自动