Add Ogg Theora support to MovieWriter

[berarma]

Supersedes #106700.

Movie Maker mode can now record files in .ogv format, which can be directly viewed in Godot's VideoStreamPlayer node along with most video players. This is a lossy format with inter-frame compression, unlike AVI + MJPEG which only performs intra-frame compression.

Thanks to @penninghlhd for the initial implementation from #98416 and @Calinou for the cleanup in #106700. 🙂

What I've done:

• Fix issues in the generated video related to the multiplexing of video and audio in an Ogg container. Reported by oggz-validate and debugged with oggz-dump.
• Fix compilation when using builtin_libtheora=no builtin_libvorbis=no builtin_libogg=no (undefined references).
• Move MovieWriterOGV to the modules/theora module it depends on, like Move MovieWriterMJPEG class to jpg module it depends on #106013.
• Add more project settings to configure the video encoding process.
• Fix video output by converting the image to RGB8 when it's in a different format. The issue can be reproduced with this demo.

I've added many settings, but it can help testing them and decide which ones we want. In the case of video_bitrate, it enables CBR and it might provide better encoding speed in some cases. Would it be really bad to provide all these options?

TODO

• Test on Windows + MSVC and make sure the Theora files are correct once they're fixed on other platforms. MSVC uses a different set of files for x86 intrinsics in Theora. (Help needed on this one.)

The message [ffmpeg/demuxer] ogg: Broken file, keyframe not correctly marked. in FFmpeg is still showing, but it's not really an error and it shouldn't affect operations. I think it's just FFmpeg being finicky.

[berarma]

I've found out the CPU/GPU times reported at the end of the recording are only for the RenderingServer without including encoding times. So I've added encoding time information to the report by measuring the time it takes to run write_frame, and also write_end since it writes the last frame in MovieWriterOGV.

In my tests with a production build, with the default settings (speed 4), OGV is as fast as MJPEG. After repeating the tests several times, OGV gets slightly faster while MJPEG gets slightly slower. File write times are measured together with the encoding time so they might cause some variance. In a debug build, the OGV encoding is a bit slower, and no change in MJPEG encoding.

Choosing speed 3 is slower but gives a good improvement in compression. It offers the best speed-compression ratio.

Using CBR with a target bitrate equal to the mean bitrate of the VBR file for equivalent compression takes almost the same time as VBR. It's even slower at speed 1. This setting might be unnecessary.

[Calinou]

Tested locally, it works as expected. Code and documentation looks good to me. I like the addition of the encoding time metric 🙂

Testing project: https://github.com/Calinou/godot-movie-maker-demo

Example output file with the default settings:

Input #0, ogg, from 'out.ogv':
  Duration: 00:00:30.00, start: 0.000000, bitrate: 27760 kb/s
  Stream #0:0: Video: theora, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 60 tbn
  Stream #0:1: Audio: vorbis, 48000 Hz, stereo, fltp, 96 kb/s

The file plays back correctly in mpv.

Encoding speeds (measured on a debug build):

OGV

Done recording movie at path: /tmp/out.ogv
1800 frames at 60 FPS (movie length: 00:00:30:00), recorded in 00:01:41 (29% of real-time speed).
CPU time: 1.40 seconds (average: 0.78 ms/frame)
GPU time: 10.53 seconds (average: 5.85 ms/frame)
Encoding time: 30.79 seconds (average: 17.10 ms/frame)

AVI

Done recording movie at path: /tmp/out.avi
1800 frames at 60 FPS (movie length: 00:00:30:00), recorded in 00:01:35 (31% of real-time speed).
CPU time: 1.39 seconds (average: 0.77 ms/frame)
GPU time: 9.57 seconds (average: 5.32 ms/frame)
Encoding time: 28.88 seconds (average: 16.04 ms/frame)

PNG/WAV

Done recording movie at path: /tmp/out.png
1800 frames at 60 FPS (movie length: 00:00:30:00), recorded in 00:04:01 (12% of real-time speed).
CPU time: 1.56 seconds (average: 0.87 ms/frame)
GPU time: 33.50 seconds (average: 18.61 ms/frame)
Encoding time: 148.37 seconds (average: 82.43 ms/frame)

You can add an OGV capitalization here so the Project Settings category uses correct casing:

godot/editor/editor_property_name_processor.cpp

Line 247 in 26df043

capitalize_string_remaps["nfc"] = "NFC";

[Calinou]

Using CBR with a target bitrate equal to the mean bitrate of the VBR file for equivalent compression takes almost the same time as VBR. It's even slower at speed 1. This setting might be unnecessary.

I agree, I suggest removing the CBR setting if we can't find a good use case for it.

[berarma]

Thanks for all the suggestions, and for saving me from my struggles when writing documentation.

I've removed the video_bitrate setting, renumbered the speed labels to use the real value instead of being off by one, and capitalized the OGV label (I was hoping for something like this to exist :) ).

And rebased it on top of the current master, now with Theora 1.2.0.

I forgot to mention it before but I'm using oggz-validate to check the generated OGV files. At first, it reported errors for Ogg pages with decreasing timestamps and the lack of end of stream markers. Now the generated files are clean.

[berarma]

I've put everything in one commit.

[berarma]

I've tried to address the issues, although I have doubts about the inclusion of CC-BY-SA 3.0 code.

I've made a few minor changes to the rgb2yuv that could improve performance slightly and make the code simpler. Should I add a comment to the file about this?

I've added a check for the video dimensions because the encoding won't work if they aren't even. Since MovieWriter doesn't check the return value from write_begin, when there's an error, it will still call write_frame as if nothing happened and more error messages will be displayed.

[akien-mga]

Looks pretty good to me!

I notice that, unlike what I might have suggested before, --movie-writer seems to be exposed in non-editor builds (export templates). I don't know if it actually works that way, but that means that with the changes done to only include the theora encoder in editor builds, OGV won't be available as a format for runtime movie encoding.

I think it's an appropriate tradeoff for now, and we can reconsider it if there's significant demand and the binary size impact is not too big.

[berarma]

I notice that, unlike what I might have suggested before, --movie-writer seems to be exposed in non-editor builds (export templates). I don't know if it actually works that way, but that means that with the changes done to only include the theora encoder in editor builds, OGV won't be available as a format for runtime movie encoding.

I didn't think to check it out. It made sense that it was editor-only.

I think it's an appropriate tradeoff for now, and we can reconsider it if there's significant demand and the binary size impact is not too big.

Don't hesitate to ask me if you change your mind now or later, it's easy to revert.

[akien-mga]

Looks great to me!

[akien-mga]

Thanks! Great work 🎉