Implement a Movie Maker mode

[reduz]

Resolves godotengine/godot-proposals#3083.

What is it?

• Allows running the game in "movie writer" mode.
• It ensures entirely stable framerate, so your run can be saved stable and with proper sound (which is impossible if your CPU/GPU can't sustain doing this in real-time).
• If disabling vsync, it can save movies faster than the game is run, but if you want to control the interaction it can get difficult.
• Implements a simple, default MJPEG writer (avi) and PNG+WAV (for lossless and transparency). More custom encoders (such as ffmpeg) can be implemented as GDExtensions by extending and registering a new MovieWriter.
• Implemented the ability to compress Image as jpg (needed for this).

This new features has two main use cases, which have high demand:

• Saving game videos in high quality and ensuring the frame rate is completely stable, always.
• Using Godot as a tool to make movies and animations (which is ideal if you want interaction, or creating them procedurally. No other software is as useful for this).

Note: This feature IS NOT for capturing real-time footage. Use something like OBS, SimpleScreenRecorder or FRAPS to achieve that, as they do a much better job at intercepting the compositor than Godot can probably do using Vulkan or OpenGL natively. If your game runs near real-time when capturing, you can still use this feature (but it will lack audible sound playback, as sound is saved directly to the video file).

Usage:

User Interace:

Enable the Movie Maker mode by toggling the new icon on the run panel:

Then set up the recording parameters in the project settings, including the output file:


For running standalone scenes, it is possible to specify custom move files for each by adding a "movie_file" metadata to the root node:

Command line:

$ godot --write-movie movie.avi [scene_file.tscn]

Missing:

• Options for configuring video writing via GLOBAL_DEF
• UI Menu for launching with this mode from the editor.
• Add to list of command line options.
• Add a feature tag to override configurations when movie writing (fantastic for saving videos with highest quality settings).

Known Limitations:

• Current MJPEG writer is limited to 4gb video files due to AVI format. AVI has extensions to overcome this limitation (basically saving multiple RIFF chunks) but they are not implemented yet. It can be done later if someone feels inspired.
• Potentially optimize the JPEG compression to do every frame on a separate thread, in order to speed up the video writing.

[YuriSizov]

Closes godotengine/godot-proposals#3083?

[reduz]

@YuriSizov yeah, thanks! Let me amend this.

[Calinou]

Implemented the ability to compress Image as jpg (needed for this).

See also #60660, which implements the same feature (but probably with some differences).

[Calinou]

I'm not sure if implicitly falling back to 60 FPS is a good idea. I would at least print a message in this case to warn the user that no fixed FPS was specified on the command line, and that the default value of 60 will be used instead.

Something like:

No `--fixed-fps <fps>` argument specified on the command line. Will be recording video at 60 FPS (non-realtime).

See also #60284, which adds a project setting to set fixed FPS (and also allows toggling it at run-time). It would be interesting to make the movie writer toggleable at run-time, but it sounds difficult to do.

[reduz]

This is only really for running from command line because it needs something. The actual writer has a project setting option and it always passed the proper fixed fps value.

[Calinou]

Should the number of buffer frames be adjustable with a project setting? This seems like a high value to default to – most other games I've seen default to 1024.

That said, for offline recording, we can probably perform latency compensation so it sounds like audio has zero latency, even if a high number of buffer frames is used.

[reduz]

This is unrelated to latency, it just needs it to to save audio temporarily.

[Calinou]

Add a feature tag to override configurations when movie writing (fantastic for saving videos with highest quality settings).

I'd probably name the feature tag movie for that purpose, then add some default overrides for graphics settings. I think the graphics settings overrides themselves can be done in a future PR still.

[lyuma]

Would it be possible to add support for image sequences (or more specifically, a folder usually full of .png files) to avoid compression loss due to jpeg encoding.

My understanding is image sequences such as png are fairly common, and Godot already supports png compression so it seems like an easy MovieWriter implementation to add (possibly as a future commit)

[Calinou]

AVI does not support lossless compression, although it does support saving uncompressed data (resulting in huge files). I'd look at the quality loss when using JPEG quality 100 first, as it may be barely noticeable in most scenarios (especially at higher input resolutions).

A PNG image sequence will therefore be smaller than a lossless AVI file, but it'll also take much longer to save as PNG encoding is notoriously slow compared to JPEG. The tradeoff is sometimes worth it, but it'll impact your rendering speed a lot unless PNG saving is moved to its own thread (or better, its own multithread pool).

For this to work, Godot would have to save a PNG image sequence and a WAV file containing the audio besides it. The naming of PNG files should be friendly to tools like FFmpeg to allow converting it to a video easily, but it should be able to handle dozens of thousands of frames if needed.

[TokageItLab]

Does this support exporting alpha channel? Or does it have the ability to output the alpha channel as a separate file? (Or can it be easily added later?)

In the use case of software like FaceRig or Live2D, when exporting video, alpha support is a very powerful tool for compositing purposes.

[Calinou]

See also godotengine/godot-proposals#1923. I think the way this PR currently handles V-Sync disabling is fine – it's just something to keep in mind for the future.

Either way, I'll definitely appreciate having this CLI argument available for games that don't expose a way to turn off V-Sync, so kudos for adding it 🙂
It's also useful to disable V-Sync on the editor without affecting the running project (at least until #48364 is merged).

[Calinou]

Does this support exporting alpha channel? Or does it have the ability to output the alpha channel as a separate file? (Or can it be easily added later?)

JPEG doesn't support saving an alpha channel, and neither does uncompressed AVI to my knowledge. A PNG image sequence would be the only way to achieve this without having to generate a separate AVI that contains just the alpha channel. (If you need this, you can probably do it by running Godot twice with different options that make it display the alpha channel only using a shader.)

[TokageItLab]

When using shaders to split the export in two, there is a possibility that transparent objects will be affected by the color of the background. For similar reasons, pre-multiplied alpha and chroma key is generally not preferred in the video compositing process, and straight alpha is preferred.

Also, perfect reproducibility is required for physics, user input, random seed and other operations. For these reasons, I believe it makes more sense to render those at the same time.

I think it is fine to have alpha channel support at the same time as the png sequence implementation, but it would be preferable if this PR is an implementation that can easily integrate that.

For example, it should not ignore the fact that the background is transparent prior to file encode process; just as the background transparent_bg option exists in the Viewport/ViewportContainer, so should it be in the video output options.

[reduz]

@lyuma @TokageItLab PNGs are enormous I tested and saved videos are like 10 times bigger. Additionally they are very slow to encode and annoying to handle if you re-run movie export because you need to delete the old ones.

As transparency or lossless video is quite a very corner case, I think it will be better if you make a small extension reimplementing MovieWriter yourself to save in the way you prefer. It really is super easy to do it.

[Structed]

As transparency or lossless video is quite a very corner case, I think it will be better if you make a small extension reimplementing MovieWriter yourself to save in the way you prefer. It really is super easy to do it.

I tend to disagree - at least for lossless. If you want to alter anything after the recording, you will want to have a lossless video to not lose even more detail.
I personally would always prefer to record in lossless, and then later compress to my own style.

Creating my own implementation of Movie Writer is an option, but again, not sure the compressed is the main use-case.

[akien-mga]

Thanks!

[golddotasksquestions]

I was asking this on reddit, but it's probably better to ask here:

Note: This feature IS NOT for capturing real-time footage. Use something like OBS, SimpleScreenRecorder or FRAPS to achieve that, as they do a much better job at intercepting the compositor than Godot can probably do using Vulkan or OpenGL natively. If your game runs near real-time when capturing, you can still use this feature (but it will lack audible sound playback, as sound is saved directly to the video file).

Hm, I'm still not sure I understand.

What I would expect from this play mode is guaranteed rendering at a certain frame rate. So if I set it to 60 fps, I would expect the resulting avi file would contain exactly 60 frames every second, rendered gameplay at regular 0.016 intervals. But I would not expect this mode to have 60 fps while playing/recording: If one frame for example would take 0.5 seconds to render while the other frames only take 0.016 seconds, I would expect to see this one frame for 0.5 seconds on screen while playing/recording in this mode, but during playback of the video file, I would not see any hickup, just smooth 60fps and no pauses in gameplay.

This means, provided the playing/recording framerate is stable because my hardware can handle it, it should be totally fine record gameplay with it. Even recommendable and advantageous over FRAPS or OBS who will also records any frame rate dips.

Even if my hardware can't cope and I therefore get freezing frames while playing/recording in this mode. If these dips are within the millisecond spectrum, and Inputs don't change radically, I don't see any negative consequence recording gameplay. The recorded video should almost always look better (while of course playing with hickup ain't as much fun). But when you are recording video footage of your project, you having fun with gameplay while doing so is not the top most priority anyway. You are creating assets to reuse in your project or for marketing.

Am I wrong here? Did I miss something obvious?

Recording interactive project footage would be my main use of this thing. If this is not what this does, why would you want to have this? Blender is a thousand times more suitable for linear movie 3D rendering and production, any Animation Software for 2D animation and rendering.

[clayjohn]

@golddotasksquestions Your description is accurate. What reduz is saying above is that this isn't useful for streaming your game online or equivalent.

[KvaGram]

Is it possible to use this feature from a compiled game?
Say you have a racing game. The game records all the data over the course of the game to the point where the entire race can be recreated. This surely already exist. But what if the player can choose to export the replay of their race as a videofile, without the need for a screenrecorder?

[Calinou]

Is it possible to use this feature from a compiled game?

Yes, the code isn't compiled out of non-editor builds and the command line argument is still available in exported projects (even in release mode).

However, this functionality cannot be toggled while the project is running. Therefore, you'll need to call OS.execute() to run the exported project a second time with the appropriate CLI arguments (such as --replay some_replay.rpl --write-movie out.avi). This assumes you've also implemented a CLI argument in your project to directly open a replay (and show nothing before, such as a splash screen). The recording project's window should be able to be minimized as long as its size is set beforehand, but I haven't tested.

Godot supports running multiple instances of the same project at once, so this should work out fine.

[Teashrock]

It's good you have impleneted .AVI, but will there also be .OGV and .MP4?

[TokisanGames]

Given the discussion above, that's not going up happen, but you can convert to any container and codec you like using ffmpeg. Or if you're adventurous and need the ability to export mp4 in your app, you can write a plugin with the codec, or a plugin that wraps ffmpeg. Ffmpeg is open source.

[ellenhp]

There's more context in the proposal on why this was implemented the way it was. Please read the context and existing Q&A in the comments here before asking more questions :) There's a lot of licensing requirements for using h264 (mp4), and I don't think we currently bundle an ogv/theora encoder meaning it would bloat the binary size which isn't good.

[IsotoxalDev]

For those who want to record fixed animations from godot 3 you can use https://github.com/simonknowsstuff/Godot-Film-Maker !!

[drcucaracha]

Hello, any way to add the alpha channel (png) only for the image sequences? since in addition to being useful for videos it would also be very useful for the creation of 2d isometric renderings like diablo 2, I know that in blender you can create a rotation script and make a screenshot every x time but having it in godot would save time to anyone who wants to use it all without leaving the engine

[Calinou]

Hello, any way to add the alpha channel (png) only for the image sequences? since in addition to being useful for videos it would also be very useful for the creation of 2d isometric renderings like diablo 2, I know that in blender you can create a rotation script and make a screenshot every x time but having it in godot would save time to anyone who wants to use it all without leaving the engine

As mentioned above, using a transparent viewport won't work as the final window is still opaque. Movie Maker only "sees" the final window, not intermediate viewports.

You have two remaining options:

• Use color keying (usually a viewport with a pure magenta or cyan background), and use this in a post-processing step to discard pixels with a given color. This is a good fit for Diablo-style renders as they typically only rely on 1-bit transparency, where chroma keying is most suited.
• Use a transparent viewport and use a custom post-processing shader on top to save the viewport's alpha channel to a grayscale image.

[YuriSizov]

Hello, any way to add the alpha channel (png) only for the image sequences? since in addition to being useful for videos it would also be very useful for the creation of 2d isometric renderings like diablo 2, I know that in blender you can create a rotation script and make a screenshot every x time but having it in godot would save time to anyone who wants to use it all without leaving the engine

You can do that in Godot already, in a similar way you can do it in Blender. Just take screenshots using viewport texture and rotate your model.

[drcucaracha]

@YuriSizov Hello, I can do that easily, what I wanted was to have a transparent background so that only the rendered 3d model would remain. I have been trying subwiport but the gray background is still saved -_-

reference videos so that you understand well what I wanted to explain

https://www.youtube.com/watch?v=St6Ji7e7OcA

[YuriSizov]

@drcucaracha Yes, I get what you need. Your viewport needs to be set to transparent background, and that information is correctly stored in its texture, but saving it to PNG for some reason renders transparent areas black.

I guess you have to follow Calinou's advice then and either generate a texture just for alpha value, or use a keying out technique.

[ghost]

In the future, you could have a built in ui node that allows the player to record gameplay and even edit it! I understand that compared to other game engines, Godot has limited devs, so I don't expect these features to be implemented now, but sometime in the future, maybe for Godot 5, yow'l could try it!

[cyberpsyche]

Could you please add a back port this important feature to 3.X version? As this is a relative independent feature and many 3.X apps will want to use this feature to output game video, but upgrading all the 3.X will take a long time, for many apps they will always been in 3.X stage.

Thanks a lot!

[Calinou]

Could you please add a back port this important feature to 3.X version?

While this feature is technically possible to backport, this won't be tackled by core contributors as they're focusing on getting 4.0 in a releasable state. The visual capabilities of Godot 3.x are also more limited, making a Movie Maker mode less interesting to have there. That said, anyone is free to try 🙂

and many 3.X apps will want to use this feature to output game video

Remember that Movie Maker mode is not designed to be used by players to record video. It's a tool for project developers first and foremost, e.g. for creating game trailers.