diff options
Diffstat (limited to 'TODO')
| -rw-r--r-- | TODO | 96 |
1 files changed, 80 insertions, 16 deletions
@@ -2,7 +2,6 @@ Check for reparent. Quickly changing workspace and back while recording under i3 breaks the screen recorder. i3 probably unmaps windows in other workspaces. See https://trac.ffmpeg.org/wiki/EncodingForStreamingSites for optimizing streaming. Look at VK_EXT_external_memory_dma_buf. -Allow setting a different output resolution than the input resolution. Use mov+faststart. Allow recording all monitors/selected monitor without nvfbc by recording the compositor proxy window and only recording the part that matches the monitor(s). Allow recording a region by recording the compositor proxy window / nvfbc window and copying part of it. @@ -10,7 +9,6 @@ Support amf and qsv. Disable flipping on nvidia? this might fix some stuttering issues on some setups. See NvCtrlGetAttribute/NvCtrlSetAttributeAndGetStatus NV_CTRL_SYNC_TO_VBLANK https://github.com/NVIDIA/nvidia-settings/blob/d5f022976368cbceb2f20b838ddb0bf992f0cfb9/src/gtk%2B-2.x/ctkopengl.c. Replays seem to have some issues with audio/video. Why? Cleanup unused gl/egl functions, macro, etc. -Add option to disable overlapping of replays (the old behavior kinda. Remove the whole replay buffer data after saving when doing this). Set audio track name to audio device name (if not merge of multiple audio devices). Add support for webcam, but only really for amd/intel because amd/intel can get drm fd access to webcam, nvidia cant. This allows us to create an opengl texture directly from the webcam fd for optimal performance. Reverse engineer nvapi so we can disable "force p2 state" on linux too (nvapi profile api with the settings id 0x50166c5e). @@ -69,16 +67,12 @@ Exit if X11/Wayland killed (if drm plane dead or something?) Use SRC_W and SRC_H for screen plane instead of crtc_w and crtc_h. -Make it possible to select which /dev/dri/card* to use, but that requires opengl to also use the same card. Not sure if that is possible for amd, intel and nvidia without using vulkan instead. - Test if p2 state can be worked around by using pure nvenc api and overwriting cuInit/cuCtxCreate* to not do anything. Cuda might be loaded when using nvenc but it might not be used, with certain record options? (such as h264 p5). nvenc uses cuda when using b frames and rgb->yuv conversion, so convert the image ourselves instead.- -Mesa doesn't support global headers (AV_CODEC_FLAG_GLOBAL_HEADER) with h264... which also breaks mkv since mkv requires global header. Right now gpu screen recorder will forcefully set video codec to hevc when h264 is requested for mkv files. - Drop frames if live streaming cant keep up with target fps, or dynamically change resolution/quality. -Support low power option (does it even work with vaapi in ffmpeg??). Would be very useful for steam deck. +Support low power option. Instead of sending a big list of drm data back to kms client, send the monitor we want to record to kms server and the server should respond with only the matching monitor, and cursor. @@ -103,12 +97,10 @@ Investigate if there is a way to do gpu->gpu copy directly without touching syst Go back to using pure vaapi without opengl for video encoding? rotation (transpose) can be done if its done after (rgb to yuv) color conversion. -Implement scaling and use lanczos resampling for better quality. Lanczos resampling can also be used for YUV chroma for better color quality on small text. +Use lanczos resampling for better scaling quality. Lanczos resampling can also be used for YUV chroma for better color quality on small text. Flac is disabled because the frame sizes are too large which causes big audio/video desync. -Add 10-bit capture option. This is good because it reduces banding and quality in very dark areas while reducing the file size compared to doing the same thing with 8-bits. - Enable b-frames. Support vfr matching games exact fps all the time. On x11 use damage tracking, on wayland? maybe there is drm plane damage tracking. But that may not be accurate as the compositor may update it every monitor hz anyways. On wayland maybe only support it for desktop portal + pipewire capture. @@ -152,17 +144,89 @@ Allow prime-run on x11 if monitor capture and the prime gpu is not nvidia. Enable 2-pass encoding. -Add cbr option. - Restart replay/update video resolution if monitor resolution changes. Fix pure vaapi copy on intel. -ffmpeg supports vulkan encoding now (h264!). Doesn't work on amd yet because mesa is missing VK_KHR_video_maintenance1, see https://gitlab.freedesktop.org/mesa/mesa/-/issues/11857. Test on nvidia! - -Test vaapi low latency mode (setenv("AMD_DEBUG", "lowlatencyenc", true);), added in mesa 24.1.4, released on july 17, 2024. Note that this forces gpu power usage to max at all times, even when recording at 2 fps. Use nvidia low latency options for better encoding times. Test ideal async_depth value. Increasing async_depth also increased gpu memory usage a lot (from 100mb to 500mb when moving from async_depth 2 to 16) at 4k resolution. Setting it to 8 increases it by 200mb which might be ok. -Replace -encoder cpu with -k h264_software?
\ No newline at end of file +Replace -encoder cpu with -k h264_software? + +Change vp8/vp9 quality options, right now the file size is too large (for vp9 at least at very_high quality). + +Support recording while in replay mode. This will be needed when enabling replay on system startup with systemd service and wanting to record a video besides that. + The harder and more bloat solution for this would be to make an IPC. + The simple solution would be to use SIGUSR2 for starting/stopping recording since SIGUSR2 is unused for replays. That would mean SIGUSR2 for pausing recording would be ignored. + It also means that the video will be created in the same directory as the replay (or have option to specify another location for that) but the filename would have to be generated automatically. + To rename the file you would have to use -sc to rename it with a script, or add an option to provide a template for the name. + +Dynamically change bitrate/resolution to match desired fps. This would be helpful when streaming for example, where the encode output speed also depends on upload speed to the streaming service. +Implement opengl injection to capture texture. This fixes VRR without having to use NvFBC direct capture and also allows perfect frame timing. +Always use direct capture with NvFBC once the capture issue in mpv fullscreen has been resolved (maybe detect if direct capture fails in nvfbc and switch to non-direct recording. NvFBC says if direct capture fails). + +Support ROI (AV_FRAME_DATA_REGIONS_OF_INTEREST). + +Default to hevc if capture size is larger than 4096 in width or height. + +Set low latency mode on vulkan encoding. + +Support recording/replay/livestreaming at the same time by allowing commands to be run on an existing gpu screen recorder instance. + +Test if `xrandr --output DP-1 --scale 1.5` captures correct size on nvidia. + +Fix cursor position and scale when scaling x11 display. + +Support surround audio in application audio recording. Right now only stereo sound is supported. + +Support application audio recording without pulseaudio combined sink. + +Support transposing (rotating) with vaapi. This isn't supported on many devices with rgb buffer, but its supported with nv12 buffer (on intel at least). + +Cleanup pipewire_audio.c (proper error handling and memory cleanup of proxies). + +Hide application audio module-null-sink by using sink_properties=media.class="Audio/Sink/Internal". + +Improve software encoding performance. + +Add option to record audio from the recorded window only. + +Add option to automatically select best video codec available. Add -k best, -k best_10bit and -k best_hdr. + +Use wayland color management protocol when it's available: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14. + +Use different exit codes for different errors. Use one for invalid -w option, another one for invalid -a option for audio devices, etc. This is to make UI error reporting better. + Document these exit codes in an exit code .md file, or finally create a manpage where this can be documented. + +Ffmpeg fixed black bars in videos on amd when using hevc and when recording at some resolutions, such as 1080p: + https://github.com/FFmpeg/FFmpeg/commit/bcfbf2bac8f9eeeedc407b40596f5c7aaa0d5b47 + https://github.com/FFmpeg/FFmpeg/commit/d0facac679faf45d3356dff2e2cb382580d7a521 + Disable gpu screen recorder black bar handling when using hevc on amd when the libavcodec version is the one that comes after those commits. + Also consider the mesa version, to see if the gpu supports this. + +Use opengl compute shader instead of graphics shader. This might allow for better performance when games are using 100% of graphics unit which might fix issue with 100% gpu usage causing gpu screen recorder to run slow when not using vaapi to convert rgb to nv12(?). + +Always disable prime run/dri prime and list all monitors to record from from all cards. + Do this instead of adding an option to choose which gpu to use. + On X11 the primary gpu will always have the framebuffer for all monitors combined. + Use randr to list all monitors and always record and encode with the primary gpu. + On Wayland each gpu will have its own list of monitors with framebuffers. + Iterate through all cards with drm and list all monitors with associated framebuffers and when choosing a monitor to record + automatically use the associated gpu card. + +Allow flv av1 if recent ffmpeg version and streaming to youtube (and twitch?) and for custom services. +Use explicit sync in pipewire video code: https://docs.pipewire.org/page_dma_buf.html. + +Support vaapi rotation. Support for it is added in mesa here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32919. + +Replay (and recording?) fails to save properly sometimes (especially for long videos). This is noticable with mp4 files since they get corrupt and become unplayable. + The entire video does seem to get saved (it's a large video file) and it seems to have the correct headers but it's not playable. + +Make it possible to save a shorter replay clip remotely. Maybe implement ipc first, to then also allow starting recording/stream while a replay is running. + +Add an option to pass http headers when streaming. Some streaming services require streaming keys to be passed in a http header instead of in the url as a parameter. + +When adding vulkan video support add VK_VIDEO_ENCODE_TUNING_MODE_LOW_LATENCY_KHR. + +Implement screenshot without invoking opengl (which is slow to start on some systems). |