diff options
Diffstat (limited to 'TODO')
| -rw-r--r-- | TODO | 84 |
1 files changed, 55 insertions, 29 deletions
@@ -6,9 +6,6 @@ Allow setting a different output resolution than the input resolution. Use mov+faststart. Allow recording all monitors/selected monitor without nvfbc by recording the compositor proxy window and only recording the part that matches the monitor(s). Allow recording a region by recording the compositor proxy window / nvfbc window and copying part of it. -Use nvenc directly, which allows removing the use of cuda. -Handle xrandr monitor change in nvfbc. -Implement follow focused in drm. Support amf and qsv. Disable flipping on nvidia? this might fix some stuttering issues on some setups. See NvCtrlGetAttribute/NvCtrlSetAttributeAndGetStatus NV_CTRL_SYNC_TO_VBLANK https://github.com/NVIDIA/nvidia-settings/blob/d5f022976368cbceb2f20b838ddb0bf992f0cfb9/src/gtk%2B-2.x/ctkopengl.c. Replays seem to have some issues with audio/video. Why? @@ -20,7 +17,7 @@ Reverse engineer nvapi so we can disable "force p2 state" on linux too (nvapi pr Support yuv444p on amd/intel. fix yuv444 for hevc. Do not allow streaming if yuv444. -Re-enable yuv444. +Re-enable yuv444 and allow yuv444 for software encoding. Good for remote desktop. But for remote desktop its more ideal to use yuv420 and when the image is not moving then send a png image instead, for clear image when the image is static. Support 10 bit output because of better gradients. May even be smaller file size. Better supported on hevc (not supported at all on h264 on my gpu). Add nvidia/(amd/intel) specific install script for ubuntu. User should run install_ubuntu.sh but it should run different install dep script depending on if /proc/driver/nvidia/version exists or not. But what about switchable graphics setup? Test different combinations of switchable graphics. Intel hybrid mode (running intel but possible to run specific applications with prime-run), running pure intel. Detect switchable graphics. @@ -31,7 +28,6 @@ https://djdallmann.github.io/GamingPCSetup/CONTENT/RESEARCH/FINDINGS/registrykey The video output will be black if if the system is suspended on nvidia and NVreg_PreserveVideoMemoryAllocations is not set to 1. This happens because I think that the driver invalidates textures/cuda buffers? To fix this we could try and recreate gsr capture when gsr_capture_capture fails (with timeout to retry again). NVreg_RegistryDwords. -Restore nvfbc screen recording on monitor reconfiguration. Window capture doesn't work properly in _control_ game after going from pause menu to in-game (and back to pause menu). There might be some x11 event we need to catch. Same for vr-video-player. Monitor capture on steam deck is slightly below the game fps, but only when capturing on the steam deck screen. If capturing on another monitor, there is no issue. @@ -44,23 +40,14 @@ Intel is a bit weird with monitor capture and multiple monitors. If one of the m Is that only the case when the primary monitor is rotated? Also the primary monitor becomes position 0, 0 so crtc (x11 randr) position doesn't match the drm pos. Maybe get monitor position and size from drm instead. How about if multiple monitors are rotated? -Support vp8/vp9. This is especially important on amd which on some distros (such as Manjaro) where hardware accelerated h264/hevc is disabled in the mesa package. - Support screen (all monitors) capture on amd/intel and nvidia wayland when no combined plane is found. Right now screen just takes the first output. Use separate plane (which has offset and pitch) from combined plane instead of the combined plane. Both twitch and youtube support variable bitrate but twitch recommends constant bitrate to reduce stream buffering/dropped frames when going from low motion to high motion: https://help.twitch.tv/s/article/broadcasting-guidelines?language=en_US. Info for youtube: https://support.google.com/youtube/answer/2853702?hl=en#zippy=%2Cvariable-bitrate-with-custom-stream-keys-in-live-control-room%2Ck-p-fps%2Cp-fps. -Limit fps recording with x damage. This is good when running replay mode 24/7 and being afk or when not much is happening on the screen. - On nvidia some games apparently causes the game to appear to stutter (without dropping fps) when recording a monitor but not using when using direct screen capture. Observed in Deus Ex and Apex Legends. -Support "screen" (all monitors) capture on wayland. This should be done by getting all drm fds and multiple EGL_DMA_BUF_PLANEX_FD_EXT to create one egl image with all fds combined. - -Support pipewire screen capture? -CPU usage is pretty high on AMD/Intel/(Nvidia(wayland)), why? opening and closing fds, creating egl, cuda association, is slow when done every frame. Test if desktop portal screencast has better performance. - Capture is broken on amd on wlroots. It's disabled at the moment and instead uses kms capture. Find out why we get a black screen in wlroots. Support vulkan video encoding. That might workaround forced p2 state nvidia driver "bug". Ffmpeg supports vulkan video encoding if it's encoding with --enable-vulkan @@ -84,10 +71,6 @@ Use SRC_W and SRC_H for screen plane instead of crtc_w and crtc_h. Make it possible to select which /dev/dri/card* to use, but that requires opengl to also use the same card. Not sure if that is possible for amd, intel and nvidia without using vulkan instead. -Support intel display framebuffer compression (I915_FORMAT_MOD_Y_TILED_CCS modifier) (and other power saving modifiers, see https://trac.ffmpeg.org/ticket/8542). The only fix may be to use desktop portal for recording. This issue doesn't appear on x11 since these modifiers are not used by xorg server. -This issue only appears on some intel iGPUs, such as Intel Iris Xe, see: https://github.com/dec05eba/gpu-screen-recorder-issues/issues/1. -Intel dedicated GPU (intel arc a750) can have a similar issue, but it's not related to compression. In that case the modifier is I915_FORMAT_MOD_4_TILED. - Test if p2 state can be worked around by using pure nvenc api and overwriting cuInit/cuCtxCreate* to not do anything. Cuda might be loaded when using nvenc but it might not be used, with certain record options? (such as h264 p5). nvenc uses cuda when using b frames and rgb->yuv conversion, so convert the image ourselves instead.- @@ -95,7 +78,7 @@ Mesa doesn't support global headers (AV_CODEC_FLAG_GLOBAL_HEADER) with h264... w Drop frames if live streaming cant keep up with target fps, or dynamically change resolution/quality. -Support low power option (does it even work with vaapi in ffmpeg??). Would be very useful for steam deck. +Support low power option. Instead of sending a big list of drm data back to kms client, send the monitor we want to record to kms server and the server should respond with only the matching monitor, and cursor. @@ -109,8 +92,6 @@ Setup hardware video context so we can query constraints and capabilities for be Use CAP_SYS_NICE in flatpak too on the main gpu screen recorder binary. It makes recording smoother, especially with constant framerate. -Show error when using compressed kms plane which isn't supported. Also do that in the gui. - Modify ffmpeg to accept opengl texture for nvenc encoding. Removes extra buffers and copies. When vulkan encode is added, mention minimum nvidia driver required. (550.54.14?). @@ -124,8 +105,6 @@ Go back to using pure vaapi without opengl for video encoding? rotation (transpo Implement scaling and use lanczos resampling for better quality. Lanczos resampling can also be used for YUV chroma for better color quality on small text. -Try fixing HDR by passing HDR+10 data as well, and in the packet. Run "ffprobe -loglevel quiet -read_intervals "%+#2" -select_streams v:0 -show_entries side_data video.mp4" to test if the file has correct metadata. - Flac is disabled because the frame sizes are too large which causes big audio/video desync. Add 10-bit capture option. This is good because it reduces banding and quality in very dark areas while reducing the file size compared to doing the same thing with 8-bits. @@ -136,14 +115,10 @@ Support vfr matching games exact fps all the time. On x11 use damage tracking, o Another method to track damage that works regardless of the display server would be to do a diff between frames with a shader. Support selecting which gpu to use. This can be done in egl with eglQueryDevicesEXT and then eglGetPlatformDisplayEXT. This will automatically work on AMD and Intel as vaapi uses the same device. On nvidia we need to use eglQueryDeviceAttribEXT with EGL_CUDA_DEVICE_NV. - Maybe on glx (nvidia x11 nvfbc) we need to use __NV_PRIME_RENDER_OFFLOAD_PROVIDER and __GLX_VENDOR_LIBRARY_NAME instead. - -Remove is_damaged and clear_damage and return a value from capture function instead that states if the image has been updated or not. + Maybe on glx (nvidia x11 nvfbc) we need to use __NV_PRIME_RENDER_OFFLOAD, __NV_PRIME_RENDER_OFFLOAD_PROVIDER, __GLX_VENDOR_LIBRARY_NAME, __VK_LAYER_NV_optimus, VK_ICD_FILENAMES instead. Just look at prime-run /usr/bin/prime-run. -Test install intel-hybrid-codec-driver-git for vp8 encoding on intel. When adding support for steam deck, add option to send video to another computer. New gpu screen recorder gui should have the option to cut the video directly, maybe running an ffmpeg command or implementing that ourselves. Only support gpu screen recorder video files. -Add hdr metadata to encoder settings metadata. Check if is software renderer by using eglQueryDisplayAttribEXT(egl_display, EGL_DEVICE_EXT..) eglQueryDeviceStringEXT(egl_device, EGL_EXTENSIONS) and check for "EGL_MESA_device_software". @@ -151,4 +126,55 @@ Use MapTexture2DINTEL for software encoding on intel. To test vulkan encode on amd set the environment variable RADV_PERFTEST=video_encode before running a program that uses vulkan encode (or queries for it, such as vulkaninfo). -Support hevc/av1 for software encoder and hdr support at the same time. Need support for yuv420p shader for that. Use libx265 for hevc and libsvtav1 for av1 (libsvtav1 is the fastest software av1 video encoder). Also support vp8/vp9 since we are not limited by hardware.
\ No newline at end of file +Support hevc/av1 for software encoder and hdr support at the same time. Need support for yuv420p shader for that. Use libx265 for hevc and libsvtav1 for av1 (libsvtav1 is the fastest software av1 video encoder). Also support vp8/vp9 since we are not limited by hardware. + +Cleanup pipewire code and add more error checks. + +Make dbus code and pipewire setup non blocking. + +Support portal (pipewire) hdr capture when pipewire adds support for it. Maybe use the result of SelectSources and then query the hdr metadata with drm. + +HDR support on x11? + +Move most kms data to kms client. We dont need root access for everything that is server from kms server right now, such as hdr metadata and drm plane properties. Only the drm plane fd really needs root access. + +Show rotated window size in monitor list when using incorrect monitor name. + +Desktop portal capture on kde plasma makes notifications not show up unless the notification is set as urgent. How to fix this? do we have to make our own notification system? + +Explicit sync is done with the drm property IN_FENCE_FD (see https://drmdb.emersion.fr/properties/4008636142/IN_FENCE_FD). Check if this needs to be used on wayland (especially on nvidia) when capturing a monitor directly without desktop portal. + +The update fps appear to be lower when recording a monitor instead of using portal on intel. Does this reflect in game framerate? + +Fix glitches when using prime-run with desktop portal. It happens when moving a window around. It's probably a syncing issue. + +Allow prime-run on x11 if monitor capture and the prime gpu is not nvidia. + +Enable 2-pass encoding. + +Add cbr option. + +Restart replay/update video resolution if monitor resolution changes. + +Fix pure vaapi copy on intel. + +ffmpeg supports vulkan encoding now (h264!). Doesn't work on amd yet because mesa is missing VK_KHR_video_maintenance1, see https://gitlab.freedesktop.org/mesa/mesa/-/issues/11857. Test on nvidia! + +Test vaapi low latency mode (setenv("AMD_DEBUG", "lowlatencyenc", true);), added in mesa 24.1.4, released on july 17, 2024. Note that this forces gpu power usage to max at all times, even when recording at 2 fps. +Use nvidia low latency options for better encoding times. + +Test ideal async_depth value. Increasing async_depth also increased gpu memory usage a lot (from 100mb to 500mb when moving from async_depth 2 to 16) at 4k resolution. Setting it to 8 increases it by 200mb which might be ok. + +Replace -encoder cpu with -k h264_software? + +Change vp8/vp9 quality options, right now the file size is too large (for vp9 at least at very_high quality). + +Support recording while in replay mode. This will be needed when enabling replay on system startup with systemd service and wanting to record a video besides that. + The harder and more bloat solution for this would be to make an IPC. + The simple solution would be to use SIGUSR2 for starting/stopping recording since SIGUSR2 is unused for replays. That would mean SIGUSR2 for pausing recording would be ignored. + It also means that the video will be created in the same directory as the replay (or have option to specify another location for that) but the filename would have to be generated automatically. + To rename the file you would have to use -sc to rename it with a script, or add an option to provide a template for the name. + +Dynamically change bitrate/resolution to match desired fps. This would be helpful when streaming for example, where the encode output speed also depends on upload speed to the streaming service. +Implement opengl injection to capture texture. This fixes VRR without having to use NvFBC direct capture and also allows perfect frame timing. +Always use direct capture with NvFBC once the capture issue in mpv fullscreen has been resolved (maybe detect if direct capture fails in nvfbc and switch to non-direct recording. NvFBC says if direct capture fails).
\ No newline at end of file |