diff options
Diffstat (limited to 'TODO')
| -rw-r--r-- | TODO | 87 |
1 files changed, 83 insertions, 4 deletions
@@ -4,7 +4,6 @@ See https://trac.ffmpeg.org/wiki/EncodingForStreamingSites for optimizing stream Look at VK_EXT_external_memory_dma_buf. Use mov+faststart. Allow recording all monitors/selected monitor without nvfbc by recording the compositor proxy window and only recording the part that matches the monitor(s). -Allow recording a region by recording the compositor proxy window / nvfbc window and copying part of it. Support amf and qsv. Disable flipping on nvidia? this might fix some stuttering issues on some setups. See NvCtrlGetAttribute/NvCtrlSetAttributeAndGetStatus NV_CTRL_SYNC_TO_VBLANK https://github.com/NVIDIA/nvidia-settings/blob/d5f022976368cbceb2f20b838ddb0bf992f0cfb9/src/gtk%2B-2.x/ctkopengl.c. Replays seem to have some issues with audio/video. Why? @@ -178,9 +177,7 @@ Test if `xrandr --output DP-1 --scale 1.5` captures correct size on nvidia. Fix cursor position and scale when scaling x11 display. -Support surround audio in application audio recording. Right now only stereo sound is supported. - -Support application audio recording without pulseaudio combined sink. +Support application audio recording without pipewire combined sink. Support transposing (rotating) with vaapi. This isn't supported on many devices with rgb buffer, but its supported with nv12 buffer (on intel at least). @@ -204,6 +201,7 @@ Ffmpeg fixed black bars in videos on amd when using hevc and when recording at s https://github.com/FFmpeg/FFmpeg/commit/d0facac679faf45d3356dff2e2cb382580d7a521 Disable gpu screen recorder black bar handling when using hevc on amd when the libavcodec version is the one that comes after those commits. Also consider the mesa version, to see if the gpu supports this. + The version is libavcodec >= 61.28.100 Use opengl compute shader instead of graphics shader. This might allow for better performance when games are using 100% of graphics unit which might fix issue with 100% gpu usage causing gpu screen recorder to run slow when not using vaapi to convert rgb to nv12(?). @@ -230,3 +228,84 @@ Add an option to pass http headers when streaming. Some streaming services requi When adding vulkan video support add VK_VIDEO_ENCODE_TUNING_MODE_LOW_LATENCY_KHR. Implement screenshot without invoking opengl (which is slow to start on some systems). + +Automatically use desktop portal on wayland when hdr is enabled (or night light) by checking if kms hdr metadata exists, if hdr video codec is not used. + Or maybe do this in the ui? + +Detect if cached portal session token is no longer valid (this can happen if the user switches to another wayland compositor). + +Support reconnecting (and setting things up again) if the audio server is restarted (for both device recording and app recording). + +Find out how nvidia-smi fixes nvenc not working on opensuse and do that ourselves instead of relying on nvidia-smi that is not always installed. + +Pulseaudio code: add "running" variable to loops to allow stopping the running code when quitting. + +Scale screenshot frame libswscale or implement lanczos shader for improved scaline for video as well. + +Support high quality scaling with -s by using lanczos. + +Support spanning multiple monitors with region capture. This would also allow the user to record multiple monitors at the same time, the same way screen-direct works on nvidia x11. + +When webcam support is added also support v4l2loopback? this is done by using avdevice_register_all(); and -c v4l2 -o /dev/video0; but it needs to output raw data as well instead of h264 and possibly yuv420p. Maybe add a -k yuv420p option to do that or -k rgb. + This would be implemented by outputting the raw data directly into the output file, without using the video encoder. + +Do proper exit, to call gsr_capture_destroy which will properly stop gsr-kms-server. Otherwise there can be zombie gsr-kms-server on error. + +Replace all scissors with clearing textures if the cursor hits the outside of the frame image. + +Cursor position might be slightly wrong on rotated monitor. + +External texture doesn't work on nvidia x11, probably because of glx context (requires gles es). External texture is not used on nvidia x11 right now so it's not an issue. + +Add option to save replay buffer on disk instead of ram. + +nvfbc capture cursor with cursor.h instead and composite that on top. This allows us to also always get a cursor in direct capture mode. This could possible give better performance as well. + +Maybe remove external shader code and make a simple external to internal texture converter (compute shader), to reduce texture sampling. Maybe this is faster? + +Fix opengl context broken after suspend on nvidia by using this: https://registry.khronos.org/OpenGL/extensions/NV/NV_robustness_video_memory_purge.txt requires glx context creation flags and GetGraphicsResetStatusARB() == PURGED_CONTEXT_RESET_NV check to recreate all graphics. + +HDR looks incorrect, brightest point gets cut off. + +Make "screen" capture the preferred monitor. + +When webcam support is added add the option to add it as a second video track, to make it easier to edit in video editors. + +Fix constant framerate not working properly on amd/intel because capture framerate gets locked to the same framerate as + game framerate, which doesn't work well when you need to encode multiple duplicate frames (AMD/Intel is slow at encoding!). + It also appears to skip audio frames on nvidia wayland? why? that should be fine, but it causes video stuttering because of audio/video sync. + +Add option to pass a fd (from socketpair) to use for rpc. In the rpc have a common header, with protocol version, data type and data in an enum. + +Add the option to set audio track name, for example with -a "track-name:blabla|device:default_output|app:firefox" + +Maybe disable qp/vbr for replay. In that case we can preallocate all replay data (for both ram and disk) and write to that directly when receiving packet (dont do that when also recording at the same time). + That could improve performance/disk write optimization and maybe even reduce ram usage because of less blocks/fragmentation. + +When rpc is added add the option to add/remove audio devices/app audio and also overlays (from new capture sources). + +Support hdr screenshot. + +Recreate opengl context on loss. This can happen if there is a gpu driver bug, causing context to need to be recreated. This is a nice improvement to not break recording even with buggy driver. + +Support saving video with surround sound. Surround sound audio capture does work, but it gets downmixed to stereo. + +Add (render) plugin support. To simplify it (and possibly best performance) create one rgba texture (with the size of the output video) that is used across all plugins. + Create a framebuffer and set this texture and the target and set the framebuffer as active before calling the plugins. + Then the plugins can render simply by doing simple opengl draw functions. + Maybe send some metadata to the plugin, such as video (and framebuffer) size. Although this data can be retrieved from the active framebuffer. + +Either support webcam support with raw yuyv, mapping the buffer directly to opengl. Or use mjpeg, mapping the buffer directly to vaapi jpeg decoder and then get then map the decoded buffer to opengl. + Some webcams dont support raw yuyv and many webcams support higher framerates for mjpeg. + +Allow medium, high, very_high and ultra quality for -bm cbr. If that is used then it will automatically estimate the best bitrate for that quality based on resolution and fps. + Maybe do this in the ui instead (or both?), to show estimated file size. + +Maybe remove shader compute code. It doesn't seem necessary anymore now that glSwapBuffer/glFinish isn't used. dbus server isn't needed anymore either, the code can be moved back to the gpu screen recorder process. + +Add proper check if opengl functions are supported. dlsym for the symbol will return a no-op function if it's not supported, so it silently fails if used. + +Colors are offset to bottom left by 1 pixel or so on steam deck in landscape mode. + +When constant framerate is used (and for audio) multiple frames need to be encoded after resuming from suspend. + The clock jumps forward by around 2-3 seconds (on my machine). Is there a way to make sure the clock doesn't jump forward?
\ No newline at end of file |