diff options
57 files changed, 2618 insertions, 1418 deletions
@@ -7,6 +7,8 @@ similar to shadowplay on windows. This is the fastest screen recording tool for This screen recorder can be used for recording your desktop offline, for live streaming and for nvidia shadowplay-like instant replay, where only the last few minutes are saved. +This is a cli-only tool, if you want an UI for this check out [GPU Screen Recorder GTK](https://git.dec05eba.com/gpu-screen-recorder-gtk/) or if you prefer a ShadowPlay-like UI then check out [GPU Screen Recorder UI](https://git.dec05eba.com/gpu-screen-recorder-ui/). + Supported video codecs: * H264 (default) * HEVC (Optionally with HDR) @@ -46,14 +48,22 @@ This should work fine on AMD/Intel X11 or Wayland. On Nvidia X11 G-SYNC only wor For example it can cause your computer to freeze when recording certain games. # Installation -If you are running an Arch Linux based distro, then you can find gpu screen recorder on aur under the name gpu-screen-recorder-git (`yay -S gpu-screen-recorder-git`).\ +If you are running an Arch Linux based distro then you can find gpu screen recorder on aur under the name gpu-screen-recorder-git (`yay -S gpu-screen-recorder-git`).\ If you are running another distro then you can run `sudo ./install.sh`, but you need to manually install the dependencies, as described below.\ You can also install gpu screen recorder ([the gtk gui version](https://git.dec05eba.com/gpu-screen-recorder-gtk/)) from [flathub](https://flathub.org/apps/details/com.dec05eba.gpu_screen_recorder), which is the easiest method to install GPU Screen Recorder on non-arch based distros.\ -The only official ways to install GPU Screen Recorder is either from source, AUR or flathub. If you install GPU Screen Recorder from somewhere else and have an issue then try installing it -from one of the official sources before reporting it as an issue. If you install GPU Screen Recorder flatpak, which is the gtk gui version then you can still run GPU Screen Recorder command line by using the flatpak command option, for example `flatpak run --command=gpu-screen-recorder com.dec05eba.gpu_screen_recorder -w screen -f 60 -o video.mp4`. Note that if you want to record your monitor on AMD/Intel then you need to install the flatpak system-wide (like so: `flatpak install flathub --system com.dec05eba.gpu_screen_recorder`). +## Unofficial install methods +The only official ways to install GPU Screen Recorder is either from source, AUR or flathub. Other sources may be out of date and missing features or may not work correctly.\ +If you install GPU Screen Recorder from somewhere else and have an issue then try installing it from one of the official sources before reporting it as an issue.\ +If you still prefer to install GPU Screen Recorder with a package manager instead of from source or as a flatpak then you may be able to find a package for your distro.\ +Here are some known unofficial packages: +* Debian/Ubuntu: [Pacstall](https://pacstall.dev/packages/gpu-screen-recorder) +* Nix: [NixOS wiki](https://wiki.nixos.org/wiki/Gpu-screen-recorder) +* openSUSE: [openSUSE software repository](https://software.opensuse.org/package/gpu-screen-recorder) +* Gentoo: [Guru](https://github.com/gentoo/guru/blob/master/media-video/gpu-screen-recorder/gpu-screen-recorder-9999.ebuild) + # Dependencies GPU Screen Recorder uses meson build system so you need to install `meson` to build GPU Screen Recorder. @@ -61,6 +71,7 @@ GPU Screen Recorder uses meson build system so you need to install `meson` to bu These are the dependencies needed to build GPU Screen Recorder: * libglvnd (which provides libgl, libglx and libegl) +* vulkan-headers * ffmpeg (libavcodec, libavformat, libavutil, libswresample, libavfilter) * x11 (libx11, libxcomposite, libxrandr, libxfixes, libxdamage) * libpulse @@ -93,7 +104,9 @@ When compiling GPU Screen Recorder with portal support (`-Dportal=true`, which i * libpipewire (and libspa which is usually part of libpipewire) # How to use -Run `gpu-screen-recorder --help` to see all options and also examples. +Run `gpu-screen-recorder --help` to see all options and also examples.\ +There is also a gui for the gpu screen recorder called [GPU Screen Recorder GTK](https://git.dec05eba.com/gpu-screen-recorder-gtk/).\ +There is also a new alternative UI for GPU Screen Recorder in the style of ShadowPlay called [GPU Screen Recorder UI](https://git.dec05eba.com/gpu-screen-recorder-ui/). ## Recording Here is an example of how to record your monitor and the default audio output: `gpu-screen-recorder -w screen -f 60 -a default_output -o ~/Videos/test_video.mp4`. Yyou can stop and save the recording with `Ctrl+C` or by running `killall -SIGINT gpu-screen-recorder`. @@ -111,15 +124,14 @@ The replay buffer is stored in ram (as encoded video), so don't use a too large To save a video in replay mode, you need to send signal SIGUSR1 to gpu screen recorder. You can do this by running `killall -SIGUSR1 gpu-screen-recorder`.\ To stop recording send SIGINT to gpu screen recorder. You can do this by running `killall -SIGINT gpu-screen-recorder` or pressing `Ctrl-C` in the terminal that runs gpu screen recorder. When recording a regular non-replay video this will also save the video.\ To pause/unpause recording send SIGUSR2 to gpu screen recorder. You can do this by running `killall -SIGUSR2 gpu-screen-recorder`. This is only applicable and useful when recording (not streaming nor replay).\ -## Finding audio device name -You can find the default output audio device (headset, speakers (in other words, desktop audio)) with the command `pactl get-default-sink`. Add `monitor` to the end of that to use that as an audio input in gpu screen recorder.\ -You can find the default input audio device (microphone) with the command `pactl get-default-source`. This input should not have `monitor` added to the end when used in gpu screen recorder.\ -Example of recording both desktop audio and microphone: `gpu-screen-recorder -w screen -f 60 -a "$(pactl get-default-sink).monitor" -a "$(pactl get-default-source)" -o ~/Videos/test_video.mp4`.\ -A name (that is visible to pipewire) can be given to an audio input device by prefixing the audio input with `<name>/`, for example `dummy/$(pactl get-default-sink).monitor`.\ -Note that if you use multiple audio inputs then they are each recorded into separate audio tracks in the video file. If you want to merge multiple audio inputs into one audio track then separate the audio inputs by "|" in one -a argument, -for example `-a "$(pactl get-default-sink).monitor|$(pactl get-default-source)"`. - -There is also a gui for the gpu screen recorder called [gpu-screen-recorder-gtk](https://git.dec05eba.com/gpu-screen-recorder-gtk/). +## Audio device name +To record the default output device (desktop audio) you can use the `default_output` option, for example `-a default_output`.\ +To record the default input device (microphone) you can use the `default_input` option, for example `-a default_input`.\ +To list all available audio devices run `gpu-screen-recorder --list-audio-devices`. The name to use with GPU Screen Recorder will be on the left side and the human readable name is on the right side.\ +To record multiple audio devices to multiple audio tracks specify the `-a` option multiple times, for example `-a default_output -a default_input`.\ +To record multiple audio devices into one audio track (merged) specify the `-a` option once split with `|` for each audio device, for example `-a "default_output|default_input"`.\ +In wireplumber the name of the audio will be in the format `gsr-<audio_device>`, but you can change that name by prefixing the audio device with a name and then a forward slash, for example: `-a "name/default_output"`. + ## Simple way to run replay without gui Run the script `scripts/start-replay.sh` to start replay and then `scripts/save-replay.sh` to save a replay and `scripts/stop-replay.sh` to stop the replay. The videos are saved to `$HOME/Videos`. You can use these scripts to start replay at system startup if you add `scripts/start-replay.sh` to startup (this can be done differently depending on your desktop environment / window manager) and then go into @@ -137,9 +149,11 @@ You have to reboot your computer after installing GPU Screen Recorder for the fi Look at the [scripts](https://git.dec05eba.com/gpu-screen-recorder/tree/scripts) directory for script examples. For example if you want to automatically save a recording/replay into a folder with the same name as the game you are recording. # Reporting bugs -Issues are reported on this Github page: [https://github.com/dec05eba/gpu-screen-recorder-issues/issues](https://github.com/dec05eba/gpu-screen-recorder-issues/issues) +Issues are reported on this Github page: [https://github.com/dec05eba/gpu-screen-recorder-issues](https://github.com/dec05eba/gpu-screen-recorder-issues). # Contributing patches -See [https://git.dec05eba.com/?p=about](https://git.dec05eba.com/?p=about) +See [https://git.dec05eba.com/?p=about](https://git.dec05eba.com/?p=about) for contribution steps. +# Donations +See [https://git.dec05eba.com/?p=about](https://git.dec05eba.com/?p=about) for donation options. # Demo [](https://www.youtube.com/watch?v=n5tm0g01n6A) @@ -166,13 +180,5 @@ Try recording to an SSD and make sure it's not using NTFS file system. Also reco You have to either record in hdr mode (-k `hevc_hdr` or -k `av1_hdr` option) to record a HDR video or record with desktop portal option (`-w portal`) to turn the HDR recording into SDR. ## GPU Screen Recorder records night light when recording in HDR mode You can record with desktop portal option (`-w portal`) instead which ignores night light, if you are ok with recording without HDR. - -# Donations -If you want to donate you can donate via bitcoin or monero. -* Bitcoin: bc1qqvuqnwrdyppf707ge27fqz2n9y9gu7lf5ypyuf -* Monero: 4An9kp2qW1C9Gah7ewv4JzcNFQ5TAX7ineGCqXWK6vQnhsGGcRpNgcn8r9EC3tMcgY7vqCKs3nSRXhejMHBaGvFdN2egYet - -# TODO -* Dynamically change bitrate/resolution to match desired fps. This would be helpful when streaming for example, where the encode output speed also depends on upload speed to the streaming service. -* Implement opengl injection to capture texture. This fixes VRR without having to use NvFBC direct capture. -* Always use direct capture with NvFBC once the capture issue in mpv fullscreen has been resolved (maybe detect if direct capture fails in nvfbc and switch to non-direct recording. NvFBC says if direct capture fails). +## Kdenlive says that the video is not usable for editing because it has variable frame rate +To fix this you can either record the video in .mkv format or constant frame rate (-fm cfr). @@ -74,11 +74,9 @@ Make it possible to select which /dev/dri/card* to use, but that requires opengl Test if p2 state can be worked around by using pure nvenc api and overwriting cuInit/cuCtxCreate* to not do anything. Cuda might be loaded when using nvenc but it might not be used, with certain record options? (such as h264 p5). nvenc uses cuda when using b frames and rgb->yuv conversion, so convert the image ourselves instead.- -Mesa doesn't support global headers (AV_CODEC_FLAG_GLOBAL_HEADER) with h264... which also breaks mkv since mkv requires global header. Right now gpu screen recorder will forcefully set video codec to hevc when h264 is requested for mkv files. - Drop frames if live streaming cant keep up with target fps, or dynamically change resolution/quality. -Support low power option (does it even work with vaapi in ffmpeg??). Would be very useful for steam deck. +Support low power option. Instead of sending a big list of drm data back to kms client, send the monitor we want to record to kms server and the server should respond with only the matching monitor, and cursor. @@ -103,12 +101,10 @@ Investigate if there is a way to do gpu->gpu copy directly without touching syst Go back to using pure vaapi without opengl for video encoding? rotation (transpose) can be done if its done after (rgb to yuv) color conversion. -Implement scaling and use lanczos resampling for better quality. Lanczos resampling can also be used for YUV chroma for better color quality on small text. +Use lanczos resampling for better scaling quality. Lanczos resampling can also be used for YUV chroma for better color quality on small text. Flac is disabled because the frame sizes are too large which causes big audio/video desync. -Add 10-bit capture option. This is good because it reduces banding and quality in very dark areas while reducing the file size compared to doing the same thing with 8-bits. - Enable b-frames. Support vfr matching games exact fps all the time. On x11 use damage tracking, on wayland? maybe there is drm plane damage tracking. But that may not be accurate as the compositor may update it every monitor hz anyways. On wayland maybe only support it for desktop portal + pipewire capture. @@ -152,15 +148,42 @@ Allow prime-run on x11 if monitor capture and the prime gpu is not nvidia. Enable 2-pass encoding. -Add cbr option. - Restart replay/update video resolution if monitor resolution changes. Fix pure vaapi copy on intel. -ffmpeg supports vulkan encoding now (h264!). Doesn't work on amd yet because mesa is missing VK_KHR_video_maintenance1, see https://gitlab.freedesktop.org/mesa/mesa/-/issues/11857. Test on nvidia! - -Test vaapi low latency mode (setenv("AMD_DEBUG", "lowlatencyenc", true);), added in mesa 24.1.4, released on july 17, 2024. Note that this forces gpu power usage to max at all times, even when recording at 2 fps. Use nvidia low latency options for better encoding times. Test ideal async_depth value. Increasing async_depth also increased gpu memory usage a lot (from 100mb to 500mb when moving from async_depth 2 to 16) at 4k resolution. Setting it to 8 increases it by 200mb which might be ok. + +Replace -encoder cpu with -k h264_software? + +Change vp8/vp9 quality options, right now the file size is too large (for vp9 at least at very_high quality). + +Support recording while in replay mode. This will be needed when enabling replay on system startup with systemd service and wanting to record a video besides that. + The harder and more bloat solution for this would be to make an IPC. + The simple solution would be to use SIGUSR2 for starting/stopping recording since SIGUSR2 is unused for replays. That would mean SIGUSR2 for pausing recording would be ignored. + It also means that the video will be created in the same directory as the replay (or have option to specify another location for that) but the filename would have to be generated automatically. + To rename the file you would have to use -sc to rename it with a script, or add an option to provide a template for the name. + +Dynamically change bitrate/resolution to match desired fps. This would be helpful when streaming for example, where the encode output speed also depends on upload speed to the streaming service. +Implement opengl injection to capture texture. This fixes VRR without having to use NvFBC direct capture and also allows perfect frame timing. +Always use direct capture with NvFBC once the capture issue in mpv fullscreen has been resolved (maybe detect if direct capture fails in nvfbc and switch to non-direct recording. NvFBC says if direct capture fails). + +Support ROI (AV_FRAME_DATA_REGIONS_OF_INTEREST). + +Default to hevc if capture size is larger than 4096 in width or height. + +Set low latency mode on vulkan encoding. + +Support pipewire audio capture which also allows capturing audio from a single application. This can also be done with pulseaudio by creating a virtual sink: + pactl load-module module-combine-sink sink_name=gsr2 slaves=$(pactl get-default-sink) sink_properties=device.description="gsr" + pactl move-sink-input 2944 gsr2 # 2944 comes from 'pactl list sink-inputs' + and then record gsr2.monitor. + Or use pa_stream_set_monitor_stream, which also takes the sink-input as input. However need to track when the sink disconnects to mute and then reconnect again. + +Support recording/replay/livestreaming at the same time by allowing commands to be run on an existing gpu screen recorder instance. + +Test if `xrandr --output DP-1 --scale 1.5` captures correct size on nvidia. + +Fix cursor position and scale when scaling x11 display.
\ No newline at end of file diff --git a/extra/gpu-screen-recorder.env b/extra/gpu-screen-recorder.env index 8ce1edb..664fa5d 100644 --- a/extra/gpu-screen-recorder.env +++ b/extra/gpu-screen-recorder.env @@ -1,6 +1,5 @@ WINDOW=screen CONTAINER=mp4 -QUALITY=very_high CODEC=h264 AUDIO_CODEC=opus AUDIO_DEVICE=default_output diff --git a/extra/gpu-screen-recorder.service b/extra/gpu-screen-recorder.service index d0912ec..1938a36 100644 --- a/extra/gpu-screen-recorder.service +++ b/extra/gpu-screen-recorder.service @@ -5,24 +5,26 @@ Description=GPU Screen Recorder Service EnvironmentFile=-%h/.config/gpu-screen-recorder.env Environment=WINDOW=screen Environment=CONTAINER=mp4 -Environment=QUALITY=very_high +Environment=QUALITY=50000 +Environment=BITRATE_MODE=cbr Environment=CODEC=auto Environment=AUDIO_CODEC=opus Environment=AUDIO_DEVICE=default_output Environment=SECONDARY_AUDIO_DEVICE= Environment=FRAMERATE=60 -Environment=REPLAYDURATION=30 +Environment=REPLAYDURATION=60 Environment=OUTPUTDIR=%h/Videos Environment=MAKEFOLDERS=no Environment=COLOR_RANGE=limited Environment=KEYINT=2 Environment=ENCODER=gpu Environment=RESTORE_PORTAL_SESSION=yes +Environment=OUTPUT_RESOLUTION=0x0 Environment=ADDITIONAL_ARGS= -ExecStart=gpu-screen-recorder -v no -w "${WINDOW}" -c "${CONTAINER}" -q "${QUALITY}" -k "${CODEC}" -ac "${AUDIO_CODEC}" -a "${AUDIO_DEVICE}" -a "${SECONDARY_AUDIO_DEVICE}" -f "${FRAMERATE}" -r "${REPLAYDURATION}" -o "${OUTPUTDIR}" -df "${MAKEFOLDERS}" $ADDITIONAL_ARGS -cr "${COLOR_RANGE}" -keyint "${KEYINT}" -restore-portal-session "${RESTORE_PORTAL_SESSION}" -encoder "${ENCODER}" +ExecStart=gpu-screen-recorder -v no -w "${WINDOW}" -s "${OUTPUT_RESOLUTION}" -c "${CONTAINER}" -q "${QUALITY}" -k "${CODEC}" -ac "${AUDIO_CODEC}" -a "${AUDIO_DEVICE}" -a "${SECONDARY_AUDIO_DEVICE}" -f "${FRAMERATE}" -r "${REPLAYDURATION}" -o "${OUTPUTDIR}" -df "${MAKEFOLDERS}" $ADDITIONAL_ARGS -cr "${COLOR_RANGE}" -keyint "${KEYINT}" -restore-portal-session "${RESTORE_PORTAL_SESSION}" -encoder "${ENCODER}" -bm "${BITRATE_MODE}" KillSignal=SIGINT Restart=on-failure RestartSec=5s [Install] -WantedBy=default.target +WantedBy=default.target
\ No newline at end of file diff --git a/include/capture/capture.h b/include/capture/capture.h index 7c8887d..dc5b7ac 100644 --- a/include/capture/capture.h +++ b/include/capture/capture.h @@ -9,9 +9,9 @@ typedef struct AVCodecContext AVCodecContext; typedef struct AVStream AVStream; typedef struct AVFrame AVFrame; -typedef struct gsr_capture gsr_capture; typedef struct AVMasteringDisplayMetadata AVMasteringDisplayMetadata; typedef struct AVContentLightMetadata AVContentLightMetadata; +typedef struct gsr_capture gsr_capture; struct gsr_capture { /* These methods should not be called manually. Call gsr_capture_* instead */ @@ -20,7 +20,6 @@ struct gsr_capture { void (*tick)(gsr_capture *cap); /* can be NULL. If there is an event then |on_event| is called before this */ bool (*should_stop)(gsr_capture *cap, bool *err); /* can be NULL. If NULL, return false */ int (*capture)(gsr_capture *cap, AVFrame *frame, gsr_color_conversion *color_conversion); - gsr_source_color (*get_source_color)(gsr_capture *cap); bool (*uses_external_image)(gsr_capture *cap); /* can be NULL. If NULL, return false */ bool (*set_hdr_metadata)(gsr_capture *cap, AVMasteringDisplayMetadata *mastering_display_metadata, AVContentLightMetadata *light_metadata); /* can be NULL. If NULL, return false */ uint64_t (*get_window_id)(gsr_capture *cap); /* can be NULL. Returns 0 if unknown */ @@ -37,7 +36,6 @@ void gsr_capture_on_event(gsr_capture *cap, gsr_egl *egl); void gsr_capture_tick(gsr_capture *cap); bool gsr_capture_should_stop(gsr_capture *cap, bool *err); int gsr_capture_capture(gsr_capture *cap, AVFrame *frame, gsr_color_conversion *color_conversion); -gsr_source_color gsr_capture_get_source_color(gsr_capture *cap); bool gsr_capture_uses_external_image(gsr_capture *cap); bool gsr_capture_set_hdr_metadata(gsr_capture *cap, AVMasteringDisplayMetadata *mastering_display_metadata, AVContentLightMetadata *light_metadata); void gsr_capture_destroy(gsr_capture *cap, AVCodecContext *video_codec_context); diff --git a/include/capture/kms.h b/include/capture/kms.h index f19e462..bf1ba62 100644 --- a/include/capture/kms.h +++ b/include/capture/kms.h @@ -10,6 +10,8 @@ typedef struct { gsr_color_range color_range; bool hdr; bool record_cursor; + int fps; + vec2i output_resolution; } gsr_capture_kms_params; gsr_capture* gsr_capture_kms_create(const gsr_capture_kms_params *params); diff --git a/include/capture/nvfbc.h b/include/capture/nvfbc.h index 95ca88d..f291f33 100644 --- a/include/capture/nvfbc.h +++ b/include/capture/nvfbc.h @@ -15,6 +15,7 @@ typedef struct { gsr_color_range color_range; bool record_cursor; bool use_software_video_encoder; + vec2i output_resolution; } gsr_capture_nvfbc_params; gsr_capture* gsr_capture_nvfbc_create(const gsr_capture_nvfbc_params *params); diff --git a/include/capture/portal.h b/include/capture/portal.h index 2e2c6f2..3989b98 100644 --- a/include/capture/portal.h +++ b/include/capture/portal.h @@ -11,6 +11,7 @@ typedef struct { bool restore_portal_session; /* If this is set to NULL then this defaults to $XDG_CONFIG_HOME/gpu-screen-recorder/restore_token ($XDG_CONFIG_HOME defaults to $HOME/.config) */ const char *portal_session_token_filepath; + vec2i output_resolution; } gsr_capture_portal_params; gsr_capture* gsr_capture_portal_create(const gsr_capture_portal_params *params); diff --git a/include/capture/xcomposite.h b/include/capture/xcomposite.h index 8c87404..45eb481 100644 --- a/include/capture/xcomposite.h +++ b/include/capture/xcomposite.h @@ -8,10 +8,10 @@ typedef struct { gsr_egl *egl; unsigned long window; bool follow_focused; /* If this is set then |window| is ignored */ - vec2i region_size; /* This is currently only used with |follow_focused| */ gsr_color_range color_range; bool record_cursor; gsr_color_depth color_depth; + vec2i output_resolution; } gsr_capture_xcomposite_params; gsr_capture* gsr_capture_xcomposite_create(const gsr_capture_xcomposite_params *params); diff --git a/include/codec_query/codec_query.h b/include/codec_query/codec_query.h new file mode 100644 index 0000000..316217d --- /dev/null +++ b/include/codec_query/codec_query.h @@ -0,0 +1,23 @@ +#ifndef GSR_CODEC_QUERY_H +#define GSR_CODEC_QUERY_H + +#include <stdbool.h> + +typedef struct { + bool supported; + bool low_power; +} gsr_supported_video_codec; + +typedef struct { + gsr_supported_video_codec h264; + gsr_supported_video_codec hevc; + gsr_supported_video_codec hevc_hdr; + gsr_supported_video_codec hevc_10bit; + gsr_supported_video_codec av1; + gsr_supported_video_codec av1_hdr; + gsr_supported_video_codec av1_10bit; + gsr_supported_video_codec vp8; + gsr_supported_video_codec vp9; +} gsr_supported_video_codecs; + +#endif /* GSR_CODEC_QUERY_H */ diff --git a/include/codec_query/nvenc.h b/include/codec_query/nvenc.h new file mode 100644 index 0000000..c01acf6 --- /dev/null +++ b/include/codec_query/nvenc.h @@ -0,0 +1,8 @@ +#ifndef GSR_CODEC_QUERY_NVENC_H +#define GSR_CODEC_QUERY_NVENC_H + +#include "codec_query.h" + +bool gsr_get_supported_video_codecs_nvenc(gsr_supported_video_codecs *video_codecs, bool cleanup); + +#endif /* GSR_CODEC_QUERY_NVENC_H */ diff --git a/include/codec_query/vaapi.h b/include/codec_query/vaapi.h new file mode 100644 index 0000000..60bdeca --- /dev/null +++ b/include/codec_query/vaapi.h @@ -0,0 +1,8 @@ +#ifndef GSR_CODEC_QUERY_VAAPI_H +#define GSR_CODEC_QUERY_VAAPI_H + +#include "codec_query.h" + +bool gsr_get_supported_video_codecs_vaapi(gsr_supported_video_codecs *video_codecs, const char *card_path, bool cleanup); + +#endif /* GSR_CODEC_QUERY_VAAPI_H */ diff --git a/include/codec_query/vulkan.h b/include/codec_query/vulkan.h new file mode 100644 index 0000000..bb06c6b --- /dev/null +++ b/include/codec_query/vulkan.h @@ -0,0 +1,8 @@ +#ifndef GSR_CODEC_QUERY_VULKAN_H +#define GSR_CODEC_QUERY_VULKAN_H + +#include "codec_query.h" + +bool gsr_get_supported_video_codecs_vulkan(gsr_supported_video_codecs *video_codecs, const char *card_path, bool cleanup); + +#endif /* GSR_CODEC_QUERY_VULKAN_H */ diff --git a/include/color_conversion.h b/include/color_conversion.h index 236bfbd..c079edd 100644 --- a/include/color_conversion.h +++ b/include/color_conversion.h @@ -33,7 +33,6 @@ typedef struct { typedef struct { gsr_egl *egl; - gsr_source_color source_color; gsr_destination_color destination_color; unsigned int destination_textures[2]; @@ -57,7 +56,7 @@ typedef struct { int gsr_color_conversion_init(gsr_color_conversion *self, const gsr_color_conversion_params *params); void gsr_color_conversion_deinit(gsr_color_conversion *self); -void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_id, vec2i source_pos, vec2i source_size, vec2i texture_pos, vec2i texture_size, float rotation, bool external_texture); +void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_id, vec2i source_pos, vec2i source_size, vec2i texture_pos, vec2i texture_size, float rotation, bool external_texture, gsr_source_color source_color); void gsr_color_conversion_clear(gsr_color_conversion *self); #endif /* GSR_COLOR_CONVERSION_H */ diff --git a/include/egl.h b/include/egl.h index 3fdbf48..82014b9 100644 --- a/include/egl.h +++ b/include/egl.h @@ -313,6 +313,9 @@ struct gsr_egl { void (*glUniform2f)(int location, float v0, float v1); void (*glDebugMessageCallback)(GLDEBUGPROC callback, const void *userParam); void (*glScissor)(int x, int y, int width, int height); + void (*glReadPixels)(int x, int y, int width, int height, unsigned int format, unsigned int type, void *pixels); + void* (*glMapBuffer)(unsigned int target, unsigned int access); + unsigned char (*glUnmapBuffer)(unsigned int target); }; bool gsr_egl_load(gsr_egl *self, Display *dpy, bool wayland, bool is_monitor_capture); diff --git a/include/encoder/video/cuda.h b/include/encoder/video/cuda.h deleted file mode 100644 index 6d32e09..0000000 --- a/include/encoder/video/cuda.h +++ /dev/null @@ -1,16 +0,0 @@ -#ifndef GSR_ENCODER_VIDEO_CUDA_H -#define GSR_ENCODER_VIDEO_CUDA_H - -#include "video.h" - -typedef struct gsr_egl gsr_egl; - -typedef struct { - gsr_egl *egl; - bool overclock; - gsr_color_depth color_depth; -} gsr_video_encoder_cuda_params; - -gsr_video_encoder* gsr_video_encoder_cuda_create(const gsr_video_encoder_cuda_params *params); - -#endif /* GSR_ENCODER_VIDEO_CUDA_H */ diff --git a/include/encoder/video/nvenc.h b/include/encoder/video/nvenc.h new file mode 100644 index 0000000..d4a906b --- /dev/null +++ b/include/encoder/video/nvenc.h @@ -0,0 +1,16 @@ +#ifndef GSR_ENCODER_VIDEO_NVENC_H +#define GSR_ENCODER_VIDEO_NVENC_H + +#include "video.h" + +typedef struct gsr_egl gsr_egl; + +typedef struct { + gsr_egl *egl; + bool overclock; + gsr_color_depth color_depth; +} gsr_video_encoder_nvenc_params; + +gsr_video_encoder* gsr_video_encoder_nvenc_create(const gsr_video_encoder_nvenc_params *params); + +#endif /* GSR_ENCODER_VIDEO_NVENC_H */ diff --git a/include/encoder/video/video.h b/include/encoder/video/video.h index 899357a..49f48bd 100644 --- a/include/encoder/video/video.h +++ b/include/encoder/video/video.h @@ -8,22 +8,9 @@ typedef struct gsr_video_encoder gsr_video_encoder; typedef struct AVCodecContext AVCodecContext; typedef struct AVFrame AVFrame; -typedef struct { - bool h264; - bool hevc; - bool hevc_hdr; - bool hevc_10bit; - bool av1; - bool av1_hdr; - bool av1_10bit; - bool vp8; - bool vp9; -} gsr_supported_video_codecs; - struct gsr_video_encoder { - gsr_supported_video_codecs (*get_supported_codecs)(gsr_video_encoder *encoder, bool cleanup); bool (*start)(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame); - void (*copy_textures_to_frame)(gsr_video_encoder *encoder, AVFrame *frame); /* Can be NULL */ + void (*copy_textures_to_frame)(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion); /* Can be NULL */ /* |textures| should be able to fit 2 elements */ void (*get_textures)(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color); void (*destroy)(gsr_video_encoder *encoder, AVCodecContext *video_codec_context); @@ -32,9 +19,8 @@ struct gsr_video_encoder { bool started; }; -gsr_supported_video_codecs gsr_video_encoder_get_supported_codecs(gsr_video_encoder *encoder, bool cleanup); bool gsr_video_encoder_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame); -void gsr_video_encoder_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame); +void gsr_video_encoder_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion); void gsr_video_encoder_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color); void gsr_video_encoder_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context); diff --git a/include/encoder/video/vulkan.h b/include/encoder/video/vulkan.h new file mode 100644 index 0000000..383fc4f --- /dev/null +++ b/include/encoder/video/vulkan.h @@ -0,0 +1,15 @@ +#ifndef GSR_ENCODER_VIDEO_VULKAN_H +#define GSR_ENCODER_VIDEO_VULKAN_H + +#include "video.h" + +typedef struct gsr_egl gsr_egl; + +typedef struct { + gsr_egl *egl; + gsr_color_depth color_depth; +} gsr_video_encoder_vulkan_params; + +gsr_video_encoder* gsr_video_encoder_vulkan_create(const gsr_video_encoder_vulkan_params *params); + +#endif /* GSR_ENCODER_VIDEO_VULKAN_H */ diff --git a/include/pipewire_audio.h b/include/pipewire_audio.h new file mode 100644 index 0000000..ae14cb3 --- /dev/null +++ b/include/pipewire_audio.h @@ -0,0 +1,4 @@ +#ifndef GSR_PIPEWIRE_AUDIO_H +#define GSR_PIPEWIRE_AUDIO_H + +#endif /* GSR_PIPEWIRE_AUDIO_H */ diff --git a/include/pipewire.h b/include/pipewire_video.h index 1908e2d..00e2835 100644 --- a/include/pipewire.h +++ b/include/pipewire_video.h @@ -1,5 +1,5 @@ -#ifndef GSR_PIPEWIRE_H -#define GSR_PIPEWIRE_H +#ifndef GSR_PIPEWIRE_VIDEO_H +#define GSR_PIPEWIRE_VIDEO_H #include <stdbool.h> #include <stdint.h> @@ -8,9 +8,9 @@ #include <spa/utils/hook.h> #include <spa/param/video/format.h> -#define GSR_PIPEWIRE_MAX_MODIFIERS 1024 -#define GSR_PIPEWIRE_NUM_VIDEO_FORMATS 6 -#define GSR_PIPEWIRE_DMABUF_MAX_PLANES 4 +#define GSR_PIPEWIRE_VIDEO_MAX_MODIFIERS 1024 +#define GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS 6 +#define GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES 4 typedef struct gsr_egl gsr_egl; @@ -18,23 +18,23 @@ typedef struct { int major; int minor; int micro; -} gsr_pipewire_data_version; +} gsr_pipewire_video_data_version; typedef struct { uint32_t fps_num; uint32_t fps_den; -} gsr_pipewire_video_info; +} gsr_pipewire_video_video_info; typedef struct { int fd; uint32_t offset; int32_t stride; -} gsr_pipewire_dmabuf_data; +} gsr_pipewire_video_dmabuf_data; typedef struct { int x, y; int width, height; -} gsr_pipewire_region; +} gsr_pipewire_video_region; typedef struct { enum spa_video_format format; @@ -82,31 +82,31 @@ typedef struct { uint32_t width, height; } crop; - gsr_video_format supported_video_formats[GSR_PIPEWIRE_NUM_VIDEO_FORMATS]; + gsr_video_format supported_video_formats[GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS]; - gsr_pipewire_data_version server_version; - gsr_pipewire_video_info video_info; - gsr_pipewire_dmabuf_data dmabuf_data[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; + gsr_pipewire_video_data_version server_version; + gsr_pipewire_video_video_info video_info; + gsr_pipewire_video_dmabuf_data dmabuf_data[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; size_t dmabuf_num_planes; bool no_modifiers_fallback; bool external_texture_fallback; - uint64_t modifiers[GSR_PIPEWIRE_MAX_MODIFIERS]; + uint64_t modifiers[GSR_PIPEWIRE_VIDEO_MAX_MODIFIERS]; size_t num_modifiers; -} gsr_pipewire; +} gsr_pipewire_video; /* |capture_cursor| only applies to when capturing a window or region. In other cases |pipewire_node|'s setup will determine if the cursor is included. Note that the cursor is not guaranteed to be shown even if set to true, it depends on the wayland compositor. */ -bool gsr_pipewire_init(gsr_pipewire *self, int pipewire_fd, uint32_t pipewire_node, int fps, bool capture_cursor, gsr_egl *egl); -void gsr_pipewire_deinit(gsr_pipewire *self); +bool gsr_pipewire_video_init(gsr_pipewire_video *self, int pipewire_fd, uint32_t pipewire_node, int fps, bool capture_cursor, gsr_egl *egl); +void gsr_pipewire_video_deinit(gsr_pipewire_video *self); -/* |dmabuf_data| should be at least GSR_PIPEWIRE_DMABUF_MAX_PLANES in size */ -bool gsr_pipewire_map_texture(gsr_pipewire *self, gsr_texture_map texture_map, gsr_pipewire_region *region, gsr_pipewire_region *cursor_region, gsr_pipewire_dmabuf_data *dmabuf_data, int *num_dmabuf_data, uint32_t *fourcc, uint64_t *modifiers, bool *using_external_image); -bool gsr_pipewire_is_damaged(gsr_pipewire *self); -void gsr_pipewire_clear_damage(gsr_pipewire *self); +/* |dmabuf_data| should be at least GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES in size */ +bool gsr_pipewire_video_map_texture(gsr_pipewire_video *self, gsr_texture_map texture_map, gsr_pipewire_video_region *region, gsr_pipewire_video_region *cursor_region, gsr_pipewire_video_dmabuf_data *dmabuf_data, int *num_dmabuf_data, uint32_t *fourcc, uint64_t *modifiers, bool *using_external_image); +bool gsr_pipewire_video_is_damaged(gsr_pipewire_video *self); +void gsr_pipewire_video_clear_damage(gsr_pipewire_video *self); -#endif /* GSR_PIPEWIRE_H */ +#endif /* GSR_PIPEWIRE_VIDEO_H */ diff --git a/include/utils.h b/include/utils.h index 92eb851..9ccd26e 100644 --- a/include/utils.h +++ b/include/utils.h @@ -51,4 +51,6 @@ bool video_codec_context_is_vaapi(AVCodecContext *video_codec_context); bool vaapi_copy_drm_planes_to_video_surface(AVCodecContext *video_codec_context, AVFrame *video_frame, vec2i source_pos, vec2i source_size, vec2i dest_pos, vec2i dest_size, uint32_t format, vec2i size, const int *fds, const uint32_t *offsets, const uint32_t *pitches, const uint64_t *modifiers, int num_planes); bool vaapi_copy_egl_image_to_video_surface(gsr_egl *egl, EGLImage image, vec2i source_pos, vec2i source_size, vec2i dest_pos, vec2i dest_size, AVCodecContext *video_codec_context, AVFrame *video_frame); +vec2i scale_keep_aspect_ratio(vec2i from, vec2i to); + #endif /* GSR_UTILS_H */ diff --git a/include/vec2.h b/include/vec2.h index 3e33cfb..8fd3858 100644 --- a/include/vec2.h +++ b/include/vec2.h @@ -9,4 +9,8 @@ typedef struct { float x, y; } vec2f; +typedef struct { + double x, y; +} vec2d; + #endif /* VEC2_H */ diff --git a/kms/client/kms_client.c b/kms/client/kms_client.c index d3e68dd..6f11244 100644 --- a/kms/client/kms_client.c +++ b/kms/client/kms_client.c @@ -146,13 +146,41 @@ static bool create_socket_path(char *output_path, size_t output_path_size) { return true; } -static void string_copy(char *dst, const char *src, int len) { - int src_len = strlen(src); - int min_len = src_len; - if(len - 1 < min_len) - min_len = len - 1; - memcpy(dst, src, min_len); - dst[min_len] = '\0'; +static bool readlink_realpath(const char *filepath, char *buffer) { + char symlinked_path[PATH_MAX]; + ssize_t bytes_written = readlink(filepath, symlinked_path, sizeof(symlinked_path) - 1); + if(bytes_written == -1 && errno == EINVAL) { + /* Not a symlink */ + snprintf(symlinked_path, sizeof(symlinked_path), "%s", filepath); + } else if(bytes_written == -1) { + return false; + } else { + symlinked_path[bytes_written] = '\0'; + } + + if(!realpath(symlinked_path, buffer)) + return false; + + return true; +} + +static bool strcat_safe(char *str, int size, const char *str_to_add) { + const int str_len = strlen(str); + const int str_to_add_len = strlen(str_to_add); + if(str_len + str_to_add_len + 1 >= size) + return false; + + memcpy(str + str_len, str_to_add, str_to_add_len); + str[str_len + str_to_add_len] = '\0'; + return true; +} + +static void file_get_directory(char *filepath) { + char *end = strrchr(filepath, '/'); + if(end == NULL) + filepath[0] = '\0'; + else + *end = '\0'; } static bool find_program_in_path(const char *program_name, char *filepath, int filepath_len) { @@ -206,11 +234,27 @@ int gsr_kms_client_init(gsr_kms_client *self, const char *card_path) { } char server_filepath[PATH_MAX]; - if(!find_program_in_path("gsr-kms-server", server_filepath, sizeof(server_filepath))) { - fprintf(stderr, "gsr error: gsr_kms_client_init: gsr-kms-server is not installed\n"); + if(!readlink_realpath("/proc/self/exe", server_filepath)) { + fprintf(stderr, "gsr error: gsr_kms_client_init: failed to resolve /proc/self/exe\n"); + return -1; + } + file_get_directory(server_filepath); + + if(!strcat_safe(server_filepath, sizeof(server_filepath), "/gsr-kms-server")) { + fprintf(stderr, "gsr error: gsr_kms_client_init: gsr-kms-server path too long\n"); return -1; } + if(access(server_filepath, F_OK) != 0) { + fprintf(stderr, "gsr info: gsr_kms_client_init: gsr-kms-server is not installed in the same directory as gpu-screen-recorder (%s not found), looking for gsr-kms-server in PATH instead\n", server_filepath); + if(!find_program_in_path("gsr-kms-server", server_filepath, sizeof(server_filepath)) || access(server_filepath, F_OK) != 0) { + fprintf(stderr, "gsr error: gsr_kms_client_init: gsr-kms-server was not found in PATH. Please install gpu-screen-recorder properly\n"); + return -1; + } + } + + fprintf(stderr, "gsr info: gsr_kms_client_init: setting up connection to %s\n", server_filepath); + const bool inside_flatpak = getenv("FLATPAK_ID") != NULL; const char *home = getenv("HOME"); if(!home) @@ -251,7 +295,7 @@ int gsr_kms_client_init(gsr_kms_client *self, const char *card_path) { } local_addr.sun_family = AF_UNIX; - string_copy(local_addr.sun_path, self->initial_socket_path, sizeof(local_addr.sun_path)); + snprintf(local_addr.sun_path, sizeof(local_addr.sun_path), "%s", (const char*)self->initial_socket_path); const mode_t prev_mask = umask(0000); const int bind_res = bind(self->initial_socket_fd, (struct sockaddr*)&local_addr, sizeof(local_addr.sun_family) + strlen(local_addr.sun_path)); @@ -400,7 +444,7 @@ int gsr_kms_client_replace_connection(gsr_kms_client *self) { } if(response.version != GSR_KMS_PROTOCOL_VERSION) { - fprintf(stderr, "gsr error: gsr_kms_client_replace_connection: expected gsr-kms-server protocol version to be %u, but it's %u\n", GSR_KMS_PROTOCOL_VERSION, response.version); + fprintf(stderr, "gsr error: gsr_kms_client_replace_connection: expected gsr-kms-server protocol version to be %u, but it's %u. please reinstall gpu screen recorder\n", GSR_KMS_PROTOCOL_VERSION, response.version); /*close_fds(response);*/ return -1; } @@ -435,7 +479,7 @@ int gsr_kms_client_get_kms(gsr_kms_client *self, gsr_kms_response *response) { } if(response->version != GSR_KMS_PROTOCOL_VERSION) { - fprintf(stderr, "gsr error: gsr_kms_client_get_kms: expected gsr-kms-server protocol version to be %u, but it's %u\n", GSR_KMS_PROTOCOL_VERSION, response->version); + fprintf(stderr, "gsr error: gsr_kms_client_get_kms: expected gsr-kms-server protocol version to be %u, but it's %u. please reinstall gpu screen recorder\n", GSR_KMS_PROTOCOL_VERSION, response->version); /*close_fds(response);*/ strcpy(response->err_msg, "mismatching protocol version"); return -1; diff --git a/kms/server/kms_server.c b/kms/server/kms_server.c index 0c1085e..b4f3378 100644 --- a/kms/server/kms_server.c +++ b/kms/server/kms_server.c @@ -1,3 +1,7 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + #include "../kms_shared.h" #include <stdio.h> @@ -6,6 +10,7 @@ #include <stdlib.h> #include <unistd.h> +#include <limits.h> #include <fcntl.h> #include <sys/socket.h> #include <sys/un.h> @@ -413,14 +418,80 @@ static double clock_get_monotonic_seconds(void) { return (double)ts.tv_sec + (double)ts.tv_nsec * 0.000000001; } -static void string_copy(char *dst, const char *src, int len) { - int src_len = strlen(src); - int min_len = src_len; - if(len - 1 < min_len) - min_len = len - 1; - memcpy(dst, src, min_len); - dst[min_len] = '\0'; -} +// static bool readlink_realpath(const char *filepath, char *buffer) { +// char symlinked_path[PATH_MAX]; +// ssize_t bytes_written = readlink(filepath, symlinked_path, sizeof(symlinked_path) - 1); +// if(bytes_written == -1 && errno == EINVAL) { +// /* Not a symlink */ +// snprintf(symlinked_path, sizeof(symlinked_path), "%s", filepath); +// } else if(bytes_written == -1) { +// return false; +// } else { +// symlinked_path[bytes_written] = '\0'; +// } + +// if(!realpath(symlinked_path, buffer)) +// return false; + +// return true; +// } + +// static void file_get_directory(char *filepath) { +// char *end = strrchr(filepath, '/'); +// if(end == NULL) +// filepath[0] = '\0'; +// else +// *end = '\0'; +// } + +// static bool string_ends_with(const char *str, const char *ends_with) { +// const int len = strlen(str); +// const int ends_with_len = strlen(ends_with); +// return len >= ends_with_len && memcmp(str + len - ends_with_len, ends_with, ends_with_len) == 0; +// } + +// This is not foolproof, but the assumption is that gsr-kms-server and gpu-screen-recorder are installed in the same directory +// in a location that only the root user can write to (usually /usr/bin or /usr/local/bin) and if the client runs from that location +// and is called gpu-screen-recorder then gsr-kms-server can only be used by a malicious program if the malicious program +// had root access, to modify that program install directory. +// static bool is_remote_peer_program_gpu_screen_recorder(int socket_fd) { +// // TODO: Use SO_PEERPIDFD on kernel >= 6.5 to avoid a race condition in the /proc/<pid> check +// struct ucred cred; +// socklen_t ucred_len = sizeof(cred); +// if(getsockopt(socket_fd, SOL_SOCKET, SO_PEERCRED, &cred, &ucred_len) == -1) { +// fprintf(stderr, "kms server error: failed to get peer credentials, error: %s\n", strerror(errno)); +// return false; +// } + +// char self_directory[PATH_MAX]; +// if(!readlink_realpath("/proc/self/exe", self_directory)) { +// fprintf(stderr, "kms server error: failed to resolve /proc/self/exe\n"); +// return false; +// } +// file_get_directory(self_directory); + +// char peer_directory[PATH_MAX]; +// char peer_exe_path[PATH_MAX]; +// snprintf(peer_exe_path, sizeof(peer_exe_path), "/proc/%d/exe", (int)cred.pid); +// if(!readlink_realpath(peer_exe_path, peer_directory)) { +// fprintf(stderr, "kms server error: failed to resolve /proc/self/exe\n"); +// return false; +// } + +// if(!string_ends_with(peer_directory, "/gpu-screen-recorder")) { +// fprintf(stderr, "kms server error: only gpu-screen-recorder can use gsr-kms-server. client program location is %s\n", peer_directory); +// return false; +// } + +// file_get_directory(peer_directory); + +// if(strcmp(self_directory, peer_directory) != 0) { +// fprintf(stderr, "kms server error: the client program is in directory %s but only programs in %s can run gsr-kms-server\n", peer_directory, self_directory); +// return false; +// } + +// return true; +// } int main(int argc, char **argv) { int res = 0; @@ -478,7 +549,7 @@ int main(int argc, char **argv) { while(clock_get_monotonic_seconds() - start_time < connect_timeout_sec) { struct sockaddr_un remote_addr = {0}; remote_addr.sun_family = AF_UNIX; - string_copy(remote_addr.sun_path, domain_socket_path, sizeof(remote_addr.sun_path)); + snprintf(remote_addr.sun_path, sizeof(remote_addr.sun_path), "%s", domain_socket_path); // TODO: Check if parent disconnected if(connect(socket_fd, (struct sockaddr*)&remote_addr, sizeof(remote_addr.sun_family) + strlen(remote_addr.sun_path)) == -1) { if(errno == ECONNREFUSED || errno == ENOENT) { @@ -505,6 +576,11 @@ int main(int argc, char **argv) { goto done; } + // if(!is_remote_peer_program_gpu_screen_recorder(socket_fd)) { + // res = 3; + // goto done; + // } + for(;;) { gsr_kms_request request; request.version = 0; @@ -528,7 +604,7 @@ int main(int argc, char **argv) { } if(request.version != GSR_KMS_PROTOCOL_VERSION) { - fprintf(stderr, "kms server error: expected gpu screen recorder protocol version to be %u, but it's %u\n", GSR_KMS_PROTOCOL_VERSION, request.version); + fprintf(stderr, "kms server error: expected gpu screen recorder protocol version to be %u, but it's %u. please reinstall gpu screen recorder\n", GSR_KMS_PROTOCOL_VERSION, request.version); /* if(request.new_connection_fd > 0) close(request.new_connection_fd); diff --git a/meson.build b/meson.build index b275812..f5c9214 100644 --- a/meson.build +++ b/meson.build @@ -1,4 +1,4 @@ -project('gpu-screen-recorder', ['c', 'cpp'], version : '4.1.8', default_options : ['warning_level=2']) +project('gpu-screen-recorder', ['c', 'cpp'], version : '4.2.6', default_options : ['warning_level=2']) add_project_arguments('-Wshadow', language : ['c', 'cpp']) if get_option('buildtype') == 'debug' @@ -14,9 +14,13 @@ src = [ 'src/capture/xcomposite.c', 'src/capture/kms.c', 'src/encoder/video/video.c', - 'src/encoder/video/cuda.c', + 'src/encoder/video/nvenc.c', 'src/encoder/video/vaapi.c', + 'src/encoder/video/vulkan.c', 'src/encoder/video/software.c', + 'src/codec_query/nvenc.c', + 'src/codec_query/vaapi.c', + 'src/codec_query/vulkan.c', 'src/egl.c', 'src/cuda.c', 'src/xnvctrl.c', @@ -53,20 +57,32 @@ dep = [ dependency('wayland-client'), ] +uses_pipewire = false + if get_option('portal') == true src += [ 'src/capture/portal.c', 'src/dbus.c', - 'src/pipewire.c', + 'src/pipewire_video.c', ] + dep += dependency('dbus-1') + add_project_arguments('-DGSR_PORTAL', language : ['c', 'cpp']) + uses_pipewire = true +endif +if get_option('app_audio') == true + src += [ + 'src/pipewire_audio.c', + ] + add_project_arguments('-DGSR_APP_AUDIO', language : ['c', 'cpp']) + uses_pipewire = true +endif + +if uses_pipewire == true dep += [ - dependency('dbus-1'), dependency('libpipewire-0.3'), dependency('libspa-0.2'), ] - - add_project_arguments('-DGSR_PORTAL', language : ['c', 'cpp']) endif add_project_arguments('-DGSR_VERSION="' + meson.project_version() + '"', language: ['c', 'cpp']) diff --git a/meson_options.txt b/meson_options.txt index 61972c6..5936927 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -1,4 +1,5 @@ option('systemd', type : 'boolean', value : true, description : 'Install systemd service file') option('capabilities', type : 'boolean', value : true, description : 'Set binary admin capability to remove password prompt when recording monitor (without desktop portal option) on amd/intel or nvidia wayland') option('nvidia_suspend_fix', type : 'boolean', value : true, description : 'Install nvidia modprobe config file to tell nvidia driver to preserve video memory on suspend. This is a workaround for an nvidia driver bug that breaks cuda (and gpu screen recorder) on suspend') -option('portal', type : 'boolean', value : true, description : 'If GPU Screen Recorder should be built with support for xdg desktop portal ScreenCast capture (wayland only)') +option('portal', type : 'boolean', value : true, description : 'Build with support for xdg desktop portal ScreenCast capture (wayland only) (-w portal option)') +option('app_audio', type : 'boolean', value : true, description : 'Build with support for recording a single audio source (-aa option). Requires pipewire')
\ No newline at end of file diff --git a/project.conf b/project.conf index 63b03a0..776df96 100644 --- a/project.conf +++ b/project.conf @@ -1,15 +1,16 @@ [package] name = "gpu-screen-recorder" type = "executable" -version = "4.1.8" +version = "4.2.6" platforms = ["posix"] [config] -ignore_dirs = ["kms/server", "build"] +ignore_dirs = ["kms/server", "build", "debug-build"] #error_on_warning = "true" [define] GSR_PORTAL = "1" +GSR_APP_AUDIO = "1" [dependencies] libavcodec = ">=58" @@ -32,3 +33,4 @@ wayland-client = ">=1" dbus-1 = ">=1" libpipewire-0.3 = ">=1" libspa-0.2 = ">=0" +#vulkan = ">=1" diff --git a/scripts/record-application-name.sh b/scripts/record-application-name.sh index 4139c9c..f8c9b0d 100755 --- a/scripts/record-application-name.sh +++ b/scripts/record-application-name.sh @@ -1,6 +1,6 @@ #!/bin/sh window=$(xdotool selectwindow) -window_name=$(xdotool getwindowclassname "$window" || xdotool getwindowname "$window" || echo "Game") +window_name=$(xdotool getwindowname "$window" || xdotool getwindowclassname "$window" || echo "Game") window_name="$(echo "$window_name" | tr '/\\' '_')" gpu-screen-recorder -w "$window" -f 60 -a default_output -o "$HOME/Videos/recording/$window_name/$(date +"Video_%Y-%m-%d_%H-%M-%S.mp4")" diff --git a/scripts/record-save-application-name.sh b/scripts/record-save-application-name.sh index b814809..c95f398 100755 --- a/scripts/record-save-application-name.sh +++ b/scripts/record-save-application-name.sh @@ -4,7 +4,7 @@ # gpu-screen-recorder -w screen -f 60 -a default_output -r 60 -sc scripts/record-save-application-name.sh -c mp4 -o "$HOME/Videos" window=$(xdotool getwindowfocus) -window_name=$(xdotool getwindowclassname "$window" || xdotool getwindowname "$window" || echo "Game") +window_name=$(xdotool getwindowname "$window" || xdotool getwindowclassname "$window" || echo "Game") window_name="$(echo "$window_name" | tr '/\\' '_')" video_dir="$HOME/Videos/Replays/$window_name" diff --git a/scripts/replay-application-name.sh b/scripts/replay-application-name.sh index 9f363f7..2a651bb 100755 --- a/scripts/replay-application-name.sh +++ b/scripts/replay-application-name.sh @@ -1,6 +1,6 @@ #!/bin/sh window=$(xdotool selectwindow) -window_name=$(xdotool getwindowclassname "$window" || xdotool getwindowname "$window" || echo "Game") +window_name=$(xdotool getwindowname "$window" || xdotool getwindowclassname "$window" || echo "Game") window_name="$(echo "$window_name" | tr '/\\' '_')" gpu-screen-recorder -w "$window" -f 60 -c mkv -a default_output -r 60 -o "$HOME/Videos/Replays/$window_name" diff --git a/scripts/start-recording.sh b/scripts/start-recording.sh index bf71d62..03fda73 100755 --- a/scripts/start-recording.sh +++ b/scripts/start-recording.sh @@ -1,5 +1,5 @@ #!/bin/sh -pidof -q gpu-screen-recorder && exit 1 +pidof -q gpu-screen-recorder && exit 0 video="$HOME/Videos/$(date +"Video_%Y-%m-%d_%H-%M-%S.mp4")" gpu-screen-recorder -w screen -f 60 -a default_output -o "$video" diff --git a/scripts/start-replay.sh b/scripts/start-replay.sh index 6c03178..860f2ce 100755 --- a/scripts/start-replay.sh +++ b/scripts/start-replay.sh @@ -1,6 +1,6 @@ #!/bin/sh -pidof -q gpu-screen-recorder && exit 1 +pidof -q gpu-screen-recorder && exit 0 video_path="$HOME/Videos" mkdir -p "$video_path" gpu-screen-recorder -w screen -f 60 -a default_output -c mkv -r 30 -o "$video_path" diff --git a/scripts/start-stop-recording.sh b/scripts/start-stop-recording.sh new file mode 100755 index 0000000..775a829 --- /dev/null +++ b/scripts/start-stop-recording.sh @@ -0,0 +1,10 @@ +#!/bin/sh + +# Simple script to start recording if it's not recording and stop recording +# if it's already recording. This script can be bound to a single hotkey +# to start/stop recording with a single hotkey. + +killall -SIGINT -q gpu-screen-recorder && exit 0 +video="$HOME/Videos/$(date +"Video_%Y-%m-%d_%H-%M-%S.mp4")" +gpu-screen-recorder -w screen -f 60 -a default_output -o "$video" +notify-send -t 2000 -u low "GPU Screen Recorder" "Video saved to $video" diff --git a/src/capture/capture.c b/src/capture/capture.c index ec10854..2a4a689 100644 --- a/src/capture/capture.c +++ b/src/capture/capture.c @@ -34,10 +34,6 @@ int gsr_capture_capture(gsr_capture *cap, AVFrame *frame, gsr_color_conversion * return cap->capture(cap, frame, color_conversion); } -gsr_source_color gsr_capture_get_source_color(gsr_capture *cap) { - return cap->get_source_color(cap); -} - bool gsr_capture_uses_external_image(gsr_capture *cap) { if(cap->uses_external_image) return cap->uses_external_image(cap); diff --git a/src/capture/kms.c b/src/capture/kms.c index 030d609..c85811e 100644 --- a/src/capture/kms.c +++ b/src/capture/kms.c @@ -54,6 +54,7 @@ typedef struct { AVCodecContext *video_codec_context; bool performance_error_shown; + bool fast_path_failed; //int drm_fd; //uint64_t prev_sequence; @@ -213,8 +214,15 @@ static int gsr_capture_kms_start(gsr_capture *cap, AVCodecContext *video_codec_c /* Disable vsync */ self->params.egl->eglSwapInterval(self->params.egl->egl_display, 0); - video_codec_context->width = FFALIGN(self->capture_size.x, 2); - video_codec_context->height = FFALIGN(self->capture_size.y, 2); + if(self->params.output_resolution.x == 0 && self->params.output_resolution.y == 0) { + self->params.output_resolution = self->capture_size; + video_codec_context->width = FFALIGN(self->capture_size.x, 2); + video_codec_context->height = FFALIGN(self->capture_size.y, 2); + } else { + self->params.output_resolution = scale_keep_aspect_ratio(self->capture_size, self->params.output_resolution); + video_codec_context->width = FFALIGN(self->params.output_resolution.x, 2); + video_codec_context->height = FFALIGN(self->params.output_resolution.y, 2); + } frame->width = video_codec_context->width; frame->height = video_codec_context->height; @@ -232,7 +240,7 @@ static void gsr_capture_kms_on_event(gsr_capture *cap, gsr_egl *egl) { gsr_cursor_on_event(&self->x11_cursor, xev); } -// TODO: This is disabled for now because we want to be able to record at a framerate highe than the monitor framerate +// TODO: This is disabled for now because we want to be able to record at a framerate higher than the monitor framerate // static void gsr_capture_kms_tick(gsr_capture *cap) { // gsr_capture_kms *self = cap->priv; @@ -428,7 +436,12 @@ static gsr_kms_response_item* find_cursor_drm_if_on_monitor(gsr_capture_kms *sel return cursor_drm_fd; } -static void render_drm_cursor(gsr_capture_kms *self, gsr_color_conversion *color_conversion, const gsr_kms_response_item *cursor_drm_fd, vec2i target_pos, float texture_rotation) { +static void render_drm_cursor(gsr_capture_kms *self, gsr_color_conversion *color_conversion, const gsr_kms_response_item *cursor_drm_fd, vec2i target_pos, float texture_rotation, vec2i output_size) { + const vec2d scale = { + self->capture_size.x == 0 ? 0 : (double)output_size.x / (double)self->capture_size.x, + self->capture_size.y == 0 ? 0 : (double)output_size.y / (double)self->capture_size.y + }; + const bool cursor_texture_id_is_external = self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_NVIDIA; const vec2i cursor_size = {cursor_drm_fd->width, cursor_drm_fd->height}; @@ -457,6 +470,9 @@ static void render_drm_cursor(gsr_capture_kms *self, gsr_color_conversion *color break; } + cursor_pos.x *= scale.x; + cursor_pos.y *= scale.y; + cursor_pos.x += target_pos.x; cursor_pos.y += target_pos.y; @@ -486,34 +502,39 @@ static void render_drm_cursor(gsr_capture_kms *self, gsr_color_conversion *color self->params.egl->eglDestroyImage(self->params.egl->egl_display, cursor_image); self->params.egl->glEnable(GL_SCISSOR_TEST); - self->params.egl->glScissor(target_pos.x, target_pos.y, self->capture_size.x, self->capture_size.y); + self->params.egl->glScissor(target_pos.x, target_pos.y, output_size.x, output_size.y); gsr_color_conversion_draw(color_conversion, self->cursor_texture_id, - cursor_pos, cursor_size, + cursor_pos, (vec2i){cursor_size.x * scale.x, cursor_size.y * scale.y}, (vec2i){0, 0}, cursor_size, - texture_rotation, cursor_texture_id_is_external); + texture_rotation, cursor_texture_id_is_external, GSR_SOURCE_COLOR_RGB); self->params.egl->glDisable(GL_SCISSOR_TEST); } -static void render_x11_cursor(gsr_capture_kms *self, gsr_color_conversion *color_conversion, vec2i capture_pos, vec2i target_pos) { +static void render_x11_cursor(gsr_capture_kms *self, gsr_color_conversion *color_conversion, vec2i capture_pos, vec2i target_pos, vec2i output_size) { if(!self->x11_cursor.visible) return; + const vec2d scale = { + self->capture_size.x == 0 ? 0 : (double)output_size.x / (double)self->capture_size.x, + self->capture_size.y == 0 ? 0 : (double)output_size.y / (double)self->capture_size.y + }; + gsr_cursor_tick(&self->x11_cursor, DefaultRootWindow(self->params.egl->x11.dpy)); const vec2i cursor_pos = { - target_pos.x + self->x11_cursor.position.x - self->x11_cursor.hotspot.x - capture_pos.x, - target_pos.y + self->x11_cursor.position.y - self->x11_cursor.hotspot.y - capture_pos.y + target_pos.x + (self->x11_cursor.position.x - self->x11_cursor.hotspot.x - capture_pos.x) * scale.x, + target_pos.y + (self->x11_cursor.position.y - self->x11_cursor.hotspot.y - capture_pos.y) * scale.y }; self->params.egl->glEnable(GL_SCISSOR_TEST); - self->params.egl->glScissor(target_pos.x, target_pos.y, self->capture_size.x, self->capture_size.y); + self->params.egl->glScissor(target_pos.x, target_pos.y, output_size.x, output_size.y); gsr_color_conversion_draw(color_conversion, self->x11_cursor.texture_id, - cursor_pos, self->x11_cursor.size, + cursor_pos, (vec2i){self->x11_cursor.size.x * scale.x, self->x11_cursor.size.y * scale.y}, (vec2i){0, 0}, self->x11_cursor.size, - 0.0f, false); + 0.0f, false, GSR_SOURCE_COLOR_RGB); self->params.egl->glDisable(GL_SCISSOR_TEST); } @@ -561,25 +582,25 @@ static int gsr_capture_kms_capture(gsr_capture *cap, AVFrame *frame, gsr_color_c " If you are experience performance problems in the video then record a single window on X11 or use portal capture option instead\n"); } - const float texture_rotation = monitor_rotation_to_radians(self->monitor_rotation); - const vec2i target_pos = { max_int(0, frame->width / 2 - self->capture_size.x / 2), max_int(0, frame->height / 2 - self->capture_size.y / 2) }; self->capture_size = rotate_capture_size_if_rotated(self, (vec2i){ drm_fd->src_w, drm_fd->src_h }); + + const bool is_scaled = self->params.output_resolution.x > 0 && self->params.output_resolution.y > 0; + vec2i output_size = is_scaled ? self->params.output_resolution : self->capture_size; + output_size = scale_keep_aspect_ratio(self->capture_size, output_size); + + const float texture_rotation = monitor_rotation_to_radians(self->monitor_rotation); + const vec2i target_pos = { max_int(0, frame->width / 2 - output_size.x / 2), max_int(0, frame->height / 2 - output_size.y / 2) }; gsr_capture_kms_update_capture_size_change(self, color_conversion, target_pos, drm_fd); vec2i capture_pos = self->capture_pos; if(!capture_is_combined_plane) capture_pos = (vec2i){drm_fd->x, drm_fd->y}; - // TODO: Hack!! cursor flickers without this when using vaapi copy on wayland. - // There is probably some sync issue between opengl and vaapi. - // Remove this when that has been figured out. Same for the below glFlush && glFinish - for(int i = 0; i < 3; ++i) { - self->params.egl->glFlush(); - self->params.egl->glFinish(); - } + self->params.egl->glFlush(); + self->params.egl->glFinish(); /* Fast opengl free path */ - if(self->monitor_rotation == GSR_MONITOR_ROT_0 && video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { + if(!self->fast_path_failed && self->monitor_rotation == GSR_MONITOR_ROT_0 && video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { int fds[4]; uint32_t offsets[4]; uint32_t pitches[4]; @@ -590,44 +611,43 @@ static int gsr_capture_kms_capture(gsr_capture *cap, AVFrame *frame, gsr_color_c pitches[i] = drm_fd->dma_buf[i].pitch; modifiers[i] = drm_fd->modifier; } - vaapi_copy_drm_planes_to_video_surface(self->video_codec_context, frame, (vec2i){capture_pos.x, capture_pos.y}, self->capture_size, target_pos, self->capture_size, drm_fd->pixel_format, (vec2i){drm_fd->width, drm_fd->height}, fds, offsets, pitches, modifiers, drm_fd->num_dma_bufs); + if(!vaapi_copy_drm_planes_to_video_surface(self->video_codec_context, frame, (vec2i){capture_pos.x, capture_pos.y}, self->capture_size, target_pos, output_size, drm_fd->pixel_format, (vec2i){drm_fd->width, drm_fd->height}, fds, offsets, pitches, modifiers, drm_fd->num_dma_bufs)) { + fprintf(stderr, "gsr error: gsr_capture_kms_capture: vaapi_copy_drm_planes_to_video_surface failed, falling back to opengl copy. Please report this as an issue at https://github.com/dec05eba/gpu-screen-recorder-issues\n"); + self->fast_path_failed = true; + } } else { + self->fast_path_failed = true; + } + + if(self->fast_path_failed) { EGLImage image = gsr_capture_kms_create_egl_image_with_fallback(self, drm_fd); if(image) { gsr_capture_kms_bind_image_to_input_texture_with_fallback(self, image); self->params.egl->eglDestroyImage(self->params.egl->egl_display, image); } - self->params.egl->glFlush(); - self->params.egl->glFinish(); - gsr_color_conversion_draw(color_conversion, self->external_texture_fallback ? self->external_input_texture_id : self->input_texture_id, - target_pos, self->capture_size, + target_pos, output_size, capture_pos, self->capture_size, - texture_rotation, self->external_texture_fallback); - } - - for(int i = 0; i < 3; ++i) { - self->params.egl->glFlush(); - self->params.egl->glFinish(); + texture_rotation, self->external_texture_fallback, GSR_SOURCE_COLOR_RGB); } if(self->params.record_cursor) { gsr_kms_response_item *cursor_drm_fd = find_cursor_drm_if_on_monitor(self, drm_fd->connector_id, capture_is_combined_plane); // The cursor is handled by x11 on x11 instead of using the cursor drm plane because on prime systems with a dedicated nvidia gpu // the cursor plane is not available when the cursor is on the monitor controlled by the nvidia device. + // TODO: This doesn't work properly with software cursor on x11 since it will draw the x11 cursor on top of the cursor already in the framebuffer. + // Detect if software cursor is used on x11 somehow. if(self->is_x11) { const vec2i cursor_monitor_offset = self->capture_pos; - render_x11_cursor(self, color_conversion, cursor_monitor_offset, target_pos); + render_x11_cursor(self, color_conversion, cursor_monitor_offset, target_pos, output_size); } else if(cursor_drm_fd) { - render_drm_cursor(self, color_conversion, cursor_drm_fd, target_pos, texture_rotation); + render_drm_cursor(self, color_conversion, cursor_drm_fd, target_pos, texture_rotation, output_size); } } - for(int i = 0; i < 3; ++i) { - self->params.egl->glFlush(); - self->params.egl->glFinish(); - } + self->params.egl->glFlush(); + self->params.egl->glFinish(); gsr_capture_kms_cleanup_kms_fds(self); @@ -641,11 +661,6 @@ static bool gsr_capture_kms_should_stop(gsr_capture *cap, bool *err) { return false; } -static gsr_source_color gsr_capture_kms_get_source_color(gsr_capture *cap) { - (void)cap; - return GSR_SOURCE_COLOR_RGB; -} - static bool gsr_capture_kms_uses_external_image(gsr_capture *cap) { (void)cap; return true; @@ -732,7 +747,6 @@ gsr_capture* gsr_capture_kms_create(const gsr_capture_kms_params *params) { //.tick = gsr_capture_kms_tick, .should_stop = gsr_capture_kms_should_stop, .capture = gsr_capture_kms_capture, - .get_source_color = gsr_capture_kms_get_source_color, .uses_external_image = gsr_capture_kms_uses_external_image, .set_hdr_metadata = gsr_capture_kms_set_hdr_metadata, //.is_damaged = gsr_capture_kms_is_damaged, diff --git a/src/capture/nvfbc.c b/src/capture/nvfbc.c index a38df63..d5a270e 100644 --- a/src/capture/nvfbc.c +++ b/src/capture/nvfbc.c @@ -102,7 +102,7 @@ static void set_func_ptr(void **dst, void *src) { } static bool gsr_capture_nvfbc_load_library(gsr_capture *cap) { - gsr_capture_nvfbc *cap_nvfbc = cap->priv; + gsr_capture_nvfbc *self = cap->priv; dlerror(); /* clear */ void *lib = dlopen("libnvidia-fbc.so.1", RTLD_LAZY); @@ -111,23 +111,23 @@ static bool gsr_capture_nvfbc_load_library(gsr_capture *cap) { return false; } - set_func_ptr((void**)&cap_nvfbc->nv_fbc_create_instance, dlsym(lib, "NvFBCCreateInstance")); - if(!cap_nvfbc->nv_fbc_create_instance) { + set_func_ptr((void**)&self->nv_fbc_create_instance, dlsym(lib, "NvFBCCreateInstance")); + if(!self->nv_fbc_create_instance) { fprintf(stderr, "gsr error: unable to resolve symbol 'NvFBCCreateInstance'\n"); dlclose(lib); return false; } - memset(&cap_nvfbc->nv_fbc_function_list, 0, sizeof(cap_nvfbc->nv_fbc_function_list)); - cap_nvfbc->nv_fbc_function_list.dwVersion = NVFBC_VERSION; - NVFBCSTATUS status = cap_nvfbc->nv_fbc_create_instance(&cap_nvfbc->nv_fbc_function_list); + memset(&self->nv_fbc_function_list, 0, sizeof(self->nv_fbc_function_list)); + self->nv_fbc_function_list.dwVersion = NVFBC_VERSION; + NVFBCSTATUS status = self->nv_fbc_create_instance(&self->nv_fbc_function_list); if(status != NVFBC_SUCCESS) { fprintf(stderr, "gsr error: failed to create NvFBC instance (status: %d)\n", status); dlclose(lib); return false; } - cap_nvfbc->library = lib; + self->library = lib; return true; } @@ -153,64 +153,64 @@ static void set_vertical_sync_enabled(gsr_egl *egl, int enabled) { fprintf(stderr, "gsr warning: setting vertical sync failed\n"); } -static void gsr_capture_nvfbc_destroy_session(gsr_capture_nvfbc *cap_nvfbc) { - if(cap_nvfbc->fbc_handle_created && cap_nvfbc->capture_session_created) { +static void gsr_capture_nvfbc_destroy_session(gsr_capture_nvfbc *self) { + if(self->fbc_handle_created && self->capture_session_created) { NVFBC_DESTROY_CAPTURE_SESSION_PARAMS destroy_capture_params; memset(&destroy_capture_params, 0, sizeof(destroy_capture_params)); destroy_capture_params.dwVersion = NVFBC_DESTROY_CAPTURE_SESSION_PARAMS_VER; - cap_nvfbc->nv_fbc_function_list.nvFBCDestroyCaptureSession(cap_nvfbc->nv_fbc_handle, &destroy_capture_params); - cap_nvfbc->capture_session_created = false; + self->nv_fbc_function_list.nvFBCDestroyCaptureSession(self->nv_fbc_handle, &destroy_capture_params); + self->capture_session_created = false; } } -static void gsr_capture_nvfbc_destroy_handle(gsr_capture_nvfbc *cap_nvfbc) { - if(cap_nvfbc->fbc_handle_created) { +static void gsr_capture_nvfbc_destroy_handle(gsr_capture_nvfbc *self) { + if(self->fbc_handle_created) { NVFBC_DESTROY_HANDLE_PARAMS destroy_params; memset(&destroy_params, 0, sizeof(destroy_params)); destroy_params.dwVersion = NVFBC_DESTROY_HANDLE_PARAMS_VER; - cap_nvfbc->nv_fbc_function_list.nvFBCDestroyHandle(cap_nvfbc->nv_fbc_handle, &destroy_params); - cap_nvfbc->fbc_handle_created = false; - cap_nvfbc->nv_fbc_handle = 0; + self->nv_fbc_function_list.nvFBCDestroyHandle(self->nv_fbc_handle, &destroy_params); + self->fbc_handle_created = false; + self->nv_fbc_handle = 0; } } -static void gsr_capture_nvfbc_destroy_session_and_handle(gsr_capture_nvfbc *cap_nvfbc) { - gsr_capture_nvfbc_destroy_session(cap_nvfbc); - gsr_capture_nvfbc_destroy_handle(cap_nvfbc); +static void gsr_capture_nvfbc_destroy_session_and_handle(gsr_capture_nvfbc *self) { + gsr_capture_nvfbc_destroy_session(self); + gsr_capture_nvfbc_destroy_handle(self); } -static int gsr_capture_nvfbc_setup_handle(gsr_capture_nvfbc *cap_nvfbc) { +static int gsr_capture_nvfbc_setup_handle(gsr_capture_nvfbc *self) { NVFBCSTATUS status; NVFBC_CREATE_HANDLE_PARAMS create_params; memset(&create_params, 0, sizeof(create_params)); create_params.dwVersion = NVFBC_CREATE_HANDLE_PARAMS_VER; create_params.bExternallyManagedContext = NVFBC_TRUE; - create_params.glxCtx = cap_nvfbc->params.egl->glx_context; - create_params.glxFBConfig = cap_nvfbc->params.egl->glx_fb_config; + create_params.glxCtx = self->params.egl->glx_context; + create_params.glxFBConfig = self->params.egl->glx_fb_config; - status = cap_nvfbc->nv_fbc_function_list.nvFBCCreateHandle(&cap_nvfbc->nv_fbc_handle, &create_params); + status = self->nv_fbc_function_list.nvFBCCreateHandle(&self->nv_fbc_handle, &create_params); if(status != NVFBC_SUCCESS) { // Reverse engineering for interoperability const uint8_t enable_key[] = { 0xac, 0x10, 0xc9, 0x2e, 0xa5, 0xe6, 0x87, 0x4f, 0x8f, 0x4b, 0xf4, 0x61, 0xf8, 0x56, 0x27, 0xe9 }; create_params.privateData = enable_key; create_params.privateDataSize = 16; - status = cap_nvfbc->nv_fbc_function_list.nvFBCCreateHandle(&cap_nvfbc->nv_fbc_handle, &create_params); + status = self->nv_fbc_function_list.nvFBCCreateHandle(&self->nv_fbc_handle, &create_params); if(status != NVFBC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", cap_nvfbc->nv_fbc_function_list.nvFBCGetLastErrorStr(cap_nvfbc->nv_fbc_handle)); + fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", self->nv_fbc_function_list.nvFBCGetLastErrorStr(self->nv_fbc_handle)); goto error_cleanup; } } - cap_nvfbc->fbc_handle_created = true; + self->fbc_handle_created = true; NVFBC_GET_STATUS_PARAMS status_params; memset(&status_params, 0, sizeof(status_params)); status_params.dwVersion = NVFBC_GET_STATUS_PARAMS_VER; - status = cap_nvfbc->nv_fbc_function_list.nvFBCGetStatus(cap_nvfbc->nv_fbc_handle, &status_params); + status = self->nv_fbc_function_list.nvFBCGetStatus(self->nv_fbc_handle, &status_params); if(status != NVFBC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", cap_nvfbc->nv_fbc_function_list.nvFBCGetLastErrorStr(cap_nvfbc->nv_fbc_handle)); + fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", self->nv_fbc_function_list.nvFBCGetLastErrorStr(self->nv_fbc_handle)); goto error_cleanup; } @@ -219,10 +219,10 @@ static int gsr_capture_nvfbc_setup_handle(gsr_capture_nvfbc *cap_nvfbc) { goto error_cleanup; } - cap_nvfbc->tracking_width = XWidthOfScreen(DefaultScreenOfDisplay(cap_nvfbc->params.egl->x11.dpy)); - cap_nvfbc->tracking_height = XHeightOfScreen(DefaultScreenOfDisplay(cap_nvfbc->params.egl->x11.dpy)); - cap_nvfbc->tracking_type = strcmp(cap_nvfbc->params.display_to_capture, "screen") == 0 ? NVFBC_TRACKING_SCREEN : NVFBC_TRACKING_OUTPUT; - if(cap_nvfbc->tracking_type == NVFBC_TRACKING_OUTPUT) { + self->tracking_width = XWidthOfScreen(DefaultScreenOfDisplay(self->params.egl->x11.dpy)); + self->tracking_height = XHeightOfScreen(DefaultScreenOfDisplay(self->params.egl->x11.dpy)); + self->tracking_type = strcmp(self->params.display_to_capture, "screen") == 0 ? NVFBC_TRACKING_SCREEN : NVFBC_TRACKING_OUTPUT; + if(self->tracking_type == NVFBC_TRACKING_OUTPUT) { if(!status_params.bXRandRAvailable) { fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: the xrandr extension is not available\n"); goto error_cleanup; @@ -233,93 +233,98 @@ static int gsr_capture_nvfbc_setup_handle(gsr_capture_nvfbc *cap_nvfbc) { goto error_cleanup; } - cap_nvfbc->output_id = get_output_id_from_display_name(status_params.outputs, status_params.dwOutputNum, cap_nvfbc->params.display_to_capture, &cap_nvfbc->tracking_width, &cap_nvfbc->tracking_height); - if(cap_nvfbc->output_id == 0) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: display '%s' not found\n", cap_nvfbc->params.display_to_capture); + self->output_id = get_output_id_from_display_name(status_params.outputs, status_params.dwOutputNum, self->params.display_to_capture, &self->tracking_width, &self->tracking_height); + if(self->output_id == 0) { + fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: display '%s' not found\n", self->params.display_to_capture); goto error_cleanup; } } + if(!self->capture_region) { + self->width = self->tracking_width; + self->height = self->tracking_height; + } + return 0; error_cleanup: - gsr_capture_nvfbc_destroy_session_and_handle(cap_nvfbc); + gsr_capture_nvfbc_destroy_session_and_handle(self); return -1; } -static int gsr_capture_nvfbc_setup_session(gsr_capture_nvfbc *cap_nvfbc) { +static int gsr_capture_nvfbc_setup_session(gsr_capture_nvfbc *self) { NVFBC_CREATE_CAPTURE_SESSION_PARAMS create_capture_params; memset(&create_capture_params, 0, sizeof(create_capture_params)); create_capture_params.dwVersion = NVFBC_CREATE_CAPTURE_SESSION_PARAMS_VER; create_capture_params.eCaptureType = NVFBC_CAPTURE_TO_GL; - create_capture_params.bWithCursor = (!cap_nvfbc->params.direct_capture || cap_nvfbc->supports_direct_cursor) ? NVFBC_TRUE : NVFBC_FALSE; - if(!cap_nvfbc->params.record_cursor) + create_capture_params.bWithCursor = (!self->params.direct_capture || self->supports_direct_cursor) ? NVFBC_TRUE : NVFBC_FALSE; + if(!self->params.record_cursor) create_capture_params.bWithCursor = false; - if(cap_nvfbc->capture_region) - create_capture_params.captureBox = (NVFBC_BOX){ cap_nvfbc->x, cap_nvfbc->y, cap_nvfbc->width, cap_nvfbc->height }; - create_capture_params.eTrackingType = cap_nvfbc->tracking_type; - create_capture_params.dwSamplingRateMs = (uint32_t)ceilf(1000.0f / (float)cap_nvfbc->params.fps); - create_capture_params.bAllowDirectCapture = cap_nvfbc->params.direct_capture ? NVFBC_TRUE : NVFBC_FALSE; - create_capture_params.bPushModel = cap_nvfbc->params.direct_capture ? NVFBC_TRUE : NVFBC_FALSE; + if(self->capture_region) + create_capture_params.captureBox = (NVFBC_BOX){ self->x, self->y, self->width, self->height }; + create_capture_params.eTrackingType = self->tracking_type; + create_capture_params.dwSamplingRateMs = (uint32_t)ceilf(1000.0f / (float)self->params.fps); + create_capture_params.bAllowDirectCapture = self->params.direct_capture ? NVFBC_TRUE : NVFBC_FALSE; + create_capture_params.bPushModel = self->params.direct_capture ? NVFBC_TRUE : NVFBC_FALSE; create_capture_params.bDisableAutoModesetRecovery = true; - if(cap_nvfbc->tracking_type == NVFBC_TRACKING_OUTPUT) - create_capture_params.dwOutputId = cap_nvfbc->output_id; + if(self->tracking_type == NVFBC_TRACKING_OUTPUT) + create_capture_params.dwOutputId = self->output_id; - NVFBCSTATUS status = cap_nvfbc->nv_fbc_function_list.nvFBCCreateCaptureSession(cap_nvfbc->nv_fbc_handle, &create_capture_params); + NVFBCSTATUS status = self->nv_fbc_function_list.nvFBCCreateCaptureSession(self->nv_fbc_handle, &create_capture_params); if(status != NVFBC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", cap_nvfbc->nv_fbc_function_list.nvFBCGetLastErrorStr(cap_nvfbc->nv_fbc_handle)); + fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", self->nv_fbc_function_list.nvFBCGetLastErrorStr(self->nv_fbc_handle)); return -1; } - cap_nvfbc->capture_session_created = true; + self->capture_session_created = true; - memset(&cap_nvfbc->setup_params, 0, sizeof(cap_nvfbc->setup_params)); - cap_nvfbc->setup_params.dwVersion = NVFBC_TOGL_SETUP_PARAMS_VER; - cap_nvfbc->setup_params.eBufferFormat = NVFBC_BUFFER_FORMAT_BGRA; + memset(&self->setup_params, 0, sizeof(self->setup_params)); + self->setup_params.dwVersion = NVFBC_TOGL_SETUP_PARAMS_VER; + self->setup_params.eBufferFormat = NVFBC_BUFFER_FORMAT_BGRA; - status = cap_nvfbc->nv_fbc_function_list.nvFBCToGLSetUp(cap_nvfbc->nv_fbc_handle, &cap_nvfbc->setup_params); + status = self->nv_fbc_function_list.nvFBCToGLSetUp(self->nv_fbc_handle, &self->setup_params); if(status != NVFBC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", cap_nvfbc->nv_fbc_function_list.nvFBCGetLastErrorStr(cap_nvfbc->nv_fbc_handle)); - gsr_capture_nvfbc_destroy_session(cap_nvfbc); + fprintf(stderr, "gsr error: gsr_capture_nvfbc_start failed: %s\n", self->nv_fbc_function_list.nvFBCGetLastErrorStr(self->nv_fbc_handle)); + gsr_capture_nvfbc_destroy_session(self); return -1; } return 0; } -static void gsr_capture_nvfbc_stop(gsr_capture_nvfbc *cap_nvfbc) { - gsr_capture_nvfbc_destroy_session_and_handle(cap_nvfbc); - if(cap_nvfbc->library) { - dlclose(cap_nvfbc->library); - cap_nvfbc->library = NULL; +static void gsr_capture_nvfbc_stop(gsr_capture_nvfbc *self) { + gsr_capture_nvfbc_destroy_session_and_handle(self); + if(self->library) { + dlclose(self->library); + self->library = NULL; } - if(cap_nvfbc->params.display_to_capture) { - free((void*)cap_nvfbc->params.display_to_capture); - cap_nvfbc->params.display_to_capture = NULL; + if(self->params.display_to_capture) { + free((void*)self->params.display_to_capture); + self->params.display_to_capture = NULL; } } static int gsr_capture_nvfbc_start(gsr_capture *cap, AVCodecContext *video_codec_context, AVFrame *frame) { - gsr_capture_nvfbc *cap_nvfbc = cap->priv; + gsr_capture_nvfbc *self = cap->priv; if(!gsr_capture_nvfbc_load_library(cap)) return -1; - cap_nvfbc->x = max_int(cap_nvfbc->params.pos.x, 0); - cap_nvfbc->y = max_int(cap_nvfbc->params.pos.y, 0); - cap_nvfbc->width = max_int(cap_nvfbc->params.size.x, 0); - cap_nvfbc->height = max_int(cap_nvfbc->params.size.y, 0); + self->x = max_int(self->params.pos.x, 0); + self->y = max_int(self->params.pos.y, 0); + self->width = max_int(self->params.size.x, 0); + self->height = max_int(self->params.size.y, 0); - cap_nvfbc->capture_region = (cap_nvfbc->x > 0 || cap_nvfbc->y > 0 || cap_nvfbc->width > 0 || cap_nvfbc->height > 0); + self->capture_region = (self->x > 0 || self->y > 0 || self->width > 0 || self->height > 0); - cap_nvfbc->supports_direct_cursor = false; + self->supports_direct_cursor = false; int driver_major_version = 0; int driver_minor_version = 0; - if(cap_nvfbc->params.direct_capture && get_driver_version(&driver_major_version, &driver_minor_version)) { + if(self->params.direct_capture && get_driver_version(&driver_major_version, &driver_minor_version)) { fprintf(stderr, "Info: detected nvidia version: %d.%d\n", driver_major_version, driver_minor_version); // TODO: if(version_at_least(driver_major_version, driver_minor_version, 515, 57) && version_less_than(driver_major_version, driver_minor_version, 520, 56)) { - cap_nvfbc->params.direct_capture = false; + self->params.direct_capture = false; fprintf(stderr, "Warning: \"screen-direct\" has temporary been disabled as it causes stuttering with driver versions >= 515.57 and < 520.56. Please update your driver if possible. Capturing \"screen\" instead.\n"); } @@ -328,68 +333,83 @@ static int gsr_capture_nvfbc_start(gsr_capture *cap, AVCodecContext *video_codec /* if(direct_capture) { if(version_at_least(driver_major_version, driver_minor_version, 515, 57)) - cap_nvfbc->supports_direct_cursor = true; + self->supports_direct_cursor = true; else fprintf(stderr, "Info: capturing \"screen-direct\" but driver version appears to be less than 515.57. Disabling capture of cursor. Please update your driver if you want to capture your cursor or record \"screen\" instead.\n"); } */ } - if(gsr_capture_nvfbc_setup_handle(cap_nvfbc) != 0) { + if(gsr_capture_nvfbc_setup_handle(self) != 0) { goto error_cleanup; } - if(gsr_capture_nvfbc_setup_session(cap_nvfbc) != 0) { + if(gsr_capture_nvfbc_setup_session(self) != 0) { goto error_cleanup; } - if(cap_nvfbc->capture_region) { - video_codec_context->width = FFALIGN(cap_nvfbc->width, 2); - video_codec_context->height = FFALIGN(cap_nvfbc->height, 2); + if(self->capture_region) { + video_codec_context->width = FFALIGN(self->width, 2); + video_codec_context->height = FFALIGN(self->height, 2); + } else { + video_codec_context->width = FFALIGN(self->tracking_width, 2); + video_codec_context->height = FFALIGN(self->tracking_height, 2); + } + + if(self->params.output_resolution.x == 0 && self->params.output_resolution.y == 0) { + self->params.output_resolution = (vec2i){video_codec_context->width, video_codec_context->height}; } else { - video_codec_context->width = FFALIGN(cap_nvfbc->tracking_width, 2); - video_codec_context->height = FFALIGN(cap_nvfbc->tracking_height, 2); + self->params.output_resolution = scale_keep_aspect_ratio((vec2i){video_codec_context->width, video_codec_context->height}, self->params.output_resolution); + video_codec_context->width = FFALIGN(self->params.output_resolution.x, 2); + video_codec_context->height = FFALIGN(self->params.output_resolution.y, 2); } frame->width = video_codec_context->width; frame->height = video_codec_context->height; /* Disable vsync */ - set_vertical_sync_enabled(cap_nvfbc->params.egl, 0); + set_vertical_sync_enabled(self->params.egl, 0); return 0; error_cleanup: - gsr_capture_nvfbc_stop(cap_nvfbc); + gsr_capture_nvfbc_stop(self); return -1; } static int gsr_capture_nvfbc_capture(gsr_capture *cap, AVFrame *frame, gsr_color_conversion *color_conversion) { - gsr_capture_nvfbc *cap_nvfbc = cap->priv; + gsr_capture_nvfbc *self = cap->priv; const double nvfbc_recreate_retry_time_seconds = 1.0; - if(cap_nvfbc->nvfbc_needs_recreate) { + if(self->nvfbc_needs_recreate) { const double now = clock_get_monotonic_seconds(); - if(now - cap_nvfbc->nvfbc_dead_start >= nvfbc_recreate_retry_time_seconds) { - cap_nvfbc->nvfbc_dead_start = now; - gsr_capture_nvfbc_destroy_session_and_handle(cap_nvfbc); + if(now - self->nvfbc_dead_start >= nvfbc_recreate_retry_time_seconds) { + self->nvfbc_dead_start = now; + gsr_capture_nvfbc_destroy_session_and_handle(self); - if(gsr_capture_nvfbc_setup_handle(cap_nvfbc) != 0) { + if(gsr_capture_nvfbc_setup_handle(self) != 0) { fprintf(stderr, "gsr error: gsr_capture_nvfbc_capture failed to recreate nvfbc handle, trying again in %f second(s)\n", nvfbc_recreate_retry_time_seconds); return -1; } - - if(gsr_capture_nvfbc_setup_session(cap_nvfbc) != 0) { + + if(gsr_capture_nvfbc_setup_session(self) != 0) { fprintf(stderr, "gsr error: gsr_capture_nvfbc_capture failed to recreate nvfbc session, trying again in %f second(s)\n", nvfbc_recreate_retry_time_seconds); return -1; } - cap_nvfbc->nvfbc_needs_recreate = false; + self->nvfbc_needs_recreate = false; } else { return 0; } } + const vec2i frame_size = (vec2i){self->width, self->height}; + const bool is_scaled = self->params.output_resolution.x > 0 && self->params.output_resolution.y > 0; + vec2i output_size = is_scaled ? self->params.output_resolution : frame_size; + output_size = scale_keep_aspect_ratio(frame_size, output_size); + + const vec2i target_pos = { max_int(0, frame->width / 2 - output_size.x / 2), max_int(0, frame->height / 2 - output_size.y / 2) }; + NVFBC_FRAME_GRAB_INFO frame_info; memset(&frame_info, 0, sizeof(frame_info)); @@ -400,37 +420,33 @@ static int gsr_capture_nvfbc_capture(gsr_capture *cap, AVFrame *frame, gsr_color grab_params.pFrameGrabInfo = &frame_info; grab_params.dwTimeoutMs = 0; - NVFBCSTATUS status = cap_nvfbc->nv_fbc_function_list.nvFBCToGLGrabFrame(cap_nvfbc->nv_fbc_handle, &grab_params); + NVFBCSTATUS status = self->nv_fbc_function_list.nvFBCToGLGrabFrame(self->nv_fbc_handle, &grab_params); if(status != NVFBC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_capture_nvfbc_capture failed: %s (%d), recreating session after %f second(s)\n", cap_nvfbc->nv_fbc_function_list.nvFBCGetLastErrorStr(cap_nvfbc->nv_fbc_handle), status, nvfbc_recreate_retry_time_seconds); - cap_nvfbc->nvfbc_needs_recreate = true; - cap_nvfbc->nvfbc_dead_start = clock_get_monotonic_seconds(); + fprintf(stderr, "gsr error: gsr_capture_nvfbc_capture failed: %s (%d), recreating session after %f second(s)\n", self->nv_fbc_function_list.nvFBCGetLastErrorStr(self->nv_fbc_handle), status, nvfbc_recreate_retry_time_seconds); + self->nvfbc_needs_recreate = true; + self->nvfbc_dead_start = clock_get_monotonic_seconds(); return 0; } - cap_nvfbc->params.egl->glFlush(); - cap_nvfbc->params.egl->glFinish(); + self->params.egl->glFlush(); + self->params.egl->glFinish(); - gsr_color_conversion_draw(color_conversion, cap_nvfbc->setup_params.dwTextures[grab_params.dwTextureIndex], - (vec2i){0, 0}, (vec2i){frame->width, frame->height}, - (vec2i){0, 0}, (vec2i){frame->width, frame->height}, - 0.0f, false); + gsr_color_conversion_draw(color_conversion, self->setup_params.dwTextures[grab_params.dwTextureIndex], + target_pos, (vec2i){output_size.x, output_size.y}, + (vec2i){0, 0}, frame_size, + 0.0f, false, GSR_SOURCE_COLOR_BGR); - cap_nvfbc->params.egl->glFlush(); - cap_nvfbc->params.egl->glFinish(); + self->params.egl->glFlush(); + self->params.egl->glFinish(); return 0; } -static gsr_source_color gsr_capture_nvfbc_get_source_color(gsr_capture *cap) { - (void)cap; - return GSR_SOURCE_COLOR_BGR; -} - static void gsr_capture_nvfbc_destroy(gsr_capture *cap, AVCodecContext *video_codec_context) { (void)video_codec_context; - gsr_capture_nvfbc *cap_nvfbc = cap->priv; - gsr_capture_nvfbc_stop(cap_nvfbc); + gsr_capture_nvfbc *self = cap->priv; + gsr_capture_nvfbc_stop(self); + free(cap->priv); free(cap); } @@ -465,13 +481,12 @@ gsr_capture* gsr_capture_nvfbc_create(const gsr_capture_nvfbc_params *params) { cap_nvfbc->params = *params; cap_nvfbc->params.display_to_capture = display_to_capture; cap_nvfbc->params.fps = max_int(cap_nvfbc->params.fps, 1); - + *cap = (gsr_capture) { .start = gsr_capture_nvfbc_start, .tick = NULL, .should_stop = NULL, .capture = gsr_capture_nvfbc_capture, - .get_source_color = gsr_capture_nvfbc_get_source_color, .uses_external_image = NULL, .destroy = gsr_capture_nvfbc_destroy, .priv = cap_nvfbc diff --git a/src/capture/portal.c b/src/capture/portal.c index 40e4581..d68e86f 100644 --- a/src/capture/portal.c +++ b/src/capture/portal.c @@ -3,7 +3,7 @@ #include "../../include/egl.h" #include "../../include/utils.h" #include "../../include/dbus.h" -#include "../../include/pipewire.h" +#include "../../include/pipewire_video.h" #include <stdlib.h> #include <stdio.h> @@ -20,12 +20,13 @@ typedef struct { gsr_dbus dbus; char *session_handle; - gsr_pipewire pipewire; + gsr_pipewire_video pipewire; vec2i capture_size; - gsr_pipewire_dmabuf_data dmabuf_data[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; + gsr_pipewire_video_dmabuf_data dmabuf_data[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; int num_dmabuf_data; AVCodecContext *video_codec_context; + bool fast_path_failed; } gsr_capture_portal; static void gsr_capture_portal_cleanup_plane_fds(gsr_capture_portal *self) { @@ -56,7 +57,7 @@ static void gsr_capture_portal_stop(gsr_capture_portal *self) { gsr_capture_portal_cleanup_plane_fds(self); - gsr_pipewire_deinit(&self->pipewire); + gsr_pipewire_video_deinit(&self->pipewire); if(self->session_handle) { free(self->session_handle); @@ -232,8 +233,8 @@ static int gsr_capture_portal_setup_dbus(gsr_capture_portal *self, int *pipewire } static bool gsr_capture_portal_get_frame_dimensions(gsr_capture_portal *self) { - gsr_pipewire_region region = {0, 0, 0, 0}; - gsr_pipewire_region cursor_region = {0, 0, 0, 0}; + gsr_pipewire_video_region region = {0, 0, 0, 0}; + gsr_pipewire_video_region cursor_region = {0, 0, 0, 0}; fprintf(stderr, "gsr info: gsr_capture_portal_start: waiting for pipewire negotiation\n"); const double start_time = clock_get_monotonic_seconds(); @@ -241,7 +242,7 @@ static bool gsr_capture_portal_get_frame_dimensions(gsr_capture_portal *self) { bool uses_external_image = false; uint32_t fourcc = 0; uint64_t modifiers = 0; - if(gsr_pipewire_map_texture(&self->pipewire, self->texture_map, ®ion, &cursor_region, self->dmabuf_data, &self->num_dmabuf_data, &fourcc, &modifiers, &uses_external_image)) { + if(gsr_pipewire_video_map_texture(&self->pipewire, self->texture_map, ®ion, &cursor_region, self->dmabuf_data, &self->num_dmabuf_data, &fourcc, &modifiers, &uses_external_image)) { gsr_capture_portal_cleanup_plane_fds(self); self->capture_size.x = region.width; self->capture_size.y = region.height; @@ -284,7 +285,7 @@ static int gsr_capture_portal_start(gsr_capture *cap, AVCodecContext *video_code fprintf(stderr, "gsr info: gsr_capture_portal_start: setting up pipewire\n"); /* TODO: support hdr when pipewire supports it */ /* gsr_pipewire closes the pipewire fd, even on failure */ - if(!gsr_pipewire_init(&self->pipewire, pipewire_fd, pipewire_node, video_codec_context->framerate.num, self->params.record_cursor, self->params.egl)) { + if(!gsr_pipewire_video_init(&self->pipewire, pipewire_fd, pipewire_node, video_codec_context->framerate.num, self->params.record_cursor, self->params.egl)) { fprintf(stderr, "gsr error: gsr_capture_portal_start: failed to setup pipewire with fd: %d, node: %" PRIu32 "\n", pipewire_fd, pipewire_node); gsr_capture_portal_stop(self); return -1; @@ -299,8 +300,15 @@ static int gsr_capture_portal_start(gsr_capture *cap, AVCodecContext *video_code /* Disable vsync */ self->params.egl->eglSwapInterval(self->params.egl->egl_display, 0); - video_codec_context->width = FFALIGN(self->capture_size.x, 2); - video_codec_context->height = FFALIGN(self->capture_size.y, 2); + if(self->params.output_resolution.x == 0 && self->params.output_resolution.y == 0) { + self->params.output_resolution = self->capture_size; + video_codec_context->width = FFALIGN(self->capture_size.x, 2); + video_codec_context->height = FFALIGN(self->capture_size.y, 2); + } else { + self->params.output_resolution = scale_keep_aspect_ratio(self->capture_size, self->params.output_resolution); + video_codec_context->width = FFALIGN(self->params.output_resolution.x, 2); + video_codec_context->height = FFALIGN(self->params.output_resolution.y, 2); + } frame->width = video_codec_context->width; frame->height = video_codec_context->height; @@ -319,12 +327,12 @@ static int gsr_capture_portal_capture(gsr_capture *cap, AVFrame *frame, gsr_colo gsr_capture_portal *self = cap->priv; /* TODO: Handle formats other than RGB(a) */ - gsr_pipewire_region region = {0, 0, 0, 0}; - gsr_pipewire_region cursor_region = {0, 0, 0, 0}; + gsr_pipewire_video_region region = {0, 0, 0, 0}; + gsr_pipewire_video_region cursor_region = {0, 0, 0, 0}; uint32_t pipewire_fourcc = 0; uint64_t pipewire_modifiers = 0; bool using_external_image = false; - if(gsr_pipewire_map_texture(&self->pipewire, self->texture_map, ®ion, &cursor_region, self->dmabuf_data, &self->num_dmabuf_data, &pipewire_fourcc, &pipewire_modifiers, &using_external_image)) { + if(gsr_pipewire_video_map_texture(&self->pipewire, self->texture_map, ®ion, &cursor_region, self->dmabuf_data, &self->num_dmabuf_data, &pipewire_fourcc, &pipewire_modifiers, &using_external_image)) { if(region.width != self->capture_size.x || region.height != self->capture_size.y) { self->capture_size.x = region.width; self->capture_size.y = region.height; @@ -334,15 +342,19 @@ static int gsr_capture_portal_capture(gsr_capture *cap, AVFrame *frame, gsr_colo return 0; } + const bool is_scaled = self->params.output_resolution.x > 0 && self->params.output_resolution.y > 0; + vec2i output_size = is_scaled ? self->params.output_resolution : self->capture_size; + output_size = scale_keep_aspect_ratio(self->capture_size, output_size); + + const vec2i target_pos = { max_int(0, frame->width / 2 - output_size.x / 2), max_int(0, frame->height / 2 - output_size.y / 2) }; + self->params.egl->glFlush(); self->params.egl->glFinish(); - - const vec2i target_pos = { max_int(0, frame->width / 2 - self->capture_size.x / 2), max_int(0, frame->height / 2 - self->capture_size.y / 2) }; // TODO: Handle region crop /* Fast opengl free path */ - if(video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { + if(!self->fast_path_failed && video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { int fds[4]; uint32_t offsets[4]; uint32_t pitches[4]; @@ -353,26 +365,38 @@ static int gsr_capture_portal_capture(gsr_capture *cap, AVFrame *frame, gsr_colo pitches[i] = self->dmabuf_data[i].stride; modifiers[i] = pipewire_modifiers; } - vaapi_copy_drm_planes_to_video_surface(self->video_codec_context, frame, (vec2i){region.x, region.y}, self->capture_size, target_pos, self->capture_size, pipewire_fourcc, self->capture_size, fds, offsets, pitches, modifiers, self->num_dmabuf_data); + if(!vaapi_copy_drm_planes_to_video_surface(self->video_codec_context, frame, (vec2i){region.x, region.y}, self->capture_size, target_pos, output_size, pipewire_fourcc, self->capture_size, fds, offsets, pitches, modifiers, self->num_dmabuf_data)) { + fprintf(stderr, "gsr error: gsr_capture_portal_capture: vaapi_copy_drm_planes_to_video_surface failed, falling back to opengl copy. Please report this as an issue at https://github.com/dec05eba/gpu-screen-recorder-issues\n"); + self->fast_path_failed = true; + } } else { + self->fast_path_failed = true; + } + + if(self->fast_path_failed) { gsr_color_conversion_draw(color_conversion, using_external_image ? self->texture_map.external_texture_id : self->texture_map.texture_id, - target_pos, self->capture_size, + target_pos, output_size, (vec2i){region.x, region.y}, self->capture_size, - 0.0f, using_external_image); + 0.0f, using_external_image, GSR_SOURCE_COLOR_RGB); } - if(self->params.record_cursor) { + if(self->params.record_cursor && self->texture_map.cursor_texture_id > 0 && cursor_region.width > 0) { + const vec2d scale = { + self->capture_size.x == 0 ? 0 : (double)output_size.x / (double)self->capture_size.x, + self->capture_size.y == 0 ? 0 : (double)output_size.y / (double)self->capture_size.y + }; + const vec2i cursor_pos = { - target_pos.x + cursor_region.x, - target_pos.y + cursor_region.y + target_pos.x + (cursor_region.x * scale.x), + target_pos.y + (cursor_region.y * scale.y) }; self->params.egl->glEnable(GL_SCISSOR_TEST); - self->params.egl->glScissor(target_pos.x, target_pos.y, self->capture_size.x, self->capture_size.y); + self->params.egl->glScissor(target_pos.x, target_pos.y, output_size.x, output_size.y); gsr_color_conversion_draw(color_conversion, self->texture_map.cursor_texture_id, - (vec2i){cursor_pos.x, cursor_pos.y}, (vec2i){cursor_region.width, cursor_region.height}, + (vec2i){cursor_pos.x, cursor_pos.y}, (vec2i){cursor_region.width * scale.x, cursor_region.height * scale.y}, (vec2i){0, 0}, (vec2i){cursor_region.width, cursor_region.height}, - 0.0f, false); + 0.0f, false, GSR_SOURCE_COLOR_RGB); self->params.egl->glDisable(GL_SCISSOR_TEST); } @@ -384,11 +408,6 @@ static int gsr_capture_portal_capture(gsr_capture *cap, AVFrame *frame, gsr_colo return 0; } -static gsr_source_color gsr_capture_portal_get_source_color(gsr_capture *cap) { - (void)cap; - return GSR_SOURCE_COLOR_RGB; -} - static bool gsr_capture_portal_uses_external_image(gsr_capture *cap) { (void)cap; return true; @@ -396,19 +415,19 @@ static bool gsr_capture_portal_uses_external_image(gsr_capture *cap) { static bool gsr_capture_portal_is_damaged(gsr_capture *cap) { gsr_capture_portal *self = cap->priv; - return gsr_pipewire_is_damaged(&self->pipewire); + return gsr_pipewire_video_is_damaged(&self->pipewire); } static void gsr_capture_portal_clear_damage(gsr_capture *cap) { gsr_capture_portal *self = cap->priv; - gsr_pipewire_clear_damage(&self->pipewire); + gsr_pipewire_video_clear_damage(&self->pipewire); } static void gsr_capture_portal_destroy(gsr_capture *cap, AVCodecContext *video_codec_context) { (void)video_codec_context; - gsr_capture_portal *cap_portal = cap->priv; + gsr_capture_portal *self = cap->priv; if(cap->priv) { - gsr_capture_portal_stop(cap_portal); + gsr_capture_portal_stop(self); free(cap->priv); cap->priv = NULL; } @@ -438,7 +457,6 @@ gsr_capture* gsr_capture_portal_create(const gsr_capture_portal_params *params) .tick = NULL, .should_stop = NULL, .capture = gsr_capture_portal_capture, - .get_source_color = gsr_capture_portal_get_source_color, .uses_external_image = gsr_capture_portal_uses_external_image, .is_damaged = gsr_capture_portal_is_damaged, .clear_damage = gsr_capture_portal_clear_damage, diff --git a/src/capture/xcomposite.c b/src/capture/xcomposite.c index 9d053bb..2867b45 100644 --- a/src/capture/xcomposite.c +++ b/src/capture/xcomposite.c @@ -36,6 +36,7 @@ typedef struct { gsr_cursor cursor; bool clear_background; + bool fast_path_failed; } gsr_capture_xcomposite; static void gsr_capture_xcomposite_stop(gsr_capture_xcomposite *self) { @@ -112,13 +113,14 @@ static int gsr_capture_xcomposite_start(gsr_capture *cap, AVCodecContext *video_ self->params.egl->glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT, &self->texture_size.y); self->params.egl->glBindTexture(GL_TEXTURE_2D, 0); - vec2i video_size = self->texture_size; - - if(self->params.region_size.x > 0 && self->params.region_size.y > 0) - video_size = self->params.region_size; - - video_codec_context->width = FFALIGN(video_size.x, 2); - video_codec_context->height = FFALIGN(video_size.y, 2); + if(self->params.output_resolution.x == 0 && self->params.output_resolution.y == 0) { + self->params.output_resolution = self->texture_size; + video_codec_context->width = FFALIGN(self->texture_size.x, 2); + video_codec_context->height = FFALIGN(self->texture_size.y, 2); + } else { + video_codec_context->width = FFALIGN(self->params.output_resolution.x, 2); + video_codec_context->height = FFALIGN(self->params.output_resolution.y, 2); + } frame->width = video_codec_context->width; frame->height = video_codec_context->height; @@ -256,36 +258,52 @@ static int gsr_capture_xcomposite_capture(gsr_capture *cap, AVFrame *frame, gsr_ gsr_color_conversion_clear(color_conversion); } - const vec2i target_pos = { max_int(0, frame->width / 2 - self->texture_size.x / 2), max_int(0, frame->height / 2 - self->texture_size.y / 2) }; + const bool is_scaled = self->params.output_resolution.x > 0 && self->params.output_resolution.y > 0; + vec2i output_size = is_scaled ? self->params.output_resolution : self->texture_size; + output_size = scale_keep_aspect_ratio(self->texture_size, output_size); + + const vec2i target_pos = { max_int(0, frame->width / 2 - output_size.x / 2), max_int(0, frame->height / 2 - output_size.y / 2) }; self->params.egl->glFlush(); self->params.egl->glFinish(); /* Fast opengl free path */ - if(video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { - vaapi_copy_egl_image_to_video_surface(self->params.egl, self->window_texture.image, (vec2i){0, 0}, self->texture_size, target_pos, self->texture_size, self->video_codec_context, frame); + if(!self->fast_path_failed && video_codec_context_is_vaapi(self->video_codec_context) && self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD) { + if(!vaapi_copy_egl_image_to_video_surface(self->params.egl, self->window_texture.image, (vec2i){0, 0}, self->texture_size, target_pos, output_size, self->video_codec_context, frame)) { + fprintf(stderr, "gsr error: gsr_capture_xcomposite_capture: vaapi_copy_egl_image_to_video_surface failed, falling back to opengl copy. Please report this as an issue at https://github.com/dec05eba/gpu-screen-recorder-issues\n"); + self->fast_path_failed = true; + } } else { + self->fast_path_failed = true; + } + + if(self->fast_path_failed) { gsr_color_conversion_draw(color_conversion, window_texture_get_opengl_texture_id(&self->window_texture), - target_pos, self->texture_size, + target_pos, output_size, (vec2i){0, 0}, self->texture_size, - 0.0f, false); + 0.0f, false, GSR_SOURCE_COLOR_RGB); } if(self->params.record_cursor && self->cursor.visible) { + const vec2d scale = { + self->texture_size.x == 0 ? 0 : (double)output_size.x / (double)self->texture_size.x, + self->texture_size.y == 0 ? 0 : (double)output_size.y / (double)self->texture_size.y + }; + gsr_cursor_tick(&self->cursor, self->window); const vec2i cursor_pos = { - target_pos.x + self->cursor.position.x - self->cursor.hotspot.x, - target_pos.y + self->cursor.position.y - self->cursor.hotspot.y + target_pos.x + (self->cursor.position.x - self->cursor.hotspot.x) * scale.x, + target_pos.y + (self->cursor.position.y - self->cursor.hotspot.y) * scale.y }; self->params.egl->glEnable(GL_SCISSOR_TEST); - self->params.egl->glScissor(target_pos.x, target_pos.y, self->texture_size.x, self->texture_size.y); + self->params.egl->glScissor(target_pos.x, target_pos.y, output_size.x, output_size.y); gsr_color_conversion_draw(color_conversion, self->cursor.texture_id, - cursor_pos, self->cursor.size, + cursor_pos, (vec2i){self->cursor.size.x * scale.x, self->cursor.size.y * scale.y}, (vec2i){0, 0}, self->cursor.size, - 0.0f, false); + 0.0f, false, GSR_SOURCE_COLOR_RGB); self->params.egl->glDisable(GL_SCISSOR_TEST); } @@ -296,11 +314,6 @@ static int gsr_capture_xcomposite_capture(gsr_capture *cap, AVFrame *frame, gsr_ return 0; } -static gsr_source_color gsr_capture_xcomposite_get_source_color(gsr_capture *cap) { - (void)cap; - return GSR_SOURCE_COLOR_RGB; -} - static uint64_t gsr_capture_xcomposite_get_window_id(gsr_capture *cap) { gsr_capture_xcomposite *self = cap->priv; return self->window; @@ -340,7 +353,6 @@ gsr_capture* gsr_capture_xcomposite_create(const gsr_capture_xcomposite_params * .tick = gsr_capture_xcomposite_tick, .should_stop = gsr_capture_xcomposite_should_stop, .capture = gsr_capture_xcomposite_capture, - .get_source_color = gsr_capture_xcomposite_get_source_color, .uses_external_image = NULL, .get_window_id = gsr_capture_xcomposite_get_window_id, .destroy = gsr_capture_xcomposite_destroy, diff --git a/src/codec_query/nvenc.c b/src/codec_query/nvenc.c new file mode 100644 index 0000000..0501851 --- /dev/null +++ b/src/codec_query/nvenc.c @@ -0,0 +1,235 @@ +#include "../../include/codec_query/nvenc.h" +#include "../../include/cuda.h" +#include "../../external/nvEncodeAPI.h" + +#include <dlfcn.h> +#include <stdio.h> +#include <string.h> + +static void* open_nvenc_library(void) { + dlerror(); /* clear */ + void *lib = dlopen("libnvidia-encode.so.1", RTLD_LAZY); + if(!lib) { + lib = dlopen("libnvidia-encode.so", RTLD_LAZY); + if(!lib) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc failed: failed to load libnvidia-encode.so/libnvidia-encode.so.1, error: %s\n", dlerror()); + return NULL; + } + } + return lib; +} + +static bool profile_is_h264(const GUID *profile_guid) { + const GUID *h264_guids[] = { + &NV_ENC_H264_PROFILE_BASELINE_GUID, + &NV_ENC_H264_PROFILE_MAIN_GUID, + &NV_ENC_H264_PROFILE_HIGH_GUID, + &NV_ENC_H264_PROFILE_PROGRESSIVE_HIGH_GUID, + &NV_ENC_H264_PROFILE_CONSTRAINED_HIGH_GUID + }; + + for(int i = 0; i < 5; ++i) { + if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) + return true; + } + + return false; +} + +static bool profile_is_hevc(const GUID *profile_guid) { + const GUID *h264_guids[] = { + &NV_ENC_HEVC_PROFILE_MAIN_GUID, + }; + + for(int i = 0; i < 1; ++i) { + if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) + return true; + } + + return false; +} + +static bool profile_is_hevc_10bit(const GUID *profile_guid) { + const GUID *h264_guids[] = { + &NV_ENC_HEVC_PROFILE_MAIN10_GUID, + }; + + for(int i = 0; i < 1; ++i) { + if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) + return true; + } + + return false; +} + +static bool profile_is_av1(const GUID *profile_guid) { + const GUID *h264_guids[] = { + &NV_ENC_AV1_PROFILE_MAIN_GUID, + }; + + for(int i = 0; i < 1; ++i) { + if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) + return true; + } + + return false; +} + +static bool encoder_get_supported_profiles(const NV_ENCODE_API_FUNCTION_LIST *function_list, void *nvenc_encoder, const GUID *encoder_guid, gsr_supported_video_codecs *supported_video_codecs) { + bool success = false; + GUID *profile_guids = NULL; + + uint32_t profile_guid_count = 0; + if(function_list->nvEncGetEncodeProfileGUIDCount(nvenc_encoder, *encoder_guid, &profile_guid_count) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncGetEncodeProfileGUIDCount failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); + goto fail; + } + + if(profile_guid_count == 0) + goto fail; + + profile_guids = calloc(profile_guid_count, sizeof(GUID)); + if(!profile_guids) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: failed to allocate %d guids\n", (int)profile_guid_count); + goto fail; + } + + if(function_list->nvEncGetEncodeProfileGUIDs(nvenc_encoder, *encoder_guid, profile_guids, profile_guid_count, &profile_guid_count) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncGetEncodeProfileGUIDs failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); + goto fail; + } + + for(uint32_t i = 0; i < profile_guid_count; ++i) { + if(profile_is_h264(&profile_guids[i])) { + supported_video_codecs->h264 = (gsr_supported_video_codec){ true, false }; + } else if(profile_is_hevc(&profile_guids[i])) { + supported_video_codecs->hevc = (gsr_supported_video_codec){ true, false }; + } else if(profile_is_hevc_10bit(&profile_guids[i])) { + supported_video_codecs->hevc_hdr = (gsr_supported_video_codec){ true, false }; + supported_video_codecs->hevc_10bit = (gsr_supported_video_codec){ true, false }; + } else if(profile_is_av1(&profile_guids[i])) { + supported_video_codecs->av1 = (gsr_supported_video_codec){ true, false }; + supported_video_codecs->av1_hdr = (gsr_supported_video_codec){ true, false }; + supported_video_codecs->av1_10bit = (gsr_supported_video_codec){ true, false }; + } + } + + success = true; + fail: + + if(profile_guids) + free(profile_guids); + + return success; +} + +static bool get_supported_video_codecs(const NV_ENCODE_API_FUNCTION_LIST *function_list, void *nvenc_encoder, gsr_supported_video_codecs *supported_video_codecs) { + bool success = false; + GUID *encoder_guids = NULL; + *supported_video_codecs = (gsr_supported_video_codecs){0}; + + uint32_t encode_guid_count = 0; + if(function_list->nvEncGetEncodeGUIDCount(nvenc_encoder, &encode_guid_count) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncGetEncodeGUIDCount failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); + goto fail; + } + + if(encode_guid_count == 0) + goto fail; + + encoder_guids = calloc(encode_guid_count, sizeof(GUID)); + if(!encoder_guids) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: failed to allocate %d guids\n", (int)encode_guid_count); + goto fail; + } + + if(function_list->nvEncGetEncodeGUIDs(nvenc_encoder, encoder_guids, encode_guid_count, &encode_guid_count) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncGetEncodeGUIDs failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); + goto fail; + } + + for(uint32_t i = 0; i < encode_guid_count; ++i) { + encoder_get_supported_profiles(function_list, nvenc_encoder, &encoder_guids[i], supported_video_codecs); + } + + success = true; + fail: + + if(encoder_guids) + free(encoder_guids); + + return success; +} + +#define NVENCAPI_VERSION_470 (11 | (1 << 24)) +#define NVENCAPI_STRUCT_VERSION_470(ver) ((uint32_t)NVENCAPI_VERSION_470 | ((ver)<<16) | (0x7 << 28)) + +bool gsr_get_supported_video_codecs_nvenc(gsr_supported_video_codecs *video_codecs, bool cleanup) { + memset(video_codecs, 0, sizeof(*video_codecs)); + + bool success = false; + void *nvenc_lib = NULL; + void *nvenc_encoder = NULL; + gsr_cuda cuda; + memset(&cuda, 0, sizeof(cuda)); + + if(!gsr_cuda_load(&cuda, NULL, false)) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: failed to load cuda\n"); + goto done; + } + + nvenc_lib = open_nvenc_library(); + if(!nvenc_lib) + goto done; + + typedef NVENCSTATUS NVENCAPI (*FUNC_NvEncodeAPICreateInstance)(NV_ENCODE_API_FUNCTION_LIST *functionList); + FUNC_NvEncodeAPICreateInstance nvEncodeAPICreateInstance = (FUNC_NvEncodeAPICreateInstance)dlsym(nvenc_lib, "NvEncodeAPICreateInstance"); + if(!nvEncodeAPICreateInstance) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: failed to find NvEncodeAPICreateInstance in libnvidia-encode.so\n"); + goto done; + } + + NV_ENCODE_API_FUNCTION_LIST function_list; + memset(&function_list, 0, sizeof(function_list)); + function_list.version = NVENCAPI_STRUCT_VERSION(2); + if(nvEncodeAPICreateInstance(&function_list) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncodeAPICreateInstance failed\n"); + goto done; + } + + NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS params; + memset(¶ms, 0, sizeof(params)); + params.version = NVENCAPI_STRUCT_VERSION(1); + params.deviceType = NV_ENC_DEVICE_TYPE_CUDA; + params.device = cuda.cu_ctx; + params.apiVersion = NVENCAPI_VERSION; + if(function_list.nvEncOpenEncodeSessionEx(¶ms, &nvenc_encoder) != NV_ENC_SUCCESS) { + // Old nvidia gpus dont support the new nvenc api (which is required for av1). + // In such cases fallback to old api version if possible and try again. + function_list.version = NVENCAPI_STRUCT_VERSION_470(2); + if(nvEncodeAPICreateInstance(&function_list) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncodeAPICreateInstance (retry) failed\n"); + goto done; + } + + params.version = NVENCAPI_STRUCT_VERSION_470(1); + params.apiVersion = NVENCAPI_VERSION_470; + if(function_list.nvEncOpenEncodeSessionEx(¶ms, &nvenc_encoder) != NV_ENC_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_nvenc: nvEncOpenEncodeSessionEx (retry) failed\n"); + goto done; + } + } + + success = get_supported_video_codecs(&function_list, nvenc_encoder, video_codecs); + + done: + if(cleanup) { + if(nvenc_encoder) + function_list.nvEncDestroyEncoder(nvenc_encoder); + if(nvenc_lib) + dlclose(nvenc_lib); + gsr_cuda_unload(&cuda); + } + + return success; +} diff --git a/src/codec_query/vaapi.c b/src/codec_query/vaapi.c new file mode 100644 index 0000000..2c74d96 --- /dev/null +++ b/src/codec_query/vaapi.c @@ -0,0 +1,203 @@ +#include "../../include/codec_query/vaapi.h" +#include "../../include/utils.h" + +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> + +#include <va/va.h> +#include <va/va_drm.h> + +static bool profile_is_h264(VAProfile profile) { + switch(profile) { + case 5: // VAProfileH264Baseline + case VAProfileH264Main: + case VAProfileH264High: + case VAProfileH264ConstrainedBaseline: + return true; + default: + return false; + } +} + +static bool profile_is_hevc_8bit(VAProfile profile) { + switch(profile) { + case VAProfileHEVCMain: + return true; + default: + return false; + } +} + +static bool profile_is_hevc_10bit(VAProfile profile) { + switch(profile) { + case VAProfileHEVCMain10: + //case VAProfileHEVCMain12: + //case VAProfileHEVCMain422_10: + //case VAProfileHEVCMain422_12: + //case VAProfileHEVCMain444: + //case VAProfileHEVCMain444_10: + //case VAProfileHEVCMain444_12: + return true; + default: + return false; + } +} + +static bool profile_is_av1(VAProfile profile) { + switch(profile) { + case VAProfileAV1Profile0: + case VAProfileAV1Profile1: + return true; + default: + return false; + } +} + +static bool profile_is_vp8(VAProfile profile) { + switch(profile) { + case VAProfileVP8Version0_3: + return true; + default: + return false; + } +} + +static bool profile_is_vp9(VAProfile profile) { + switch(profile) { + case VAProfileVP9Profile0: + case VAProfileVP9Profile1: + case VAProfileVP9Profile2: + case VAProfileVP9Profile3: + return true; + default: + return false; + } +} + +static bool profile_supports_video_encoding(VADisplay va_dpy, VAProfile profile, bool *low_power) { + *low_power = false; + int num_entrypoints = vaMaxNumEntrypoints(va_dpy); + if(num_entrypoints <= 0) + return false; + + VAEntrypoint *entrypoint_list = calloc(num_entrypoints, sizeof(VAEntrypoint)); + if(!entrypoint_list) + return false; + + bool supports_encoding = false; + bool supports_low_power_encoding = false; + if(vaQueryConfigEntrypoints(va_dpy, profile, entrypoint_list, &num_entrypoints) == VA_STATUS_SUCCESS) { + for(int i = 0; i < num_entrypoints; ++i) { + if(entrypoint_list[i] == VAEntrypointEncSlice) + supports_encoding = true; + else if(entrypoint_list[i] == VAEntrypointEncSliceLP) + supports_low_power_encoding = true; + } + } + + if(!supports_encoding && supports_low_power_encoding) + *low_power = true; + + free(entrypoint_list); + return supports_encoding || supports_low_power_encoding; +} + +static bool get_supported_video_codecs(VADisplay va_dpy, gsr_supported_video_codecs *video_codecs, bool cleanup) { + *video_codecs = (gsr_supported_video_codecs){0}; + bool success = false; + VAProfile *profile_list = NULL; + + vaSetInfoCallback(va_dpy, NULL, NULL); + + int va_major = 0; + int va_minor = 0; + if(vaInitialize(va_dpy, &va_major, &va_minor) != VA_STATUS_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vaapi: vaInitialize failed\n"); + goto fail; + } + + int num_profiles = vaMaxNumProfiles(va_dpy); + if(num_profiles <= 0) + goto fail; + + profile_list = calloc(num_profiles, sizeof(VAProfile)); + if(!profile_list || vaQueryConfigProfiles(va_dpy, profile_list, &num_profiles) != VA_STATUS_SUCCESS) + goto fail; + + for(int i = 0; i < num_profiles; ++i) { + bool low_power = false; + if(profile_is_h264(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) { + video_codecs->h264 = (gsr_supported_video_codec){ true, low_power }; + } + } else if(profile_is_hevc_8bit(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) + video_codecs->hevc = (gsr_supported_video_codec){ true, low_power }; + } else if(profile_is_hevc_10bit(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) { + video_codecs->hevc_hdr = (gsr_supported_video_codec){ true, low_power }; + video_codecs->hevc_10bit = (gsr_supported_video_codec){ true, low_power }; + } + } else if(profile_is_av1(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) { + video_codecs->av1 = (gsr_supported_video_codec){ true, low_power }; + video_codecs->av1_hdr = (gsr_supported_video_codec){ true, low_power }; + video_codecs->av1_10bit = (gsr_supported_video_codec){ true, low_power }; + } + } else if(profile_is_vp8(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) + video_codecs->vp8 = (gsr_supported_video_codec){ true, low_power }; + } else if(profile_is_vp9(profile_list[i])) { + if(profile_supports_video_encoding(va_dpy, profile_list[i], &low_power)) + video_codecs->vp9 = (gsr_supported_video_codec){ true, low_power }; + } + } + + success = true; + fail: + if(profile_list) + free(profile_list); + + if(cleanup) + vaTerminate(va_dpy); + + return success; +} + +bool gsr_get_supported_video_codecs_vaapi(gsr_supported_video_codecs *video_codecs, const char *card_path, bool cleanup) { + memset(video_codecs, 0, sizeof(*video_codecs)); + bool success = false; + int drm_fd = -1; + + char render_path[128]; + if(!gsr_card_path_get_render_path(card_path, render_path)) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vaapi: failed to get /dev/dri/renderDXXX file from %s\n", card_path); + goto done; + } + + drm_fd = open(render_path, O_RDWR); + if(drm_fd == -1) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vaapi: failed to open device %s\n", render_path); + goto done; + } + + VADisplay va_dpy = vaGetDisplayDRM(drm_fd); + if(va_dpy) { + if(!get_supported_video_codecs(va_dpy, video_codecs, cleanup)) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vaapi: failed to query supported video codecs for device %s\n", render_path); + goto done; + } + success = true; + } + + done: + if(cleanup) { + if(drm_fd > 0) + close(drm_fd); + } + + return success; +} diff --git a/src/codec_query/vulkan.c b/src/codec_query/vulkan.c new file mode 100644 index 0000000..15dd98b --- /dev/null +++ b/src/codec_query/vulkan.c @@ -0,0 +1,156 @@ +#include "../../include/codec_query/vulkan.h" + +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <xf86drm.h> +#define VK_NO_PROTOTYPES +//#include <vulkan/vulkan.h> + +#define MAX_PHYSICAL_DEVICES 32 + +static const char *required_device_extensions[] = { + "VK_KHR_external_memory_fd", + "VK_KHR_external_semaphore_fd", + "VK_KHR_video_encode_queue", + "VK_KHR_video_queue", + "VK_KHR_video_maintenance1", + "VK_EXT_external_memory_dma_buf", + "VK_EXT_external_memory_host", + "VK_EXT_image_drm_format_modifier" +}; +static int num_required_device_extensions = 8; + +bool gsr_get_supported_video_codecs_vulkan(gsr_supported_video_codecs *video_codecs, const char *card_path, bool cleanup) { + memset(video_codecs, 0, sizeof(*video_codecs)); +#if 0 + bool success = false; + VkInstance instance = NULL; + VkPhysicalDevice physical_devices[MAX_PHYSICAL_DEVICES]; + VkDevice device = NULL; + VkExtensionProperties *device_extensions = NULL; + + const VkApplicationInfo app_info = { + .sType = VK_STRUCTURE_TYPE_APPLICATION_INFO, + .pApplicationName = "GPU Screen Recorder", + .applicationVersion = VK_MAKE_VERSION(1, 0, 0), + .pEngineName = "GPU Screen Recorder", + .engineVersion = VK_MAKE_VERSION(1, 0, 0), + .apiVersion = VK_API_VERSION_1_3, + }; + + const VkInstanceCreateInfo instance_create_info = { + .sType = VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO, + .pApplicationInfo = &app_info + }; + + if(vkCreateInstance(&instance_create_info, NULL, &instance) != VK_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkCreateInstance failed\n"); + goto done; + } + + uint32_t num_devices = 0; + if(vkEnumeratePhysicalDevices(instance, &num_devices, NULL) != VK_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkEnumeratePhysicalDevices (query num devices) failed\n"); + goto done; + } + + if(num_devices == 0) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: no vulkan capable device found\n"); + goto done; + } + + if(num_devices > MAX_PHYSICAL_DEVICES) + num_devices = MAX_PHYSICAL_DEVICES; + + if(vkEnumeratePhysicalDevices(instance, &num_devices, physical_devices) != VK_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkEnumeratePhysicalDevices (get data) failed\n"); + goto done; + } + + VkPhysicalDevice physical_device = NULL; + char device_card_path[128]; + for(uint32_t i = 0; i < num_devices; ++i) { + VkPhysicalDeviceDrmPropertiesEXT device_drm_properties = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DRM_PROPERTIES_EXT + }; + + VkPhysicalDeviceProperties2 device_properties = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2, + .pNext = &device_drm_properties + }; + vkGetPhysicalDeviceProperties2(physical_devices[i], &device_properties); + + if(!device_drm_properties.hasPrimary) + continue; + + snprintf(device_card_path, sizeof(device_card_path), DRM_DEV_NAME, DRM_DIR_NAME, (int)device_drm_properties.primaryMinor); + if(strcmp(device_card_path, card_path) == 0) { + physical_device = physical_devices[i]; + break; + } + } + + if(!physical_device) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: failed to find a vulkan device that matches opengl device %s\n", card_path); + goto done; + } + + const VkDeviceCreateInfo device_create_info = { + .sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO, + .enabledExtensionCount = num_required_device_extensions, + .ppEnabledExtensionNames = required_device_extensions + }; + + if(vkCreateDevice(physical_device, &device_create_info, NULL, &device) != VK_SUCCESS) { + //fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkCreateDevice failed. Device %s likely doesn't support vulkan video encoding\n", card_path); + goto done; + } + + uint32_t num_device_extensions = 0; + if(vkEnumerateDeviceExtensionProperties(physical_device, NULL, &num_device_extensions, NULL) != VK_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkEnumerateDeviceExtensionProperties (query num device extensions) failed\n"); + goto done; + } + + device_extensions = calloc(num_device_extensions, sizeof(VkExtensionProperties)); + if(!device_extensions) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: failed to allocate %d device extensions\n", num_device_extensions); + goto done; + } + + if(vkEnumerateDeviceExtensionProperties(physical_device, NULL, &num_device_extensions, device_extensions) != VK_SUCCESS) { + fprintf(stderr, "gsr error: gsr_get_supported_video_codecs_vulkan: vkEnumerateDeviceExtensionProperties (get data) failed\n"); + goto done; + } + + for(uint32_t i = 0; i < num_device_extensions; ++i) { + if(strcmp(device_extensions[i].extensionName, "VK_KHR_video_encode_h264") == 0) { + video_codecs->h264 = true; + } else if(strcmp(device_extensions[i].extensionName, "VK_KHR_video_encode_h265") == 0) { + // TODO: Verify if 10bit and hdr are actually supported + video_codecs->hevc = true; + video_codecs->hevc_10bit = true; + video_codecs->hevc_hdr = true; + } + } + + success = true; + + done: + if(cleanup) { + if(device) + vkDestroyDevice(device, NULL); + if(instance) + vkDestroyInstance(instance, NULL); + } + if(device_extensions) + free(device_extensions); + return success; +#else + // TODO: Low power query + video_codecs->h264 = (gsr_supported_video_codec){ true, false }; + video_codecs->hevc = (gsr_supported_video_codec){ true, false }; + return true; +#endif +} diff --git a/src/color_conversion.c b/src/color_conversion.c index 3962dd4..f34512c 100644 --- a/src/color_conversion.c +++ b/src/color_conversion.c @@ -325,22 +325,22 @@ void gsr_color_conversion_deinit(gsr_color_conversion *self) { self->params.egl = NULL; } -static void gsr_color_conversion_swizzle_texture_source(gsr_color_conversion *self) { - if(self->params.source_color == GSR_SOURCE_COLOR_BGR) { - const int swizzle_mask[] = { GL_BLUE, GL_GREEN, GL_RED, 1 }; +static void gsr_color_conversion_swizzle_texture_source(gsr_color_conversion *self, gsr_source_color source_color) { + if(source_color == GSR_SOURCE_COLOR_BGR) { + const int swizzle_mask[] = { GL_BLUE, GL_GREEN, GL_RED, GL_ALPHA }; self->params.egl->glTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_SWIZZLE_RGBA, swizzle_mask); } } -static void gsr_color_conversion_swizzle_reset(gsr_color_conversion *self) { - if(self->params.source_color == GSR_SOURCE_COLOR_BGR) { +static void gsr_color_conversion_swizzle_reset(gsr_color_conversion *self, gsr_source_color source_color) { + if(source_color == GSR_SOURCE_COLOR_BGR) { const int swizzle_mask[] = { GL_RED, GL_GREEN, GL_BLUE, GL_ALPHA }; self->params.egl->glTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_SWIZZLE_RGBA, swizzle_mask); } } /* |source_pos| is in pixel coordinates and |source_size| */ -void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_id, vec2i source_pos, vec2i source_size, vec2i texture_pos, vec2i texture_size, float rotation, bool external_texture) { +void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_id, vec2i source_pos, vec2i source_size, vec2i texture_pos, vec2i texture_size, float rotation, bool external_texture, gsr_source_color source_color) { // TODO: Remove this crap rotation = M_PI*2.0f - rotation; @@ -402,7 +402,7 @@ void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_ -1.0f + 0.0f + size_norm.x, -1.0f + 0.0f + size_norm.y, texture_pos_norm.x + texture_size_norm.x, texture_pos_norm.y + texture_size_norm.y }; - gsr_color_conversion_swizzle_texture_source(self); + gsr_color_conversion_swizzle_texture_source(self, source_color); self->params.egl->glBindVertexArray(self->vertex_array_object_id); self->params.egl->glViewport(0, 0, dest_texture_size.x, dest_texture_size.y); @@ -438,7 +438,7 @@ void gsr_color_conversion_draw(gsr_color_conversion *self, unsigned int texture_ self->params.egl->glBindTexture(texture_target, 0); self->params.egl->glBindFramebuffer(GL_FRAMEBUFFER, 0); - gsr_color_conversion_swizzle_reset(self); + gsr_color_conversion_swizzle_reset(self, source_color); } void gsr_color_conversion_clear(gsr_color_conversion *self) { diff --git a/src/cursor.c b/src/cursor.c index 3dca0c6..56b9694 100644 --- a/src/cursor.c +++ b/src/cursor.c @@ -43,10 +43,12 @@ static bool gsr_cursor_set_from_x11_cursor_image(gsr_cursor *self, XFixesCursorI *visible = true; } - *out++ = (unsigned)*in++ * 255/alpha; - *out++ = (unsigned)*in++ * 255/alpha; - *out++ = (unsigned)*in++ * 255/alpha; - *out++ = *in++; + out[0] = (float)in[2] * 255.0/(float)alpha; + out[1] = (float)in[1] * 255.0/(float)alpha; + out[2] = (float)in[0] * 255.0/(float)alpha; + out[3] = in[3]; + out += 4; + in += 4; } } diff --git a/src/damage.c b/src/damage.c index 8e62762..10f4ccd 100644 --- a/src/damage.c +++ b/src/damage.c @@ -188,6 +188,7 @@ static void gsr_damage_on_output_change(gsr_damage *self, XEvent *xev) { if(!screen_res) return; + // TODO: What about scaled output? look at for_each_active_monitor_output_x11_not_cached XRROutputInfo *out_info = XRRGetOutputInfo(self->egl->x11.dpy, screen_res, rr_output_change_event->output); if(out_info && out_info->crtc && out_info->crtc == self->monitor.monitor_identifier) { XRRCrtcInfo *crtc_info = XRRGetCrtcInfo(self->egl->x11.dpy, screen_res, out_info->crtc); @@ -223,7 +223,7 @@ static bool gsr_egl_create_window(gsr_egl *self, bool wayland) { if(wayland) { self->wayland.dpy = wl_display_connect(NULL); if(!self->wayland.dpy) { - fprintf(stderr, "gsr error: gsr_egl_create_window failed: wl_display_connect failed\n"); + fprintf(stderr, "gsr error: gsr_egl_create_window failed: failed to connect to the Wayland server\n"); goto fail; } @@ -262,12 +262,12 @@ static bool gsr_egl_create_window(gsr_egl *self, bool wayland) { fprintf(stderr, "gsr error: gsr_egl_create_window failed: eglInitialize failed\n"); goto fail; } - + if(!self->eglChooseConfig(self->egl_display, attr, &ecfg, 1, &num_config) || num_config != 1) { fprintf(stderr, "gsr error: gsr_egl_create_window failed: failed to find a matching config\n"); goto fail; } - + self->egl_context = self->eglCreateContext(self->egl_display, ecfg, NULL, ctxattr); if(!self->egl_context) { fprintf(stderr, "gsr error: gsr_egl_create_window failed: failed to create egl context\n"); @@ -499,6 +499,9 @@ static bool gsr_egl_load_gl(gsr_egl *self, void *library) { { (void**)&self->glUniform2f, "glUniform2f" }, { (void**)&self->glDebugMessageCallback, "glDebugMessageCallback" }, { (void**)&self->glScissor, "glScissor" }, + { (void**)&self->glReadPixels, "glReadPixels" }, + { (void**)&self->glMapBuffer, "glMapBuffer" }, + { (void**)&self->glUnmapBuffer, "glUnmapBuffer" }, { NULL, NULL } }; @@ -708,6 +711,10 @@ bool gsr_egl_process_event(gsr_egl *self) { } void gsr_egl_swap_buffers(gsr_egl *self) { + /* This uses less cpu than swap buffer on nvidia */ + // TODO: Do these and remove swap + //self->glFlush(); + //self->glFinish(); if(self->egl_display) { self->eglSwapBuffers(self->egl_display, self->egl_surface); } else if(self->x11.window) { diff --git a/src/encoder/video/cuda.c b/src/encoder/video/cuda.c deleted file mode 100644 index a323940..0000000 --- a/src/encoder/video/cuda.c +++ /dev/null @@ -1,460 +0,0 @@ -#include "../../../include/encoder/video/cuda.h" -#include "../../../include/egl.h" -#include "../../../include/cuda.h" -#include "../../../external/nvEncodeAPI.h" - -#include <libavcodec/avcodec.h> -#include <libavutil/hwcontext_cuda.h> - -#include <stdlib.h> -#include <dlfcn.h> - -typedef struct { - gsr_video_encoder_cuda_params params; - - unsigned int target_textures[2]; - - AVBufferRef *device_ctx; - - gsr_cuda cuda; - CUgraphicsResource cuda_graphics_resources[2]; - CUarray mapped_arrays[2]; - CUstream cuda_stream; -} gsr_video_encoder_cuda; - -static bool gsr_video_encoder_cuda_setup_context(gsr_video_encoder_cuda *self, AVCodecContext *video_codec_context) { - self->device_ctx = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_CUDA); - if(!self->device_ctx) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_context failed: failed to create hardware device context\n"); - return false; - } - - AVHWDeviceContext *hw_device_context = (AVHWDeviceContext*)self->device_ctx->data; - AVCUDADeviceContext *cuda_device_context = (AVCUDADeviceContext*)hw_device_context->hwctx; - cuda_device_context->cuda_ctx = self->cuda.cu_ctx; - if(av_hwdevice_ctx_init(self->device_ctx) < 0) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_context failed: failed to create hardware device context\n"); - av_buffer_unref(&self->device_ctx); - return false; - } - - AVBufferRef *frame_context = av_hwframe_ctx_alloc(self->device_ctx); - if(!frame_context) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_context failed: failed to create hwframe context\n"); - av_buffer_unref(&self->device_ctx); - return false; - } - - AVHWFramesContext *hw_frame_context = (AVHWFramesContext*)frame_context->data; - hw_frame_context->width = video_codec_context->width; - hw_frame_context->height = video_codec_context->height; - hw_frame_context->sw_format = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12; - hw_frame_context->format = video_codec_context->pix_fmt; - hw_frame_context->device_ctx = (AVHWDeviceContext*)self->device_ctx->data; - - if (av_hwframe_ctx_init(frame_context) < 0) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_context failed: failed to initialize hardware frame context " - "(note: ffmpeg version needs to be > 4.0)\n"); - av_buffer_unref(&self->device_ctx); - //av_buffer_unref(&frame_context); - return false; - } - - self->cuda_stream = cuda_device_context->stream; - video_codec_context->hw_frames_ctx = av_buffer_ref(frame_context); - av_buffer_unref(&frame_context); - return true; -} - -static unsigned int gl_create_texture(gsr_egl *egl, int width, int height, int internal_format, unsigned int format) { - unsigned int texture_id = 0; - egl->glGenTextures(1, &texture_id); - egl->glBindTexture(GL_TEXTURE_2D, texture_id); - egl->glTexImage2D(GL_TEXTURE_2D, 0, internal_format, width, height, 0, format, GL_UNSIGNED_BYTE, NULL); - - egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); - egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); - egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); - egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); - - egl->glBindTexture(GL_TEXTURE_2D, 0); - return texture_id; -} - -static bool cuda_register_opengl_texture(gsr_cuda *cuda, CUgraphicsResource *cuda_graphics_resource, CUarray *mapped_array, unsigned int texture_id) { - CUresult res; - res = cuda->cuGraphicsGLRegisterImage(cuda_graphics_resource, texture_id, GL_TEXTURE_2D, CU_GRAPHICS_REGISTER_FLAGS_NONE); - if (res != CUDA_SUCCESS) { - const char *err_str = "unknown"; - cuda->cuGetErrorString(res, &err_str); - fprintf(stderr, "gsr error: cuda_register_opengl_texture: cuGraphicsGLRegisterImage failed, error: %s, texture " "id: %u\n", err_str, texture_id); - return false; - } - - res = cuda->cuGraphicsResourceSetMapFlags(*cuda_graphics_resource, CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE); - res = cuda->cuGraphicsMapResources(1, cuda_graphics_resource, 0); - - res = cuda->cuGraphicsSubResourceGetMappedArray(mapped_array, *cuda_graphics_resource, 0, 0); - return true; -} - -static bool gsr_video_encoder_cuda_setup_textures(gsr_video_encoder_cuda *self, AVCodecContext *video_codec_context, AVFrame *frame) { - const int res = av_hwframe_get_buffer(video_codec_context->hw_frames_ctx, frame, 0); - if(res < 0) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_textures: av_hwframe_get_buffer failed: %d\n", res); - return false; - } - - const unsigned int internal_formats_nv12[2] = { GL_R8, GL_RG8 }; - const unsigned int internal_formats_p010[2] = { GL_R16, GL_RG16 }; - const unsigned int formats[2] = { GL_RED, GL_RG }; - const int div[2] = {1, 2}; // divide UV texture size by 2 because chroma is half size - - for(int i = 0; i < 2; ++i) { - self->target_textures[i] = gl_create_texture(self->params.egl, video_codec_context->width / div[i], video_codec_context->height / div[i], self->params.color_depth == GSR_COLOR_DEPTH_8_BITS ? internal_formats_nv12[i] : internal_formats_p010[i], formats[i]); - if(self->target_textures[i] == 0) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_textures: failed to create opengl texture\n"); - return false; - } - - if(!cuda_register_opengl_texture(&self->cuda, &self->cuda_graphics_resources[i], &self->mapped_arrays[i], self->target_textures[i])) { - return false; - } - } - - return true; -} - -static void* open_nvenc_library(void) { - dlerror(); /* clear */ - void *lib = dlopen("libnvidia-encode.so.1", RTLD_LAZY); - if(!lib) { - lib = dlopen("libnvidia-encode.so", RTLD_LAZY); - if(!lib) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs failed: failed to load libnvidia-encode.so/libnvidia-encode.so.1, error: %s\n", dlerror()); - return NULL; - } - } - return lib; -} - -static bool profile_is_h264(const GUID *profile_guid) { - const GUID *h264_guids[] = { - &NV_ENC_H264_PROFILE_BASELINE_GUID, - &NV_ENC_H264_PROFILE_MAIN_GUID, - &NV_ENC_H264_PROFILE_HIGH_GUID, - &NV_ENC_H264_PROFILE_PROGRESSIVE_HIGH_GUID, - &NV_ENC_H264_PROFILE_CONSTRAINED_HIGH_GUID - }; - - for(int i = 0; i < 5; ++i) { - if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) - return true; - } - - return false; -} - -static bool profile_is_hevc(const GUID *profile_guid) { - const GUID *h264_guids[] = { - &NV_ENC_HEVC_PROFILE_MAIN_GUID, - }; - - for(int i = 0; i < 1; ++i) { - if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) - return true; - } - - return false; -} - -static bool profile_is_hevc_10bit(const GUID *profile_guid) { - const GUID *h264_guids[] = { - &NV_ENC_HEVC_PROFILE_MAIN10_GUID, - }; - - for(int i = 0; i < 1; ++i) { - if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) - return true; - } - - return false; -} - -static bool profile_is_av1(const GUID *profile_guid) { - const GUID *h264_guids[] = { - &NV_ENC_AV1_PROFILE_MAIN_GUID, - }; - - for(int i = 0; i < 1; ++i) { - if(memcmp(profile_guid, h264_guids[i], sizeof(GUID)) == 0) - return true; - } - - return false; -} - -static bool encoder_get_supported_profiles(const NV_ENCODE_API_FUNCTION_LIST *function_list, void *nvenc_encoder, const GUID *encoder_guid, gsr_supported_video_codecs *supported_video_codecs) { - bool success = false; - GUID *profile_guids = NULL; - - uint32_t profile_guid_count = 0; - if(function_list->nvEncGetEncodeProfileGUIDCount(nvenc_encoder, *encoder_guid, &profile_guid_count) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncGetEncodeProfileGUIDCount failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); - goto fail; - } - - if(profile_guid_count == 0) - goto fail; - - profile_guids = calloc(profile_guid_count, sizeof(GUID)); - if(!profile_guids) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: failed to allocate %d guids\n", (int)profile_guid_count); - goto fail; - } - - if(function_list->nvEncGetEncodeProfileGUIDs(nvenc_encoder, *encoder_guid, profile_guids, profile_guid_count, &profile_guid_count) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncGetEncodeProfileGUIDs failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); - goto fail; - } - - for(uint32_t i = 0; i < profile_guid_count; ++i) { - if(profile_is_h264(&profile_guids[i])) { - supported_video_codecs->h264 = true; - } else if(profile_is_hevc(&profile_guids[i])) { - supported_video_codecs->hevc = true; - } else if(profile_is_hevc_10bit(&profile_guids[i])) { - supported_video_codecs->hevc_hdr = true; - supported_video_codecs->hevc_10bit = true; - } else if(profile_is_av1(&profile_guids[i])) { - supported_video_codecs->av1 = true; - supported_video_codecs->av1_hdr = true; - supported_video_codecs->av1_10bit = true; - } - } - - success = true; - fail: - - if(profile_guids) - free(profile_guids); - - return success; -} - -static bool get_supported_video_codecs(const NV_ENCODE_API_FUNCTION_LIST *function_list, void *nvenc_encoder, gsr_supported_video_codecs *supported_video_codecs) { - bool success = false; - GUID *encoder_guids = NULL; - *supported_video_codecs = (gsr_supported_video_codecs){0}; - - uint32_t encode_guid_count = 0; - if(function_list->nvEncGetEncodeGUIDCount(nvenc_encoder, &encode_guid_count) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncGetEncodeGUIDCount failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); - goto fail; - } - - if(encode_guid_count == 0) - goto fail; - - encoder_guids = calloc(encode_guid_count, sizeof(GUID)); - if(!encoder_guids) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: failed to allocate %d guids\n", (int)encode_guid_count); - goto fail; - } - - if(function_list->nvEncGetEncodeGUIDs(nvenc_encoder, encoder_guids, encode_guid_count, &encode_guid_count) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncGetEncodeGUIDs failed, error: %s\n", function_list->nvEncGetLastErrorString(nvenc_encoder)); - goto fail; - } - - for(uint32_t i = 0; i < encode_guid_count; ++i) { - encoder_get_supported_profiles(function_list, nvenc_encoder, &encoder_guids[i], supported_video_codecs); - } - - success = true; - fail: - - if(encoder_guids) - free(encoder_guids); - - return success; -} - -#define NVENCAPI_VERSION_470 (11 | (1 << 24)) - -static gsr_supported_video_codecs gsr_video_encoder_cuda_get_supported_codecs(gsr_video_encoder *encoder, bool cleanup) { - (void)encoder; - - void *nvenc_lib = NULL; - gsr_cuda cuda; - memset(&cuda, 0, sizeof(cuda)); - gsr_supported_video_codecs supported_video_codecs = {0}; - - if(!gsr_cuda_load(&cuda, NULL, false)) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: failed to load cuda\n"); - goto done; - } - - nvenc_lib = open_nvenc_library(); - if(!nvenc_lib) - goto done; - - typedef NVENCSTATUS NVENCAPI (*FUNC_NvEncodeAPICreateInstance)(NV_ENCODE_API_FUNCTION_LIST *functionList); - FUNC_NvEncodeAPICreateInstance nvEncodeAPICreateInstance = (FUNC_NvEncodeAPICreateInstance)dlsym(nvenc_lib, "NvEncodeAPICreateInstance"); - if(!nvEncodeAPICreateInstance) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: failed to find NvEncodeAPICreateInstance in libnvidia-encode.so\n"); - goto done; - } - - NV_ENCODE_API_FUNCTION_LIST function_list; - memset(&function_list, 0, sizeof(function_list)); - function_list.version = NV_ENCODE_API_FUNCTION_LIST_VER; - if(nvEncodeAPICreateInstance(&function_list) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncodeAPICreateInstance failed\n"); - goto done; - } - - NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS params; - memset(¶ms, 0, sizeof(params)); - params.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER; - params.deviceType = NV_ENC_DEVICE_TYPE_CUDA; - params.device = cuda.cu_ctx; - params.apiVersion = NVENCAPI_VERSION; - - void *nvenc_encoder = NULL; - if(function_list.nvEncOpenEncodeSessionEx(¶ms, &nvenc_encoder) != NV_ENC_SUCCESS) { - // Old nvidia gpus dont support the new nvenc api (which is required for av1). - // In such cases fallback to old api version if possible and try again. - params.apiVersion = NVENCAPI_VERSION_470; - if(function_list.nvEncOpenEncodeSessionEx(¶ms, &nvenc_encoder) != NV_ENC_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_get_supported_codecs: nvEncOpenEncodeSessionEx failed\n"); - goto done; - } - } - - get_supported_video_codecs(&function_list, nvenc_encoder, &supported_video_codecs); - - done: - if(cleanup) { - function_list.nvEncDestroyEncoder(nvenc_encoder); - if(nvenc_lib) - dlclose(nvenc_lib); - gsr_cuda_unload(&cuda); - } - - return supported_video_codecs; -} - -static void gsr_video_encoder_cuda_stop(gsr_video_encoder_cuda *self, AVCodecContext *video_codec_context); - -static bool gsr_video_encoder_cuda_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { - gsr_video_encoder_cuda *encoder_cuda = encoder->priv; - - const bool overclock = gsr_egl_get_display_server(encoder_cuda->params.egl) == GSR_DISPLAY_SERVER_X11 ? encoder_cuda->params.overclock : false; - if(!gsr_cuda_load(&encoder_cuda->cuda, encoder_cuda->params.egl->x11.dpy, overclock)) { - fprintf(stderr, "gsr error: gsr_video_encoder_cuda_start: failed to load cuda\n"); - gsr_video_encoder_cuda_stop(encoder_cuda, video_codec_context); - return false; - } - - if(!gsr_video_encoder_cuda_setup_context(encoder_cuda, video_codec_context)) { - gsr_video_encoder_cuda_stop(encoder_cuda, video_codec_context); - return false; - } - - if(!gsr_video_encoder_cuda_setup_textures(encoder_cuda, video_codec_context, frame)) { - gsr_video_encoder_cuda_stop(encoder_cuda, video_codec_context); - return false; - } - - return true; -} - -void gsr_video_encoder_cuda_stop(gsr_video_encoder_cuda *self, AVCodecContext *video_codec_context) { - self->params.egl->glDeleteTextures(2, self->target_textures); - self->target_textures[0] = 0; - self->target_textures[1] = 0; - - if(video_codec_context->hw_frames_ctx) - av_buffer_unref(&video_codec_context->hw_frames_ctx); - if(self->device_ctx) - av_buffer_unref(&self->device_ctx); - - if(self->cuda.cu_ctx) { - for(int i = 0; i < 2; ++i) { - if(self->cuda_graphics_resources[i]) { - self->cuda.cuGraphicsUnmapResources(1, &self->cuda_graphics_resources[i], 0); - self->cuda.cuGraphicsUnregisterResource(self->cuda_graphics_resources[i]); - self->cuda_graphics_resources[i] = 0; - } - } - } - - gsr_cuda_unload(&self->cuda); -} - -static void gsr_video_encoder_cuda_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame) { - gsr_video_encoder_cuda *encoder_cuda = encoder->priv; - const int div[2] = {1, 2}; // divide UV texture size by 2 because chroma is half size - for(int i = 0; i < 2; ++i) { - CUDA_MEMCPY2D memcpy_struct; - memcpy_struct.srcXInBytes = 0; - memcpy_struct.srcY = 0; - memcpy_struct.srcMemoryType = CU_MEMORYTYPE_ARRAY; - - memcpy_struct.dstXInBytes = 0; - memcpy_struct.dstY = 0; - memcpy_struct.dstMemoryType = CU_MEMORYTYPE_DEVICE; - - memcpy_struct.srcArray = encoder_cuda->mapped_arrays[i]; - memcpy_struct.srcPitch = frame->width / div[i]; - memcpy_struct.dstDevice = (CUdeviceptr)frame->data[i]; - memcpy_struct.dstPitch = frame->linesize[i]; - memcpy_struct.WidthInBytes = frame->width * (encoder_cuda->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? 2 : 1); - memcpy_struct.Height = frame->height / div[i]; - // TODO: Remove this copy if possible - encoder_cuda->cuda.cuMemcpy2DAsync_v2(&memcpy_struct, encoder_cuda->cuda_stream); - } - - // TODO: needed? - encoder_cuda->cuda.cuStreamSynchronize(encoder_cuda->cuda_stream); -} - -static void gsr_video_encoder_cuda_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { - gsr_video_encoder_cuda *encoder_cuda = encoder->priv; - textures[0] = encoder_cuda->target_textures[0]; - textures[1] = encoder_cuda->target_textures[1]; - *num_textures = 2; - *destination_color = encoder_cuda->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; -} - -static void gsr_video_encoder_cuda_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context) { - gsr_video_encoder_cuda_stop(encoder->priv, video_codec_context); - free(encoder->priv); - free(encoder); -} - -gsr_video_encoder* gsr_video_encoder_cuda_create(const gsr_video_encoder_cuda_params *params) { - gsr_video_encoder *encoder = calloc(1, sizeof(gsr_video_encoder)); - if(!encoder) - return NULL; - - gsr_video_encoder_cuda *encoder_cuda = calloc(1, sizeof(gsr_video_encoder_cuda)); - if(!encoder_cuda) { - free(encoder); - return NULL; - } - - encoder_cuda->params = *params; - - *encoder = (gsr_video_encoder) { - .get_supported_codecs = gsr_video_encoder_cuda_get_supported_codecs, - .start = gsr_video_encoder_cuda_start, - .copy_textures_to_frame = gsr_video_encoder_cuda_copy_textures_to_frame, - .get_textures = gsr_video_encoder_cuda_get_textures, - .destroy = gsr_video_encoder_cuda_destroy, - .priv = encoder_cuda - }; - - return encoder; -} diff --git a/src/encoder/video/nvenc.c b/src/encoder/video/nvenc.c new file mode 100644 index 0000000..718560d --- /dev/null +++ b/src/encoder/video/nvenc.c @@ -0,0 +1,237 @@ +#include "../../../include/encoder/video/nvenc.h" +#include "../../../include/egl.h" +#include "../../../include/cuda.h" + +#include <libavcodec/avcodec.h> +#include <libavutil/hwcontext_cuda.h> + +#include <stdlib.h> + +typedef struct { + gsr_video_encoder_nvenc_params params; + + unsigned int target_textures[2]; + + AVBufferRef *device_ctx; + + gsr_cuda cuda; + CUgraphicsResource cuda_graphics_resources[2]; + CUarray mapped_arrays[2]; + CUstream cuda_stream; +} gsr_video_encoder_nvenc; + +static bool gsr_video_encoder_nvenc_setup_context(gsr_video_encoder_nvenc *self, AVCodecContext *video_codec_context) { + self->device_ctx = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_CUDA); + if(!self->device_ctx) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_context failed: failed to create hardware device context\n"); + return false; + } + + AVHWDeviceContext *hw_device_context = (AVHWDeviceContext*)self->device_ctx->data; + AVCUDADeviceContext *cuda_device_context = (AVCUDADeviceContext*)hw_device_context->hwctx; + cuda_device_context->cuda_ctx = self->cuda.cu_ctx; + if(av_hwdevice_ctx_init(self->device_ctx) < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_context failed: failed to create hardware device context\n"); + av_buffer_unref(&self->device_ctx); + return false; + } + + AVBufferRef *frame_context = av_hwframe_ctx_alloc(self->device_ctx); + if(!frame_context) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_context failed: failed to create hwframe context\n"); + av_buffer_unref(&self->device_ctx); + return false; + } + + AVHWFramesContext *hw_frame_context = (AVHWFramesContext*)frame_context->data; + hw_frame_context->width = video_codec_context->width; + hw_frame_context->height = video_codec_context->height; + hw_frame_context->sw_format = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12; + hw_frame_context->format = video_codec_context->pix_fmt; + hw_frame_context->device_ctx = (AVHWDeviceContext*)self->device_ctx->data; + + if (av_hwframe_ctx_init(frame_context) < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_context failed: failed to initialize hardware frame context " + "(note: ffmpeg version needs to be > 4.0)\n"); + av_buffer_unref(&self->device_ctx); + //av_buffer_unref(&frame_context); + return false; + } + + self->cuda_stream = cuda_device_context->stream; + video_codec_context->hw_frames_ctx = av_buffer_ref(frame_context); + av_buffer_unref(&frame_context); + return true; +} + +static unsigned int gl_create_texture(gsr_egl *egl, int width, int height, int internal_format, unsigned int format) { + unsigned int texture_id = 0; + egl->glGenTextures(1, &texture_id); + egl->glBindTexture(GL_TEXTURE_2D, texture_id); + egl->glTexImage2D(GL_TEXTURE_2D, 0, internal_format, width, height, 0, format, GL_UNSIGNED_BYTE, NULL); + + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); + + egl->glBindTexture(GL_TEXTURE_2D, 0); + return texture_id; +} + +static bool cuda_register_opengl_texture(gsr_cuda *cuda, CUgraphicsResource *cuda_graphics_resource, CUarray *mapped_array, unsigned int texture_id) { + CUresult res; + res = cuda->cuGraphicsGLRegisterImage(cuda_graphics_resource, texture_id, GL_TEXTURE_2D, CU_GRAPHICS_REGISTER_FLAGS_NONE); + if (res != CUDA_SUCCESS) { + const char *err_str = "unknown"; + cuda->cuGetErrorString(res, &err_str); + fprintf(stderr, "gsr error: cuda_register_opengl_texture: cuGraphicsGLRegisterImage failed, error: %s, texture " "id: %u\n", err_str, texture_id); + return false; + } + + res = cuda->cuGraphicsResourceSetMapFlags(*cuda_graphics_resource, CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE); + res = cuda->cuGraphicsMapResources(1, cuda_graphics_resource, 0); + + res = cuda->cuGraphicsSubResourceGetMappedArray(mapped_array, *cuda_graphics_resource, 0, 0); + return true; +} + +static bool gsr_video_encoder_nvenc_setup_textures(gsr_video_encoder_nvenc *self, AVCodecContext *video_codec_context, AVFrame *frame) { + const int res = av_hwframe_get_buffer(video_codec_context->hw_frames_ctx, frame, 0); + if(res < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_textures: av_hwframe_get_buffer failed: %d\n", res); + return false; + } + + const unsigned int internal_formats_nv12[2] = { GL_R8, GL_RG8 }; + const unsigned int internal_formats_p010[2] = { GL_R16, GL_RG16 }; + const unsigned int formats[2] = { GL_RED, GL_RG }; + const int div[2] = {1, 2}; // divide UV texture size by 2 because chroma is half size + + for(int i = 0; i < 2; ++i) { + self->target_textures[i] = gl_create_texture(self->params.egl, video_codec_context->width / div[i], video_codec_context->height / div[i], self->params.color_depth == GSR_COLOR_DEPTH_8_BITS ? internal_formats_nv12[i] : internal_formats_p010[i], formats[i]); + if(self->target_textures[i] == 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_setup_textures: failed to create opengl texture\n"); + return false; + } + + if(!cuda_register_opengl_texture(&self->cuda, &self->cuda_graphics_resources[i], &self->mapped_arrays[i], self->target_textures[i])) { + return false; + } + } + + return true; +} + +static void gsr_video_encoder_nvenc_stop(gsr_video_encoder_nvenc *self, AVCodecContext *video_codec_context); + +static bool gsr_video_encoder_nvenc_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { + gsr_video_encoder_nvenc *self = encoder->priv; + + const bool overclock = gsr_egl_get_display_server(self->params.egl) == GSR_DISPLAY_SERVER_X11 ? self->params.overclock : false; + if(!gsr_cuda_load(&self->cuda, self->params.egl->x11.dpy, overclock)) { + fprintf(stderr, "gsr error: gsr_video_encoder_nvenc_start: failed to load cuda\n"); + gsr_video_encoder_nvenc_stop(self, video_codec_context); + return false; + } + + if(!gsr_video_encoder_nvenc_setup_context(self, video_codec_context)) { + gsr_video_encoder_nvenc_stop(self, video_codec_context); + return false; + } + + if(!gsr_video_encoder_nvenc_setup_textures(self, video_codec_context, frame)) { + gsr_video_encoder_nvenc_stop(self, video_codec_context); + return false; + } + + return true; +} + +void gsr_video_encoder_nvenc_stop(gsr_video_encoder_nvenc *self, AVCodecContext *video_codec_context) { + self->params.egl->glDeleteTextures(2, self->target_textures); + self->target_textures[0] = 0; + self->target_textures[1] = 0; + + if(video_codec_context->hw_frames_ctx) + av_buffer_unref(&video_codec_context->hw_frames_ctx); + if(self->device_ctx) + av_buffer_unref(&self->device_ctx); + + if(self->cuda.cu_ctx) { + for(int i = 0; i < 2; ++i) { + if(self->cuda_graphics_resources[i]) { + self->cuda.cuGraphicsUnmapResources(1, &self->cuda_graphics_resources[i], 0); + self->cuda.cuGraphicsUnregisterResource(self->cuda_graphics_resources[i]); + self->cuda_graphics_resources[i] = 0; + } + } + } + + gsr_cuda_unload(&self->cuda); +} + +static void gsr_video_encoder_nvenc_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion) { + gsr_video_encoder_nvenc *self = encoder->priv; + const int div[2] = {1, 2}; // divide UV texture size by 2 because chroma is half size + for(int i = 0; i < 2; ++i) { + CUDA_MEMCPY2D memcpy_struct; + memcpy_struct.srcXInBytes = 0; + memcpy_struct.srcY = 0; + memcpy_struct.srcMemoryType = CU_MEMORYTYPE_ARRAY; + + memcpy_struct.dstXInBytes = 0; + memcpy_struct.dstY = 0; + memcpy_struct.dstMemoryType = CU_MEMORYTYPE_DEVICE; + + memcpy_struct.srcArray = self->mapped_arrays[i]; + memcpy_struct.srcPitch = frame->width / div[i]; + memcpy_struct.dstDevice = (CUdeviceptr)frame->data[i]; + memcpy_struct.dstPitch = frame->linesize[i]; + memcpy_struct.WidthInBytes = frame->width * (self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? 2 : 1); + memcpy_struct.Height = frame->height / div[i]; + // TODO: Remove this copy if possible + self->cuda.cuMemcpy2DAsync_v2(&memcpy_struct, self->cuda_stream); + } + + // TODO: needed? + self->cuda.cuStreamSynchronize(self->cuda_stream); +} + +static void gsr_video_encoder_nvenc_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { + gsr_video_encoder_nvenc *self = encoder->priv; + textures[0] = self->target_textures[0]; + textures[1] = self->target_textures[1]; + *num_textures = 2; + *destination_color = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; +} + +static void gsr_video_encoder_nvenc_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context) { + gsr_video_encoder_nvenc_stop(encoder->priv, video_codec_context); + free(encoder->priv); + free(encoder); +} + +gsr_video_encoder* gsr_video_encoder_nvenc_create(const gsr_video_encoder_nvenc_params *params) { + gsr_video_encoder *encoder = calloc(1, sizeof(gsr_video_encoder)); + if(!encoder) + return NULL; + + gsr_video_encoder_nvenc *encoder_cuda = calloc(1, sizeof(gsr_video_encoder_nvenc)); + if(!encoder_cuda) { + free(encoder); + return NULL; + } + + encoder_cuda->params = *params; + + *encoder = (gsr_video_encoder) { + .start = gsr_video_encoder_nvenc_start, + .copy_textures_to_frame = gsr_video_encoder_nvenc_copy_textures_to_frame, + .get_textures = gsr_video_encoder_nvenc_get_textures, + .destroy = gsr_video_encoder_nvenc_destroy, + .priv = encoder_cuda + }; + + return encoder; +} diff --git a/src/encoder/video/software.c b/src/encoder/video/software.c index 4a4b78a..be227f2 100644 --- a/src/encoder/video/software.c +++ b/src/encoder/video/software.c @@ -58,26 +58,10 @@ static bool gsr_video_encoder_software_setup_textures(gsr_video_encoder_software return true; } -static gsr_supported_video_codecs gsr_video_encoder_software_get_supported_codecs(gsr_video_encoder *encoder, bool cleanup) { - (void)encoder; - (void)cleanup; - return (gsr_supported_video_codecs) { - .h264 = true, - .hevc = false, - .hevc_hdr = false, - .hevc_10bit = false, - .av1 = false, - .av1_hdr = false, - .av1_10bit = false, - .vp8 = false, - .vp9 = false - }; -} - static void gsr_video_encoder_software_stop(gsr_video_encoder_software *self, AVCodecContext *video_codec_context); static bool gsr_video_encoder_software_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { - gsr_video_encoder_software *encoder_software = encoder->priv; + gsr_video_encoder_software *self = encoder->priv; video_codec_context->width = FFALIGN(video_codec_context->width, LINESIZE_ALIGNMENT); video_codec_context->height = FFALIGN(video_codec_context->height, 2); @@ -85,8 +69,8 @@ static bool gsr_video_encoder_software_start(gsr_video_encoder *encoder, AVCodec frame->width = video_codec_context->width; frame->height = video_codec_context->height; - if(!gsr_video_encoder_software_setup_textures(encoder_software, video_codec_context, frame)) { - gsr_video_encoder_software_stop(encoder_software, video_codec_context); + if(!gsr_video_encoder_software_setup_textures(self, video_codec_context, frame)) { + gsr_video_encoder_software_stop(self, video_codec_context); return false; } @@ -100,29 +84,29 @@ void gsr_video_encoder_software_stop(gsr_video_encoder_software *self, AVCodecCo self->target_textures[1] = 0; } -static void gsr_video_encoder_software_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame) { - gsr_video_encoder_software *encoder_software = encoder->priv; +static void gsr_video_encoder_software_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion) { + gsr_video_encoder_software *self = encoder->priv; // TODO: hdr support const unsigned int formats[2] = { GL_RED, GL_RG }; for(int i = 0; i < 2; ++i) { - encoder_software->params.egl->glBindTexture(GL_TEXTURE_2D, encoder_software->target_textures[i]); + self->params.egl->glBindTexture(GL_TEXTURE_2D, self->target_textures[i]); // We could use glGetTexSubImage and then we wouldn't have to use a specific linesize (LINESIZE_ALIGNMENT) that adds padding, // but glGetTexSubImage is only available starting from opengl 4.5. - encoder_software->params.egl->glGetTexImage(GL_TEXTURE_2D, 0, formats[i], GL_UNSIGNED_BYTE, frame->data[i]); + self->params.egl->glGetTexImage(GL_TEXTURE_2D, 0, formats[i], GL_UNSIGNED_BYTE, frame->data[i]); } - encoder_software->params.egl->glBindTexture(GL_TEXTURE_2D, 0); + self->params.egl->glBindTexture(GL_TEXTURE_2D, 0); // cap_kms->kms.base.egl->eglSwapBuffers(cap_kms->kms.base.egl->egl_display, cap_kms->kms.base.egl->egl_surface); - encoder_software->params.egl->glFlush(); - encoder_software->params.egl->glFinish(); + self->params.egl->glFlush(); + self->params.egl->glFinish(); } static void gsr_video_encoder_software_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { - gsr_video_encoder_software *encoder_software = encoder->priv; - textures[0] = encoder_software->target_textures[0]; - textures[1] = encoder_software->target_textures[1]; + gsr_video_encoder_software *self = encoder->priv; + textures[0] = self->target_textures[0]; + textures[1] = self->target_textures[1]; *num_textures = 2; - *destination_color = encoder_software->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; + *destination_color = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; } static void gsr_video_encoder_software_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context) { @@ -145,7 +129,6 @@ gsr_video_encoder* gsr_video_encoder_software_create(const gsr_video_encoder_sof encoder_software->params = *params; *encoder = (gsr_video_encoder) { - .get_supported_codecs = gsr_video_encoder_software_get_supported_codecs, .start = gsr_video_encoder_software_start, .copy_textures_to_frame = gsr_video_encoder_software_copy_textures_to_frame, .get_textures = gsr_video_encoder_software_get_textures, diff --git a/src/encoder/video/vaapi.c b/src/encoder/video/vaapi.c index 03218cb..d558785 100644 --- a/src/encoder/video/vaapi.c +++ b/src/encoder/video/vaapi.c @@ -4,9 +4,9 @@ #include <libavcodec/avcodec.h> #include <libavutil/hwcontext_vaapi.h> +#include <libavutil/intreadwrite.h> #include <va/va_drmcommon.h> -#include <va/va_drm.h> #include <stdlib.h> #include <unistd.h> @@ -102,6 +102,7 @@ static bool gsr_video_encoder_vaapi_setup_textures(gsr_video_encoder_vaapi *self uint32_t pitches[4]; uint64_t modifiers[4]; for(uint32_t j = 0; j < self->prime.layers[layer].num_planes; ++j) { + // TODO: Close these? in _stop, using self->prime fds[j] = self->prime.objects[self->prime.layers[layer].object_index[j]].fd; offsets[j] = self->prime.layers[layer].offset[j]; pitches[j] = self->prime.layers[layer].pitch[j]; @@ -147,194 +148,16 @@ static bool gsr_video_encoder_vaapi_setup_textures(gsr_video_encoder_vaapi *self } } -static bool profile_is_h264(VAProfile profile) { - switch(profile) { - case 5: // VAProfileH264Baseline - case VAProfileH264Main: - case VAProfileH264High: - case VAProfileH264ConstrainedBaseline: - return true; - default: - return false; - } -} - -static bool profile_is_hevc_8bit(VAProfile profile) { - switch(profile) { - case VAProfileHEVCMain: - return true; - default: - return false; - } -} - -static bool profile_is_hevc_10bit(VAProfile profile) { - switch(profile) { - case VAProfileHEVCMain10: - //case VAProfileHEVCMain12: - //case VAProfileHEVCMain422_10: - //case VAProfileHEVCMain422_12: - //case VAProfileHEVCMain444: - //case VAProfileHEVCMain444_10: - //case VAProfileHEVCMain444_12: - return true; - default: - return false; - } -} - -static bool profile_is_av1(VAProfile profile) { - switch(profile) { - case VAProfileAV1Profile0: - case VAProfileAV1Profile1: - return true; - default: - return false; - } -} - -static bool profile_is_vp8(VAProfile profile) { - switch(profile) { - case VAProfileVP8Version0_3: - return true; - default: - return false; - } -} - -static bool profile_is_vp9(VAProfile profile) { - switch(profile) { - case VAProfileVP9Profile0: - case VAProfileVP9Profile1: - case VAProfileVP9Profile2: - case VAProfileVP9Profile3: - return true; - default: - return false; - } -} - -static bool profile_supports_video_encoding(VADisplay va_dpy, VAProfile profile) { - int num_entrypoints = vaMaxNumEntrypoints(va_dpy); - if(num_entrypoints <= 0) - return false; - - VAEntrypoint *entrypoint_list = calloc(num_entrypoints, sizeof(VAEntrypoint)); - if(!entrypoint_list) - return false; - - bool supported = false; - if(vaQueryConfigEntrypoints(va_dpy, profile, entrypoint_list, &num_entrypoints) == VA_STATUS_SUCCESS) { - for(int i = 0; i < num_entrypoints; ++i) { - if(entrypoint_list[i] == VAEntrypointEncSlice) { - supported = true; - break; - } - } - } - - free(entrypoint_list); - return supported; -} - -static bool get_supported_video_codecs(VADisplay va_dpy, gsr_supported_video_codecs *video_codecs, bool cleanup) { - *video_codecs = (gsr_supported_video_codecs){0}; - bool success = false; - VAProfile *profile_list = NULL; - - vaSetInfoCallback(va_dpy, NULL, NULL); - - int va_major = 0; - int va_minor = 0; - if(vaInitialize(va_dpy, &va_major, &va_minor) != VA_STATUS_SUCCESS) { - fprintf(stderr, "gsr error: gsr_video_encoder_vaapi_get_supported_codecs: vaInitialize failed\n"); - goto fail; - } - - int num_profiles = vaMaxNumProfiles(va_dpy); - if(num_profiles <= 0) - goto fail; - - profile_list = calloc(num_profiles, sizeof(VAProfile)); - if(!profile_list || vaQueryConfigProfiles(va_dpy, profile_list, &num_profiles) != VA_STATUS_SUCCESS) - goto fail; - - for(int i = 0; i < num_profiles; ++i) { - if(profile_is_h264(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) - video_codecs->h264 = true; - } else if(profile_is_hevc_8bit(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) - video_codecs->hevc = true; - } else if(profile_is_hevc_10bit(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) { - video_codecs->hevc_hdr = true; - video_codecs->hevc_10bit = true; - } - } else if(profile_is_av1(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) { - video_codecs->av1 = true; - video_codecs->av1_hdr = true; - video_codecs->av1_10bit = true; - } - } else if(profile_is_vp8(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) - video_codecs->vp8 = true; - } else if(profile_is_vp9(profile_list[i])) { - if(profile_supports_video_encoding(va_dpy, profile_list[i])) - video_codecs->vp9 = true; - } - } - - success = true; - fail: - if(profile_list) - free(profile_list); - - if(cleanup) - vaTerminate(va_dpy); - - return success; -} - -static gsr_supported_video_codecs gsr_video_encoder_vaapi_get_supported_codecs(gsr_video_encoder *encoder, bool cleanup) { - gsr_video_encoder_vaapi *encoder_vaapi = encoder->priv; - gsr_supported_video_codecs supported_video_codecs = {0}; - - char render_path[128]; - if(!gsr_card_path_get_render_path(encoder_vaapi->params.egl->card_path, render_path)) { - fprintf(stderr, "gsr error: gsr_video_encoder_vaapi_get_supported_codecs: failed to get /dev/dri/renderDXXX file from %s\n", encoder_vaapi->params.egl->card_path); - return supported_video_codecs; - } - - const int drm_fd = open(render_path, O_RDWR); - if(drm_fd == -1) { - fprintf(stderr, "gsr error: gsr_video_encoder_vaapi_get_supported_codecs: failed to open device %s\n", render_path); - return supported_video_codecs; - } - - VADisplay va_dpy = vaGetDisplayDRM(drm_fd); - if(va_dpy) { - if(!get_supported_video_codecs(va_dpy, &supported_video_codecs, cleanup)) - fprintf(stderr, "gsr error: gsr_video_encoder_vaapi_get_supported_codecs: failed to query supported video codecs for device %s\n", render_path); - } - - if(cleanup) - close(drm_fd); - - return supported_video_codecs; -} - static void gsr_video_encoder_vaapi_stop(gsr_video_encoder_vaapi *self, AVCodecContext *video_codec_context); static bool gsr_video_encoder_vaapi_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { - gsr_video_encoder_vaapi *encoder_vaapi = encoder->priv; + gsr_video_encoder_vaapi *self = encoder->priv; - if(encoder_vaapi->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD && video_codec_context->codec_id == AV_CODEC_ID_HEVC) { + if(self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD && video_codec_context->codec_id == AV_CODEC_ID_HEVC) { // TODO: dont do this if using ffmpeg reports that this is not needed (AMD driver bug that was fixed recently) video_codec_context->width = FFALIGN(video_codec_context->width, 64); video_codec_context->height = FFALIGN(video_codec_context->height, 16); - } else if(encoder_vaapi->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD && video_codec_context->codec_id == AV_CODEC_ID_AV1) { + } else if(self->params.egl->gpu_info.vendor == GSR_GPU_VENDOR_AMD && video_codec_context->codec_id == AV_CODEC_ID_AV1) { // TODO: Dont do this for VCN 5 and forward which should fix this hardware bug video_codec_context->width = FFALIGN(video_codec_context->width, 64); // AMD driver has special case handling for 1080 height to set it to 1082 instead of 1088 (1080 aligned to 16). @@ -346,20 +169,40 @@ static bool gsr_video_encoder_vaapi_start(gsr_video_encoder *encoder, AVCodecCon } } - if(video_codec_context->width != frame->width || video_codec_context->height != frame->height) { + const int crop_top = (video_codec_context->height - frame->height) / 2; + const int crop_left = (video_codec_context->width - frame->width) / 2; + if(crop_top != 0 || crop_left != 0) { fprintf(stderr, "gsr warning: gsr_video_encoder_vaapi_start: black bars have been added to the video because of a bug in AMD drivers/hardware. Record with h264 codec instead (-k h264) to get around this issue\n"); +#if 0 + #if LIBAVCODEC_VERSION_INT >= AV_VERSION_INT(61, 10, 100) + const int crop_bottom = crop_top; + const int crop_right = crop_left; + fprintf(stderr, "gsr info: cropping metadata has been added to the file to try and workaround this issue. Video players that support this will remove the black bars when the video is playing\n"); + const int frame_cropping_data_size = 4 * sizeof(uint32_t); + uint8_t *frame_cropping = av_malloc(frame_cropping_data_size); + if(frame_cropping) { + AV_WL32(frame_cropping + 0, crop_top); + AV_WL32(frame_cropping + 4, crop_bottom); + AV_WL32(frame_cropping + 8, crop_left); + AV_WL32(frame_cropping + 12, crop_right); + const bool sidedata_added = av_packet_side_data_add(&video_stream->codecpar->coded_side_data, &video_stream->codecpar->nb_coded_side_data, AV_PKT_DATA_FRAME_CROPPING, frame_cropping, frame_cropping_data_size, 0) != NULL; + if(!sidedata_added) + av_free(frame_cropping); + } + #endif +#endif } frame->width = video_codec_context->width; frame->height = video_codec_context->height; - if(!gsr_video_encoder_vaapi_setup_context(encoder_vaapi, video_codec_context)) { - gsr_video_encoder_vaapi_stop(encoder_vaapi, video_codec_context); + if(!gsr_video_encoder_vaapi_setup_context(self, video_codec_context)) { + gsr_video_encoder_vaapi_stop(self, video_codec_context); return false; } - if(!gsr_video_encoder_vaapi_setup_textures(encoder_vaapi, video_codec_context, frame)) { - gsr_video_encoder_vaapi_stop(encoder_vaapi, video_codec_context); + if(!gsr_video_encoder_vaapi_setup_textures(self, video_codec_context, frame)) { + gsr_video_encoder_vaapi_stop(self, video_codec_context); return false; } @@ -385,11 +228,11 @@ void gsr_video_encoder_vaapi_stop(gsr_video_encoder_vaapi *self, AVCodecContext } static void gsr_video_encoder_vaapi_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { - gsr_video_encoder_vaapi *encoder_vaapi = encoder->priv; - textures[0] = encoder_vaapi->target_textures[0]; - textures[1] = encoder_vaapi->target_textures[1]; + gsr_video_encoder_vaapi *self = encoder->priv; + textures[0] = self->target_textures[0]; + textures[1] = self->target_textures[1]; *num_textures = 2; - *destination_color = encoder_vaapi->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; + *destination_color = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; } static void gsr_video_encoder_vaapi_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context) { @@ -412,9 +255,7 @@ gsr_video_encoder* gsr_video_encoder_vaapi_create(const gsr_video_encoder_vaapi_ encoder_vaapi->params = *params; *encoder = (gsr_video_encoder) { - .get_supported_codecs = gsr_video_encoder_vaapi_get_supported_codecs, .start = gsr_video_encoder_vaapi_start, - .copy_textures_to_frame = NULL, .get_textures = gsr_video_encoder_vaapi_get_textures, .destroy = gsr_video_encoder_vaapi_destroy, .priv = encoder_vaapi diff --git a/src/encoder/video/video.c b/src/encoder/video/video.c index daaf537..76d53b0 100644 --- a/src/encoder/video/video.c +++ b/src/encoder/video/video.c @@ -1,10 +1,6 @@ #include "../../../include/encoder/video/video.h" #include <assert.h> -gsr_supported_video_codecs gsr_video_encoder_get_supported_codecs(gsr_video_encoder *encoder, bool cleanup) { - return encoder->get_supported_codecs(encoder, cleanup); -} - bool gsr_video_encoder_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { assert(!encoder->started); bool res = encoder->start(encoder, video_codec_context, frame); @@ -13,10 +9,10 @@ bool gsr_video_encoder_start(gsr_video_encoder *encoder, AVCodecContext *video_c return res; } -void gsr_video_encoder_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame) { +void gsr_video_encoder_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion) { assert(encoder->started); if(encoder->copy_textures_to_frame) - encoder->copy_textures_to_frame(encoder, frame); + encoder->copy_textures_to_frame(encoder, frame, color_conversion); } void gsr_video_encoder_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { diff --git a/src/encoder/video/vulkan.c b/src/encoder/video/vulkan.c new file mode 100644 index 0000000..0b6c380 --- /dev/null +++ b/src/encoder/video/vulkan.c @@ -0,0 +1,313 @@ +#include "../../../include/encoder/video/vulkan.h" +#include "../../../include/utils.h" +#include "../../../include/egl.h" + +#include <libavcodec/avcodec.h> +#define VK_NO_PROTOTYPES +#include <libavutil/hwcontext_vulkan.h> + +//#include <vulkan/vulkan_core.h> + +#define GL_TEXTURE_TILING_EXT 0x9580 +#define GL_OPTIMAL_TILING_EXT 0x9584 +#define GL_LINEAR_TILING_EXT 0x9585 + +#define GL_PIXEL_PACK_BUFFER 0x88EB +#define GL_PIXEL_UNPACK_BUFFER 0x88EC +#define GL_STREAM_READ 0x88E1 +#define GL_STREAM_DRAW 0x88E0 +#define GL_READ_ONLY 0x88B8 +#define GL_WRITE_ONLY 0x88B9 +#define GL_READ_FRAMEBUFFER 0x8CA8 + +typedef struct { + gsr_video_encoder_vulkan_params params; + unsigned int target_textures[2]; + AVBufferRef *device_ctx; + AVVulkanDeviceContext* vv; + unsigned int pbo_y[2]; + unsigned int pbo_uv[2]; + AVFrame *sw_frame; +} gsr_video_encoder_vulkan; + +static bool gsr_video_encoder_vulkan_setup_context(gsr_video_encoder_vulkan *self, AVCodecContext *video_codec_context) { + AVDictionary *options = NULL; + //av_dict_set(&options, "linear_images", "1", 0); + //av_dict_set(&options, "disable_multiplane", "1", 0); + + // TODO: Use correct device + if(av_hwdevice_ctx_create(&self->device_ctx, AV_HWDEVICE_TYPE_VULKAN, NULL, options, 0) < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_vulkan_setup_context: failed to create hardware device context\n"); + return false; + } + + AVBufferRef *frame_context = av_hwframe_ctx_alloc(self->device_ctx); + if(!frame_context) { + fprintf(stderr, "gsr error: gsr_video_encoder_vulkan_setup_context: failed to create hwframe context\n"); + av_buffer_unref(&self->device_ctx); + return false; + } + + AVHWFramesContext *hw_frame_context = (AVHWFramesContext*)frame_context->data; + hw_frame_context->width = video_codec_context->width; + hw_frame_context->height = video_codec_context->height; + hw_frame_context->sw_format = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12; + hw_frame_context->format = video_codec_context->pix_fmt; + hw_frame_context->device_ctx = (AVHWDeviceContext*)self->device_ctx->data; + + //AVVulkanFramesContext *vk_frame_ctx = (AVVulkanFramesContext*)hw_frame_context->hwctx; + //hw_frame_context->initial_pool_size = 20; + + if (av_hwframe_ctx_init(frame_context) < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_vulkan_setup_context: failed to initialize hardware frame context " + "(note: ffmpeg version needs to be > 4.0)\n"); + av_buffer_unref(&self->device_ctx); + //av_buffer_unref(&frame_context); + return false; + } + + video_codec_context->hw_frames_ctx = av_buffer_ref(frame_context); + av_buffer_unref(&frame_context); + return true; +} + +static unsigned int gl_create_texture(gsr_egl *egl, int width, int height, int internal_format, unsigned int format) { + unsigned int texture_id = 0; + egl->glGenTextures(1, &texture_id); + egl->glBindTexture(GL_TEXTURE_2D, texture_id); + //egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_TILING_EXT, GL_OPTIMAL_TILING_EXT); + egl->glTexImage2D(GL_TEXTURE_2D, 0, internal_format, width, height, 0, format, GL_UNSIGNED_BYTE, NULL); + + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); + egl->glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); + + egl->glBindTexture(GL_TEXTURE_2D, 0); + return texture_id; +} + +static AVVulkanDeviceContext* video_codec_context_get_vulkan_data(AVCodecContext *video_codec_context) { + AVBufferRef *hw_frames_ctx = video_codec_context->hw_frames_ctx; + if(!hw_frames_ctx) + return NULL; + + AVHWFramesContext *hw_frame_context = (AVHWFramesContext*)hw_frames_ctx->data; + AVHWDeviceContext *device_context = (AVHWDeviceContext*)hw_frame_context->device_ctx; + if(device_context->type != AV_HWDEVICE_TYPE_VULKAN) + return NULL; + + return (AVVulkanDeviceContext*)device_context->hwctx; +} + +static bool gsr_video_encoder_vulkan_setup_textures(gsr_video_encoder_vulkan *self, AVCodecContext *video_codec_context, AVFrame *frame) { + const int res = av_hwframe_get_buffer(video_codec_context->hw_frames_ctx, frame, 0); + if(res < 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_vulkan_setup_textures: av_hwframe_get_buffer failed: %d\n", res); + return false; + } + + //AVVkFrame *target_surface_id = (AVVkFrame*)frame->data[0]; + self->vv = video_codec_context_get_vulkan_data(video_codec_context); + + const unsigned int internal_formats_nv12[2] = { GL_RGBA8, GL_RGBA8 }; + const unsigned int internal_formats_p010[2] = { GL_R16, GL_RG16 }; + const unsigned int formats[2] = { GL_RED, GL_RG }; + const int div[2] = {1, 2}; // divide UV texture size by 2 because chroma is half size + + for(int i = 0; i < 2; ++i) { + self->target_textures[i] = gl_create_texture(self->params.egl, video_codec_context->width / div[i], video_codec_context->height / div[i], self->params.color_depth == GSR_COLOR_DEPTH_8_BITS ? internal_formats_nv12[i] : internal_formats_p010[i], formats[i]); + if(self->target_textures[i] == 0) { + fprintf(stderr, "gsr error: gsr_video_encoder_cuda_setup_textures: failed to create opengl texture\n"); + return false; + } + } + + self->params.egl->glGenBuffers(2, self->pbo_y); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[0]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, frame->width * frame->height, 0, GL_STREAM_READ); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[1]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, frame->width * frame->height, 0, GL_STREAM_READ); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); + + self->params.egl->glGenBuffers(2, self->pbo_uv); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_uv[0]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, (frame->width/2 * frame->height/2) * 2, 0, GL_STREAM_READ); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_uv[1]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, (frame->width/2 * frame->height/2) * 2, 0, GL_STREAM_READ); + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); + + self->sw_frame = av_frame_alloc(); + self->sw_frame->format = AV_PIX_FMT_NV12; + self->sw_frame->width = frame->width; + self->sw_frame->height = frame->height; + + // TODO: Remove + if(av_frame_get_buffer(self->sw_frame, 0) < 0) { + fprintf(stderr, "failed to allocate sw frame\n"); + } + + // TODO: Remove + if(av_frame_make_writable(self->sw_frame) < 0) { + fprintf(stderr, "failed to make writable\n"); + } + return true; +} + +static void gsr_video_encoder_vulkan_stop(gsr_video_encoder_vulkan *self, AVCodecContext *video_codec_context); + +static bool gsr_video_encoder_vulkan_start(gsr_video_encoder *encoder, AVCodecContext *video_codec_context, AVFrame *frame) { + gsr_video_encoder_vulkan *self = encoder->priv; + + if(!gsr_video_encoder_vulkan_setup_context(self, video_codec_context)) { + gsr_video_encoder_vulkan_stop(self, video_codec_context); + return false; + } + + if(!gsr_video_encoder_vulkan_setup_textures(self, video_codec_context, frame)) { + gsr_video_encoder_vulkan_stop(self, video_codec_context); + return false; + } + + return true; +} + +void gsr_video_encoder_vulkan_stop(gsr_video_encoder_vulkan *self, AVCodecContext *video_codec_context) { + self->params.egl->glDeleteTextures(2, self->target_textures); + self->target_textures[0] = 0; + self->target_textures[1] = 0; + + if(video_codec_context->hw_frames_ctx) + av_buffer_unref(&video_codec_context->hw_frames_ctx); + if(self->device_ctx) + av_buffer_unref(&self->device_ctx); +} + +static void nop_free(void *opaque, uint8_t *data) { + +} + +static void gsr_video_encoder_vulkan_copy_textures_to_frame(gsr_video_encoder *encoder, AVFrame *frame, gsr_color_conversion *color_conversion) { + gsr_video_encoder_vulkan *self = encoder->priv; + + static int counter = 0; + ++counter; + + // AVBufferRef *av_buffer_create(uint8_t *data, size_t size, + // void (*free)(void *opaque, uint8_t *data), + // void *opaque, int flags); + + while(self->params.egl->glGetError()){} + self->params.egl->glBindFramebuffer(GL_READ_FRAMEBUFFER, color_conversion->framebuffers[0]); + //fprintf(stderr, "1 gl err: %d\n", self->params.egl->glGetError()); + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[counter % 2]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, frame->width * frame->height, 0, GL_STREAM_READ); + self->params.egl->glReadPixels(0, 0, frame->width, frame->height, GL_RED, GL_UNSIGNED_BYTE, 0); + //fprintf(stderr, "2 gl err: %d\n", self->params.egl->glGetError()); + + const int next_pbo_y = (counter + 1) % 2; + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[next_pbo_y]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, frame->width * frame->height, 0, GL_STREAM_READ); + //fprintf(stderr, "3 gl err: %d\n", self->params.egl->glGetError()); + uint8_t *ptr_y = (uint8_t*)self->params.egl->glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY); + //fprintf(stderr, "4 gl err: %d\n", self->params.egl->glGetError()); + if(!ptr_y) { + fprintf(stderr, "failed to map buffer y!\n"); + } + + while(self->params.egl->glGetError()){} + self->params.egl->glBindFramebuffer(GL_READ_FRAMEBUFFER, color_conversion->framebuffers[1]); + //fprintf(stderr, "5 gl err: %d\n", self->params.egl->glGetError()); + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_uv[counter % 2]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, (frame->width/2 * frame->height/2) * 2, 0, GL_STREAM_READ); + //fprintf(stderr, "5.5 gl err: %d\n", self->params.egl->glGetError()); + self->params.egl->glReadPixels(0, 0, frame->width/2, frame->height/2, GL_RG, GL_UNSIGNED_BYTE, 0); + //fprintf(stderr, "6 gl err: %d\n", self->params.egl->glGetError()); + + const int next_pbo_uv = (counter + 1) % 2; + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_uv[next_pbo_uv]); + self->params.egl->glBufferData(GL_PIXEL_PACK_BUFFER, (frame->width/2 * frame->height/2) * 2, 0, GL_STREAM_READ); + //fprintf(stderr, "7 gl err: %d\n", self->params.egl->glGetError()); + uint8_t *ptr_uv = (uint8_t*)self->params.egl->glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY); + //fprintf(stderr, "8 gl err: %d\n", self->params.egl->glGetError()); + if(!ptr_uv) { + fprintf(stderr, "failed to map buffer uv!\n"); + } + + //self->sw_frame->buf[0] = av_buffer_create(ptr_y, 3840 * 2160, nop_free, NULL, 0); + //self->sw_frame->buf[1] = av_buffer_create(ptr_uv, 1920 * 1080 * 2, nop_free, NULL, 0); + //self->sw_frame->data[0] = self->sw_frame->buf[0]->data; + //self->sw_frame->data[1] = self->sw_frame->buf[1]->data; + //self->sw_frame->extended_data[0] = self->sw_frame->data[0]; + //self->sw_frame->extended_data[1] = self->sw_frame->data[1]; + + self->sw_frame->data[0] = ptr_y; + self->sw_frame->data[1] = ptr_uv; + + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); + self->params.egl->glBindFramebuffer(GL_READ_FRAMEBUFFER, 0); + + //self->params.egl->glBindTexture(GL_TEXTURE_2D, self->target_textures[1]); + //self->params.egl->glGetTexImage(GL_TEXTURE_2D, 0, GL_RG, GL_UNSIGNED_BYTE, sw_frame->data[1]); + + //self->params.egl->glBindTexture(GL_TEXTURE_2D, 0); + + int ret = av_hwframe_transfer_data(frame, self->sw_frame, 0); + if(ret < 0) { + fprintf(stderr, "transfer data failed, error: %s\n", av_err2str(ret)); + } + + //av_buffer_unref(&self->sw_frame->buf[0]); + //av_buffer_unref(&self->sw_frame->buf[1]); + + //av_frame_free(&sw_frame); + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[next_pbo_y]); + self->params.egl->glUnmapBuffer(GL_PIXEL_PACK_BUFFER); + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, self->pbo_y[next_pbo_uv]); + self->params.egl->glUnmapBuffer(GL_PIXEL_PACK_BUFFER); + self->params.egl->glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); +} + +static void gsr_video_encoder_vulkan_get_textures(gsr_video_encoder *encoder, unsigned int *textures, int *num_textures, gsr_destination_color *destination_color) { + gsr_video_encoder_vulkan *self = encoder->priv; + textures[0] = self->target_textures[0]; + textures[1] = self->target_textures[1]; + *num_textures = 2; + *destination_color = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? GSR_DESTINATION_COLOR_P010 : GSR_DESTINATION_COLOR_NV12; +} + +static void gsr_video_encoder_vulkan_destroy(gsr_video_encoder *encoder, AVCodecContext *video_codec_context) { + gsr_video_encoder_vulkan_stop(encoder->priv, video_codec_context); + free(encoder->priv); + free(encoder); +} + +gsr_video_encoder* gsr_video_encoder_vulkan_create(const gsr_video_encoder_vulkan_params *params) { + gsr_video_encoder *encoder = calloc(1, sizeof(gsr_video_encoder)); + if(!encoder) + return NULL; + + gsr_video_encoder_vulkan *encoder_vulkan = calloc(1, sizeof(gsr_video_encoder_vulkan)); + if(!encoder_vulkan) { + free(encoder); + return NULL; + } + + encoder_vulkan->params = *params; + + *encoder = (gsr_video_encoder) { + .start = gsr_video_encoder_vulkan_start, + .copy_textures_to_frame = gsr_video_encoder_vulkan_copy_textures_to_frame, + .get_textures = gsr_video_encoder_vulkan_get_textures, + .destroy = gsr_video_encoder_vulkan_destroy, + .priv = encoder_vulkan + }; + + return encoder; +} diff --git a/src/main.cpp b/src/main.cpp index 020deb7..21425d9 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -6,9 +6,13 @@ extern "C" { #include "../include/capture/portal.h" #include "../include/dbus.h" #endif -#include "../include/encoder/video/cuda.h" +#include "../include/encoder/video/nvenc.h" #include "../include/encoder/video/vaapi.h" +#include "../include/encoder/video/vulkan.h" #include "../include/encoder/video/software.h" +#include "../include/codec_query/nvenc.h" +#include "../include/codec_query/vaapi.h" +#include "../include/codec_query/vulkan.h" #include "../include/egl.h" #include "../include/utils.h" #include "../include/damage.h" @@ -28,6 +32,7 @@ extern "C" { #include <sys/stat.h> #include <unistd.h> #include <sys/wait.h> +#include <inttypes.h> #include <libgen.h> #include "../include/sound.hpp" @@ -100,7 +105,9 @@ enum class VideoCodec { AV1_HDR, AV1_10BIT, VP8, - VP9 + VP9, + H264_VULKAN, + HEVC_VULKAN }; enum class AudioCodec { @@ -122,7 +129,8 @@ enum class FramerateMode { enum class BitrateMode { QP, - VBR + VBR, + CBR }; static int x11_error_handler(Display*, XErrorEvent*) { @@ -134,6 +142,7 @@ static int x11_io_error_handler(Display*) { } static bool video_codec_is_hdr(VideoCodec video_codec) { + // TODO: Vulkan switch(video_codec) { case VideoCodec::HEVC_HDR: case VideoCodec::AV1_HDR: @@ -144,6 +153,7 @@ static bool video_codec_is_hdr(VideoCodec video_codec) { } static VideoCodec hdr_video_codec_to_sdr_video_codec(VideoCodec video_codec) { + // TODO: Vulkan switch(video_codec) { case VideoCodec::HEVC_HDR: return VideoCodec::HEVC; @@ -155,6 +165,7 @@ static VideoCodec hdr_video_codec_to_sdr_video_codec(VideoCodec video_codec) { } static gsr_color_depth video_codec_to_bit_depth(VideoCodec video_codec) { + // TODO: Vulkan switch(video_codec) { case VideoCodec::HEVC_HDR: case VideoCodec::HEVC_10BIT: @@ -167,6 +178,7 @@ static gsr_color_depth video_codec_to_bit_depth(VideoCodec video_codec) { } // static bool video_codec_is_hevc(VideoCodec video_codec) { +// TODO: Vulkan // switch(video_codec) { // case VideoCodec::HEVC: // case VideoCodec::HEVC_HDR: @@ -178,6 +190,7 @@ static gsr_color_depth video_codec_to_bit_depth(VideoCodec video_codec) { // } static bool video_codec_is_av1(VideoCodec video_codec) { + // TODO: Vulkan switch(video_codec) { case VideoCodec::AV1: case VideoCodec::AV1_HDR: @@ -188,6 +201,16 @@ static bool video_codec_is_av1(VideoCodec video_codec) { } } +static bool video_codec_is_vulkan(VideoCodec video_codec) { + switch(video_codec) { + case VideoCodec::H264_VULKAN: + case VideoCodec::HEVC_VULKAN: + return true; + default: + return false; + } +} + struct PacketData { PacketData() {} PacketData(const PacketData&) = delete; @@ -379,7 +402,7 @@ static AVSampleFormat audio_format_to_sample_format(const AudioFormat audio_form return AV_SAMPLE_FMT_S16; } -static AVCodecContext* create_audio_codec_context(int fps, AudioCodec audio_codec, bool mix_audio, int audio_bitrate) { +static AVCodecContext* create_audio_codec_context(int fps, AudioCodec audio_codec, bool mix_audio, int64_t audio_bitrate) { (void)fps; const AVCodec *codec = avcodec_find_encoder(audio_codec_get_id(audio_codec)); if (!codec) { @@ -466,7 +489,7 @@ static int vbr_get_quality_parameter(AVCodecContext *codec_context, VideoQuality static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, VideoQuality video_quality, int fps, const AVCodec *codec, bool low_latency, gsr_gpu_vendor vendor, FramerateMode framerate_mode, - bool hdr, gsr_color_range color_range, float keyint, bool use_software_video_encoder, BitrateMode bitrate_mode) { + bool hdr, gsr_color_range color_range, float keyint, bool use_software_video_encoder, BitrateMode bitrate_mode, VideoCodec video_codec, int64_t bitrate) { AVCodecContext *codec_context = avcodec_alloc_context3(codec); @@ -510,7 +533,13 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, if(codec->id == AV_CODEC_ID_HEVC) codec_context->codec_tag = MKTAG('h', 'v', 'c', '1'); // QuickTime on MacOS requires this or the video wont be playable - if(bitrate_mode == BitrateMode::VBR) { + if(bitrate_mode == BitrateMode::CBR) { + codec_context->bit_rate = bitrate; + codec_context->rc_max_rate = codec_context->bit_rate; + //codec_context->rc_min_rate = codec_context->bit_rate; + codec_context->rc_buffer_size = codec_context->bit_rate;//codec_context->bit_rate / 10; + codec_context->rc_initial_buffer_occupancy = 0;//codec_context->bit_rate;//codec_context->bit_rate * 1000; + } else if(bitrate_mode == BitrateMode::VBR) { const int quality = vbr_get_quality_parameter(codec_context, video_quality, hdr); switch(video_quality) { case VideoQuality::MEDIUM: @@ -536,15 +565,17 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, } codec_context->rc_max_rate = codec_context->bit_rate; - codec_context->rc_min_rate = codec_context->bit_rate; + //codec_context->rc_min_rate = codec_context->bit_rate; codec_context->rc_buffer_size = codec_context->bit_rate;//codec_context->bit_rate / 10; - codec_context->rc_initial_buffer_occupancy = 100000;//codec_context->bit_rate * 1000; + codec_context->rc_initial_buffer_occupancy = codec_context->bit_rate;//codec_context->bit_rate * 1000; + } else { + //codec_context->rc_buffer_size = 50000 * 1000; } //codec_context->profile = FF_PROFILE_H264_MAIN; if (codec_context->codec_id == AV_CODEC_ID_MPEG1VIDEO) codec_context->mb_decision = 2; - if(!use_software_video_encoder && vendor != GSR_GPU_VENDOR_NVIDIA) { + if(!use_software_video_encoder && vendor != GSR_GPU_VENDOR_NVIDIA && bitrate_mode != BitrateMode::CBR) { // 8 bit / 10 bit = 80%, and increase it even more const float quality_multiply = hdr ? (8.0f/10.0f * 0.7f) : 1.0f; if(codec_context->codec_id == AV_CODEC_ID_AV1 || codec_context->codec_id == AV_CODEC_ID_H264 || codec_context->codec_id == AV_CODEC_ID_HEVC) { @@ -556,7 +587,7 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, codec_context->global_quality = 120 * quality_multiply; break; case VideoQuality::VERY_HIGH: - codec_context->global_quality = 100 * quality_multiply; + codec_context->global_quality = 115 * quality_multiply; break; case VideoQuality::ULTRA: codec_context->global_quality = 90 * quality_multiply; @@ -571,7 +602,7 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, codec_context->global_quality = 30 * quality_multiply; break; case VideoQuality::VERY_HIGH: - codec_context->global_quality = 20 * quality_multiply; + codec_context->global_quality = 25 * quality_multiply; break; case VideoQuality::ULTRA: codec_context->global_quality = 10 * quality_multiply; @@ -586,7 +617,7 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, codec_context->global_quality = 30 * quality_multiply; break; case VideoQuality::VERY_HIGH: - codec_context->global_quality = 20 * quality_multiply; + codec_context->global_quality = 25 * quality_multiply; break; case VideoQuality::ULTRA: codec_context->global_quality = 10 * quality_multiply; @@ -601,10 +632,35 @@ static AVCodecContext *create_video_codec_context(AVPixelFormat pix_fmt, if(vendor != GSR_GPU_VENDOR_NVIDIA) { // TODO: More options, better options //codec_context->bit_rate = codec_context->width * codec_context->height; - if(bitrate_mode == BitrateMode::QP) - av_opt_set(codec_context->priv_data, "rc_mode", "CQP", 0); - else - av_opt_set(codec_context->priv_data, "rc_mode", "VBR", 0); + switch(bitrate_mode) { + case BitrateMode::QP: { + if(video_codec_is_vulkan(video_codec)) + av_opt_set(codec_context->priv_data, "rc_mode", "cqp", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_opt_set(codec_context->priv_data, "rc", "constqp", 0); + else + av_opt_set(codec_context->priv_data, "rc_mode", "CQP", 0); + break; + } + case BitrateMode::VBR: { + if(video_codec_is_vulkan(video_codec)) + av_opt_set(codec_context->priv_data, "rc_mode", "vbr", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_opt_set(codec_context->priv_data, "rc", "vbr", 0); + else + av_opt_set(codec_context->priv_data, "rc_mode", "VBR", 0); + break; + } + case BitrateMode::CBR: { + if(video_codec_is_vulkan(video_codec)) + av_opt_set(codec_context->priv_data, "rc_mode", "cbr", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_opt_set(codec_context->priv_data, "rc", "cbr", 0); + else + av_opt_set(codec_context->priv_data, "rc_mode", "CBR", 0); + break; + } + } //codec_context->global_quality = 4; //codec_context->compression_level = 2; } @@ -654,6 +710,50 @@ static AVFrame* create_audio_frame(AVCodecContext *audio_codec_context) { return frame; } +static void dict_set_profile(AVCodecContext *codec_context, gsr_gpu_vendor vendor, gsr_color_depth color_depth, AVDictionary **options) { + #if LIBAVCODEC_VERSION_INT < AV_VERSION_INT(61, 17, 100) + if(codec_context->codec_id == AV_CODEC_ID_H264) { + // TODO: Only for vaapi + //if(color_depth == GSR_COLOR_DEPTH_10_BITS) + // av_dict_set(options, "profile", "high10", 0); + //else + av_dict_set(options, "profile", "high", 0); + } else if(codec_context->codec_id == AV_CODEC_ID_AV1) { + if(vendor == GSR_GPU_VENDOR_NVIDIA) { + if(color_depth == GSR_COLOR_DEPTH_10_BITS) + av_dict_set_int(options, "highbitdepth", 1, 0); + } else { + av_dict_set(options, "profile", "main", 0); // TODO: use professional instead? + } + } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { + if(color_depth == GSR_COLOR_DEPTH_10_BITS) + av_dict_set(options, "profile", "main10", 0); + else + av_dict_set(options, "profile", "main", 0); + } + #else + if(codec_context->codec_id == AV_CODEC_ID_H264) { + // TODO: Only for vaapi + //if(color_depth == GSR_COLOR_DEPTH_10_BITS) + // av_dict_set_int(options, "profile", AV_PROFILE_H264_HIGH_10, 0); + //else + av_dict_set_int(options, "profile", AV_PROFILE_H264_HIGH, 0); + } else if(codec_context->codec_id == AV_CODEC_ID_AV1) { + if(vendor == GSR_GPU_VENDOR_NVIDIA) { + if(color_depth == GSR_COLOR_DEPTH_10_BITS) + av_dict_set_int(options, "highbitdepth", 1, 0); + } else { + av_dict_set_int(options, "profile", AV_PROFILE_AV1_MAIN, 0); // TODO: use professional instead? + } + } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { + if(color_depth == GSR_COLOR_DEPTH_10_BITS) + av_dict_set_int(options, "profile", AV_PROFILE_HEVC_MAIN_10, 0); + else + av_dict_set_int(options, "profile", AV_PROFILE_HEVC_MAIN, 0); + } + #endif +} + static void video_software_set_qp(AVCodecContext *codec_context, VideoQuality video_quality, bool hdr, AVDictionary **options) { // 8 bit / 10 bit = 80% const float qp_multiply = hdr ? 8.0f/10.0f : 1.0f; @@ -681,10 +781,10 @@ static void video_software_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 23 * qp_multiply, 0); + av_dict_set_int(options, "qp", 25 * qp_multiply, 0); break; case VideoQuality::ULTRA: - av_dict_set_int(options, "qp", 20 * qp_multiply, 0); + av_dict_set_int(options, "qp", 22 * qp_multiply, 0); break; } } else { @@ -712,14 +812,9 @@ static void open_video_software(AVCodecContext *codec_context, VideoQuality vide if(bitrate_mode == BitrateMode::QP) video_software_set_qp(codec_context, video_quality, hdr, &options); - av_dict_set(&options, "preset", "medium", 0); - if(color_depth == GSR_COLOR_DEPTH_10_BITS) { - av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH_10, 0); - } else { - av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH, 0); - } - // TODO: If streaming or piping output set this to zerolatency - av_dict_set(&options, "tune", "fastdecode", 0); + av_dict_set(&options, "preset", "veryfast", 0); + av_dict_set(&options, "tune", "film", 0); + dict_set_profile(codec_context, GSR_GPU_VENDOR_INTEL, color_depth, &options); if(codec_context->codec_id == AV_CODEC_ID_H264) { av_dict_set(&options, "coder", "cabac", 0); // TODO: cavlc is faster than cabac but worse compression. Which to use? @@ -734,6 +829,38 @@ static void open_video_software(AVCodecContext *codec_context, VideoQuality vide } } +static void video_set_rc(VideoCodec video_codec, gsr_gpu_vendor vendor, BitrateMode bitrate_mode, AVDictionary **options) { + switch(bitrate_mode) { + case BitrateMode::QP: { + if(video_codec_is_vulkan(video_codec)) + av_dict_set(options, "rc_mode", "cqp", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_dict_set(options, "rc", "constqp", 0); + else + av_dict_set(options, "rc_mode", "CQP", 0); + break; + } + case BitrateMode::VBR: { + if(video_codec_is_vulkan(video_codec)) + av_dict_set(options, "rc_mode", "vbr", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_dict_set(options, "rc", "vbr", 0); + else + av_dict_set(options, "rc_mode", "VBR", 0); + break; + } + case BitrateMode::CBR: { + if(video_codec_is_vulkan(video_codec)) + av_dict_set(options, "rc_mode", "cbr", 0); + else if(vendor == GSR_GPU_VENDOR_NVIDIA) + av_dict_set(options, "rc", "cbr", 0); + else + av_dict_set(options, "rc_mode", "CBR", 0); + break; + } + } +} + static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality video_quality, gsr_gpu_vendor vendor, bool hdr, AVDictionary **options) { // 8 bit / 10 bit = 80% const float qp_multiply = hdr ? 8.0f/10.0f : 1.0f; @@ -748,7 +875,7 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 25 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: av_dict_set_int(options, "qp", 22 * qp_multiply, 0); @@ -757,16 +884,16 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi } else if(codec_context->codec_id == AV_CODEC_ID_H264) { switch(video_quality) { case VideoQuality::MEDIUM: - av_dict_set_int(options, "qp", 34 * qp_multiply, 0); + av_dict_set_int(options, "qp", 35 * qp_multiply, 0); break; case VideoQuality::HIGH: av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 23 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: - av_dict_set_int(options, "qp", 20 * qp_multiply, 0); + av_dict_set_int(options, "qp", 22 * qp_multiply, 0); break; } } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { @@ -778,7 +905,7 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 25 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: av_dict_set_int(options, "qp", 22 * qp_multiply, 0); @@ -793,31 +920,29 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 25 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: av_dict_set_int(options, "qp", 22 * qp_multiply, 0); break; } } - - av_dict_set(options, "rc", "constqp", 0); } else { if(codec_context->codec_id == AV_CODEC_ID_AV1) { // Using global_quality option } else if(codec_context->codec_id == AV_CODEC_ID_H264) { switch(video_quality) { case VideoQuality::MEDIUM: - av_dict_set_int(options, "qp", 34 * qp_multiply, 0); + av_dict_set_int(options, "qp", 35 * qp_multiply, 0); break; case VideoQuality::HIGH: av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 23 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: - av_dict_set_int(options, "qp", 20 * qp_multiply, 0); + av_dict_set_int(options, "qp", 22 * qp_multiply, 0); break; } } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { @@ -829,7 +954,7 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 25 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: av_dict_set_int(options, "qp", 22 * qp_multiply, 0); @@ -844,51 +969,54 @@ static void video_hardware_set_qp(AVCodecContext *codec_context, VideoQuality vi av_dict_set_int(options, "qp", 30 * qp_multiply, 0); break; case VideoQuality::VERY_HIGH: - av_dict_set_int(options, "qp", 25 * qp_multiply, 0); + av_dict_set_int(options, "qp", 27 * qp_multiply, 0); break; case VideoQuality::ULTRA: av_dict_set_int(options, "qp", 22 * qp_multiply, 0); break; } } - - av_dict_set(options, "rc_mode", "CQP", 0); } } -static void open_video_hardware(AVCodecContext *codec_context, VideoQuality video_quality, bool very_old_gpu, gsr_gpu_vendor vendor, PixelFormat pixel_format, bool hdr, gsr_color_depth color_depth, BitrateMode bitrate_mode) { +static void open_video_hardware(AVCodecContext *codec_context, VideoQuality video_quality, bool very_old_gpu, gsr_gpu_vendor vendor, PixelFormat pixel_format, bool hdr, gsr_color_depth color_depth, BitrateMode bitrate_mode, VideoCodec video_codec, bool low_power) { (void)very_old_gpu; AVDictionary *options = nullptr; - if(bitrate_mode == BitrateMode::QP) { + if(bitrate_mode == BitrateMode::QP) video_hardware_set_qp(codec_context, video_quality, vendor, hdr, &options); - } else { - if(vendor == GSR_GPU_VENDOR_NVIDIA) { - av_dict_set(&options, "rc", "vbr", 0); - } else { - av_dict_set(&options, "rc_mode", "VBR", 0); - } - } + + video_set_rc(video_codec, vendor, bitrate_mode, &options); + + // TODO: Enable multipass + + // TODO: Set "usage" option to "record"/"stream" and "content" option to "rendered" for vulkan encoding if(vendor == GSR_GPU_VENDOR_NVIDIA) { + // TODO: These dont seem to be necessary + // av_dict_set_int(&options, "zerolatency", 1, 0); + // if(codec_context->codec_id == AV_CODEC_ID_AV1) { + // av_dict_set(&options, "tune", "ll", 0); + // } else if(codec_context->codec_id == AV_CODEC_ID_H264 || codec_context->codec_id == AV_CODEC_ID_HEVC) { + // av_dict_set(&options, "preset", "llhq", 0); + // av_dict_set(&options, "tune", "ll", 0); + // } av_dict_set(&options, "tune", "hq", 0); - // TODO: Enable multipass + dict_set_profile(codec_context, vendor, color_depth, &options); if(codec_context->codec_id == AV_CODEC_ID_H264) { // TODO: h264 10bit? - switch(pixel_format) { - case PixelFormat::YUV420: - av_dict_set(&options, "profile", "high", 0); - break; - case PixelFormat::YUV444: - av_dict_set(&options, "profile", "high444p", 0); - break; - } + // TODO: + // switch(pixel_format) { + // case PixelFormat::YUV420: + // av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH, 0); + // break; + // case PixelFormat::YUV444: + // av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH_444, 0); + // break; + // } } else if(codec_context->codec_id == AV_CODEC_ID_AV1) { - if(color_depth == GSR_COLOR_DEPTH_10_BITS) - av_dict_set_int(&options, "highbitdepth", 1, 0); - switch(pixel_format) { case PixelFormat::YUV420: av_dict_set(&options, "rgb_mode", "yuv420", 0); @@ -899,34 +1027,20 @@ static void open_video_hardware(AVCodecContext *codec_context, VideoQuality vide } } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { //av_dict_set(&options, "pix_fmt", "yuv420p16le", 0); - if(color_depth == GSR_COLOR_DEPTH_10_BITS) - av_dict_set_int(&options, "profile", AV_PROFILE_HEVC_MAIN_10, 0); - else - av_dict_set_int(&options, "profile", AV_PROFILE_HEVC_MAIN, 0); } } else { // TODO: More quality options - //av_dict_set_int(&options, "low_power", 1, 0); + if(low_power) + av_dict_set_int(&options, "low_power", 1, 0); // Improves performance but increases vram - av_dict_set_int(&options, "async_depth", 8, 0); + //av_dict_set_int(&options, "async_depth", 8, 0); if(codec_context->codec_id == AV_CODEC_ID_H264) { - // TODO: - if(color_depth == GSR_COLOR_DEPTH_10_BITS) - av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH_10, 0); - else - av_dict_set_int(&options, "profile", AV_PROFILE_H264_HIGH, 0); // Removed because it causes stutter in games for some people //av_dict_set_int(&options, "quality", 5, 0); // quality preset } else if(codec_context->codec_id == AV_CODEC_ID_AV1) { - av_dict_set_int(&options, "profile", AV_PROFILE_AV1_MAIN, 0); // TODO: use professional instead? av_dict_set(&options, "tier", "main", 0); } else if(codec_context->codec_id == AV_CODEC_ID_HEVC) { - if(color_depth == GSR_COLOR_DEPTH_10_BITS) - av_dict_set_int(&options, "profile", AV_PROFILE_HEVC_MAIN_10, 0); - else - av_dict_set_int(&options, "profile", AV_PROFILE_HEVC_MAIN, 0); - if(hdr) av_dict_set(&options, "sei", "hdr", 0); } @@ -950,7 +1064,7 @@ static void open_video_hardware(AVCodecContext *codec_context, VideoQuality vide static void usage_header() { const bool inside_flatpak = getenv("FLATPAK_ID") != NULL; const char *program_name = inside_flatpak ? "flatpak run --command=gpu-screen-recorder com.dec05eba.gpu_screen_recorder" : "gpu-screen-recorder"; - fprintf(stderr, "usage: %s -w <window_id|monitor|focused|portal> [-c <container_format>] [-s WxH] -f <fps> [-a <audio_input>] [-q <quality>] [-r <replay_buffer_size_sec>] [-k h264|hevc|av1|vp8|vp9|hevc_hdr|av1_hdr|hevc_10bit|av1_10bit] [-ac aac|opus|flac] [-ab <bitrate>] [-oc yes|no] [-fm cfr|vfr|content] [-bm auto|qp|vbr] [-cr limited|full] [-df yes|no] [-sc <script_path>] [-cursor yes|no] [-keyint <value>] [-restore-portal-session yes|no] [-portal-session-token-filepath filepath] [-encoder gpu|cpu] [-o <output_file>] [-v yes|no] [--version] [-h|--help]\n", program_name); + fprintf(stderr, "usage: %s -w <window_id|monitor|focused|portal> [-c <container_format>] [-s WxH] -f <fps> [-a <audio_input>] [-aa <application_name>] [-q <quality>] [-r <replay_buffer_size_sec>] [-k h264|hevc|av1|vp8|vp9|hevc_hdr|av1_hdr|hevc_10bit|av1_10bit] [-ac aac|opus|flac] [-ab <bitrate>] [-oc yes|no] [-fm cfr|vfr|content] [-bm auto|qp|vbr|cbr] [-cr limited|full] [-df yes|no] [-sc <script_path>] [-cursor yes|no] [-keyint <value>] [-restore-portal-session yes|no] [-portal-session-token-filepath filepath] [-encoder gpu|cpu] [-o <output_file>] [-v yes|no] [--version] [-h|--help]\n", program_name); } // TODO: Update with portal info @@ -972,7 +1086,9 @@ static void usage_full() { fprintf(stderr, " If an output file is specified and -c is not used then the container format is determined from the output filename extension.\n"); fprintf(stderr, " Only containers that support h264, hevc, av1, vp8 or vp9 are supported, which means that only mp4, mkv, flv, webm (and some others) are supported.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -s The size (area) to record at in the format WxH, for example 1920x1080. This option is only supported (and required) when -w is \"focused\".\n"); + fprintf(stderr, " -s The output resolution limit of the video in the format WxH, for example 1920x1080. If this is 0x0 then the original resolution is used. Optional, except when -w is \"focused\".\n"); + fprintf(stderr, " Note: the captured content is scaled to this size. The output resolution might not be exactly as specified by this option. The original aspect ratio is respected so the resolution will match that.\n"); + fprintf(stderr, " The video encoder might also need to add padding, which will result in black bars on the sides of the video. This is especially an issue on AMD.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -f Frame rate to record at. Recording will only capture frames at this target frame rate.\n"); fprintf(stderr, " For constant frame rate mode this option is the frame rate every frame will be captured at and if the capture frame rate is below this target frame rate then the frames will be duplicated.\n"); @@ -986,15 +1102,27 @@ static void usage_full() { fprintf(stderr, " If the audio device is an empty string then the audio device is ignored.\n"); fprintf(stderr, " Optional, no audio track is added by default.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -q Video quality. Should be either 'medium', 'high', 'very_high' or 'ultra'. 'high' is the recommended option when live streaming or when you have a slower harddrive.\n"); - fprintf(stderr, " Optional, set to 'very_high' be default.\n"); +#ifdef GSR_APP_AUDIO + fprintf(stderr, " -aa Audio device to record from (pulse audio device). Can be specified multiple times. Each time this is specified a new audio track is added for the specified audio device.\n"); + fprintf(stderr, " A name can be given to the audio input device by prefixing the audio input with <name>/, for example \"dummy/alsa_output.pci-0000_00_1b.0.analog-stereo.monitor\".\n"); + fprintf(stderr, " Multiple audio devices can be merged into one audio track by using \"|\" as a separator into one -a argument, for example: -a \"alsa_output1|alsa_output2\".\n"); + fprintf(stderr, " The audio device can also be \"default_output\" in which case the default output device is used, or \"default_input\" in which case the default input device is used.\n"); + fprintf(stderr, " If the audio device is an empty string then the audio device is ignored.\n"); + fprintf(stderr, " Optional, no audio track is added by default.\n"); + fprintf(stderr, "\n"); +#endif + fprintf(stderr, " -q Video quality. Should be either 'medium', 'high', 'very_high' or 'ultra' when using '-bm qp' or '-bm vbr' options, and '-bm qp' is the default option used.\n"); + fprintf(stderr, " 'high' is the recommended option when live streaming or when you have a slower harddrive.\n"); + fprintf(stderr, " When using '-bm cbr' option then this is option is instead used to specify the video bitrate in kbps.\n"); + fprintf(stderr, " Optional when using '-bm qp' or '-bm vbr' options, set to 'very_high' be default.\n"); + fprintf(stderr, " Required when using '-bm cbr' option.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -r Replay buffer size in seconds. If this is set, then only the last seconds as set by this option will be stored\n"); fprintf(stderr, " and the video will only be saved when the gpu-screen-recorder is closed. This feature is similar to Nvidia's instant replay feature.\n"); fprintf(stderr, " This option has be between 5 and 1200. Note that the replay buffer size will not always be precise, because of keyframes. Optional, disabled by default.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -k Video codec to use. Should be either 'auto', 'h264', 'hevc', 'av1', 'vp8', 'vp9', 'hevc_hdr', 'av1_hdr', 'hevc_10bit' or 'av1_10bit'. Optional, set to 'auto' by default which defaults to 'h264'.\n"); - fprintf(stderr, " Forcefully set to 'h264' if the file container type is 'flv'.\n"); + fprintf(stderr, " -k Video codec to use. Should be either 'auto', 'h264', 'hevc', 'av1', 'vp8', 'vp9', 'hevc_hdr', 'av1_hdr', 'hevc_10bit' or 'av1_10bit'.\n"); + fprintf(stderr, " Optional, set to 'auto' by default which defaults to 'h264'. Forcefully set to 'h264' if the file container type is 'flv'.\n"); fprintf(stderr, " 'hevc_hdr' and 'av1_hdr' option is not available on X11 nor when using the portal capture option.\n"); fprintf(stderr, " 'hevc_10bit' and 'av1_10bit' options allow you to select 10 bit color depth which can reduce banding and improve quality in darker areas, but not all video players support 10 bit color depth\n"); fprintf(stderr, " and if you upload the video to a website the website might reduce 10 bit to 8 bit.\n"); @@ -1004,8 +1132,8 @@ static void usage_full() { fprintf(stderr, " 'opus' and 'flac' is only supported by .mp4/.mkv files. 'opus' is recommended for best performance and smallest audio size.\n"); fprintf(stderr, " Flac audio codec is option is disable at the moment because of a temporary issue.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -ab Audio bitrate to use. If this is set to 0 then it's the same as if it's absent, in which case the bitrate is determined automatically depending on the audio codec.\n"); - fprintf(stderr, " Optional, by default the bitrate is 128000 for opus and flac and 160000 for aac.\n"); + fprintf(stderr, " -ab Audio bitrate in kbps. If this is set to 0 then it's the same as if it's absent, in which case the bitrate is determined automatically depending on the audio codec.\n"); + fprintf(stderr, " Optional, by default the bitrate is 128kbps for opus and flac and 160kbps for aac.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -oc Overclock memory transfer rate to the maximum performance level. This only applies to NVIDIA on X11 and exists to overcome a bug in NVIDIA driver where performance level\n"); fprintf(stderr, " is dropped when you record a game. Only needed if you are recording a game that is bottlenecked by GPU. The same issue exists on Wayland but overclocking is not possible on Wayland.\n"); @@ -1015,9 +1143,9 @@ static void usage_full() { fprintf(stderr, " 'vfr' is recommended for recording for less issue with very high system load but some applications such as video editors may not support it properly.\n"); fprintf(stderr, " 'content' is currently only supported on X11 or when using portal capture option. The 'content' option matches the recording frame rate to the captured content.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -bm Bitrate mode. Should be either 'auto', 'qp' (constant quality) or 'vbr' (variable bitrate). Optional, set to 'auto' by default which defaults to 'qp' on all devices\n"); + fprintf(stderr, " -bm Bitrate mode. Should be either 'auto', 'qp' (constant quality), 'vbr' (variable bitrate) or 'cbr' (constant bitrate). Optional, set to 'auto' by default which defaults to 'qp' on all devices\n"); fprintf(stderr, " except steam deck that has broken drivers and doesn't support qp.\n"); - fprintf(stderr, " 'vbr' option is not supported when using '-encoder cpu' option.\n"); + fprintf(stderr, " Note: 'vbr' option is not supported when using '-encoder cpu' option.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -cr Color range. Should be either 'limited' (aka mpeg) or 'full' (aka jpeg). Optional, set to 'limited' by default.\n"); fprintf(stderr, " Limited color range means that colors are in range 16-235 (4112-60395 for hdr) while full color range means that colors are in range 0-255 (0-65535 for hdr).\n"); @@ -1049,12 +1177,12 @@ static void usage_full() { fprintf(stderr, " Note: the directory to the portal session token file is created automatically if it doesn't exist.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -encoder\n"); - fprintf(stderr, " Which device should be used for video encoding. Should either be 'gpu' or 'cpu'. Does currently only work with h264 codec option (-k).\n"); + fprintf(stderr, " Which device should be used for video encoding. Should either be 'gpu' or 'cpu'. 'cpu' option currently only work with h264 codec option (-k).\n"); fprintf(stderr, " Optional, set to 'gpu' by default.\n"); fprintf(stderr, "\n"); fprintf(stderr, " --info\n"); fprintf(stderr, " List info about the system (for use by GPU Screen Recorder UI). Lists the following information (prints them to stdout and exits):\n"); - fprintf(stderr, " Supported video codecs (h264, h264_software, hevc, hevc_hdr, hevc_10bit, av1, av1_hdr, av1_10bit, vp8, vp9, (if supported)).\n"); + fprintf(stderr, " Supported video codecs (h264, h264_software, hevc, hevc_hdr, hevc_10bit, av1, av1_hdr, av1_10bit, vp8, vp9 (if supported)).\n"); fprintf(stderr, " Supported capture options (window, focused, screen, monitors and portal, if supported by the system).\n"); fprintf(stderr, " If opengl initialization fails then the program exits with 22, if no usable drm device is found then it exits with 23. On success it exits with 0.\n"); fprintf(stderr, "\n"); @@ -1064,6 +1192,7 @@ static void usage_full() { fprintf(stderr, " For example:\n"); fprintf(stderr, " bluez_input.88:C9:E8:66:A2:27|WH-1000XM4\n"); fprintf(stderr, " The <audio_device_name> is the name to pass to GPU Screen Recorder in a -a option.\n"); + fprintf(stderr, "\n"); fprintf(stderr, " --version\n"); fprintf(stderr, " Print version (%s) and exit\n", GSR_VERSION); fprintf(stderr, "\n"); @@ -1072,7 +1201,7 @@ static void usage_full() { fprintf(stderr, " In replay mode this has to be a directory instead of a file.\n"); fprintf(stderr, " Note: the directory to the file is created automatically if it doesn't already exist.\n"); fprintf(stderr, "\n"); - fprintf(stderr, " -v Prints per second, fps updates. Optional, set to 'yes' by default.\n"); + fprintf(stderr, " -v Prints fps and damage info once per second. Optional, set to 'yes' by default.\n"); fprintf(stderr, "\n"); fprintf(stderr, " -h, --help\n"); fprintf(stderr, " Show this help.\n"); @@ -1084,10 +1213,12 @@ static void usage_full() { fprintf(stderr, "\n"); fprintf(stderr, "EXAMPLES:\n"); fprintf(stderr, " %s -w screen -f 60 -a default_output -o \"$HOME/Videos/video.mp4\"\n", program_name); + fprintf(stderr, " %s -w screen -f 60 -a default_output -a default_input -o \"$HOME/Videos/video.mp4\"\n", program_name); fprintf(stderr, " %s -w screen -f 60 -a \"default_output|default_input\" -o \"$HOME/Videos/video.mp4\"\n", program_name); fprintf(stderr, " %s -w screen -f 60 -a default_output -c mkv -r 60 -o \"$HOME/Videos\"\n", program_name); fprintf(stderr, " %s -w screen -f 60 -a default_output -c mkv -sc script.sh -r 60 -o \"$HOME/Videos\"\n", program_name); fprintf(stderr, " %s -w portal -f 60 -a default_output -restore-portal-session yes -o \"$HOME/Videos/video.mp4\"\n", program_name); + fprintf(stderr, " %s -w screen -f 60 -a default_output -bm cbr -q 15000 -o \"$HOME/Videos/video.mp4\"\n", program_name); //fprintf(stderr, " gpu-screen-recorder -w screen -f 60 -q ultra -pixfmt yuv444 -o video.mp4\n"); _exit(1); } @@ -1260,8 +1391,14 @@ struct AudioTrack { static bool add_hdr_metadata_to_video_stream(gsr_capture *cap, AVStream *video_stream) { size_t light_metadata_size = 0; + size_t mastering_display_metadata_size = 0; AVContentLightMetadata *light_metadata = av_content_light_metadata_alloc(&light_metadata_size); + #if LIBAVUTIL_VERSION_INT < AV_VERSION_INT(59, 37, 100) AVMasteringDisplayMetadata *mastering_display_metadata = av_mastering_display_metadata_alloc(); + mastering_display_metadata_size = sizeof(*mastering_display_metadata); + #else + AVMasteringDisplayMetadata *mastering_display_metadata = av_mastering_display_metadata_alloc_size(&mastering_display_metadata_size); + #endif if(!light_metadata || !mastering_display_metadata) { if(light_metadata) @@ -1282,17 +1419,24 @@ static bool add_hdr_metadata_to_video_stream(gsr_capture *cap, AVStream *video_s // TODO: More error checking #if LIBAVCODEC_VERSION_INT < AV_VERSION_INT(60, 31, 102) - const bool added_light_metadata = av_stream_add_side_data(video_stream, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, (uint8_t*)light_metadata, light_metadata_size); + const bool content_light_level_added = av_stream_add_side_data(video_stream, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, (uint8_t*)light_metadata, light_metadata_size) == 0; #else - av_packet_side_data_add(&video_stream->codecpar->coded_side_data, &video_stream->codecpar->nb_coded_side_data, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, light_metadata, light_metadata_size, 0); + const bool content_light_level_added = av_packet_side_data_add(&video_stream->codecpar->coded_side_data, &video_stream->codecpar->nb_coded_side_data, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, light_metadata, light_metadata_size, 0) != NULL; #endif #if LIBAVCODEC_VERSION_INT < AV_VERSION_INT(60, 31, 102) - const bool added_display_metadata = av_stream_add_side_data(video_stream, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, (uint8_t*)mastering_display_metadata, sizeof(*mastering_display_metadata)); + const bool mastering_display_metadata_added = av_stream_add_side_data(video_stream, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, (uint8_t*)mastering_display_metadata, mastering_display_metadata_size) == 0; #else - av_packet_side_data_add(&video_stream->codecpar->coded_side_data, &video_stream->codecpar->nb_coded_side_data, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, mastering_display_metadata, sizeof(*mastering_display_metadata), 0); + const bool mastering_display_metadata_added = av_packet_side_data_add(&video_stream->codecpar->coded_side_data, &video_stream->codecpar->nb_coded_side_data, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, mastering_display_metadata, mastering_display_metadata_size, 0) != NULL; #endif + if(!content_light_level_added) + av_freep(light_metadata); + + if(!mastering_display_metadata_added) + av_freep(mastering_display_metadata); + + // Return true even on failure because we dont want to retry adding hdr metadata on failure return true; } @@ -1453,16 +1597,27 @@ static void split_string(const std::string &str, char delimiter, std::function<b } } -static std::vector<AudioInput> parse_audio_input_arg(const char *str) { +static const AudioInput* get_audio_device_by_name(const std::vector<AudioInput> &audio_inputs, const std::string &name) { + for(const auto &audio_input : audio_inputs) { + if(audio_input.name == name) + return &audio_input; + } + return nullptr; +} + +static std::vector<AudioInput> parse_audio_input_arg(const char *str, const AudioDevices &audio_devices) { std::vector<AudioInput> audio_inputs; - split_string(str, '|', [&audio_inputs](const char *sub, size_t size) { + split_string(str, '|', [&](const char *sub, size_t size) { AudioInput audio_input; audio_input.name.assign(sub, size); + + const bool name_is_existing_audio_device = get_audio_device_by_name(audio_devices.audio_inputs, audio_input.name); const size_t index = audio_input.name.find('/'); - if(index != std::string::npos) { + if(!name_is_existing_audio_device && index != std::string::npos) { audio_input.description = audio_input.name.substr(0, index); audio_input.name.erase(audio_input.name.begin(), audio_input.name.begin() + index + 1); } + audio_inputs.push_back(std::move(audio_input)); return true; }); @@ -1592,7 +1747,7 @@ static int init_filter_graph(AVCodecContext *audio_codec_context, AVFilterGraph return 0; } -static gsr_video_encoder* create_video_encoder(gsr_egl *egl, bool overclock, gsr_color_depth color_depth, bool use_software_video_encoder) { +static gsr_video_encoder* create_video_encoder(gsr_egl *egl, bool overclock, gsr_color_depth color_depth, bool use_software_video_encoder, VideoCodec video_codec) { gsr_video_encoder *video_encoder = nullptr; if(use_software_video_encoder) { @@ -1603,6 +1758,14 @@ static gsr_video_encoder* create_video_encoder(gsr_egl *egl, bool overclock, gsr return video_encoder; } + if(video_codec_is_vulkan(video_codec)) { + gsr_video_encoder_vulkan_params params; + params.egl = egl; + params.color_depth = color_depth; + video_encoder = gsr_video_encoder_vulkan_create(¶ms); + return video_encoder; + } + switch(egl->gpu_info.vendor) { case GSR_GPU_VENDOR_AMD: case GSR_GPU_VENDOR_INTEL: { @@ -1613,11 +1776,11 @@ static gsr_video_encoder* create_video_encoder(gsr_egl *egl, bool overclock, gsr break; } case GSR_GPU_VENDOR_NVIDIA: { - gsr_video_encoder_cuda_params params; + gsr_video_encoder_nvenc_params params; params.egl = egl; params.overclock = overclock; params.color_depth = color_depth; - video_encoder = gsr_video_encoder_cuda_create(¶ms); + video_encoder = gsr_video_encoder_nvenc_create(¶ms); break; } } @@ -1625,6 +1788,28 @@ static gsr_video_encoder* create_video_encoder(gsr_egl *egl, bool overclock, gsr return video_encoder; } +static bool get_supported_video_codecs(gsr_egl *egl, VideoCodec video_codec, bool use_software_video_encoder, bool cleanup, gsr_supported_video_codecs *video_codecs) { + memset(video_codecs, 0, sizeof(*video_codecs)); + + if(use_software_video_encoder) { + video_codecs->h264.supported = true; + return true; + } + + if(video_codec_is_vulkan(video_codec)) + return gsr_get_supported_video_codecs_vulkan(video_codecs, egl->card_path, cleanup); + + switch(egl->gpu_info.vendor) { + case GSR_GPU_VENDOR_AMD: + case GSR_GPU_VENDOR_INTEL: + return gsr_get_supported_video_codecs_vaapi(video_codecs, egl->card_path, cleanup); + case GSR_GPU_VENDOR_NVIDIA: + return gsr_get_supported_video_codecs_nvenc(video_codecs, cleanup); + } + + return false; +} + static void xwayland_check_callback(const gsr_monitor *monitor, void *userdata) { bool *xwayland_found = (bool*)userdata; if(monitor->name_len >= 8 && strncmp(monitor->name, "XWAYLAND", 8) == 0) @@ -1689,65 +1874,84 @@ static const AVCodec* get_ffmpeg_video_codec(VideoCodec video_codec, gsr_gpu_ven return avcodec_find_encoder_by_name(vendor == GSR_GPU_VENDOR_NVIDIA ? "vp8_nvenc" : "vp8_vaapi"); case VideoCodec::VP9: return avcodec_find_encoder_by_name(vendor == GSR_GPU_VENDOR_NVIDIA ? "vp9_nvenc" : "vp9_vaapi"); + case VideoCodec::H264_VULKAN: + return avcodec_find_encoder_by_name("h264_vulkan"); + case VideoCodec::HEVC_VULKAN: + return avcodec_find_encoder_by_name("hevc_vulkan"); } return nullptr; } -static void set_supported_video_codecs_ffmpeg(gsr_supported_video_codecs *supported_video_codecs, gsr_gpu_vendor vendor) { +static void set_supported_video_codecs_ffmpeg(gsr_supported_video_codecs *supported_video_codecs, gsr_supported_video_codecs *supported_video_codecs_vulkan, gsr_gpu_vendor vendor) { if(!get_ffmpeg_video_codec(VideoCodec::H264, vendor)) { - supported_video_codecs->h264 = false; + supported_video_codecs->h264.supported = false; } if(!get_ffmpeg_video_codec(VideoCodec::HEVC, vendor)) { - supported_video_codecs->hevc = false; - supported_video_codecs->hevc_hdr = false; - supported_video_codecs->hevc_10bit = false; + supported_video_codecs->hevc.supported = false; + supported_video_codecs->hevc_hdr.supported = false; + supported_video_codecs->hevc_10bit.supported = false; } if(!get_ffmpeg_video_codec(VideoCodec::AV1, vendor)) { - supported_video_codecs->av1 = false; - supported_video_codecs->av1_hdr = false; - supported_video_codecs->av1_10bit = false; + supported_video_codecs->av1.supported = false; + supported_video_codecs->av1_hdr.supported = false; + supported_video_codecs->av1_10bit.supported = false; } if(!get_ffmpeg_video_codec(VideoCodec::VP8, vendor)) { - supported_video_codecs->vp8 = false; + supported_video_codecs->vp8.supported = false; } if(!get_ffmpeg_video_codec(VideoCodec::VP9, vendor)) { - supported_video_codecs->vp9 = false; + supported_video_codecs->vp9.supported = false; + } + + if(!get_ffmpeg_video_codec(VideoCodec::H264_VULKAN, vendor)) { + supported_video_codecs_vulkan->h264.supported = false; + } + + if(!get_ffmpeg_video_codec(VideoCodec::HEVC_VULKAN, vendor)) { + supported_video_codecs_vulkan->hevc.supported = false; + supported_video_codecs_vulkan->hevc_hdr.supported = false; + supported_video_codecs_vulkan->hevc_10bit.supported = false; } } static void list_supported_video_codecs(gsr_egl *egl, bool wayland) { // Dont clean it up on purpose to increase shutdown speed - gsr_video_encoder *video_encoder = create_video_encoder(egl, false, GSR_COLOR_DEPTH_8_BITS, false); - if(!video_encoder) - return; - - gsr_supported_video_codecs supported_video_codecs = gsr_video_encoder_get_supported_codecs(video_encoder, false); - set_supported_video_codecs_ffmpeg(&supported_video_codecs, egl->gpu_info.vendor); + gsr_supported_video_codecs supported_video_codecs; + get_supported_video_codecs(egl, VideoCodec::H264, false, false, &supported_video_codecs); + + gsr_supported_video_codecs supported_video_codecs_vulkan; + get_supported_video_codecs(egl, VideoCodec::H264_VULKAN, false, false, &supported_video_codecs_vulkan); - if(supported_video_codecs.h264) + set_supported_video_codecs_ffmpeg(&supported_video_codecs, &supported_video_codecs_vulkan, egl->gpu_info.vendor); + + if(supported_video_codecs.h264.supported) puts("h264"); if(avcodec_find_encoder_by_name("libx264")) puts("h264_software"); - if(supported_video_codecs.hevc) + if(supported_video_codecs.hevc.supported) puts("hevc"); - if(supported_video_codecs.hevc_hdr && wayland) + if(supported_video_codecs.hevc_hdr.supported && wayland) puts("hevc_hdr"); - if(supported_video_codecs.hevc_10bit) + if(supported_video_codecs.hevc_10bit.supported) puts("hevc_10bit"); - if(supported_video_codecs.av1) + if(supported_video_codecs.av1.supported) puts("av1"); - if(supported_video_codecs.av1_hdr && wayland) + if(supported_video_codecs.av1_hdr.supported && wayland) puts("av1_hdr"); - if(supported_video_codecs.av1_10bit) + if(supported_video_codecs.av1_10bit.supported) puts("av1_10bit"); - if(supported_video_codecs.vp8) + if(supported_video_codecs.vp8.supported) puts("vp8"); - if(supported_video_codecs.vp9) + if(supported_video_codecs.vp9.supported) puts("vp9"); + //if(supported_video_codecs_vulkan.h264.supported) + // puts("h264_vulkan"); + //if(supported_video_codecs_vulkan.hevc.supported) + // puts("hevc_vulkan"); // TODO: hdr, 10 bit } static bool monitor_capture_use_drm(gsr_egl *egl, bool wayland) { @@ -1887,11 +2091,10 @@ static void list_audio_devices_command() { _exit(0); } -static gsr_capture* create_capture_impl(std::string &window_str, const char *screen_region, bool wayland, gsr_egl *egl, int fps, VideoCodec video_codec, gsr_color_range color_range, +static gsr_capture* create_capture_impl(std::string &window_str, vec2i output_resolution, bool wayland, gsr_egl *egl, int fps, VideoCodec video_codec, gsr_color_range color_range, bool record_cursor, bool use_software_video_encoder, bool restore_portal_session, const char *portal_session_token_filepath, gsr_color_depth color_depth) { - vec2i region_size = { 0, 0 }; Window src_window_id = None; bool follow_focused = false; @@ -1902,18 +2105,8 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr _exit(2); } - if(!screen_region) { - fprintf(stderr, "Error: option -s is required when using -w focused\n"); - usage(); - } - - if(sscanf(screen_region, "%dx%d", ®ion_size.x, ®ion_size.y) != 2) { - fprintf(stderr, "Error: invalid value for option -s '%s', expected a value in format WxH\n", screen_region); - usage(); - } - - if(region_size.x <= 0 || region_size.y <= 0) { - fprintf(stderr, "Error: invalud value for option -s '%s', expected width and height to be greater than 0\n", screen_region); + if(output_resolution.x <= 0 || output_resolution.y <= 0) { + fprintf(stderr, "Error: invalid value for option -s '%dx%d' when using -w focused option. expected width and height to be greater than 0\n", output_resolution.x, output_resolution.y); usage(); } @@ -1933,11 +2126,12 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr portal_params.record_cursor = record_cursor; portal_params.restore_portal_session = restore_portal_session; portal_params.portal_session_token_filepath = portal_session_token_filepath; + portal_params.output_resolution = output_resolution; capture = gsr_capture_portal_create(&portal_params); if(!capture) _exit(1); #else - fprintf(stderr, "Error: option '-w portal' used but GPU Screen Recorder was compiled without desktop portal support\n"); + fprintf(stderr, "Error: option '-w portal' used but GPU Screen Recorder was compiled without desktop portal support. Please recompile GPU Screen recorder with the -Dportal=true option\n"); _exit(2); #endif } else if(contains_non_hex_number(window_str.c_str())) { @@ -2007,6 +2201,7 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr nvfbc_params.color_range = color_range; nvfbc_params.record_cursor = record_cursor; nvfbc_params.use_software_video_encoder = use_software_video_encoder; + nvfbc_params.output_resolution = output_resolution; capture = gsr_capture_nvfbc_create(&nvfbc_params); if(!capture) _exit(1); @@ -2018,6 +2213,8 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr kms_params.color_range = color_range; kms_params.record_cursor = record_cursor; kms_params.hdr = video_codec_is_hdr(video_codec); + kms_params.fps = fps; + kms_params.output_resolution = output_resolution; capture = gsr_capture_kms_create(&kms_params); if(!capture) _exit(1); @@ -2041,10 +2238,10 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr xcomposite_params.egl = egl; xcomposite_params.window = src_window_id; xcomposite_params.follow_focused = follow_focused; - xcomposite_params.region_size = region_size; xcomposite_params.color_range = color_range; xcomposite_params.record_cursor = record_cursor; xcomposite_params.color_depth = color_depth; + xcomposite_params.output_resolution = output_resolution; capture = gsr_capture_xcomposite_create(&xcomposite_params); if(!capture) _exit(1); @@ -2053,11 +2250,14 @@ static gsr_capture* create_capture_impl(std::string &window_str, const char *scr return capture; } -static AVPixelFormat get_pixel_format(gsr_gpu_vendor vendor, bool use_software_video_encoder) { +static AVPixelFormat get_pixel_format(VideoCodec video_codec, gsr_gpu_vendor vendor, bool use_software_video_encoder) { if(use_software_video_encoder) { return AV_PIX_FMT_NV12; } else { - return vendor == GSR_GPU_VENDOR_NVIDIA ? AV_PIX_FMT_CUDA : AV_PIX_FMT_VAAPI; + if(video_codec_is_vulkan(video_codec)) + return AV_PIX_FMT_VULKAN; + else + return vendor == GSR_GPU_VENDOR_NVIDIA ? AV_PIX_FMT_CUDA : AV_PIX_FMT_VAAPI; } } @@ -2084,7 +2284,7 @@ static std::vector<MergedAudioInputs> parse_audio_inputs(const AudioDevices &aud if(!audio_input || audio_input[0] == '\0') continue; - requested_audio_inputs.push_back({parse_audio_input_arg(audio_input)}); + requested_audio_inputs.push_back({parse_audio_input_arg(audio_input, audio_devices)}); if(requested_audio_inputs.back().audio_inputs.size() > 1) uses_amix = true; @@ -2105,14 +2305,11 @@ static std::vector<MergedAudioInputs> parse_audio_inputs(const AudioDevices &aud match = true; } - for(const auto &existing_audio_input : audio_devices.audio_inputs) { - if(request_audio_input.name == existing_audio_input.name) { - if(request_audio_input.description.empty()) - request_audio_input.description = "gsr-" + existing_audio_input.description; - - match = true; - break; - } + const AudioInput* existing_audio_input = get_audio_device_by_name(audio_devices.audio_inputs, request_audio_input.name); + if(existing_audio_input) { + if(request_audio_input.description.empty()) + request_audio_input.description = "gsr-" + existing_audio_input->description; + match = true; } if(!match) { @@ -2121,8 +2318,8 @@ static std::vector<MergedAudioInputs> parse_audio_inputs(const AudioDevices &aud fprintf(stderr, " default_output (Default output)\n"); if(!audio_devices.default_input.empty()) fprintf(stderr, " default_input (Default input)\n"); - for(const auto &existing_audio_input : audio_devices.audio_inputs) { - fprintf(stderr, " %s (%s)\n", existing_audio_input.name.c_str(), existing_audio_input.description.c_str()); + for(const auto &audio_device_input : audio_devices.audio_inputs) { + fprintf(stderr, " %s (%s)\n", audio_device_input.name.c_str(), audio_device_input.description.c_str()); } _exit(2); } @@ -2174,80 +2371,107 @@ static AudioCodec select_audio_codec_with_fallback(AudioCodec audio_codec, const } static const char* video_codec_to_string(VideoCodec video_codec) { - switch(video_codec) { - case VideoCodec::H264: return "h264"; - case VideoCodec::HEVC: return "hevc"; - case VideoCodec::HEVC_HDR: return "hevc_hdr"; - case VideoCodec::HEVC_10BIT: return "hevc_10bit"; - case VideoCodec::AV1: return "av1"; - case VideoCodec::AV1_HDR: return "av1_hdr"; - case VideoCodec::AV1_10BIT: return "av1_10bit"; - case VideoCodec::VP8: return "vp8"; - case VideoCodec::VP9: return "vp9"; + switch(video_codec) { + case VideoCodec::H264: return "h264"; + case VideoCodec::HEVC: return "hevc"; + case VideoCodec::HEVC_HDR: return "hevc_hdr"; + case VideoCodec::HEVC_10BIT: return "hevc_10bit"; + case VideoCodec::AV1: return "av1"; + case VideoCodec::AV1_HDR: return "av1_hdr"; + case VideoCodec::AV1_10BIT: return "av1_10bit"; + case VideoCodec::VP8: return "vp8"; + case VideoCodec::VP9: return "vp9"; + case VideoCodec::H264_VULKAN: return "h264_vulkan"; + case VideoCodec::HEVC_VULKAN: return "hevc_vulkan"; } return ""; } -static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bool use_software_video_encoder, bool video_codec_auto, const char *video_codec_to_use, bool is_flv) { +static bool video_codec_only_supports_low_power_mode(const gsr_supported_video_codecs &supported_video_codecs, VideoCodec video_codec) { + switch(video_codec) { + case VideoCodec::H264: return supported_video_codecs.h264.low_power; + case VideoCodec::HEVC: return supported_video_codecs.hevc.low_power; + case VideoCodec::HEVC_HDR: return supported_video_codecs.hevc_hdr.low_power; + case VideoCodec::HEVC_10BIT: return supported_video_codecs.hevc_10bit.low_power; + case VideoCodec::AV1: return supported_video_codecs.av1.low_power; + case VideoCodec::AV1_HDR: return supported_video_codecs.av1_hdr.low_power; + case VideoCodec::AV1_10BIT: return supported_video_codecs.av1_10bit.low_power; + case VideoCodec::VP8: return supported_video_codecs.vp8.low_power; + case VideoCodec::VP9: return supported_video_codecs.vp9.low_power; + case VideoCodec::H264_VULKAN: return supported_video_codecs.h264.low_power; + case VideoCodec::HEVC_VULKAN: return supported_video_codecs.hevc.low_power; // TODO: hdr, 10 bit + } + return false; +} + +static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bool use_software_video_encoder, bool video_codec_auto, const char *video_codec_to_use, bool is_flv, bool *low_power) { // TODO: software encoder for hevc, av1, vp8 and vp9 + *low_power = false; - gsr_video_encoder *video_encoder = create_video_encoder(egl, false, GSR_COLOR_DEPTH_8_BITS, use_software_video_encoder); - if(!video_encoder) { - fprintf(stderr, "Error: failed to create video encoder\n"); - _exit(1); + gsr_supported_video_codecs supported_video_codecs; + if(!get_supported_video_codecs(egl, *video_codec, use_software_video_encoder, true, &supported_video_codecs)) { + fprintf(stderr, "Error: failed to query for supported video codecs\n"); + _exit(11); } - const gsr_supported_video_codecs supported_video_codecs = gsr_video_encoder_get_supported_codecs(video_encoder, true); const AVCodec *video_codec_f = nullptr; - // TODO: Cleanup - // gsr_video_encoder_destroy - switch(*video_codec) { case VideoCodec::H264: { if(use_software_video_encoder) video_codec_f = avcodec_find_encoder_by_name("libx264"); - else if(supported_video_codecs.h264) + else if(supported_video_codecs.h264.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::HEVC: { - if(supported_video_codecs.hevc) + if(supported_video_codecs.hevc.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::HEVC_HDR: { - if(supported_video_codecs.hevc_hdr) + if(supported_video_codecs.hevc_hdr.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::HEVC_10BIT: { - if(supported_video_codecs.hevc_10bit) + if(supported_video_codecs.hevc_10bit.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::AV1: { - if(supported_video_codecs.av1) + if(supported_video_codecs.av1.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::AV1_HDR: { - if(supported_video_codecs.av1_hdr) + if(supported_video_codecs.av1_hdr.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::AV1_10BIT: { - if(supported_video_codecs.av1_10bit) + if(supported_video_codecs.av1_10bit.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::VP8: { - if(supported_video_codecs.vp8) + if(supported_video_codecs.vp8.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } case VideoCodec::VP9: { - if(supported_video_codecs.vp9) + if(supported_video_codecs.vp9.supported) + video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); + break; + } + case VideoCodec::H264_VULKAN: { + if(supported_video_codecs.h264.supported) + video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); + break; + } + case VideoCodec::HEVC_VULKAN: { + // TODO: hdr, 10 bit + if(supported_video_codecs.hevc.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } @@ -2258,7 +2482,7 @@ static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bo case VideoCodec::H264: { fprintf(stderr, "Warning: selected video codec h264 is not supported, trying hevc instead\n"); video_codec_to_use = "hevc"; - if(supported_video_codecs.hevc) + if(supported_video_codecs.hevc.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } @@ -2268,7 +2492,7 @@ static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bo fprintf(stderr, "Warning: selected video codec hevc is not supported, trying h264 instead\n"); video_codec_to_use = "h264"; *video_codec = VideoCodec::H264; - if(supported_video_codecs.h264) + if(supported_video_codecs.h264.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } @@ -2278,7 +2502,7 @@ static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bo fprintf(stderr, "Warning: selected video codec av1 is not supported, trying h264 instead\n"); video_codec_to_use = "h264"; *video_codec = VideoCodec::H264; - if(supported_video_codecs.h264) + if(supported_video_codecs.h264.supported) video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); break; } @@ -2286,6 +2510,32 @@ static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bo case VideoCodec::VP9: // TODO: Cant fallback to other codec because webm only supports vp8/vp9 break; + case VideoCodec::H264_VULKAN: { + fprintf(stderr, "Warning: selected video codec h264_vulkan is not supported, trying h264 instead\n"); + video_codec_to_use = "h264"; + *video_codec = VideoCodec::H264; + // Need to do a query again because this time it's without vulkan + if(!get_supported_video_codecs(egl, *video_codec, use_software_video_encoder, true, &supported_video_codecs)) { + fprintf(stderr, "Error: failed to query for supported video codecs\n"); + _exit(11); + } + if(supported_video_codecs.h264.supported) + video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); + break; + } + case VideoCodec::HEVC_VULKAN: { + fprintf(stderr, "Warning: selected video codec hevc_vulkan is not supported, trying hevc instead\n"); + video_codec_to_use = "hevc"; + *video_codec = VideoCodec::HEVC; + // Need to do a query again because this time it's without vulkan + if(!get_supported_video_codecs(egl, *video_codec, use_software_video_encoder, true, &supported_video_codecs)) { + fprintf(stderr, "Error: failed to query for supported video codecs\n"); + _exit(11); + } + if(supported_video_codecs.hevc.supported) + video_codec_f = get_ffmpeg_video_codec(*video_codec, egl->gpu_info.vendor); + break; + } } } @@ -2306,10 +2556,12 @@ static const AVCodec* pick_video_codec(VideoCodec *video_codec, gsr_egl *egl, bo _exit(2); } + *low_power = video_codec_only_supports_low_power_mode(supported_video_codecs, *video_codec); + return video_codec_f; } -static const AVCodec* select_video_codec_with_fallback(VideoCodec *video_codec, const char *video_codec_to_use, const char *file_extension, bool use_software_video_encoder, gsr_egl *egl) { +static const AVCodec* select_video_codec_with_fallback(VideoCodec *video_codec, const char *video_codec_to_use, const char *file_extension, bool use_software_video_encoder, gsr_egl *egl, bool *low_power) { const bool video_codec_auto = strcmp(video_codec_to_use, "auto") == 0; if(video_codec_auto) { if(strcmp(file_extension, "webm") == 0) { @@ -2359,10 +2611,12 @@ static const AVCodec* select_video_codec_with_fallback(VideoCodec *video_codec, usage(); } - return pick_video_codec(video_codec, egl, use_software_video_encoder, video_codec_auto, video_codec_to_use, is_flv); + return pick_video_codec(video_codec, egl, use_software_video_encoder, video_codec_auto, video_codec_to_use, is_flv, low_power); } int main(int argc, char **argv) { + setlocale(LC_ALL, "C"); // Sigh... stupid C + signal(SIGINT, stop_handler); signal(SIGUSR1, save_replay_handler); signal(SIGUSR2, toggle_pause_handler); @@ -2372,6 +2626,13 @@ int main(int argc, char **argv) { // If this is set to 1 then cuGraphicsGLRegisterImage will fail for egl context with error: invalid OpenGL or DirectX context, // so we overwrite it setenv("__GL_THREADED_OPTIMIZATIONS", "0", true); + // Forces low latency encoding mode. Use this environment variable until vaapi supports setting this as a parameter. + // The downside of this is that it always uses maximum power, which is not ideal for replay mode that runs on system startup. + // This option was added in mesa 24.1.4, released in july 17, 2024. + // TODO: Add an option to enable/disable this? + // Seems like the performance issue is not in encoding, but rendering the frame. + // Some frames end up taking 10 times longer. Seems to be an issue with amd gpu power management when letting the application sleep on the cpu side? + setenv("AMD_DEBUG", "lowlatencyenc", true); // Some people set this to nvidia (for nvdec) or vdpau (for nvidia vdpau), which breaks gpu screen recorder since // nvidia doesn't support vaapi and nvidia-vaapi-driver doesn't support encoding yet. // Let vaapi find the match vaapi driver instead of forcing a specific one. @@ -2482,6 +2743,10 @@ int main(int argc, char **argv) { video_codec = VideoCodec::VP8; } else if(strcmp(video_codec_to_use, "vp9") == 0) { video_codec = VideoCodec::VP9; + //} else if(strcmp(video_codec_to_use, "h264_vulkan") == 0) { + // video_codec = VideoCodec::H264_VULKAN; + //} else if(strcmp(video_codec_to_use, "hevc_vulkan") == 0) { + // video_codec = VideoCodec::HEVC_VULKAN; } else if(strcmp(video_codec_to_use, "auto") != 0) { fprintf(stderr, "Error: -k should either be either 'auto', 'h264', 'hevc', 'av1', 'vp8', 'vp9', 'hevc_hdr', 'av1_hdr', 'hevc_10bit' or 'av1_10bit', got: '%s'\n", video_codec_to_use); usage(); @@ -2509,13 +2774,25 @@ int main(int argc, char **argv) { audio_codec = AudioCodec::OPUS; } - int audio_bitrate = 0; + int64_t audio_bitrate = 0; const char *audio_bitrate_str = args["-ab"].value(); if(audio_bitrate_str) { - if(sscanf(audio_bitrate_str, "%d", &audio_bitrate) != 1) { + if(sscanf(audio_bitrate_str, "%" PRIi64, &audio_bitrate) != 1) { fprintf(stderr, "Error: -ab argument \"%s\" is not an integer\n", audio_bitrate_str); usage(); } + + if(audio_bitrate < 0) { + fprintf(stderr, "Error: -ab is expected to be 0 or larger, got %" PRIi64 "\n", audio_bitrate); + usage(); + } + + if(audio_bitrate > 50000) { + fprintf(stderr, "Error: audio bitrate %" PRIi64 "is too high. It's expected to be in kbps, normally in the range 54-300\n", audio_bitrate); + usage(); + } + + audio_bitrate *= 1000LL; } float keyint = 2.0; @@ -2527,7 +2804,7 @@ int main(int argc, char **argv) { } if(keyint < 0) { - fprintf(stderr, "Error: -keyint is expected to be 0 or larger\n"); + fprintf(stderr, "Error: -keyint is expected to be 0 or larger, got %f\n", keyint); usage(); } } @@ -2677,24 +2954,6 @@ int main(int argc, char **argv) { if(fps < 1) fps = 1; - VideoQuality quality = VideoQuality::VERY_HIGH; - const char *quality_str = args["-q"].value(); - if(!quality_str) - quality_str = "very_high"; - - if(strcmp(quality_str, "medium") == 0) { - quality = VideoQuality::MEDIUM; - } else if(strcmp(quality_str, "high") == 0) { - quality = VideoQuality::HIGH; - } else if(strcmp(quality_str, "very_high") == 0) { - quality = VideoQuality::VERY_HIGH; - } else if(strcmp(quality_str, "ultra") == 0) { - quality = VideoQuality::ULTRA; - } else { - fprintf(stderr, "Error: -q should either be either 'medium', 'high', 'very_high' or 'ultra', got: '%s'\n", quality_str); - usage(); - } - int replay_buffer_size_secs = -1; const char *replay_buffer_size_secs_str = args["-r"].value(); if(replay_buffer_size_secs_str) { @@ -2782,10 +3041,10 @@ int main(int argc, char **argv) { } } - if(wayland && is_monitor_capture) { - fprintf(stderr, "gsr warning: it's not possible to sync video to recorded monitor exactly on wayland when recording a monitor." - " If you experience stutter in the video then record with portal capture option instead (-w portal) or use X11 instead\n"); - } + // if(wayland && is_monitor_capture) { + // fprintf(stderr, "gsr warning: it's not possible to sync video to recorded monitor exactly on wayland when recording a monitor." + // " If you experience stutter in the video then record with portal capture option instead (-w portal) or use X11 instead\n"); + // } // TODO: Fix constant framerate not working properly on amd/intel because capture framerate gets locked to the same framerate as // game framerate, which doesn't work well when you need to encode multiple duplicate frames (AMD/Intel is slow at encoding!). @@ -2820,8 +3079,10 @@ int main(int argc, char **argv) { bitrate_mode = BitrateMode::QP; } else if(strcmp(bitrate_mode_str, "vbr") == 0) { bitrate_mode = BitrateMode::VBR; + } else if(strcmp(bitrate_mode_str, "cbr") == 0) { + bitrate_mode = BitrateMode::CBR; } else if(strcmp(bitrate_mode_str, "auto") != 0) { - fprintf(stderr, "Error: -bm should either be either 'auto', 'qp', 'vbr', got: '%s'\n", bitrate_mode_str); + fprintf(stderr, "Error: -bm should either be either 'auto', 'qp', 'vbr' or 'cbr', got: '%s'\n", bitrate_mode_str); usage(); } @@ -2830,11 +3091,55 @@ int main(int argc, char **argv) { bitrate_mode = egl.gpu_info.is_steam_deck ? BitrateMode::VBR : BitrateMode::QP; } - if(use_software_video_encoder && bitrate_mode != BitrateMode::QP) { + if(egl.gpu_info.is_steam_deck && bitrate_mode == BitrateMode::QP) { + fprintf(stderr, "Warning: qp bitrate mode is not supported on Steam Deck because of Steam Deck driver bugs. Using vbr instead\n"); + bitrate_mode = BitrateMode::VBR; + } + + if(use_software_video_encoder && bitrate_mode == BitrateMode::VBR) { fprintf(stderr, "Warning: bitrate mode has been forcefully set to qp because software encoding option doesn't support vbr option\n"); bitrate_mode = BitrateMode::QP; } + const char *quality_str = args["-q"].value(); + VideoQuality quality = VideoQuality::VERY_HIGH; + int64_t video_bitrate = 0; + + if(bitrate_mode == BitrateMode::CBR) { + if(!quality_str) { + fprintf(stderr, "Error: option '-q' is required when using '-bm cbr' option\n"); + usage(); + } + + if(sscanf(quality_str, "%" PRIi64, &video_bitrate) != 1) { + fprintf(stderr, "Error: -q argument \"%s\" is not an integer value. When using '-bm cbr' option '-q' is expected to be an integer value\n", quality_str); + usage(); + } + + if(video_bitrate < 0) { + fprintf(stderr, "Error: -q is expected to be 0 or larger, got %" PRIi64 "\n", video_bitrate); + usage(); + } + + video_bitrate *= 1000LL; + } else { + if(!quality_str) + quality_str = "very_high"; + + if(strcmp(quality_str, "medium") == 0) { + quality = VideoQuality::MEDIUM; + } else if(strcmp(quality_str, "high") == 0) { + quality = VideoQuality::HIGH; + } else if(strcmp(quality_str, "very_high") == 0) { + quality = VideoQuality::VERY_HIGH; + } else if(strcmp(quality_str, "ultra") == 0) { + quality = VideoQuality::ULTRA; + } else { + fprintf(stderr, "Error: -q should either be either 'medium', 'high', 'very_high' or 'ultra', got: '%s'\n", quality_str); + usage(); + } + } + gsr_color_range color_range = GSR_COLOR_RANGE_LIMITED; const char *color_range_str = args["-cr"].value(); if(!color_range_str) @@ -2849,13 +3154,25 @@ int main(int argc, char **argv) { usage(); } - const char *screen_region = args["-s"].value(); - - if(screen_region && strcmp(window_str.c_str(), "focused") != 0) { - fprintf(stderr, "Error: option -s is only available when using -w focused\n"); + const char *output_resolution_str = args["-s"].value(); + if(!output_resolution_str && strcmp(window_str.c_str(), "focused") == 0) { + fprintf(stderr, "Error: option -s is required when using -w focused option\n"); usage(); } + vec2i output_resolution = {0, 0}; + if(output_resolution_str) { + if(sscanf(output_resolution_str, "%dx%d", &output_resolution.x, &output_resolution.y) != 2) { + fprintf(stderr, "Error: invalid value for option -s '%s', expected a value in format WxH\n", output_resolution_str); + usage(); + } + + if(output_resolution.x < 0 || output_resolution.y < 0) { + fprintf(stderr, "Error: invalud value for option -s '%s', expected width and height to be greater or equal to 0\n", output_resolution_str); + usage(); + } + } + bool is_livestream = false; const char *filename = args["-o"].value(); if(filename) { @@ -2936,10 +3253,11 @@ int main(int argc, char **argv) { } audio_codec = select_audio_codec_with_fallback(audio_codec, file_extension, uses_amix); - const AVCodec *video_codec_f = select_video_codec_with_fallback(&video_codec, video_codec_to_use, file_extension.c_str(), use_software_video_encoder, &egl); + bool low_power = false; + const AVCodec *video_codec_f = select_video_codec_with_fallback(&video_codec, video_codec_to_use, file_extension.c_str(), use_software_video_encoder, &egl, &low_power); const gsr_color_depth color_depth = video_codec_to_bit_depth(video_codec); - gsr_capture *capture = create_capture_impl(window_str, screen_region, wayland, &egl, fps, video_codec, color_range, record_cursor, use_software_video_encoder, restore_portal_session, portal_session_token_filepath, color_depth); + gsr_capture *capture = create_capture_impl(window_str, output_resolution, wayland, &egl, fps, video_codec, color_range, record_cursor, use_software_video_encoder, restore_portal_session, portal_session_token_filepath, color_depth); // (Some?) livestreaming services require at least one audio track to work. // If not audio is provided then create one silent audio track. @@ -2960,7 +3278,8 @@ int main(int argc, char **argv) { const bool hdr = video_codec_is_hdr(video_codec); const bool low_latency_recording = is_livestream || is_output_piped; - AVCodecContext *video_codec_context = create_video_codec_context(get_pixel_format(egl.gpu_info.vendor, use_software_video_encoder), quality, fps, video_codec_f, low_latency_recording, egl.gpu_info.vendor, framerate_mode, hdr, color_range, keyint, use_software_video_encoder, bitrate_mode); + const enum AVPixelFormat video_pix_fmt = get_pixel_format(video_codec, egl.gpu_info.vendor, use_software_video_encoder); + AVCodecContext *video_codec_context = create_video_codec_context(video_pix_fmt, quality, fps, video_codec_f, low_latency_recording, egl.gpu_info.vendor, framerate_mode, hdr, color_range, keyint, use_software_video_encoder, bitrate_mode, video_codec, video_bitrate); if(replay_buffer_size_secs == -1) video_stream = create_stream(av_format_context, video_codec_context); @@ -2984,7 +3303,7 @@ int main(int argc, char **argv) { _exit(capture_result); } - gsr_video_encoder *video_encoder = create_video_encoder(&egl, overclock, color_depth, use_software_video_encoder); + gsr_video_encoder *video_encoder = create_video_encoder(&egl, overclock, color_depth, use_software_video_encoder, video_codec); if(!video_encoder) { fprintf(stderr, "Error: failed to create video encoder\n"); _exit(1); @@ -2999,7 +3318,6 @@ int main(int argc, char **argv) { memset(&color_conversion_params, 0, sizeof(color_conversion_params)); color_conversion_params.color_range = color_range; color_conversion_params.egl = &egl; - color_conversion_params.source_color = gsr_capture_get_source_color(capture); color_conversion_params.load_external_image_shader = gsr_capture_uses_external_image(capture); gsr_video_encoder_get_textures(video_encoder, color_conversion_params.destination_textures, &color_conversion_params.num_destination_textures, &color_conversion_params.destination_color); @@ -3014,7 +3332,7 @@ int main(int argc, char **argv) { if(use_software_video_encoder) { open_video_software(video_codec_context, quality, pixel_format, hdr, color_depth, bitrate_mode); } else { - open_video_hardware(video_codec_context, quality, very_old_gpu, egl.gpu_info.vendor, pixel_format, hdr, color_depth, bitrate_mode); + open_video_hardware(video_codec_context, quality, very_old_gpu, egl.gpu_info.vendor, pixel_format, hdr, color_depth, bitrate_mode, video_codec, low_power); } if(video_stream) avcodec_parameters_from_context(video_stream->codecpar, video_codec_context); @@ -3126,6 +3444,7 @@ int main(int argc, char **argv) { } double fps_start_time = clock_get_monotonic_seconds(); + //double frame_timer_start = fps_start_time; int fps_counter = 0; int damage_fps_counter = 0; @@ -3337,7 +3656,7 @@ int main(int argc, char **argv) { } // Set update_fps to 24 to test if duplicate/delayed frames cause video/audio desync or too fast/slow video. - const double update_fps = fps; + //const double update_fps = fps + 190; bool should_stop_error = false; int64_t video_pts_counter = 0; @@ -3359,12 +3678,15 @@ int main(int argc, char **argv) { if(is_monitor_capture) gsr_damage_set_target_monitor(&damage, window_str.c_str()); + double last_capture_seconds = record_start_time; + bool wait_until_frame_time_elapsed = false; + while(running) { const double frame_start = clock_get_monotonic_seconds(); while(gsr_egl_process_event(&egl)) { - gsr_capture_on_event(capture, &egl); gsr_damage_on_event(&damage, gsr_egl_get_event_data(&egl)); + gsr_capture_on_event(capture, &egl); } gsr_damage_tick(&damage); gsr_capture_tick(capture); @@ -3392,11 +3714,16 @@ int main(int argc, char **argv) { else damaged = true; + // TODO: Readd wayland sync warning when removing this + if(framerate_mode != FramerateMode::CONTENT) + damaged = true; + if(damaged) ++damage_fps_counter; ++fps_counter; - double time_now = clock_get_monotonic_seconds(); + const double time_now = clock_get_monotonic_seconds(); + //const double frame_timer_elapsed = time_now - frame_timer_start; const double elapsed = time_now - fps_start_time; if (elapsed >= 1.0) { if(verbose) { @@ -3408,26 +3735,42 @@ int main(int argc, char **argv) { } const double this_video_frame_time = clock_get_monotonic_seconds() - paused_time_offset; - const int64_t expected_frames = std::round((this_video_frame_time - record_start_time) / target_fps); - const int num_frames = std::max((int64_t)0LL, expected_frames - video_pts_counter); - const double num_frames_seconds = num_frames * target_fps; - if((damaged || (framerate_mode == FramerateMode::CONSTANT && num_frames > 0) || (framerate_mode != FramerateMode::CONSTANT && num_frames_seconds >= damage_timeout_seconds)) && !paused) { + const double time_since_last_frame_captured_seconds = this_video_frame_time - last_capture_seconds; + double frame_time_overflow = time_since_last_frame_captured_seconds - target_fps; + const bool frame_timeout = frame_time_overflow >= 0.0; + + bool force_frame_capture = wait_until_frame_time_elapsed && frame_timeout; + bool allow_capture = !wait_until_frame_time_elapsed || force_frame_capture; + if(framerate_mode == FramerateMode::CONTENT) { + force_frame_capture = false; + allow_capture = frame_timeout; + } + + bool frame_captured = false; + if((damaged || force_frame_capture) && allow_capture && !paused) { + frame_captured = true; + frame_time_overflow = std::min(std::max(0.0, frame_time_overflow), target_fps); + last_capture_seconds = this_video_frame_time - frame_time_overflow; + wait_until_frame_time_elapsed = false; + gsr_damage_clear(&damage); if(capture->clear_damage) capture->clear_damage(capture); - if(damaged || video_pts_counter == 0) { - egl.glClear(0); - gsr_capture_capture(capture, video_frame, &color_conversion); - gsr_egl_swap_buffers(&egl); - gsr_video_encoder_copy_textures_to_frame(video_encoder, video_frame); - } + // TODO: Dont do this if no damage? + egl.glClear(0); + gsr_capture_capture(capture, video_frame, &color_conversion); + gsr_egl_swap_buffers(&egl); + gsr_video_encoder_copy_textures_to_frame(video_encoder, video_frame, &color_conversion); if(hdr && !hdr_metadata_set && replay_buffer_size_secs == -1 && add_hdr_metadata_to_video_stream(capture, video_stream)) hdr_metadata_set = true; + const int64_t expected_frames = std::round((this_video_frame_time - record_start_time) / target_fps); + const int num_missed_frames = std::max((int64_t)1LL, expected_frames - video_pts_counter); + // TODO: Check if duplicate frame can be saved just by writing it with a different pts instead of sending it again - const int num_frames_to_encode = framerate_mode == FramerateMode::CONSTANT ? num_frames : 1; + const int num_frames_to_encode = framerate_mode == FramerateMode::CONSTANT ? num_missed_frames : 1; for(int i = 0; i < num_frames_to_encode; ++i) { if(framerate_mode == FramerateMode::CONSTANT) { video_frame->pts = video_pts_counter + i; @@ -3449,7 +3792,7 @@ int main(int argc, char **argv) { } } - video_pts_counter += num_frames; + video_pts_counter += num_frames_to_encode; } if(toggle_pause == 1) { @@ -3482,13 +3825,28 @@ int main(int argc, char **argv) { } const double frame_end = clock_get_monotonic_seconds(); - const double frame_sleep_fps = 1.0 / update_fps; - const double sleep_time = frame_sleep_fps - (frame_end - frame_start); - if(sleep_time > 0.0) { - if(damaged) - av_usleep(sleep_time * 1000.0 * 1000.0); - else - av_usleep(2 * 1000.0); // 2 milliseconds + const double time_at_frame_end = frame_end - paused_time_offset; + const double time_elapsed_total = time_at_frame_end - record_start_time; + const int64_t frames_elapsed = (int64_t)(time_elapsed_total / target_fps); + const double time_at_next_frame = (frames_elapsed + 1) * target_fps; + double time_to_next_frame = time_at_next_frame - time_elapsed_total; + if(time_to_next_frame > target_fps*1.1) + time_to_next_frame = target_fps; + + const double frame_time = frame_end - frame_start; + const bool frame_deadline_missed = frame_time > target_fps; + if(time_to_next_frame >= 0.0 && !frame_deadline_missed && frame_captured) + av_usleep(time_to_next_frame * 1000.0 * 1000.0); + else { + if(paused) + av_usleep(20.0 * 1000.0); // 20 milliseconds + else if(frame_deadline_missed) + {} + else if(framerate_mode == FramerateMode::CONTENT || !frame_captured) + av_usleep(2.8 * 1000.0); // 2.8 milliseconds + else if(!frame_captured) + av_usleep(1.0 * 1000.0); // 1 milliseconds + wait_until_frame_time_elapsed = true; } } diff --git a/src/pipewire_audio.c b/src/pipewire_audio.c new file mode 100644 index 0000000..2c18432 --- /dev/null +++ b/src/pipewire_audio.c @@ -0,0 +1 @@ +#include "../include/pipewire_audio.h"
\ No newline at end of file diff --git a/src/pipewire.c b/src/pipewire_video.c index 3bf54db..b5f1562 100644 --- a/src/pipewire.c +++ b/src/pipewire_video.c @@ -1,4 +1,4 @@ -#include "../include/pipewire.h" +#include "../include/pipewire_video.h" #include "../include/egl.h" #include "../include/utils.h" @@ -13,7 +13,7 @@ /* This code is partially based on xr-video-player pipewire implementation which is based on obs-studio's pipewire implementation */ -/* TODO: Make gsr_pipewire_init asynchronous */ +/* TODO: Make gsr_pipewire_video_init asynchronous */ /* TODO: Support 10-bit capture (hdr) when pipewire supports it */ /* TODO: Test all of the image formats */ @@ -25,12 +25,12 @@ (sizeof(struct spa_meta_cursor) + sizeof(struct spa_meta_bitmap) + \ width * height * 4) -static bool parse_pw_version(gsr_pipewire_data_version *dst, const char *version) { +static bool parse_pw_version(gsr_pipewire_video_data_version *dst, const char *version) { const int n_matches = sscanf(version, "%d.%d.%d", &dst->major, &dst->minor, &dst->micro); return n_matches == 3; } -static bool check_pw_version(const gsr_pipewire_data_version *pw_version, int major, int minor, int micro) { +static bool check_pw_version(const gsr_pipewire_video_data_version *pw_version, int major, int minor, int micro) { if (pw_version->major != major) return pw_version->major > major; if (pw_version->minor != minor) @@ -38,7 +38,7 @@ static bool check_pw_version(const gsr_pipewire_data_version *pw_version, int ma return pw_version->micro >= micro; } -static void update_pw_versions(gsr_pipewire *self, const char *version) { +static void update_pw_versions(gsr_pipewire_video *self, const char *version) { fprintf(stderr, "gsr info: pipewire: server version: %s\n", version); fprintf(stderr, "gsr info: pipewire: library version: %s\n", pw_get_library_version()); fprintf(stderr, "gsr info: pipewire: header version: %s\n", pw_get_headers_version()); @@ -47,18 +47,18 @@ static void update_pw_versions(gsr_pipewire *self, const char *version) { } static void on_core_info_cb(void *user_data, const struct pw_core_info *info) { - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; update_pw_versions(self, info->version); } static void on_core_error_cb(void *user_data, uint32_t id, int seq, int res, const char *message) { - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; fprintf(stderr, "gsr error: pipewire: error id:%u seq:%d res:%d: %s\n", id, seq, res, message); pw_thread_loop_signal(self->thread_loop, false); } static void on_core_done_cb(void *user_data, uint32_t id, int seq) { - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; if (id == PW_ID_CORE && self->server_version_sync == seq) pw_thread_loop_signal(self->thread_loop, false); } @@ -86,7 +86,7 @@ static const struct pw_core_events core_events = { }; static void on_process_cb(void *user_data) { - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; struct spa_meta_cursor *cursor = NULL; //struct spa_meta *video_damage = NULL; @@ -122,8 +122,8 @@ static void on_process_cb(void *user_data) { } self->dmabuf_num_planes = buffer->n_datas; - if(self->dmabuf_num_planes > GSR_PIPEWIRE_DMABUF_MAX_PLANES) - self->dmabuf_num_planes = GSR_PIPEWIRE_DMABUF_MAX_PLANES; + if(self->dmabuf_num_planes > GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES) + self->dmabuf_num_planes = GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES; for(size_t i = 0; i < self->dmabuf_num_planes; ++i) { self->dmabuf_data[i].fd = dup(buffer->datas[i].fd); @@ -206,7 +206,7 @@ read_metadata: } static void on_param_changed_cb(void *user_data, uint32_t id, const struct spa_pod *param) { - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; if (!param || id != SPA_PARAM_Format) return; @@ -276,7 +276,7 @@ static void on_param_changed_cb(void *user_data, uint32_t id, const struct spa_p static void on_state_changed_cb(void *user_data, enum pw_stream_state old, enum pw_stream_state state, const char *error) { (void)old; - gsr_pipewire *self = user_data; + gsr_pipewire_video *self = user_data; fprintf(stderr, "gsr info: pipewire: stream %p state: \"%s\" (error: %s)\n", (void*)self->stream, pw_stream_state_as_string(state), @@ -291,7 +291,7 @@ static const struct pw_stream_events stream_events = { }; static inline struct spa_pod *build_format(struct spa_pod_builder *b, - const gsr_pipewire_video_info *ovi, + const gsr_pipewire_video_video_info *ovi, uint32_t format, const uint64_t *modifiers, size_t modifier_count) { @@ -358,13 +358,13 @@ static const enum spa_video_format video_formats[] = { SPA_VIDEO_FORMAT_RGB, }; -static bool gsr_pipewire_build_format_params(gsr_pipewire *self, struct spa_pod_builder *pod_builder, struct spa_pod **params, uint32_t *num_params) { +static bool gsr_pipewire_video_build_format_params(gsr_pipewire_video *self, struct spa_pod_builder *pod_builder, struct spa_pod **params, uint32_t *num_params) { *num_params = 0; if(!check_pw_version(&self->server_version, 0, 3, 33)) return false; - for(size_t i = 0; i < GSR_PIPEWIRE_NUM_VIDEO_FORMATS; i++) { + for(size_t i = 0; i < GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS; i++) { if(self->supported_video_formats[i].modifiers_size == 0) continue; params[i] = build_format(pod_builder, &self->video_info, self->supported_video_formats[i].format, self->modifiers + self->supported_video_formats[i].modifiers_index, self->supported_video_formats[i].modifiers_size); @@ -376,15 +376,15 @@ static bool gsr_pipewire_build_format_params(gsr_pipewire *self, struct spa_pod_ static void renegotiate_format(void *data, uint64_t expirations) { (void)expirations; - gsr_pipewire *self = (gsr_pipewire*)data; + gsr_pipewire_video *self = (gsr_pipewire_video*)data; pw_thread_loop_lock(self->thread_loop); - struct spa_pod *params[GSR_PIPEWIRE_NUM_VIDEO_FORMATS]; + struct spa_pod *params[GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS]; uint32_t num_video_formats = 0; uint8_t params_buffer[2048]; struct spa_pod_builder pod_builder = SPA_POD_BUILDER_INIT(params_buffer, sizeof(params_buffer)); - if (!gsr_pipewire_build_format_params(self, &pod_builder, params, &num_video_formats)) { + if (!gsr_pipewire_video_build_format_params(self, &pod_builder, params, &num_video_formats)) { pw_thread_loop_unlock(self->thread_loop); return; } @@ -393,7 +393,7 @@ static void renegotiate_format(void *data, uint64_t expirations) { pw_thread_loop_unlock(self->thread_loop); } -static bool spa_video_format_get_modifiers(gsr_pipewire *self, const enum spa_video_format format, uint64_t *modifiers, int32_t max_modifiers, int32_t *num_modifiers) { +static bool spa_video_format_get_modifiers(gsr_pipewire_video *self, const enum spa_video_format format, uint64_t *modifiers, int32_t max_modifiers, int32_t *num_modifiers) { *num_modifiers = 0; if(max_modifiers == 0) { @@ -430,36 +430,36 @@ static bool spa_video_format_get_modifiers(gsr_pipewire *self, const enum spa_vi return true; } -static void gsr_pipewire_init_modifiers(gsr_pipewire *self) { - for(size_t i = 0; i < GSR_PIPEWIRE_NUM_VIDEO_FORMATS; i++) { +static void gsr_pipewire_video_init_modifiers(gsr_pipewire_video *self) { + for(size_t i = 0; i < GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS; i++) { self->supported_video_formats[i].format = video_formats[i]; int32_t num_modifiers = 0; - spa_video_format_get_modifiers(self, self->supported_video_formats[i].format, self->modifiers + self->num_modifiers, GSR_PIPEWIRE_MAX_MODIFIERS - self->num_modifiers, &num_modifiers); + spa_video_format_get_modifiers(self, self->supported_video_formats[i].format, self->modifiers + self->num_modifiers, GSR_PIPEWIRE_VIDEO_MAX_MODIFIERS - self->num_modifiers, &num_modifiers); self->supported_video_formats[i].modifiers_index = self->num_modifiers; self->supported_video_formats[i].modifiers_size = num_modifiers; } } -static bool gsr_pipewire_setup_stream(gsr_pipewire *self) { - struct spa_pod *params[GSR_PIPEWIRE_NUM_VIDEO_FORMATS]; +static bool gsr_pipewire_video_setup_stream(gsr_pipewire_video *self) { + struct spa_pod *params[GSR_PIPEWIRE_VIDEO_NUM_VIDEO_FORMATS]; uint32_t num_video_formats = 0; uint8_t params_buffer[2048]; struct spa_pod_builder pod_builder = SPA_POD_BUILDER_INIT(params_buffer, sizeof(params_buffer)); self->thread_loop = pw_thread_loop_new("PipeWire thread loop", NULL); if(!self->thread_loop) { - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to create pipewire thread\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to create pipewire thread\n"); goto error; } self->context = pw_context_new(pw_thread_loop_get_loop(self->thread_loop), NULL, 0); if(!self->context) { - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to create pipewire context\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to create pipewire context\n"); goto error; } if(pw_thread_loop_start(self->thread_loop) < 0) { - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to start thread\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to start thread\n"); goto error; } @@ -469,20 +469,20 @@ static bool gsr_pipewire_setup_stream(gsr_pipewire *self) { self->core = pw_context_connect_fd(self->context, fcntl(self->fd, F_DUPFD_CLOEXEC, 5), NULL, 0); if(!self->core) { pw_thread_loop_unlock(self->thread_loop); - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to connect to fd %d\n", self->fd); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to connect to fd %d\n", self->fd); goto error; } // TODO: Error check pw_core_add_listener(self->core, &self->core_listener, &core_events, self); - gsr_pipewire_init_modifiers(self); + gsr_pipewire_video_init_modifiers(self); // TODO: Cleanup? self->reneg = pw_loop_add_event(pw_thread_loop_get_loop(self->thread_loop), renegotiate_format, self); if(!self->reneg) { pw_thread_loop_unlock(self->thread_loop); - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: pw_loop_add_event failed\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: pw_loop_add_event failed\n"); goto error; } @@ -495,14 +495,14 @@ static bool gsr_pipewire_setup_stream(gsr_pipewire *self) { PW_KEY_MEDIA_ROLE, "Screen", NULL)); if(!self->stream) { pw_thread_loop_unlock(self->thread_loop); - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to create stream\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to create stream\n"); goto error; } pw_stream_add_listener(self->stream, &self->stream_listener, &stream_events, self); - if(!gsr_pipewire_build_format_params(self, &pod_builder, params, &num_video_formats)) { + if(!gsr_pipewire_video_build_format_params(self, &pod_builder, params, &num_video_formats)) { pw_thread_loop_unlock(self->thread_loop); - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to build format params\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to build format params\n"); goto error; } @@ -512,7 +512,7 @@ static bool gsr_pipewire_setup_stream(gsr_pipewire *self) { num_video_formats) < 0) { pw_thread_loop_unlock(self->thread_loop); - fprintf(stderr, "gsr error: gsr_pipewire_setup_stream: failed to connect stream\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_setup_stream: failed to connect stream\n"); goto error; } @@ -549,7 +549,7 @@ static bool gsr_pipewire_setup_stream(gsr_pipewire *self) { } static int pw_init_counter = 0; -bool gsr_pipewire_init(gsr_pipewire *self, int pipewire_fd, uint32_t pipewire_node, int fps, bool capture_cursor, gsr_egl *egl) { +bool gsr_pipewire_video_init(gsr_pipewire_video *self, int pipewire_fd, uint32_t pipewire_node, int fps, bool capture_cursor, gsr_egl *egl) { if(pw_init_counter == 0) pw_init(NULL, NULL); ++pw_init_counter; @@ -559,8 +559,8 @@ bool gsr_pipewire_init(gsr_pipewire *self, int pipewire_fd, uint32_t pipewire_no self->fd = pipewire_fd; self->node = pipewire_node; if(pthread_mutex_init(&self->mutex, NULL) != 0) { - fprintf(stderr, "gsr error: gsr_pipewire_init: failed to initialize mutex\n"); - gsr_pipewire_deinit(self); + fprintf(stderr, "gsr error: gsr_pipewire_video_init: failed to initialize mutex\n"); + gsr_pipewire_video_deinit(self); return false; } self->mutex_initialized = true; @@ -568,15 +568,15 @@ bool gsr_pipewire_init(gsr_pipewire *self, int pipewire_fd, uint32_t pipewire_no self->video_info.fps_den = 1; self->cursor.visible = capture_cursor; - if(!gsr_pipewire_setup_stream(self)) { - gsr_pipewire_deinit(self); + if(!gsr_pipewire_video_setup_stream(self)) { + gsr_pipewire_video_deinit(self); return false; } return true; } -void gsr_pipewire_deinit(gsr_pipewire *self) { +void gsr_pipewire_video_deinit(gsr_pipewire_video *self) { if(self->thread_loop) { //pw_thread_loop_wait(self->thread_loop); pw_thread_loop_stop(self->thread_loop); @@ -636,7 +636,7 @@ void gsr_pipewire_deinit(gsr_pipewire *self) { } } -static EGLImage gsr_pipewire_create_egl_image(gsr_pipewire *self, const int *fds, const uint32_t *offsets, const uint32_t *pitches, const uint64_t *modifiers, bool use_modifiers) { +static EGLImage gsr_pipewire_video_create_egl_image(gsr_pipewire_video *self, const int *fds, const uint32_t *offsets, const uint32_t *pitches, const uint64_t *modifiers, bool use_modifiers) { intptr_t img_attr[44]; setup_dma_buf_attrs(img_attr, spa_video_format_to_drm_format(self->format.info.raw.format), self->format.info.raw.size.width, self->format.info.raw.size.height, fds, offsets, pitches, modifiers, self->dmabuf_num_planes, use_modifiers); @@ -650,11 +650,11 @@ static EGLImage gsr_pipewire_create_egl_image(gsr_pipewire *self, const int *fds return image; } -static EGLImage gsr_pipewire_create_egl_image_with_fallback(gsr_pipewire *self) { - int fds[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; - uint32_t offsets[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; - uint32_t pitches[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; - uint64_t modifiers[GSR_PIPEWIRE_DMABUF_MAX_PLANES]; +static EGLImage gsr_pipewire_video_create_egl_image_with_fallback(gsr_pipewire_video *self) { + int fds[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; + uint32_t offsets[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; + uint32_t pitches[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; + uint64_t modifiers[GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES]; for(size_t i = 0; i < self->dmabuf_num_planes; ++i) { fds[i] = self->dmabuf_data[i].fd; offsets[i] = self->dmabuf_data[i].offset; @@ -664,19 +664,19 @@ static EGLImage gsr_pipewire_create_egl_image_with_fallback(gsr_pipewire *self) EGLImage image = NULL; if(self->no_modifiers_fallback) { - image = gsr_pipewire_create_egl_image(self, fds, offsets, pitches, modifiers, false); + image = gsr_pipewire_video_create_egl_image(self, fds, offsets, pitches, modifiers, false); } else { - image = gsr_pipewire_create_egl_image(self, fds, offsets, pitches, modifiers, true); + image = gsr_pipewire_video_create_egl_image(self, fds, offsets, pitches, modifiers, true); if(!image) { - fprintf(stderr, "gsr error: gsr_pipewire_create_egl_image_with_fallback: failed to create egl image with modifiers, trying without modifiers\n"); + fprintf(stderr, "gsr error: gsr_pipewire_video_create_egl_image_with_fallback: failed to create egl image with modifiers, trying without modifiers\n"); self->no_modifiers_fallback = true; - image = gsr_pipewire_create_egl_image(self, fds, offsets, pitches, modifiers, false); + image = gsr_pipewire_video_create_egl_image(self, fds, offsets, pitches, modifiers, false); } } return image; } -static bool gsr_pipewire_bind_image_to_texture(gsr_pipewire *self, EGLImage image, unsigned int texture_id, bool external_texture) { +static bool gsr_pipewire_video_bind_image_to_texture(gsr_pipewire_video *self, EGLImage image, unsigned int texture_id, bool external_texture) { const int texture_target = external_texture ? GL_TEXTURE_EXTERNAL_OES : GL_TEXTURE_2D; while(self->egl->glGetError() != 0){} self->egl->glBindTexture(texture_target, texture_id); @@ -686,19 +686,19 @@ static bool gsr_pipewire_bind_image_to_texture(gsr_pipewire *self, EGLImage imag return success; } -static void gsr_pipewire_bind_image_to_texture_with_fallback(gsr_pipewire *self, gsr_texture_map texture_map, EGLImage image) { +static void gsr_pipewire_video_bind_image_to_texture_with_fallback(gsr_pipewire_video *self, gsr_texture_map texture_map, EGLImage image) { if(self->external_texture_fallback) { - gsr_pipewire_bind_image_to_texture(self, image, texture_map.external_texture_id, true); + gsr_pipewire_video_bind_image_to_texture(self, image, texture_map.external_texture_id, true); } else { - if(!gsr_pipewire_bind_image_to_texture(self, image, texture_map.texture_id, false)) { - fprintf(stderr, "gsr error: gsr_pipewire_map_texture: failed to bind image to texture, trying with external texture\n"); + if(!gsr_pipewire_video_bind_image_to_texture(self, image, texture_map.texture_id, false)) { + fprintf(stderr, "gsr error: gsr_pipewire_video_map_texture: failed to bind image to texture, trying with external texture\n"); self->external_texture_fallback = true; - gsr_pipewire_bind_image_to_texture(self, image, texture_map.external_texture_id, true); + gsr_pipewire_video_bind_image_to_texture(self, image, texture_map.external_texture_id, true); } } } -static void gsr_pipewire_update_cursor_texture(gsr_pipewire *self, gsr_texture_map texture_map) { +static void gsr_pipewire_video_update_cursor_texture(gsr_pipewire_video *self, gsr_texture_map texture_map) { if(!self->cursor.data) return; @@ -715,9 +715,9 @@ static void gsr_pipewire_update_cursor_texture(gsr_pipewire *self, gsr_texture_m self->cursor.data = NULL; } -bool gsr_pipewire_map_texture(gsr_pipewire *self, gsr_texture_map texture_map, gsr_pipewire_region *region, gsr_pipewire_region *cursor_region, gsr_pipewire_dmabuf_data *dmabuf_data, int *num_dmabuf_data, uint32_t *fourcc, uint64_t *modifiers, bool *using_external_image) { - for(int i = 0; i < GSR_PIPEWIRE_DMABUF_MAX_PLANES; ++i) { - memset(&dmabuf_data[i], 0, sizeof(gsr_pipewire_dmabuf_data)); +bool gsr_pipewire_video_map_texture(gsr_pipewire_video *self, gsr_texture_map texture_map, gsr_pipewire_video_region *region, gsr_pipewire_video_region *cursor_region, gsr_pipewire_video_dmabuf_data *dmabuf_data, int *num_dmabuf_data, uint32_t *fourcc, uint64_t *modifiers, bool *using_external_image) { + for(int i = 0; i < GSR_PIPEWIRE_VIDEO_DMABUF_MAX_PLANES; ++i) { + memset(&dmabuf_data[i], 0, sizeof(gsr_pipewire_video_dmabuf_data)); } *num_dmabuf_data = 0; *using_external_image = self->external_texture_fallback; @@ -730,14 +730,14 @@ bool gsr_pipewire_map_texture(gsr_pipewire *self, gsr_texture_map texture_map, g return false; } - EGLImage image = gsr_pipewire_create_egl_image_with_fallback(self); + EGLImage image = gsr_pipewire_video_create_egl_image_with_fallback(self); if(image) { - gsr_pipewire_bind_image_to_texture_with_fallback(self, texture_map, image); + gsr_pipewire_video_bind_image_to_texture_with_fallback(self, texture_map, image); *using_external_image = self->external_texture_fallback; self->egl->eglDestroyImage(self->egl->egl_display, image); } - gsr_pipewire_update_cursor_texture(self, texture_map); + gsr_pipewire_video_update_cursor_texture(self, texture_map); region->x = 0; region->y = 0; @@ -773,7 +773,7 @@ bool gsr_pipewire_map_texture(gsr_pipewire *self, gsr_texture_map texture_map, g return true; } -bool gsr_pipewire_is_damaged(gsr_pipewire *self) { +bool gsr_pipewire_video_is_damaged(gsr_pipewire_video *self) { bool damaged = false; pthread_mutex_lock(&self->mutex); damaged = self->damaged; @@ -781,7 +781,7 @@ bool gsr_pipewire_is_damaged(gsr_pipewire *self) { return damaged; } -void gsr_pipewire_clear_damage(gsr_pipewire *self) { +void gsr_pipewire_video_clear_damage(gsr_pipewire_video *self) { pthread_mutex_lock(&self->mutex); self->damaged = false; pthread_mutex_unlock(&self->mutex); diff --git a/src/sound.cpp b/src/sound.cpp index d0f2a80..aea5b91 100644 --- a/src/sound.cpp +++ b/src/sound.cpp @@ -340,16 +340,18 @@ static void pa_server_info_cb(pa_context*, const pa_server_info *server_info, vo } static void get_pulseaudio_default_inputs(AudioDevices &audio_devices) { - pa_mainloop *main_loop = pa_mainloop_new(); - - pa_context *ctx = pa_context_new(pa_mainloop_get_api(main_loop), "gpu-screen-recorder-gtk"); - pa_context_connect(ctx, NULL, PA_CONTEXT_NOFLAGS, NULL); int state = 0; int pa_ready = 0; - pa_context_set_state_callback(ctx, pa_state_cb, &pa_ready); - pa_operation *pa_op = NULL; + pa_mainloop *main_loop = pa_mainloop_new(); + + pa_context *ctx = pa_context_new(pa_mainloop_get_api(main_loop), "gpu-screen-recorder"); + if(pa_context_connect(ctx, NULL, PA_CONTEXT_NOFLAGS, NULL) < 0) + goto done; + + pa_context_set_state_callback(ctx, pa_state_cb, &pa_ready); + for(;;) { // Not ready if(pa_ready == 0) { @@ -366,23 +368,25 @@ static void get_pulseaudio_default_inputs(AudioDevices &audio_devices) { } // Couldn't get connection to the server - if(pa_ready == 2 || (state == 1 && pa_op && pa_operation_get_state(pa_op) == PA_OPERATION_DONE)) { - if(pa_op) - pa_operation_unref(pa_op); - pa_context_disconnect(ctx); - pa_context_unref(ctx); - pa_mainloop_free(main_loop); - return; - } + if(pa_ready == 2 || (state == 1 && pa_op && pa_operation_get_state(pa_op) == PA_OPERATION_DONE)) + break; pa_mainloop_iterate(main_loop, 1, NULL); } + done: + if(pa_op) + pa_operation_unref(pa_op); + pa_context_disconnect(ctx); + pa_context_unref(ctx); pa_mainloop_free(main_loop); } AudioDevices get_pulseaudio_inputs() { AudioDevices audio_devices; + int state = 0; + int pa_ready = 0; + pa_operation *pa_op = NULL; // TODO: Do this in the same connection below instead of two separate connections get_pulseaudio_default_inputs(audio_devices); @@ -390,12 +394,10 @@ AudioDevices get_pulseaudio_inputs() { pa_mainloop *main_loop = pa_mainloop_new(); pa_context *ctx = pa_context_new(pa_mainloop_get_api(main_loop), "gpu-screen-recorder"); - pa_context_connect(ctx, NULL, PA_CONTEXT_NOFLAGS, NULL); - int state = 0; - int pa_ready = 0; - pa_context_set_state_callback(ctx, pa_state_cb, &pa_ready); + if(pa_context_connect(ctx, NULL, PA_CONTEXT_NOFLAGS, NULL) < 0) + goto done; - pa_operation *pa_op = NULL; + pa_context_set_state_callback(ctx, pa_state_cb, &pa_ready); for(;;) { // Not ready @@ -413,17 +415,17 @@ AudioDevices get_pulseaudio_inputs() { } // Couldn't get connection to the server - if(pa_ready == 2 || (state == 1 && pa_op && pa_operation_get_state(pa_op) == PA_OPERATION_DONE)) { - if(pa_op) - pa_operation_unref(pa_op); - pa_context_disconnect(ctx); - pa_context_unref(ctx); + if(pa_ready == 2 || (state == 1 && pa_op && pa_operation_get_state(pa_op) == PA_OPERATION_DONE)) break; - } pa_mainloop_iterate(main_loop, 1, NULL); } + done: + if(pa_op) + pa_operation_unref(pa_op); + pa_context_disconnect(ctx); + pa_context_unref(ctx); pa_mainloop_free(main_loop); return audio_devices; } diff --git a/src/utils.c b/src/utils.c index b4e34fd..a8e4dc2 100644 --- a/src/utils.c +++ b/src/utils.c @@ -71,6 +71,16 @@ static uint32_t x11_output_get_connector_id(Display *dpy, RROutput output, Atom return result; } +static vec2i get_monitor_size_rotated(int width, int height, gsr_monitor_rotation rotation) { + vec2i size = { .x = width, .y = height }; + if(rotation == GSR_MONITOR_ROT_90 || rotation == GSR_MONITOR_ROT_270) { + int tmp_x = size.x; + size.x = size.y; + size.y = tmp_x; + } + return size; +} + void for_each_active_monitor_output_x11_not_cached(Display *display, active_monitor_callback callback, void *userdata) { XRRScreenResources *screen_res = XRRGetScreenResources(display, DefaultRootWindow(display)); if(!screen_res) @@ -84,16 +94,23 @@ void for_each_active_monitor_output_x11_not_cached(Display *display, active_moni if(out_info && out_info->crtc && out_info->connection == RR_Connected) { XRRCrtcInfo *crt_info = XRRGetCrtcInfo(display, screen_res, out_info->crtc); if(crt_info && crt_info->mode) { + // We want to use the current mode info width/height (mode_info->width/height) instead of crtc info width/height (crt_info->width/height) because crtc info + // is scaled if the monitor is scaled (xrandr --output DP-1 --scale 1.5). Normally this is not an issue for x11 applications, + // but gpu screen recorder captures the drm framebuffer instead of x11 api. This drm framebuffer which doesn't increase in size when using xrandr scaling. + // Maybe a better option would be to get the drm crtc size instead. const XRRModeInfo *mode_info = get_mode_info(screen_res, crt_info->mode); if(mode_info && out_info->nameLen < (int)sizeof(display_name)) { snprintf(display_name, sizeof(display_name), "%.*s", (int)out_info->nameLen, out_info->name); + const gsr_monitor_rotation rotation = x11_rotation_to_gsr_rotation(crt_info->rotation); + const vec2i monitor_size = get_monitor_size_rotated(mode_info->width, mode_info->height, rotation); + const gsr_monitor monitor = { .name = display_name, .name_len = out_info->nameLen, .pos = { .x = crt_info->x, .y = crt_info->y }, - .size = { .x = (int)crt_info->width, .y = (int)crt_info->height }, + .size = monitor_size, .connector_id = x11_output_get_connector_id(display, screen_res->outputs[i], randr_connector_id_atom), - .rotation = x11_rotation_to_gsr_rotation(crt_info->rotation), + .rotation = rotation, .monitor_identifier = out_info->crtc }; callback(&monitor, userdata); @@ -165,7 +182,7 @@ static bool connector_get_property_by_name(int drmfd, drmModeConnectorPtr props, return false; } -/* TODO: Support more connector types*/ +/* TODO: Support more connector types */ static int get_connector_type_by_name(const char *name) { int len = strlen(name); if(len >= 5 && strncmp(name, "HDMI-", 5) == 0) @@ -400,6 +417,8 @@ bool gl_get_gpu_info(gsr_egl *egl, gsr_gpu_info *info) { if(strstr((const char*)gl_vendor, "AMD")) info->vendor = GSR_GPU_VENDOR_AMD; + else if(strstr((const char*)gl_vendor, "Mesa") && gl_renderer && strstr((const char*)gl_renderer, "AMD")) + info->vendor = GSR_GPU_VENDOR_AMD; else if(strstr((const char*)gl_vendor, "Intel")) info->vendor = GSR_GPU_VENDOR_INTEL; else if(strstr((const char*)gl_vendor, "NVIDIA")) @@ -464,23 +483,14 @@ static bool try_card_has_valid_plane(const char *card_path) { return false; } -static void string_copy(char *dst, const char *src, int len) { - int src_len = strlen(src); - int min_len = src_len; - if(len - 1 < min_len) - min_len = len - 1; - memcpy(dst, src, min_len); - dst[min_len] = '\0'; -} - bool gsr_get_valid_card_path(gsr_egl *egl, char *output, bool is_monitor_capture) { if(egl->dri_card_path) { - string_copy(output, egl->dri_card_path, 127); + snprintf(output, 128, "%s", egl->dri_card_path); return is_monitor_capture ? try_card_has_valid_plane(output) : true; } for(int i = 0; i < 10; ++i) { - snprintf(output, 127, DRM_DEV_NAME, DRM_DIR_NAME, i); + snprintf(output, 128, DRM_DEV_NAME, DRM_DIR_NAME, i); if(try_card_has_valid_plane(output)) return true; } @@ -494,7 +504,7 @@ bool gsr_card_path_get_render_path(const char *card_path, char *render_path) { char *render_path_tmp = drmGetRenderDeviceNameFromFd(fd); if(render_path_tmp) { - string_copy(render_path, render_path_tmp, 127); + snprintf(render_path, 128, "%s", render_path_tmp); free(render_path_tmp); close(fd); return true; @@ -745,6 +755,8 @@ bool vaapi_copy_drm_planes_to_video_surface(AVCodecContext *video_codec_context, .height = dest_size.y }; + const bool scaled = dest_size.x != source_size.x || dest_size.y != source_size.y; + // Copying a surface to another surface will automatically perform the color conversion. Thanks vaapi! VAProcPipelineParameterBuffer params = {0}; params.surface = input_surface_id; @@ -752,7 +764,7 @@ bool vaapi_copy_drm_planes_to_video_surface(AVCodecContext *video_codec_context, params.surface_region = &source_region; params.output_region = &output_region; params.output_background_color = 0; - params.filter_flags = VA_FRAME_PICTURE; + params.filter_flags = scaled ? (VA_FILTER_SCALING_HQ | VA_FILTER_INTERPOLATION_BILINEAR) : 0; params.pipeline_flags = VA_PROC_PIPELINE_FAST; params.input_color_properties.colour_primaries = 1; @@ -884,3 +896,20 @@ bool vaapi_copy_egl_image_to_video_surface(gsr_egl *egl, EGLImage image, vec2i s return success; } + +vec2i scale_keep_aspect_ratio(vec2i from, vec2i to) { + if(from.x == 0 || from.y == 0) + return (vec2i){0, 0}; + + const double height_to_width_ratio = (double)from.y / (double)from.x; + from.x = to.x; + from.y = from.x * height_to_width_ratio; + + if(from.y > to.y) { + const double width_height_ratio = (double)from.x / (double)from.y; + from.y = to.y; + from.x = from.y * width_height_ratio; + } + + return from; +} |