Faster mangakatana search on exact match

9 files changed, 71 insertions, 47 deletions

diff --git a/TODO b/TODO
index 4e1ed73..b241b45 100644
--- a/TODO
+++ b/TODO
@@ -22,7 +22,7 @@ Show filename at the bottom when viewing an image/video on 4chan.
 Use https://github.com/simdjson/simdjson as a json library in other parts than matrix.
 Sanitize check: do not allow pasting more than 2gb of text.
 Add autocomplete for matrix commands.
-Add option to disable autosearch and search when pressing enter instead or something? this would be needed for mobile phones where typing is slow.
+Add option to disable autosearch and search when pressing enter instead or something? this would be needed for mobile phones where typing is slow. Or mobile should have longer timeout for all searches.
 Render view to a rendertexture and render that instead of redrawing every time every time.
 Provide a way to specify when notifications should be received (using matrix api) and also read the notification config from matrix. Also provide a way to disable notifications globally.
 Use quickmedia to show image in matrix rooms, instead of mpv.
@@ -67,7 +67,6 @@ Modify matrix sync to download and parse json but not handle it, and then add a
 Fetch replies/pinned message using multiple threads.
 Show in room tags list when there is a message in any of the rooms in the tag.
 Show images while they download by showing them as scanlines starting from the top. Needed for slow websites such as 4chan.
-Use curl parallel download instead of downloading with multiple threads. This can be done with multiple -O parameters.
 Add functionality to ignore users in matrix. This is done with an ignore request and we wont get messages and invites from that user anymore. Also add option to ignore in the invites page.
 Add keybind to go to invites page from any page.
 Show marker beside pinned messages tab name if there are new pinned messages.
@@ -97,10 +96,8 @@ Allow removing reactions.
 When fetching previous messages in matrix, fetching until there are 0 messages or until there is at least 1 visible item. This is needed because right now we could fetch 10 messages, all which are delete/edit/react and it will look like there are no more messages to fetch to the user.
 QuickMedia crashes if you enter a room that you have left before. This could happen if cache is loaded that has a room that is then getting removed when sync is finished.
 Limit size of Entry and scroll content instead.
-Instead of doing a GET for the first N bytes to check if a video is streamable, start streaming the video and if the first bytes doesn't contain MOOV then wait until the whole video has downloaded before playing it.
 Implement matrix spoiler, see: https://github.com/matrix-org/matrix-doc/blob/master/proposals/2010-spoilers.md.
-Replace sfml font glyph loading completely with FreeType.
-Add arguments to pipe plugin to pass input and output fifo for sending commands to QuickMedia and receiving events.
+Replace sfml font glyph loading completely with stb_freetype.
 Create a workaround for dwm terminal swallow patch stealing mpv when moving QuickMedia to another monitor sometimes. Maybe check for structure notify events on mpv and reparent and select input on the mpv window again?
 Add option to decline and mute user in invites. This is to combat invite spam, where muted users cant invite you.
 Add an option to select video resolution, if we want to use less power and less bandwidth for example.
@@ -124,8 +121,7 @@ Update timestamp of messages posted with matrix (and move them to the correct pl
 When sfml dependency is removed use libvips for image manipulation. Its a very fast library, especially for thumbnail creation.
 Notification race condition when fetching the first notifications page and receiving a notification immediately after the first sync? we might end up with a duplicate notification.
 Readd copying of Text in Body copy constructor. Find out why we need to make the text dirty on copy.
-Body items that are no longer visible should stop their thumbnail download/creation (moving to bottom of file-manager is very slow).
-Fix body flickering when moving up and there is a new local image to load. It happens because we have no idea how large the thumbnail is before loading it.
+Body items that are no longer visible should stop their thumbnail download.
 Make body width the same as the window width (for the main body when there isn't a room list beside it) and pass margin to body draw. This is needed to allow swiping body from the window (screen) edge.
 Restrict sf::View viewport in Body to x axis as well, to hide the body in the end when swiping right.
 Ctrl+F to either bring up search that searches in the body (filtering, of searching for all messages in the room in matrix) or does an autocomplete search. For youtube that would be a google autocomplete search.
@@ -141,8 +137,6 @@ When synapse adds support for media download http request range then quickmedia
 Ctrl+S when a body item is selected on youtube/xxxplugins should show an option to download the video, instead of having to press ctrl+s after the video is playing.
 Show a subscribe button for channels, which should be red and say "subscribe" if we are not subscribed and it should be gray and say "unsubscribe" if we already are subscribed.
 Display youtube playlists differently and support downloading the whole playlist at once.
-To fix jitter in body when items are added before the selected item, we should increase page_scroll by the height of the new item.
-Cancel current search when search is updated.
 Support 4chan archive.
 Use old method of rendering body where rendering items is done up and down, starting from the selected item. This will make quickmedia run as fast when displaying 100 000 items as when displaying 10 items.
 Use correct spacing when in grid (card) mode. Should be row spacing instead of body spacing.
@@ -153,18 +147,13 @@ Embedding elements in rich text: first byte should be an invalid utf8 character
 Do not set fps to monitor hz if no key is pressed and cursor is not moving (for example when the computer is sleeping). (Maybe just detect if any sfml event is received), but maybe that wouldn't work with accelerometer.
 Ctrl+arrow key to move to previous/next video.
 Add keybinding to view file-manager images in fullscreen to preview them. Also add keybinding to create/delete directories.
-Update item height when it switches from not being merged with previous to being merged with previous. This happens when loading previous messages in matrix and the message is the top one.
 Reload youtube video url if the video is idle for too long. The video url is only valid for a specific amount of time (the valid duration is in the json).
 Improve live stream startup time by downloading the video formats in parts instead of the hls/dash manifest? (use YoutubeLiveStreamMediaProxy).
-Disable drop shadow on pinephone.
 Load the next page in chapter list when reaching the bottom (when going to previous chapters in image view).
 Loading image background should be rounded.
-//Workaround mpv issue where video is frozen after seeking (with and without cache enabled, but more often with cache enabled). This happens because of audio. Reloading audio fixes this but audio will then be gone.
 Better deal with reading from file errors. This could happen when reading a file while its being modified. See read_file_as_json.
 Allow ctrl+r for video when the video is loading.
-Youtube download gets stuck sometimes because of audio. Find a workaround for this.
 Dynamically change youtube video quality by modifying the itags (and other params?) if download is buffering or if the video is lagging.
-Use the new media proxy for downloading youtube videos as well.
 PgUp/PgDown shouldn't move body by the number of visible items. It should instead move by the height of the body.
 Add option to view dead link in 4chan with 4chan archive and navigate to crossboard links.
 Show latest message before sync is done for a room when the latest message is an edit. Right now it has to fetch previous messages until the first non-edit message.
diff --git a/include/DownloadUtils.hpp b/include/DownloadUtils.hpp
index 8edcd5a..7e46d8b 100644
--- a/include/DownloadUtils.hpp
+++ b/include/DownloadUtils.hpp
@@ -25,7 +25,7 @@ namespace QuickMedia {
     // Returns the remote name from the content-disposition header or tries to extract the file name from url. Can return empty name
     DownloadResult url_get_remote_name(const std::string &url, std::string &result, bool use_browser_useragent);
     // Note: if |cloudflare_bypass| is set to true then tls is limited to version 1.1 and the user agent is changed
-    DownloadResult download_to_string(const std::string &url, std::string &result, const std::vector<CommandArg> &additional_args, bool use_browser_useragent = false, bool fail_on_error = true, bool cloudflare_bypass = false, std::string *header = nullptr, int download_limit = 1024 * 1024 * 100); // 100mb download limit
+    DownloadResult download_to_string(const std::string &url, std::string &result, const std::vector<CommandArg> &additional_args, bool use_browser_useragent = false, bool fail_on_error = true, bool cloudflare_bypass = false, std::vector<std::string> *headers = nullptr, int download_limit = 1024 * 1024 * 100); // 100mb download limit
     // Note: This function saves the content to the file atomically
     DownloadResult download_to_string_cache(const std::string &url, std::string &result, const std::vector<CommandArg> &additional_args, bool use_browser_useragent = false, DownloadErrorHandler error_handler = nullptr, Path cache_path = "");
     // Note: This function saves the content to the file atomically.
diff --git a/include/StringUtils.hpp b/include/StringUtils.hpp
index cc2acf4..acc7305 100644
--- a/include/StringUtils.hpp
+++ b/include/StringUtils.hpp
@@ -20,6 +20,7 @@ namespace QuickMedia {
     bool string_ends_with(const std::string &str, const std::string &ends_with_str);
     size_t str_find_case_insensitive(const std::string &str, size_t start_index, const char *substr, size_t substr_len);
     char to_upper(char c);
+    bool strncase_equals(const char *str1, const char *str2, size_t length);
     bool strcase_equals(const char *str1, const char *str2);
     bool to_num(const char *str, size_t size, int &num);
 }
\ No newline at end of file
diff --git a/src/DownloadUtils.cpp b/src/DownloadUtils.cpp
index 5f0c547..a56a56e 100644
--- a/src/DownloadUtils.cpp
+++ b/src/DownloadUtils.cpp
@@ -12,10 +12,10 @@
 
 namespace QuickMedia {
     struct DownloadUserdata {
-        std::string *header = nullptr;
+        std::vector<std::string> *headers = nullptr;
         std::string *body = nullptr;
         int64_t download_limit = 1024 * 1024 * 100; // 100mb
-        bool header_finished = false;
+        bool headers_finished = false;
         int64_t total_downloaded_size = 0;
     };
 
@@ -37,19 +37,27 @@ namespace QuickMedia {
     static int accumulate_string_with_header(char *data, int size, void *userdata) {
         DownloadUserdata *download_userdata = (DownloadUserdata*)userdata;
 
-        if(download_userdata->header_finished) {
+        if(download_userdata->headers_finished) {
             download_userdata->body->append(data, size);
         } else {
-            download_userdata->header->append(data, size);
+            if(download_userdata->headers->empty())
+                download_userdata->headers->push_back("");
+
+            std::string *current_header = &download_userdata->headers->back();
+            current_header->append(data, size);
+
             bool end_of_header_found = false;
-            size_t end_of_headers_index = download_userdata->header->find("\r\n\r\n");
+            size_t end_of_headers_index = current_header->find("\r\n\r\n");
             if(end_of_headers_index != std::string::npos) {
                 while(true) {
-                    const bool is_redirect = http_is_redirect(download_userdata->header->c_str(), end_of_headers_index);
+                    const bool is_redirect = http_is_redirect(current_header->c_str(), end_of_headers_index);
                     end_of_headers_index += 4;
-                    if(is_redirect && download_userdata->header->size() - end_of_headers_index > 0) {
-                        download_userdata->header->erase(download_userdata->header->begin(), download_userdata->header->begin() + end_of_headers_index);
-                        end_of_headers_index = download_userdata->header->find("\r\n\r\n");
+                    if(is_redirect) {
+                        std::string header_after_this_header = current_header->substr(end_of_headers_index);
+                        current_header->erase(current_header->begin() + end_of_headers_index, current_header->end());
+                        download_userdata->headers->push_back(std::move(header_after_this_header));
+                        current_header = &download_userdata->headers->back();
+                        end_of_headers_index = current_header->find("\r\n\r\n");
                         if(end_of_headers_index == std::string::npos)
                             break;
                     } else {
@@ -60,9 +68,9 @@ namespace QuickMedia {
             }
 
             if(end_of_header_found) {
-                download_userdata->body->append(download_userdata->header->begin() + end_of_headers_index, download_userdata->header->end());
-                download_userdata->header->erase(download_userdata->header->begin() + end_of_headers_index, download_userdata->header->end());
-                download_userdata->header_finished = true;
+                download_userdata->body->append(current_header->begin() + end_of_headers_index, current_header->end());
+                current_header->erase(current_header->begin() + end_of_headers_index, current_header->end());
+                download_userdata->headers_finished = true;
             }
         }
 
@@ -185,7 +193,7 @@ namespace QuickMedia {
     }
 
     // TODO: Add timeout
-    DownloadResult download_to_string(const std::string &url, std::string &result, const std::vector<CommandArg> &additional_args, bool use_browser_useragent, bool fail_on_error, bool cloudflare_bypass, std::string *header, int download_limit) {
+    DownloadResult download_to_string(const std::string &url, std::string &result, const std::vector<CommandArg> &additional_args, bool use_browser_useragent, bool fail_on_error, bool cloudflare_bypass, std::vector<std::string> *headers, int download_limit) {
         result.clear();
         sf::Clock timer;
         std::vector<const char*> args;
@@ -203,7 +211,7 @@ namespace QuickMedia {
         }
         if(cloudflare_bypass)
             args.insert(args.end(), { "--tls-max", "1.1", "-A", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FSL 7.0.6" });
-        if(header)
+        if(headers)
             args.push_back("-i");
         args.push_back("--");
         args.push_back(url.c_str());
@@ -218,10 +226,10 @@ namespace QuickMedia {
         }
 
         DownloadUserdata download_userdata;
-        download_userdata.header = header;
+        download_userdata.headers = headers;
         download_userdata.body = &result;
         download_userdata.download_limit = download_limit;
-        download_userdata.header_finished = !header;
+        download_userdata.headers_finished = !headers;
 
         if(exec_program(args.data(), accumulate_string_with_header, &download_userdata) != 0)
             return DownloadResult::NET_ERR;
diff --git a/src/NetUtils.cpp b/src/NetUtils.cpp
index 0f957d5..9cf88c7 100644
--- a/src/NetUtils.cpp
+++ b/src/NetUtils.cpp
@@ -311,7 +311,7 @@ namespace QuickMedia {
         std::string result;
         string_split(header, '\n', [&type, &result](const char *str, size_t size) {
             while(size > 0 && (*str == ' ' || *str == '\t')) { ++str; --size; }
-            if(size < type.size() || strncasecmp(str, type.c_str(), type.size()) != 0 || size == type.size())
+            if(size < type.size() || !strncase_equals(str, type.c_str(), type.size()) || size == type.size())
                 return true;
 
             str += type.size();
@@ -325,7 +325,9 @@ namespace QuickMedia {
             str += (colon_offset + 1);
             size -= (colon_offset + 1);
 
+            // lstrip space
             while(size > 0 && (*str == ' ' || *str == '\t')) { ++str; --size; }
+            // rstrip whitespace
             while(size > 0 && (str[size - 1] == ' ' || str[size - 1] == '\t' || str[size - 1] == '\r' || str[size - 1] == '\n')) { --size; }
 
             result.assign(str, size);
diff --git a/src/StringUtils.cpp b/src/StringUtils.cpp
index 5dfeca9..8a3a0ef 100644
--- a/src/StringUtils.cpp
+++ b/src/StringUtils.cpp
@@ -117,14 +117,35 @@ namespace QuickMedia {
             return c;
     }
 
+    bool strncase_equals(const char *str1, const char *str2, size_t length) {
+        size_t i = 0;
+        for(;;) {
+            if(i == length)
+                return true;
+            ++i;
+
+            const char c1 = *str1;
+            const char c2 = *str2;
+            if(to_upper(c1) != to_upper(c2))
+                return false;
+            else if(c1 == '\0')
+                return true;
+
+            ++str1;
+            ++str2;
+        }
+    }
+
     bool strcase_equals(const char *str1, const char *str2) {
         for(;;) {
             const char c1 = *str1;
             const char c2 = *str2;
+
             if(to_upper(c1) != to_upper(c2))
                 return false;
             else if(c1 == '\0')
                 return true;
+
             ++str1;
             ++str2;
         }
diff --git a/src/VideoPlayer.cpp b/src/VideoPlayer.cpp
index 98d187f..ac18151 100644
--- a/src/VideoPlayer.cpp
+++ b/src/VideoPlayer.cpp
@@ -154,9 +154,12 @@ namespace QuickMedia {
             "--cache-on-disk=yes",
             "--cache-secs=86400", // 24 hours
             "--sub-font-size=40",
+            //"--sub-font=DejaVuSans-Bold",
+            //"--sub-bold=yes",
             "--sub-margin-y=45",
             "--sub-border-size=1.95",
-            "--sub-border-color=0.01",
+            "--sub-shadow-offset=1.25",
+            "--sub-shadow-color=0.2/0.9",
             cache_dir.c_str(),
             input_conf.c_str(),
             wid_arg.c_str()
diff --git a/src/plugins/MangaGeneric.cpp b/src/plugins/MangaGeneric.cpp
index 738c6dc..87b4f1d 100644
--- a/src/plugins/MangaGeneric.cpp
+++ b/src/plugins/MangaGeneric.cpp
@@ -169,7 +169,8 @@ namespace QuickMedia {
 
         std::string target_url;
         std::string website_data;
-        if(download_to_string(url, website_data, args, true, fail_on_http_error) != DownloadResult::OK)
+        std::vector<std::string> website_headers;
+        if(download_to_string(url, website_data, args, true, fail_on_http_error, false, &website_headers) != DownloadResult::OK)
             return PluginResult::NET_ERR;
 
         if(website_data.empty())
@@ -199,17 +200,10 @@ namespace QuickMedia {
                 goto cleanup;
 
             if(!text_query.url_field && !new_result_items.empty()) {
-                if(target_url.empty()) {
-                    std::string response_headers;
-                    DownloadResult download_result = download_head_to_string(url, response_headers, true); 
-                    if(download_result != DownloadResult::OK) {
-                        result = -1;
-                        goto cleanup;
-                    }
-
-                    target_url = header_extract_value(response_headers, "location");
+                if(target_url.empty() && !website_headers.empty()) {
+                    target_url = header_extract_value(website_headers.front(), "location");
                     if(target_url.empty()) {
-                        fprintf(stderr, "Failed to extract target location from %s HEAD\n", url.c_str());
+                        fprintf(stderr, "Failed to extract target location from %s\n", url.c_str());
                         result = -1;
                         goto cleanup;
                     }
diff --git a/src/plugins/Youtube.cpp b/src/plugins/Youtube.cpp
index 4934c43..b490e97 100644
--- a/src/plugins/Youtube.cpp
+++ b/src/plugins/Youtube.cpp
@@ -237,20 +237,26 @@ namespace QuickMedia {
         std::vector<CommandArg> additional_args = get_cookies();
         additional_args.push_back({"--no-buffer", ""});
 
+        std::vector<std::string> response_headers;
         const int max_redirects = 5;
         for(int i = 0; i < max_redirects; ++i) {
             std::string response_body;
-            std::string response_headers;
+            response_headers.clear();
             download_to_string(playback_url, response_body, additional_args, true, true, false, &response_headers, 4096);
 
-            std::string content_type = header_extract_value(response_headers, "content-type");
+            if(response_headers.empty()) {
+                fprintf(stderr, "Youtube video header not found\n");
+                return "";
+            }
+
+            std::string content_type = header_extract_value(response_headers.back(), "content-type");
             if(content_type.empty()) {
                 fprintf(stderr, "Failed to find content-type in youtube video header\n");
                 return "";
             }
 
             if(string_starts_with(content_type, "video") || string_starts_with(content_type, "audio")) {
-                std::string content_length_str = header_extract_value(response_headers, "content-length");
+                std::string content_length_str = header_extract_value(response_headers.back(), "content-length");
                 if(content_length_str.empty())
                     return "";