This commit adds cross-device sync ability for two plugins: reading statistics and vocabulary builder. It relies on user setting up a Cloud server (DropBox and WebDAV but not FTP though) and designating a path. Behind the curtains sqlite databases are being passed around and updated.
UI-wise, for the statistics plugin, two new menu items Synchronize now and Cloud sync to set it up (might not be the best wording) are added. As for vocabulary builder, a similar Cloud sync button is added to the menu and a shortcut icon button to Synchronize now is pinned at the bottom corner.
CloudStorage new features: WebDAV creating folders and uploading files. And a new widget-like sync server chooser. In the end I decided not to add automatic sync, as the SQL commands part seem a bit much.
If this is not done, the URL when the file is downloaded will be
something like hostdir/path, rather than host/dir/path.
Also add a debug log to make it more clear when a bogus URL
is being fetched,
* Add a new socketutil module with a few helper functions that allow us to:
* Always use a sane User-Agent (previously, only Wikipedia did so)
* Set timeouts in an almost sane manner. Doing it explicitly prevents an interaction with KOSync that does crazy stuff I don't even want to try to understand.
* Unified said timeouts based on the request's intended usage (except for Wikipedia, which already had meaningful timeout values).
* Stopped using LuaSec directly, LuaSocket defers to LuaSec sanely on its own. Everything now transparently supports HTTPS without code duplication.
This change to the parser in `cloudstorage.lua` adds support for the [Apache WebDAV module][1]
It was manually tested using the [bytemark/webdav][2] docker container.
I developed this in Windows, with a docker container that hosted an extracted AppImage and a VNC server that I viewed using a VNC client.
I will write up my work flow at a later point.
Changes have not been tested against other webdav servers (what was this originally tested against?). Please could someone test against other webdav servers?
I also noticed a logic inversion error where we were looking for a slash at the end of the URL and if it exists, then we explicitly set `has_trailing_slash=false` - so I fixed it to set to `true`. I had to do this so that we weren't visiting the URL without a trailing slash - apache sends back a 301 redirect with a `location` header with a trailing slash, if you don't put a trailing slash.
As a side note, I think we should consider replacing this regexp pattern matching parser with the [XML parser in the newsreader plugin[3]
[1]: https://httpd.apache.org/docs/2.4/mod/mod_dav.html
[2]: https://github.com/BytemarkHosting/docker-webdav
[3]: https://github.com/koreader/koreader/blob/master/plugins/newsdownloader.koplugin/lib/xml.lua
Option to show all files in cloudstorage also not supported by KOReader. We can also download any file.
To show all files in CS we need to enable option Show unsupported files in file manager (introducing in #5129)
Close: #5006
Addition of WebDav to the CloudStorage. The functionality is the same as with Dropbox and FTP, it is possible to browse the files on the server and download a copy.
Tested with:
NextCloud
HubZilla
