Commit Graph

497 Commits

Author SHA1 Message Date
Côme Chilliet 5090848b25 Store size in int|float for 32bit support
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-09-19 13:43:01 +02:00
Julius Härtl 13b9dd2267 fix: Only read unencrypted_size when file is actually encrypted
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-09-06 07:25:28 +02:00
Robin Appelman 9ea03a9534 reuse dependencies from wrapped cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-08-18 11:14:52 +02:00
Ferdinand Thiessen 9c04c07622 fix: Prevent PHP warnings when optional CacheEntry attributes are unset
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2023-08-16 14:12:51 +02:00
Robin Appelman 658aed27ea select the fileid first when looking for incomplete files
this seems to improve mariadbs index selection

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-08-14 18:08:01 +02:00
John Molakvoæ 7b7d1e3c09 fix: simplify sourceData check
Co-authored-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
Signed-off-by: John Molakvoæ <skjnldsv@users.noreply.github.com>
2023-08-05 12:05:36 +02:00
Josh Richards deecb48e54 Catch more invalid cache source storage paths
OC\Files\Cache\Cache::get can return string|false|null, not just string|false.

- nextcloud/server#26270 added handling of false, but null is needed too.
- Well, or we change the default $resullt to false, but I'm not sure if that has other ramifications and the real need here is to simply catch situations where the cache source storage path is not valid for whatever reason

Related: nextcloud/server#19009

Signed-off-by: Josh Richards <josh.t.richards@gmail.com>
2023-08-03 09:53:46 -04:00
Robin Appelman c912a3f0f6 Merge pull request #38860 from nextcloud/scanner-change-summary
add summary of detected changes to files:scan output
2023-07-20 18:37:07 +02:00
Christoph Wurst 14719110b9 chore: Replace \OC::$server->query with \OCP\Server::get in /lib
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2023-07-06 15:21:22 +02:00
Julius Härtl eddb64f8c3 Merge pull request #37969 from nextcloud/poc/noid/systemtags-perf-tag-endpoint 2023-06-28 07:53:35 +02:00
Robin Appelman 39a0554c09 Merge pull request #38858 from nextcloud/scanner-performance3
file scanner performance improvements
2023-06-26 17:01:18 +02:00
Arthur Schiwon 49db546f78 fix: include invisible tags for admins
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 20:28:01 +02:00
Arthur Schiwon 783e32a64d fix: favorites view and universal search against tags
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon b0d1cf5730 fix: change if with conditionless else to switch; and a parameter value
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon 58f7fd2370 use efficient tag retrieval on DAV report request
- uses DAV search approach against valid files joined by systemtag selector
- reduced table join for tag/systemtag search
- supports pagination
- no changes to the output formats or similar

Example request body:

<?xml version="1.0"?>
<oc:filter-files xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns" xmlns:ocs="http://open-collaboration-services.org/ns">
  <d:prop>
    <d:getcontentlength/>
    <d:getcontenttype/>
    <d:getetag/>
    <d:getlastmodified/>
    <d:resourcetype/>
    <nc:face-detections/>
    <nc:file-metadata-size/>
    <nc:has-preview/>
    <nc:realpath/>
    <oc:favorite/>
    <oc:fileid/>
    <oc:permissions/>
    <nc:nbItems/>
  </d:prop>
  <oc:filter-rules>
    <oc:systemtag>32</oc:systemtag>
  </oc:filter-rules>
  <d:limit>
    <d:nresults>50</d:nresults>
    <nc:firstresult>0</nc:firstresult>
  </d:limit>
</oc:filter-files>

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon 71e41ffeeb refactor: declare getMount() and getMountsIn() at IRootFolder
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:20 +02:00
Arthur Schiwon f6ac874dbf chore: ugly type juggling
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:20 +02:00
Arthur Schiwon eb50b6c6aa fix: expect interface, not a specific implementation
- fixes a regression when deleting folders while music app was enabled,
  for a LazyRoot was passed to this method.

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:13 +02:00
Robin Appelman 292c0e53f8 add summary of detected changes to files:scan output
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-06-16 17:56:34 +02:00
Robin Appelman 561536fc58 file scanner performance improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-06-16 16:31:48 +02:00
Christoph Wurst 872c181c74 chore: Drop dead private methods in /lib
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2023-06-06 11:01:58 +02:00
Simon L 702196dd86 Merge pull request #38478 from nextcloud/backport/38440/master
[master] fix: Catch Deadlock properly as execute throws Doctrine exceptions not our wrapped ones
2023-06-05 23:22:40 +02:00
Robin Appelman 9f1d497a0b Merge pull request #38261 from fsamapoor/replace_strpos_calls_in_lib_private
Refactors "strpos" calls in  lib/private to improve code readability.
2023-06-01 23:10:00 +02:00
Julius Härtl 9453c2d40d fix: execute throws docrine exceptions not our wrapped ones
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-05-26 06:56:55 +00:00
Côme Chilliet 8362eea14e Merge pull request #38196 from nextcloud/fix/fix-32bits-freespace-and-sizes
Get rid of more int casts in file size manipulations
2023-05-15 17:05:46 +02:00
Faraz Samapoor e7cc7653b8 Refactors "strpos" calls in lib/private to improve code readability.
Signed-off-by: Faraz Samapoor <fsamapoor@gmail.com>
2023-05-15 15:17:19 +03:30
Côme Chilliet ae525e1935 Fix return types in phpdoc
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-05-15 12:24:42 +02:00
Julius Härtl e18f97fc95 Merge pull request #37709 from nextcloud/bugfix/deadlock 2023-05-13 13:13:04 +02:00
Côme Chilliet a90581b405 Get rid of more int casts in file size manipulations
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-05-11 12:46:16 +02:00
Arthur Schiwon b6c034ac57 Merge pull request #37961 from nextcloud/poc/noid/systemtags-perf
SystemTags endpoint to return tags used by a user with meta data
2023-05-11 10:16:03 +02:00
Robin Appelman 2ea41dab93 repair -1 folder sizes for object store background scan
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-05-10 19:33:26 +02:00
Arthur Schiwon df662f50bd refactor: remove where specification from SELECT getter
- search constraints are now fully in control of
  SystemTagsInFilesDetector::detectAssignedSystemTagsIn(), avoids
  duplication of a WHERE statement

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-10 18:48:06 +02:00
Arthur Schiwon dbfd2f936a refactor: remove SystemTag logic from Folder into QuerySearchHelper
- adds OC\SystemTag\SystemTagsInFilesDetector where the search logic is
  moved to

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:52 +02:00
Arthur Schiwon 2489a2d63a feat: specify media type via url path: systemtags-current/$mediaType
- only the media part of the mime type can be search, but not the full
  mime type. It can be added, should it become necessary.
- thus fixes previously hardcoded selector for image/ types
- also fixes a return type hint
- adds a return type hint

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:51 +02:00
Arthur Schiwon 6bff6a5667 PoC: SystemTags endpoint to return tags used by a user with meta data
Target case is photos app: when visiting the tags category, all systemtags
of the whole cloud are retrieved. In subequent steps the next tag is
requested until the browser view is filled with tag tiles (i.e. previews
are requested just as well).

With this approach, we incorpoate the dav search and look for user related
tags that are used by them, and already returns the statistics (number of
files tagged with the respective tag) as well as a file id for the purpose
to load the preview. This defaults to the file with the highest id.

Call:
curl -s -u 'user:password' \
  'https://my.nc.srv/remote.php/dav/systemtags-current' \
  -X PROPFIND -H 'Accept: text/plain' \
  -H 'Accept-Language: en-US,en;q=0.5'  -H 'Depth: 1' \
  -H 'Content-Type: text/plain;charset=UTF-8' \
  --data @/home/doe/request-systemtag-props.xml

With request-systemtag-props.xml:
<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
        <d:prop xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
                <oc:id/>
                <oc:display-name/>
                <oc:user-visible/>
                <oc:user-assignable/>
                <oc:can-assign/>
                <nc:files-assigned/>
                <nc:reference-fileid/>
        </d:prop>
</d:propfind>

Example output:
  …
  <d:response>
    <d:href>/master/remote.php/dav/systemtags/84</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>84</oc:id>
        <oc:display-name>Computer</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>42</nc:files-assigned>
        <nc:reference-fileid>924022</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/systemtags/97</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>97</oc:id>
        <oc:display-name>Bear</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>1</nc:files-assigned>
        <nc:reference-fileid>923422</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  …

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:48 +02:00
Julius Härtl 3bdd770129 fix: Check for wrapped retriable exceptions
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-05-02 19:42:11 +02:00
Robin Appelman 4953296b9e Merge pull request #36857 from nextcloud/scan-repair-unencrypted-size
fix unencrypted_size for folders when scanning the filesystem with encryption enabled
2023-04-24 15:24:50 +02:00
Robin Appelman 0ac7230b7f Merge pull request #37820 from nextcloud/parent-storage-mtime-ignore-error
ignore errors while trying to update parent storage_mtime
2023-04-21 19:11:27 +02:00
Robin Appelman 4b52dafbf9 ignore errors while trying to update parent storage_mtime
in the worst case this should only cause an extra rescan later

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-20 17:16:03 +02:00
Côme Chilliet b294edad80 Merge branch 'master' into enh/type-iconfig-getter-calls
Signed-off-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
2023-04-20 16:52:38 +02:00
Robin Appelman b1f352c614 improve handling of files we can't access in the scanner
instead of erroring, remove the items from the cache.

this situation can be triggered if a user has access to a file but looses it afterwards

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-05 14:44:55 +02:00
Côme Chilliet 426c0341ff Use typed version of IConfig::getSystemValue as much as possible
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-05 12:50:08 +02:00
Robin Appelman 5267a74027 fix type hints
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-04 17:05:45 +02:00
Robin Appelman 8c9197d65e fix unencrypted folder size when no children with unencrypted size set are left
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-04 16:48:11 +02:00
Robin Appelman 649bed5154 fix unencrypted_size for files when scanning
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-04 16:48:11 +02:00
Robin Appelman 63fb33538c fix unencrypted_size for folders when scanning the filesystem with encryption enabled
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-04 16:48:10 +02:00
jld3103 dcf6ebebab Fix inherited parameter names
Signed-off-by: jld3103 <jld3103yt@gmail.com>
2023-04-03 08:29:33 +02:00
Robin Appelman c9e4e30523 clear encrypted flag when moving away from encrypted storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-01 15:36:37 +00:00
Côme Chilliet c60182065f Merge pull request #34773 from nextcloud/artonge/feat/dispatch_entry_removed_event_for_all_entries
Dispatch event for all removed entries
2023-03-21 11:42:18 +01:00
Joas Schilling 4bdc5834fa Make sure name and path are strings
Otherwise Oracle returns NULL for empty strings and PHP 8.2
throws on null in string functions like trim() and md5()

Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-02-02 12:05:22 +01:00