Commit Graph

146 Commits

Author SHA1 Message Date
Morris Jobke
638e9438a9 Merge pull request #21728 from nextcloud/bugfix/noid/s3-external-mtime-cache
Reuse cache for directory mtime/size if filesystem changes can be ignored
2020-07-15 18:26:48 +02:00
Robin Appelman
e357d46863 filter files containing a hash in the path for ftp storages
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.

By filtering out paths containing a hash we can at least stop the external storage from breaking completely

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-10 14:18:40 +02:00
Julius Härtl
f5cc34c9c2 Reuse cache for directory mtime/size if filesystem changes can be ignored
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-07-07 10:22:22 +02:00
Julius Härtl
69f4c50032 Remove rescanDelay from directory mtime
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-06-22 14:15:57 +02:00
Robin Appelman
6f2e920a5b normalize sftp path in read and write stream
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-05-25 15:26:04 +02:00
Christoph Wurst
cb057829f7 Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Robin Appelman
23e2baeb95 Add option to check share ACL's when listing directories
If a file or folder in a directory doesn't have read permissions they
will not be shown

Note that enabling this option incurs a performance penalty additional
requests need to be made to get all the acl.

Additionally the acl resolving logic is fairly primitive at the moment
and might not work correctly in all setups (it should error to showing
the entry)

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-21 16:26:12 +02:00
Robin Appelman
9735b5d61f Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Christoph Wurst
28f8eb5dba Add visibility to all constants
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:54:27 +02:00
Christoph Wurst
1584c9ae9c Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst
caff1023ea Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst
14c996d982 Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Christoph Wurst
bcd13296de Make sure every file ends with an empty newline
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:11:50 +02:00
Christoph Wurst
44577e4345 Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00
Christoph Wurst
afbd9c4e6e Unify function spacing to PSR2 recommendation
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 13:54:22 +02:00
Christoph Wurst
41b5e5923a Use exactly one empty line after the namespace declaration
For PSR2

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:48:10 +02:00
Christoph Wurst
85e369cddb Fix multiline comments
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-08 22:24:54 +02:00
Christoph Wurst
1a9330cd69 Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
John Molakvoæ
fa4162e86f Merge pull request #20208 from nextcloud/smb-folder-contents-normalize-path
normalize smb path while listing folder contents
2020-03-30 10:06:39 +02:00
Robin Appelman
ef7bb5225a normalize smb path while listing folder contents
currently when listing `'/'`, it will include a double slash in the paths used for the listing logic
this causes unneeeded cache misses in the statCache

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-27 17:04:53 +01:00
Christoph Wurst
b80ebc9674 Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Robin Appelman
223ee42a53 faster implementation of SFTP write stream
using mostly the same techniques as the read stream

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-10 16:18:40 +01:00
Robin Appelman
fe4e80093e Faster implementation of sftp read stream
- Keeps a single file handle open instead of creating a new one for
every stream_read
- Read from sftp in large chunks instead of the 8k internal php size
- Request a read for the next chunk while we're still processing the
current one

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-06 16:57:18 +01:00
Christoph Wurst
5bf3d1bb38 Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Roeland Jago Douma
68748d4f85 Some php-cs fixes
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-22 20:52:10 +01:00
Daniel Kesselberg
230f685987 Remove superfluous fclose
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-10-21 11:31:07 +02:00
Roeland Jago Douma
88b6dc5eaf Merge pull request #17258 from nextcloud/smb-timeout-config
add (hidden) option to configure smb timeout
2019-09-27 12:29:56 +02:00
Robin Appelman
194edfc64e add (hidden) option to configure smb timeout
hidden from ui to prevent clutter

```occ files_external:config <mount> timeout 30```

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-09-25 13:30:40 +02:00
Daniel Kesselberg
72d22a4828 Use local variable for directory name
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
75b8944061 Don't strip path from directory prefix
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
80340a8d46 Fill directory cache from opendir
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
12863e0d24 Change visibility to private (from protected)
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg
963d35c78c Rename $location to $path
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg
010473ca1f Add method to check if directory exists
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Arthur Schiwon
43bc31bacb set a storage availability delay on auth issues to avoid lock out
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-22 12:40:15 +02:00
Arthur Schiwon
cd62b38cc8 catch auth issues also when php-smbclient is in use
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-22 11:12:43 +02:00
Morris Jobke
adc2ab4bb2 Ignore forbidden exception when fetching share mtime on SMB for symlinks
See https://github.com/nextcloud/server/pull/7556#issuecomment-499028026

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2019-07-08 21:43:24 +02:00
Robin Appelman
6d5579863e add option to show hidden files in SMB shares
Note hidden files can mean different things in smb and the option the the files web ui,
the webui only counts files starting with '.' as hidden, while smb files
can be marked as hidden regardless, any files that are marked as hidden
on smb will thus be shown in the webui regardless of the setting in the files app.

Fixes #15644

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-23 21:23:56 +02:00
Michael Zamot
dc35a8af83 Convert timestamp key value of metadata in Swift files_external
modified:   apps/files_external/lib/Lib/Storage/Swift.php

Signed-off-by: Michael Zamot <michael@zamot.io>

Convert value of timestamp metadata key in swift files_external to string
2019-03-01 12:10:37 -07:00
Morris Jobke
7fc68fe93e Merge pull request #14211 from nextcloud/bugfix/prefix-path-to-filecache-s3
Prefix $path for filename for internal file cache
2019-02-19 12:01:08 +01:00
Daniel Kesselberg
0d667d18bb Call headObject if file not in filecache 🙈
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-02-14 18:34:04 +01:00
Daniel Kesselberg
659ed3ecda Prefix $path for filename for internal file cache
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-02-14 17:06:14 +01:00
Arthur Schiwon
792bcb82ae add LDAP ConfigHandler for external storages and "$home" var
* handler registered upon OCA\\Files_External::loadAdditionalBackends
  event as user_ldap is loaded before files_external
* new configuration field "ldapExtStorageHomeAttribute" (not in GUI yet)

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-02-14 15:22:22 +01:00
Robin Appelman
2c1f6331a4 Add interface to allow storages from opting out of encryption
As opposed to hard-coding a list of excluded storages

Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-29 16:31:34 +01:00
Daniel Kesselberg
4cbea5f7e1 Remove value from cache on delete/clear
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-10-08 16:40:39 +02:00
Daniel Kesselberg
5cf38254bb Use helper method for content length and last modified
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-10-01 20:18:36 +02:00
Daniel Kesselberg
67d5380b91 Reuse information from ListObjects for stat / filetype
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-09-30 21:05:53 +02:00
Morris Jobke
8563ab94aa Merge pull request #10589 from nextcloud/smb-stat-retry
retry smb stat on timeout
2018-08-24 07:00:40 +02:00
Robin Appelman
bdc6d0b5a1 log entries that are hidden during file listing
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 01:20:05 +02:00
Robin Appelman
24aaa3bc97 retry stat on timeout
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-08 16:07:41 +02:00