Commit Graph

695 Commits

Author SHA1 Message Date
lui87kw 3fe3d1fca3 Extra white space to follow other ifs
Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-07-15 10:18:47 +02:00
bladewing 7dafd5831f Avoid substr() error when strpos returns false
"Exception: substr() expects parameter 3 to be int, bool given" can occur on Line 378 $mimePart = substr($icon, 0, strpos($icon, '-'));
This happens, when '-' is not found and strpos returns false instead of an int.
When this occurs, e.g., Activity hangs.

Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-07-15 10:18:47 +02:00
Robin Appelman e357d46863 filter files containing a hash in the path for ftp storages
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.

By filtering out paths containing a hash we can at least stop the external storage from breaking completely

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-10 14:18:40 +02:00
Robin Appelman 70906a359f ensure home storage is initialized on first setup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-09 11:14:45 +02:00
Morris Jobke e900f424ce Merge pull request #21636 from nextcloud/lock-exception-readable-path
add proper paths to locking exceptions
2020-07-05 22:37:12 +02:00
Christoph Wurst d3a4bc91f7 Fix static method call for s3 bucket compat check
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-07-03 09:02:58 +02:00
Roeland Jago Douma d53e005458 Merge pull request #21360 from nextcloud/root-storage-prevent-scanning
prevent the root storage from accidentally scanning user folders
2020-07-02 21:07:16 +02:00
Roeland Jago Douma 873be16c8d Merge pull request #21588 from nextcloud/bug/21214/method-signature
Update presign method to match with interface again.
2020-06-30 20:36:30 +02:00
Robin Appelman da2d425044 add proper paths to locking exceptions
while some code paths do wrap the "raw" locking exception into one with a proper path, not all of them do
by adding the proper path to the original exception we ensure that we always have the usefull information in out logs

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-30 18:10:42 +02:00
Daniel Kesselberg e984357cfc Set retry = false for forbidden exception.
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-06-25 23:34:41 +02:00
Daniel Kesselberg 8aa99aa8ff Update presign method to match with interface again.
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-06-25 22:56:28 +02:00
Robin Appelman 35d75dbb38 relax permissions mask check for detecting part file rename
with files drop uploads the original file name isn't always used for the '.ocTransferId' source path

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-24 13:06:14 +02:00
Christoph Wurst 9d392891be Disable Client-Side Monitoring on AWS storage
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-06-15 09:13:08 +02:00
Robin Appelman e9b5514186 prevent the root storage from accidentally scanning user folders
while this scan *should* never be triggered, it's good to have some failsafe to ensure
that the users home contents don't end up getting scanned in the root storage

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-11 15:41:33 +02:00
Roeland Jago Douma 8bee8e1317 Merge pull request #21317 from FlorentCoppint/s3_upload_part_size
Upload part size as S3 parameter instead of constant value
2020-06-09 19:28:58 +02:00
Florent 3594ba6971 Upload part size as S3 parameter instead of constant value
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
2020-06-09 09:18:42 +02:00
Robin Appelman e75797ad67 increase log level for storage self-test
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-08 18:12:49 +02:00
Roeland Jago Douma 6aa6ab3e02 Add lazy events for the Node API
Right now if you want to get events via the Node API you have to have a
real instance of the Root. Which in turns sets up the whole FS.

We should make sure this is done lazy. Else enabling the preview
generator for example makes you setup the whole FS on each and every
authenticated call.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-05-28 12:35:45 +02:00
Roeland Jago Douma 6b26744787 Clear the statscache before fetching the metadata
Else if a lot of writes happen. It might happen that an old stat result
is used. Resulting in a wrong file size for the file. For example the
text app when a lot of people edit at the same time.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-05-22 09:21:53 +02:00
Robin Appelman f7153a3d0c generalize the "lazy folder" concept
makes it easy to use lazy intialized folder in other places

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-05-11 15:36:13 +02:00
Roeland Jago Douma d5850eb28f Merge pull request #18955 from adrb/swift_upload_large_objects
Large Object support for OpenStack Swift
2020-05-06 13:03:24 +02:00
Robin Appelman 4e6d3d1880 Fix getDirectoryContent implementation for Jail wrapper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-30 15:27:09 +02:00
Roeland Jago Douma f6b435d0d9 Run the cs fixer for green ci
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-04-30 10:28:44 +02:00
Roeland Jago Douma a1c1b354fc Merge pull request #18883 from adrb/optimize_swift_14116
Optimize Openstack Swift files download
2020-04-30 09:06:19 +02:00
Christoph Wurst cb057829f7 Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Adrian Brzezinski e107519295 Large Object support for OpenStack Swift.
Until now, you wouldn't be able to create
objects larger that 5GB.

It's somewhat related with pull #18883

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
2020-04-27 14:23:59 +02:00
Adrian Brzezinski 354c0996d8 It's a fix for issue #14116,
Improves efficiency when downloading files from Swift storage.
Before, files were downloaded and then pushed back to user.
That behaevior causes all kinds of performance problems.

Now, files are streamed directly to user.

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
2020-04-27 14:16:50 +02:00
Robin Appelman 436908ee3f filter out blacklisted files in getDirectoryContent
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-23 13:34:55 +02:00
Robin Appelman 9735b5d61f Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Roeland Jago Douma 79cf13e349 Merge pull request #16946 from exussum12/patch-1
Increase max read
2020-04-16 10:37:10 +02:00
Scott Dutton 65e134c13a Update View.php 2020-04-15 19:10:06 +01:00
Morris Jobke 7fe8477830 Run cs:fix and fix HashWrapper code
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-04-15 14:08:56 +02:00
Morris Jobke d8557d0d8a Merge pull request #19540 from jlutran/fixKeystoneV3Auth
Fix issues with Keystone auth v3 in files_external app
2020-04-15 14:05:32 +02:00
Christoph Wurst 301327135c Merge pull request #20361 from nextcloud/write-stream-close-on-exception
Close the streams in `writeStream` even when there is an exception
2020-04-15 10:48:24 +02:00
Julien Lutran 3e849daf11 Fix issues with Keystone auth v3 in files_external app
Signed-off-by: Julien Lutran <julien.lutran@corp.ovh.com>
2020-04-15 10:33:08 +02:00
Robin Appelman c2677682c4 Return hashes of uploaded content for dav uploads
hashes are set in "X-Hash-MD5", "X-Hash-SHA1" and "X-Hash-SHA256" headers.

these headers are set for file uploads and the MOVE request at the end of a multipart upload.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-13 12:57:27 +02:00
Scott Dutton 75b99e85ff Increase max read
8kb is very low, especially given this will be local files

Signed-off-by: Scott Dutton <scott@exussum.co.uk>
2020-04-12 13:01:29 +01:00
Christoph Wurst 734c62bee0 Format code according to PSR2
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:56:50 +02:00
Christoph Wurst 28f8eb5dba Add visibility to all constants
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:54:27 +02:00
Christoph Wurst 1584c9ae9c Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst a7c8d26d31 Add visibility to all properties and move static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:48:31 +02:00
Christoph Wurst caff1023ea Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst 14c996d982 Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Roeland Jago Douma fb64fda4d8 Merge pull request #20385 from nextcloud/skeeable-http-harden
harden seekable http stream a bit against failures
2020-04-10 08:34:52 +02:00
Christoph Wurst 44577e4345 Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00
Christoph Wurst 42625a46be Remove spaces after method or function call
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:05:56 +02:00
Robin Appelman 60de74ac40 harden seekable http stream a bit against failures
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-09 14:37:01 +02:00
Christoph Wurst afbd9c4e6e Unify function spacing to PSR2 recommendation
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 13:54:22 +02:00
Christoph Wurst 41b5e5923a Use exactly one empty line after the namespace declaration
For PSR2

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:48:10 +02:00
Robin Appelman ccbf3059ba Close the streams in writeStream even when there is an exception
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-08 16:40:56 +02:00