averello/nextcloud-server-mirror
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-02-27 18:37:17 +01:00
Code Issues Packages Projects Releases Wiki Activity
84,313 Commits 689 Branches 1,186 Tags
fixup
Commit Graph

321 Commits

Author SHA1 Message Date
Robin Appelman
f525ae0ff7 feat: don't gate perBucket object store configuration behind multibucket 
a setup can have multiple bucket without having `multibucket` enabled trough things like per-groupfolder buckets

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2026-01-05 15:50:23 +00:00
Andy Scherzinger
264c982cc0 Merge pull request #56573 from nextcloud/backport/56515/stable32 
[stable32] Encode s3 metadata as base64 if needed
 2025-12-04 08:09:41 +01:00
Daniel Kesselberg
9af9395d5c fix(s3): make data integrity protections opt-in 
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2025-12-03 19:52:17 +01:00
nfebe
3e2582c4e6 feat(objectstore): add configurable S3 retry attempts 
Add retriesMaxAttempts parameter to S3 objectstore configuration
to allow customization of AWS SDK retry behavior for handling
unreliable network conditions or proxy issues.

Defaults to 5 retries (AWS SDK default) if not specified.

Signed-off-by: nfebe <fenn25.fn@gmail.com>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2025-12-03 12:48:52 +01:00
provokateurin
57734fbd01 feat(ObjectStore): Allow overriding arguments per bucket 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2025-12-01 16:44:55 +00:00
Andy Scherzinger
8c09cdd07d Merge pull request #56766 from nextcloud/backport/52749/stable32 
[stable32] feat(CertificateManager): Add option to specify the default certificates bundle path
 2025-12-01 14:54:10 +01:00
provokateurin
6b2c19df98 feat(CertificateManager): Add option to specify the default certificates bundle path 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2025-12-01 11:55:11 +01:00
Kent Delante
e8802a57f4 feat: emit an event when an S3 bucket is created 
Signed-off-by: Kent Delante <kent.delante@proton.me>
 2025-11-26 08:18:09 +08:00
Robin Appelman
d2b176fee8 fix: encode s3 metadata as base64 if needed 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-11-21 16:18:22 +00:00
Fiehe Christoph
91434f1893 fix(s3): expose request_checksum_calculation and response_checksum_validation 
Fix https://github.com/nextcloud/server/issues/56077

This commit makes the configuration settings 'request_checksum_calculation' and 'response_checksum_validation' of the S3Client from the AWS SDK for PHP configurable.

Signed-off-by: Fiehe Christoph <c.fiehe@eurodata.de>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2025-11-03 16:07:08 +01:00
Louis Chmn
2ad19b99f9 fix(S3): Only append streams if non-seekable 
Later, when we rewind the stream in `writeMultiPart` during retry, both streams were rewinded, so the resulting stream was bigger than expected.

Inspired by https://github.com/aws/aws-sdk-php/blob/master/src/S3/ObjectUploader.php#L136-L146

Signed-off-by: Louis Chmn <louis@chmn.me>
 2025-10-10 12:43:32 +00:00
yemkareems
8633dd58b1 fix: small typo fix 
Signed-off-by: yemkareems <yemkareems@gmail.com>
 2025-09-29 12:17:22 +00:00
Robin Appelman
a8ee28ba62 fix: ensure all object store configuration have distict bucket names 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-09-08 13:23:47 +00:00
Robin Appelman
02f4a82088 fix: make bucket mapper work with new multi-object-store config 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-28 17:51:25 +02:00
Robin Appelman
b3c53c7436 feat: allow object store configuration aliases for easier migrations 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-14 13:06:51 +02:00
Robin Appelman
2d4bba7b0c feat: add command to get user objectstore config mappings 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-14 13:06:51 +02:00
Robin Appelman
385dd36ff8 feat: allow configuring multiple objectstore configurations 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-14 13:06:51 +02:00
Robin Appelman
97efc95efc fix: better object store write error propagation 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-05 19:17:58 +02:00
Robin Appelman
83b8a390cd fix: always do stream counting for object store upload 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-05 19:17:58 +02:00
Robin Appelman
398b106f0c fix: validate written size for s3 multipart uploads 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-05 19:17:58 +02:00
Ferdinand Thiessen
5981b7eb51 chore: apply new CSFixer rules 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>

# Conflicts:
#	apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
 2025-07-01 16:26:50 +02:00
Robin Appelman
2e1c316582 fix: remove double cache write in ObjectStoreStorage::touch 
the file_put_contents already puts all of those values into the cache

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-06-18 22:22:51 +02:00
Kent Delante
ea98e4b3f5 Merge pull request #53419 from nextcloud/leftybournes/fix/object_storage_large_uploads 
fix(s3): retry failed multipart uploads with decreased concurrency
 2025-06-18 21:07:06 +08:00
Daniel Calviño Sánchez
d7ae952dc8 fix(ObjectStore): Make S3 "connect_timeout" option configurable 
The hardcoded connection timeout of 5 seconds may not be enough in some
cases, so now it is got from the ObjectStore arguments in Nextcloud
configuration, falling back to 5 if not set.

The connection timeout is set in seconds, but decimal precision can be
used for subsecond accuracy (for example, 4.2 for 4200 milliseconds).

Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
 2025-06-16 20:17:50 +02:00
Kent Delante
06b5ce5859 fix(s3): retry failed multipart uploads with decreased concurrency 
Signed-off-by: Kent Delante <kent.delante@proton.me>
 2025-06-10 17:39:59 +08:00
Louis
f4cb78b905 Merge pull request #52242 from nextcloud/artonge/fix/copy_subfolders_s3 2025-06-02 11:55:54 +02:00
Robin Appelman
5b3f4cd8fe perf: set the folder size in the same query as we create it 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-28 15:52:00 +02:00
Louis Chemineau
1f8b823992 fix(S3): Use original folder size during copy 
This prevent having copied folders with a wrongly set size of 0KB.

- Fix https://github.com/nextcloud/server/issues/51916

Signed-off-by: Louis Chemineau <louis@chmn.me>
 2025-05-28 15:52:00 +02:00
Robin Appelman
7599162c7b feat: move primary object store configuration to a single place 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-28 15:32:31 +02:00
Kate
9592390070 Merge pull request #51779 from nextcloud/object-store-filename 2025-05-06 15:21:01 +02:00
Anna Larch
49baa914cf fix(objectstorage): add retry attempts to S3 connection 
Signed-off-by: Anna Larch <anna@nextcloud.com>
 2025-04-23 12:15:06 +02:00
Louis Chemineau
8fdf2a7eae fix: Transfer ownership with S3 as primary 
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:

`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`

The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.

- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.

- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)
- `oneTimeUserSetup(...)`: 687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)
- `mkdir(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)
- `moveFromStorage(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)

Delete pre-existing folder in `moveFromStorage(...)`

Signed-off-by: Louis Chemineau <louis@chmn.me>
 2025-04-01 13:53:56 +02:00
Robin Appelman
8aae332622 feat: store original storage id and path in object store metadata 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 15:10:58 +02:00
Robin Appelman
94114b99f7 feat: more generic way of passing metadata to object storage backends for new objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 14:55:18 +02:00
Robin Appelman
fcde776683 feat: add command to list objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 14:30:40 +02:00
Robin Appelman
c3bc362f48 feat: add command to get object metadata 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 14:30:40 +02:00
Hector Manuel
12a92a2e8b Merge branch 'master' into feat/support-aws-session-token 2025-03-04 18:27:54 +01:00
Benjamin Gaussorgues
9bdee76535 perf(files): faster query to fetch incomplete directories 
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
 2025-02-13 10:11:02 +01:00
Hector Valcarcel
6b4c859a41 feat(object_store): Add support for session token in AWS credentials 
- Pass session token, either null or with value, to the AWS Credentials constructor

Signed-off-by: Hector Valcarcel <hmvalcarcel@gmail.com>
 2025-02-06 16:24:19 +01:00
Richard Steinmetz
f1c025dfd3 fix(s3): treat empty sse_c_key as not set 
Co-authored-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
 2025-01-28 22:59:59 +01:00
Ferdinand Thiessen
b48ee2e924 fix: Harden files scanner for invalid null access 
Co-authored-by: Ferdinand Thiessen <opensource@fthiessen.de>
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2025-01-28 20:08:46 +01:00
Robin Appelman
5896f8c37f chore: cleanup leftover debug statement 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-09 15:40:01 +01:00
Robin Appelman
6cf66f95ce fix: get object size from stream where possible 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-05 15:16:22 +01:00
Robin Appelman
ca547bb913 fix: don't perform the extra buffering in s3 stream write when the stream size is known 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-05 14:07:40 +01:00
Robin Appelman
7da8951c57 fix: use php://temp instead of php://memory for multi-part upload buffer 
this should reduce potential memory issues if the limit is set very high

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-05 14:07:37 +01:00
Robin Appelman
df3a9e221d fix: throw correct exception type when we can't verify if an s3 bucket exists 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-11-26 16:49:55 +01:00
provokateurin
0de4843b73 refactor(Storage): Align all Storage constructors 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-10-23 13:24:18 +02:00
provokateurin
f28e74b7a8 refactor(Storage): Make all parameter types strong types 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-10-07 15:00:05 +02:00
Robin Appelman
3e12e1e789 fix: rework move into object store to better preserve fileids 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-27 14:46:08 +02:00
Robin Appelman
888d06dff9 fix: preserve fileid when moving from objectstore to non-objectstore 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-27 14:45:42 +02:00