Commit Graph

110 Commits

Author SHA1 Message Date
Côme Chilliet 1ab09ec753 chore: Apply new coding standard to all files
The diff can be checked using: git diff --ignore-all-space --ignore-blank-lines
To see only the changes not related to blank lines.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2026-06-01 13:46:39 +02:00
Ferdinand Thiessen d8e8703796 chore: add missing Override attribute to app code
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2026-04-28 21:29:28 +02:00
Kent Delante 6ce657f2ee fix(s3): ignore prefixes with repeating delimiters
Signed-off-by: Kent Delante <kent@delante.me>

Amazon's hosted S3 service allows repeating delimiters in keys
(e.g. 'path/to//file.txt' or 'path/to///file.txt') and we get
repeating directories in the filecache as a result (based on the
previous examples we get 'path/to/to/file.txt' or
'path/to/to/to/file.txt'). This ignores it and its contents for S3 external storage.
2026-04-22 16:07:45 +08:00
Stephen Cuppett cdaeed02b6 feat(objectstore): Add AWS SSE-KMS encryption support for S3 storage
Add support for Server-Side Encryption with AWS Key Management Service
(SSE-KMS) for S3 object storage. This allows Nextcloud to encrypt data
at rest in S3 using AWS-managed keys.

Key features:
- New config options: sse_kms_enabled and sse_kms_key_id
- Backward compatible with existing SSE-C (customer-provided keys)
- SSE-C takes precedence when both SSE-C and SSE-KMS are configured

Implementation details:
- Added getServerSideEncryptionParameters() method to centralize
  encryption parameter logic for both SSE-C and SSE-KMS
- Updated multipart uploads to use unified encryption parameters
- Added comprehensive PHPUnit tests for SSE-KMS scenarios
- Tested with AWS bucket and KMS keys in us-east-1 region

Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Stephen Cuppett <steve@cuppett.com>
2026-04-16 13:21:53 -04:00
Josh 958e67687b fix(s3): Fix 0 folder handling in External Storage
Signed-off-by: Josh <josh.t.richards@gmail.com>
2026-03-02 13:44:19 +01:00
Côme Chilliet e7c4dbf2cb Merge pull request #58642 from nextcloud/jtr/perf-s3-external-cache-sizing
perf(files_external): Increase S3 cache caps
2026-03-02 13:36:01 +01:00
Josh bde043d13b chore(files_external/s3): drop no-op $this->filesCache use sites
Signed-off-by: Josh <josh.t.richards@gmail.com>
2026-03-01 12:53:57 -05:00
Josh 08b465d219 refactor(files_external/S3): drop unused methods
getContentLength() and getLastModified() are dead code. Originally added in PR #11518 and made defunct in PR #29220.

Signed-off-by: Josh <josh.t.richards@gmail.com>
2026-03-01 12:45:01 -05:00
Josh 430975ce96 perf(files_external): Increase S3 metadata cache caps
Signed-off-by: Josh <josh.t.richards@gmail.com>
2026-02-28 09:08:00 -05:00
Kent Delante 8d1cb50048 fix: pass only object key to deleteObjects call
Some S3-compatible object storage hosts don't like the ETag being included in
the request and return a MalformedXML response. In the AWS API documentation,
only the object key is required so just pass that in.

Signed-off-by: Kent Delante <kent.delante@proton.me>
2026-02-27 10:54:23 +08:00
Carl Schwan 2ea1bd4cdc feat(movie-preview): Use getDirectDownloadById for generating preview
Allow to speed-up considerably the creation of previews for movies
stored on S3.

Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-01-02 17:13:41 +01:00
invario ffe91b48dc feat(previews): allow ffmpeg to connect direct for AWS S3 buckets
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2026-01-02 10:48:39 +01:00
Kent Delante b57db10814 fix: pass only necessary information when deleting mulitple objects
DeleteObjects currently fails when the request includes all the
information returned by ListObjects. Send only the necessary
information in the request. Note: 'Size' and 'DateModified' is now
only supported by directory buckets.

Signed-off-by: Kent Delante <kent.delante@proton.me>
2025-11-18 12:07:53 +08:00
Ferdinand Thiessen 5981b7eb51 chore: apply new CSFixer rules
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>

# Conflicts:
#	apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
2025-07-01 16:26:50 +02:00
Côme Chilliet 64863c9d46 chore: Apply new rector configuration to apps folder
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2025-02-13 11:46:42 +01:00
Christian Becker c25c5bbc00 feat(S3): add SSE-C support in S3 External Storage
Just need to add the parameter and fix two calls. All other logic is already implemented and implicitly called through the S3ConnectionTrait

fixes #33283

Signed-off-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2025-01-28 22:57:59 +01:00
provokateurin 0de4843b73 refactor(Storage): Align all Storage constructors
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-23 13:24:18 +02:00
Côme Chilliet 1580c8612b chore(apps): Apply new rector configuration to autouse classes
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-10-15 10:40:25 +02:00
provokateurin 14e6734547 refactor(files_external): Add Storage parameter strong types
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-07 15:00:07 +02:00
provokateurin 1302b48ca2 fix(files_external): Fix all IStorage return types
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-26 17:29:48 +02:00
Côme Chilliet 1a4978c4ea chore: Apply rector configuration to apps folder
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-09-20 17:51:00 +02:00
Daniel Kesselberg af6de04e9e style: update codestyle for coding-standard 1.2.3
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-08-25 19:34:58 +02:00
Robin Appelman c2b1f079b1 feat: add a specialized writeStream implementation for s3 external storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-07-24 17:37:20 +02:00
Andy Scherzinger c1555fc33e chore: Add SPDX header
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
2024-06-06 18:05:37 +02:00
Joas Schilling aa5f037af7 chore: apply changes from Nextcloud coding standards 1.1.1
Signed-off-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-23 10:36:13 +01:00
Louis Chemineau e66e8bad1a Return correct status if touch failed
Signed-off-by: Louis Chemineau <louis@chmn.me>
2023-10-23 15:26:43 +02:00
Robin Appelman eab8dd3ca0 use multipart copy for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-09-20 17:28:48 +02:00
Robin Appelman 7c9edbff32 handle more exceptions in AmazonS3::fopen
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-07-13 15:37:42 +02:00
James Guo bb173e092b Update AmazonS3.php
Signed-off-by: James Guo <i@ze3kr.com>
2023-04-06 14:34:08 +08:00
François Ménabé 710f3fd405 Support of s3 storage class in External Storage application
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
2023-01-12 14:59:13 +01:00
Robin Appelman ed8fedc685 fix s3 directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-15 17:17:59 +01:00
Carl Schwan a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +02:00
Carl Schwan 732badf552 Additional type hinting found by psalm
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 12:28:34 +02:00
Carl Schwan cf1381a6f9 Port away from deprecated clearBucket call
This was removed a long time ago in the library that we use, but never
detected since this was inside a try catch block...

Remove the call for now and we should investigate newer api in the
future.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:49 +02:00
Carl Schwan 1d90f84141 Port to LoggerInterface
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:28 +02:00
Matthieu Gallien 54c05bcdb9 handle AccessDenied error when checking if external s3 support versions
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2022-07-28 09:20:12 +00:00
Carl Schwan d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +02:00
Carl Schwan ec6b83cc18 Add stricter psalm type for CappedMemoryCache
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-12 15:06:18 +02:00
Carl Schwan f945c0cbc6 Add a public replacement for OC::$server->get
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-10 18:51:12 +02:00
Erik van Velzen 17f3e5e4b5 AmazonS3: allow not implemented versioning
In case the S3 implementation does not implement versioning, set it to
false. Versioning was introduced in Nexcloud in commit
09ffac5e6d

This is needed for compatibility with the Storj gateway.

Signed-off-by: Erik van Velzen <erik@evanv.nl>
2022-03-15 00:40:38 +01:00
Julius Härtl 4e57146617 Use unique combination of hostname/bucket/key for external storages
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-11-04 11:34:13 +01:00
Robin Appelman a1ca901e58 cache versioning enabled status
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 15:03:22 +02:00
Robin Appelman eb6e6e3a85 minor directory detect improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 14:58:40 +02:00
Robin Appelman 247e12da96 always set Key field in headObject
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:43:50 +02:00
Robin Appelman 4bd08af2ad more reliable hasUpdated for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:02:57 +02:00
Robin Appelman 34637697e1 remove old migration method
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:19 +02:00
Robin Appelman d3bd0b5a1b optimize filetype for s3 directories a bit
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:16 +02:00
Robin Appelman 294af4275c more reliable directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:09 +02:00
Robin Appelman 5e3c8b3af2 doesDirectoryExist fixes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:06 +02:00
Robin Appelman 09ffac5e6d s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +02:00