feat: Add automated bookmark backup feature (#2182)

* feat: Add automated bookmark backup system

Implements a comprehensive automated backup feature for user bookmarks with the following capabilities:

Database Schema:
- Add backupSettings table to store user backup preferences (enabled, frequency, retention)
- Add backups table to track backup records with status and metadata
- Add BACKUP asset type for storing compressed backup files
- Add migration 0066_add_backup_tables.sql

Background Workers:
- Implement BackupSchedulingWorker cron job (runs daily at midnight UTC)
- Create BackupWorker to process individual backup jobs
- Deterministic scheduling spreads backup jobs across 24 hours based on user ID hash
- Support for daily and weekly backup frequencies
- Automated retention cleanup to delete old backups based on user settings

Export & Compression:
- Reuse existing export functionality for bookmark data
- Compress exports using Node.js built-in zlib (gzip level 9)
- Store compressed backups as assets with proper metadata
- Track backup size and bookmark count for statistics

tRPC API:
- backups.getSettings - Retrieve user backup configuration
- backups.updateSettings - Update backup preferences
- backups.list - List all user backups with metadata
- backups.get - Get specific backup details
- backups.delete - Delete a backup
- backups.download - Download backup file (base64 encoded)
- backups.triggerBackup - Manually trigger backup creation

UI Components:
- BackupSettings component with configuration form
- Enable/disable automatic backups toggle
- Frequency selection (daily/weekly)
- Retention period configuration (1-365 days)
- Backup list table with download and delete actions
- Manual backup trigger button
- Display backup stats (size, bookmark count, status)
- Added backups page to settings navigation

Technical Details:
- Uses Restate queue system for distributed job processing
- Implements idempotency keys to prevent duplicate backups
- Background worker concurrency: 2 jobs at a time
- 10-minute timeout for large backup exports
- Proper error handling and logging throughout
- Type-safe implementation with Zod schemas

* refactor: simplify backup settings and asset handling

- Move backup settings from separate table to user table columns
- Update BackupSettings model to use static methods with users table
- Remove download mutation in favor of direct asset links
- Implement proper quota checks using QuotaService.checkStorageQuota
- Update UI to use new property names and direct asset downloads
- Update shared types to match new schema

Key changes:
- backupSettingsTable removed, settings now in users table
- Backup downloads use direct /api/assets/{id} links
- Quota properly validated before creating backup assets
- Cleaner separation of concerns in tRPC models

* migration

* use zip instead of gzip

* fix drizzle

* fix settings

* streaming json

* remove more dead code

* add e2e tests

* return backup

* poll for backups

* more fixes

* more fixes

* fix test

* fix UI

* fix delete asset

* fix ui

* redirect for backup download

* cleanups

* fix idempotency

* fix tests

* add ratelimit

* add error handling for background backups

* i18n

* model changes

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Mohamed Bassem
2025-11-29 14:53:31 +00:00
committed by GitHub
parent e67c33e466
commit 86a4b39665
32 changed files with 5698 additions and 9 deletions

View File

@@ -0,0 +1,17 @@
"use client";
import BackupSettings from "@/components/settings/BackupSettings";
import { useTranslation } from "@/lib/i18n/client";
export default function BackupsPage() {
const { t } = useTranslation();
return (
<div className="flex flex-col gap-4">
<h1 className="text-3xl font-bold">{t("settings.backups.page_title")}</h1>
<p className="text-muted-foreground">
{t("settings.backups.page_description")}
</p>
<BackupSettings />
</div>
);
}

View File

@@ -7,6 +7,7 @@ import { TFunction } from "i18next";
import {
ArrowLeft,
BarChart3,
CloudDownload,
CreditCard,
Download,
GitBranch,
@@ -67,6 +68,11 @@ const settingsSidebarItems = (
icon: <Rss size={18} />,
path: "/settings/feeds",
},
{
name: t("settings.backups.backups"),
icon: <CloudDownload size={18} />,
path: "/settings/backups",
},
{
name: t("settings.import.import_export"),
icon: <Download size={18} />,

View File

@@ -0,0 +1,423 @@
"use client";
import React from "react";
import Link from "next/link";
import { ActionButton } from "@/components/ui/action-button";
import {
Form,
FormControl,
FormDescription,
FormField,
FormItem,
FormLabel,
FormMessage,
} from "@/components/ui/form";
import { FullPageSpinner } from "@/components/ui/full-page-spinner";
import { Input } from "@/components/ui/input";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { Switch } from "@/components/ui/switch";
import { toast } from "@/components/ui/use-toast";
import { useTranslation } from "@/lib/i18n/client";
import { api } from "@/lib/trpc";
import { useUserSettings } from "@/lib/userSettings";
import { zodResolver } from "@hookform/resolvers/zod";
import {
CheckCircle,
Download,
Play,
Save,
Trash2,
XCircle,
} from "lucide-react";
import { useForm } from "react-hook-form";
import { z } from "zod";
import { useUpdateUserSettings } from "@karakeep/shared-react/hooks/users";
import { zBackupSchema } from "@karakeep/shared/types/backups";
import { zUpdateBackupSettingsSchema } from "@karakeep/shared/types/users";
import { getAssetUrl } from "@karakeep/shared/utils/assetUtils";
import ActionConfirmingDialog from "../ui/action-confirming-dialog";
import { Button } from "../ui/button";
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from "../ui/table";
import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
function BackupConfigurationForm() {
const { t } = useTranslation();
const settings = useUserSettings();
const { mutate: updateSettings, isPending: isUpdating } =
useUpdateUserSettings({
onSuccess: () => {
toast({
description: t("settings.info.user_settings.user_settings_updated"),
});
},
onError: () => {
toast({
description: t("common.something_went_wrong"),
variant: "destructive",
});
},
});
const form = useForm<z.infer<typeof zUpdateBackupSettingsSchema>>({
resolver: zodResolver(zUpdateBackupSettingsSchema),
values: settings
? {
backupsEnabled: settings.backupsEnabled,
backupsFrequency: settings.backupsFrequency,
backupsRetentionDays: settings.backupsRetentionDays,
}
: undefined,
});
return (
<div className="rounded-md border bg-background p-4">
<h3 className="mb-4 text-lg font-medium">
{t("settings.backups.configuration.title")}
</h3>
<Form {...form}>
<form
className="space-y-4"
onSubmit={form.handleSubmit((value) => {
updateSettings(value);
})}
>
<FormField
control={form.control}
name="backupsEnabled"
render={({ field }) => (
<FormItem className="flex flex-row items-center justify-between rounded-lg border p-3">
<div className="space-y-0.5">
<FormLabel>
{t(
"settings.backups.configuration.enable_automatic_backups",
)}
</FormLabel>
<FormDescription>
{t(
"settings.backups.configuration.enable_automatic_backups_description",
)}
</FormDescription>
</div>
<FormControl>
<Switch
checked={field.value}
onCheckedChange={field.onChange}
/>
</FormControl>
</FormItem>
)}
/>
<FormField
control={form.control}
name="backupsFrequency"
render={({ field }) => (
<FormItem>
<FormLabel>
{t("settings.backups.configuration.backup_frequency")}
</FormLabel>
<FormControl>
<Select
onValueChange={field.onChange}
defaultValue={field.value}
{...field}
>
<SelectTrigger>
<SelectValue
placeholder={t(
"settings.backups.configuration.select_frequency",
)}
/>
</SelectTrigger>
<SelectContent>
<SelectItem value="daily">
{t("settings.backups.configuration.frequency.daily")}
</SelectItem>
<SelectItem value="weekly">
{t("settings.backups.configuration.frequency.weekly")}
</SelectItem>
</SelectContent>
</Select>
</FormControl>
<FormDescription>
{t(
"settings.backups.configuration.backup_frequency_description",
)}
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="backupsRetentionDays"
render={({ field }) => (
<FormItem>
<FormLabel>
{t("settings.backups.configuration.retention_period")}
</FormLabel>
<FormControl>
<Input
type="number"
min={1}
max={365}
{...field}
onChange={(e) => field.onChange(parseInt(e.target.value))}
/>
</FormControl>
<FormDescription>
{t(
"settings.backups.configuration.retention_period_description",
)}
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<ActionButton
type="submit"
loading={isUpdating}
className="items-center"
>
<Save className="mr-2 size-4" />
{t("settings.backups.configuration.save_settings")}
</ActionButton>
</form>
</Form>
</div>
);
}
function BackupRow({ backup }: { backup: z.infer<typeof zBackupSchema> }) {
const { t } = useTranslation();
const apiUtils = api.useUtils();
const { mutate: deleteBackup, isPending: isDeleting } =
api.backups.delete.useMutation({
onSuccess: () => {
toast({
description: t("settings.backups.toasts.backup_deleted"),
});
apiUtils.backups.list.invalidate();
},
onError: (error) => {
toast({
description: `Error: ${error.message}`,
variant: "destructive",
});
},
});
const formatSize = (bytes: number) => {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(2)} KB`;
return `${(bytes / (1024 * 1024)).toFixed(2)} MB`;
};
return (
<TableRow>
<TableCell>{backup.createdAt.toLocaleString()}</TableCell>
<TableCell>
{backup.status === "pending"
? "-"
: backup.bookmarkCount.toLocaleString()}
</TableCell>
<TableCell>
{backup.status === "pending" ? "-" : formatSize(backup.size)}
</TableCell>
<TableCell>
{backup.status === "success" ? (
<span
title={t("settings.backups.list.status.success")}
className="flex items-center gap-1"
>
<CheckCircle className="size-4 text-green-600" />
{t("settings.backups.list.status.success")}
</span>
) : backup.status === "failure" ? (
<Tooltip>
<TooltipTrigger asChild>
<span
title={
backup.errorMessage ||
t("settings.backups.list.status.failed")
}
className="flex items-center gap-1"
>
<XCircle className="size-4 text-red-600" />
{t("settings.backups.list.status.failed")}
</span>
</TooltipTrigger>
<TooltipContent>{backup.errorMessage}</TooltipContent>
</Tooltip>
) : (
<span className="flex items-center gap-1">
<div className="size-4 animate-spin rounded-full border-2 border-gray-300 border-t-gray-600" />
{t("settings.backups.list.status.pending")}
</span>
)}
</TableCell>
<TableCell className="flex items-center gap-2">
{backup.assetId && (
<Tooltip>
<TooltipTrigger asChild>
<Button
asChild
variant="ghost"
className="items-center"
disabled={backup.status !== "success"}
>
<Link
href={getAssetUrl(backup.assetId)}
download
prefetch={false}
className={
backup.status !== "success"
? "pointer-events-none opacity-50"
: ""
}
>
<Download className="size-4" />
</Link>
</Button>
</TooltipTrigger>
<TooltipContent>
{t("settings.backups.list.actions.download_backup")}
</TooltipContent>
</Tooltip>
)}
<ActionConfirmingDialog
title={t("settings.backups.dialogs.delete_backup_title")}
description={t("settings.backups.dialogs.delete_backup_description")}
actionButton={() => (
<ActionButton
loading={isDeleting}
variant="destructive"
onClick={() => deleteBackup({ backupId: backup.id })}
className="items-center"
type="button"
>
<Trash2 className="mr-2 size-4" />
{t("settings.backups.list.actions.delete_backup")}
</ActionButton>
)}
>
<Button variant="ghost" disabled={isDeleting}>
<Trash2 className="size-4" />
</Button>
</ActionConfirmingDialog>
</TableCell>
</TableRow>
);
}
function BackupsList() {
const { t } = useTranslation();
const apiUtils = api.useUtils();
const { data: backups, isLoading } = api.backups.list.useQuery(undefined, {
refetchInterval: (query) => {
const data = query.state.data;
// Poll every 3 seconds if there's a pending backup, otherwise don't poll
return data?.backups.some((backup) => backup.status === "pending")
? 3000
: false;
},
});
const { mutate: triggerBackup, isPending: isTriggering } =
api.backups.triggerBackup.useMutation({
onSuccess: () => {
toast({
description: t("settings.backups.toasts.backup_queued"),
});
apiUtils.backups.list.invalidate();
},
onError: (error) => {
toast({
description: `Error: ${error.message}`,
variant: "destructive",
});
},
});
return (
<div className="rounded-md border bg-background p-4">
<div className="flex flex-col gap-2">
<div className="flex items-center justify-between">
<span className="text-lg font-medium">
{t("settings.backups.list.title")}
</span>
<ActionButton
onClick={() => triggerBackup()}
loading={isTriggering}
variant="default"
className="items-center"
>
<Play className="mr-2 size-4" />
{t("settings.backups.list.create_backup_now")}
</ActionButton>
</div>
{isLoading && <FullPageSpinner />}
{backups && backups.backups.length === 0 && (
<p className="rounded-md bg-muted p-2 text-sm text-muted-foreground">
{t("settings.backups.list.no_backups")}
</p>
)}
{backups && backups.backups.length > 0 && (
<Table>
<TableHeader>
<TableRow>
<TableHead>
{t("settings.backups.list.table.created_at")}
</TableHead>
<TableHead>
{t("settings.backups.list.table.bookmarks")}
</TableHead>
<TableHead>{t("settings.backups.list.table.size")}</TableHead>
<TableHead>{t("settings.backups.list.table.status")}</TableHead>
<TableHead>
{t("settings.backups.list.table.actions")}
</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{backups.backups.map((backup) => (
<BackupRow key={backup.id} backup={backup} />
))}
</TableBody>
</Table>
)}
</div>
</div>
);
}
export default function BackupSettings() {
return (
<div className="space-y-6">
<BackupConfigurationForm />
<BackupsList />
</div>
);
}

View File

@@ -359,6 +359,55 @@
"delete_dialog_title": "Delete Import Session",
"delete_dialog_description": "Are you sure you want to delete \"{{name}}\"? This action cannot be undone. The bookmarks themselves will not be deleted.",
"delete_session": "Delete Session"
},
"backups": {
"backups": "Backups",
"page_title": "Backups",
"page_description": "Automatically create and manage backups of your bookmarks. Backups are compressed and stored securely.",
"configuration": {
"title": "Backup Configuration",
"enable_automatic_backups": "Enable Automatic Backups",
"enable_automatic_backups_description": "Automatically create backups of your bookmarks",
"backup_frequency": "Backup Frequency",
"backup_frequency_description": "How often backups should be created",
"retention_period": "Retention Period (days)",
"retention_period_description": "How many days to keep backups before deleting them",
"frequency": {
"daily": "Daily",
"weekly": "Weekly"
},
"select_frequency": "Select frequency",
"save_settings": "Save Settings"
},
"list": {
"title": "Your Backups",
"create_backup_now": "Create Backup Now",
"no_backups": "You don't have any backups yet. Enable automatic backups or create one manually.",
"table": {
"created_at": "Created At",
"bookmarks": "Bookmarks",
"size": "Size",
"status": "Status",
"actions": "Actions"
},
"status": {
"success": "Success",
"failed": "Failed",
"pending": "Pending"
},
"actions": {
"download_backup": "Download Backup",
"delete_backup": "Delete Backup"
}
},
"dialogs": {
"delete_backup_title": "Delete Backup?",
"delete_backup_description": "Are you sure you want to delete this backup? This action cannot be undone."
},
"toasts": {
"backup_queued": "Backup job has been queued! It will be processed shortly.",
"backup_deleted": "Backup has been deleted!"
}
}
},
"admin": {

View File

@@ -10,6 +10,9 @@ export const UserSettingsContext = createContext<ZUserSettings>({
bookmarkClickAction: "open_original_link",
archiveDisplayBehaviour: "show",
timezone: "UTC",
backupsEnabled: false,
backupsFrequency: "daily",
backupsRetentionDays: 7,
});
export function UserSettingsContextProvider({

View File

@@ -13,6 +13,7 @@ import logger from "@karakeep/shared/logger";
import { shutdownPromise } from "./exit";
import { AdminMaintenanceWorker } from "./workers/adminMaintenanceWorker";
import { AssetPreprocessingWorker } from "./workers/assetPreprocessingWorker";
import { BackupSchedulingWorker, BackupWorker } from "./workers/backupWorker";
import { CrawlerWorker } from "./workers/crawlerWorker";
import { FeedRefreshingWorker, FeedWorker } from "./workers/feedWorker";
import { OpenAiWorker } from "./workers/inference/inferenceWorker";
@@ -31,6 +32,7 @@ const workerBuilders = {
assetPreprocessing: () => AssetPreprocessingWorker.build(),
webhook: () => WebhookWorker.build(),
ruleEngine: () => RuleEngineWorker.build(),
backup: () => BackupWorker.build(),
} as const;
type WorkerName = keyof typeof workerBuilders;
@@ -69,6 +71,10 @@ async function main() {
FeedRefreshingWorker.start();
}
if (workers.some((w) => w.name === "backup")) {
BackupSchedulingWorker.start();
}
await Promise.any([
Promise.all([
...workers.map(({ worker }) => worker.run()),
@@ -84,6 +90,9 @@ async function main() {
if (workers.some((w) => w.name === "feed")) {
FeedRefreshingWorker.stop();
}
if (workers.some((w) => w.name === "backup")) {
BackupSchedulingWorker.stop();
}
for (const { worker } of workers) {
worker.stop();
}

View File

@@ -15,6 +15,7 @@
"@karakeep/tsconfig": "workspace:^0.1.0",
"@mozilla/readability": "^0.6.0",
"@tsconfig/node22": "^22.0.0",
"archiver": "^7.0.1",
"async-mutex": "^0.4.1",
"dompurify": "^3.2.4",
"dotenv": "^16.4.1",
@@ -57,6 +58,7 @@
},
"devDependencies": {
"@karakeep/prettier-config": "workspace:^0.1.0",
"@types/archiver": "^7.0.0",
"@types/jsdom": "^21.1.6",
"@types/node-cron": "^3.0.11",
"tsdown": "^0.12.9"

View File

@@ -0,0 +1,431 @@
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { stat, unlink } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createId } from "@paralleldrive/cuid2";
import archiver from "archiver";
import { eq } from "drizzle-orm";
import { workerStatsCounter } from "metrics";
import cron from "node-cron";
import type { ZBackupRequest } from "@karakeep/shared-server";
import { db } from "@karakeep/db";
import { assets, AssetTypes, users } from "@karakeep/db/schema";
import { BackupQueue, QuotaService } from "@karakeep/shared-server";
import { saveAssetFromFile } from "@karakeep/shared/assetdb";
import { toExportFormat } from "@karakeep/shared/import-export";
import logger from "@karakeep/shared/logger";
import { DequeuedJob, getQueueClient } from "@karakeep/shared/queueing";
import { AuthedContext } from "@karakeep/trpc";
import { Backup } from "@karakeep/trpc/models/backups";
import { buildImpersonatingAuthedContext } from "../trpc";
import { fetchBookmarksInBatches } from "./utils/fetchBookmarks";
// Run daily at midnight UTC
export const BackupSchedulingWorker = cron.schedule(
"0 0 * * *",
async () => {
logger.info("[backup] Scheduling daily backup jobs ...");
try {
const usersWithBackups = await db.query.users.findMany({
columns: {
id: true,
backupsFrequency: true,
},
where: eq(users.backupsEnabled, true),
});
logger.info(
`[backup] Found ${usersWithBackups.length} users with backups enabled`,
);
const now = new Date();
const currentDay = now.toISOString().split("T")[0]; // YYYY-MM-DD
for (const user of usersWithBackups) {
// Deterministically schedule backups throughout the day based on user ID
// This spreads the load across 24 hours
const hash = createHash("sha256").update(user.id).digest("hex");
const hashNum = parseInt(hash.substring(0, 8), 16);
// For daily: schedule within 24 hours
// For weekly: only schedule on the user's designated day of week
let shouldSchedule = false;
let delayMs = 0;
if (user.backupsFrequency === "daily") {
shouldSchedule = true;
// Spread across 24 hours (86400000 ms)
delayMs = hashNum % 86400000;
} else if (user.backupsFrequency === "weekly") {
// Use hash to determine day of week (0-6)
const userDayOfWeek = hashNum % 7;
const currentDayOfWeek = now.getDay();
if (userDayOfWeek === currentDayOfWeek) {
shouldSchedule = true;
// Spread across 24 hours
delayMs = hashNum % 86400000;
}
}
if (shouldSchedule) {
const idempotencyKey = `${user.id}-${currentDay}`;
await BackupQueue.enqueue(
{
userId: user.id,
},
{
delayMs,
idempotencyKey,
},
);
logger.info(
`[backup] Scheduled backup for user ${user.id} with delay ${Math.round(delayMs / 1000 / 60)} minutes`,
);
}
}
logger.info("[backup] Finished scheduling backup jobs");
} catch (error) {
logger.error(`[backup] Error scheduling backup jobs: ${error}`);
}
},
{
runOnInit: false,
scheduled: false,
},
);
export class BackupWorker {
static async build() {
logger.info("Starting backup worker ...");
const worker = (await getQueueClient())!.createRunner<ZBackupRequest>(
BackupQueue,
{
run: run,
onComplete: async (job) => {
workerStatsCounter.labels("backup", "completed").inc();
const jobId = job.id;
logger.info(`[backup][${jobId}] Completed successfully`);
},
onError: async (job) => {
workerStatsCounter.labels("backup", "failed").inc();
if (job.numRetriesLeft == 0) {
workerStatsCounter.labels("backup", "failed_permanent").inc();
}
const jobId = job.id;
logger.error(
`[backup][${jobId}] Backup job failed: ${job.error}\n${job.error?.stack}`,
);
// Mark backup as failed
if (job.data?.backupId && job.data?.userId) {
try {
const authCtx = await buildImpersonatingAuthedContext(
job.data.userId,
);
const backup = await Backup.fromId(authCtx, job.data.backupId);
await backup.update({
status: "failure",
errorMessage: job.error?.message || "Unknown error",
});
} catch (err) {
logger.error(
`[backup][${jobId}] Failed to mark backup as failed: ${err}`,
);
}
}
},
},
{
concurrency: 2, // Process 2 backups at a time
pollIntervalMs: 5000,
timeoutSecs: 600, // 10 minutes timeout for large exports
},
);
return worker;
}
}
async function run(req: DequeuedJob<ZBackupRequest>) {
const jobId = req.id;
const userId = req.data.userId;
const backupId = req.data.backupId;
logger.info(`[backup][${jobId}] Starting backup for user ${userId} ...`);
// Fetch user settings to check if backups are enabled and get retention
const user = await db.query.users.findFirst({
columns: {
id: true,
backupsRetentionDays: true,
},
where: eq(users.id, userId),
});
if (!user) {
logger.info(`[backup][${jobId}] User not found: ${userId}. Skipping.`);
return;
}
const timestamp = new Date().toISOString().replace(/[:.]/g, "-");
const tempJsonPath = join(
tmpdir(),
`karakeep-backup-${userId}-${timestamp}.json`,
);
const tempZipPath = join(
tmpdir(),
`karakeep-backup-${userId}-${timestamp}.zip`,
);
let backup: Backup | null = null;
try {
// Step 1: Stream bookmarks to JSON file
const ctx = await buildImpersonatingAuthedContext(userId);
const backupInstance = await (backupId
? Backup.fromId(ctx, backupId)
: Backup.create(ctx));
backup = backupInstance;
// Ensure backupId is attached to job data so error handler can mark failure.
req.data.backupId = backupInstance.id;
const bookmarkCount = await streamBookmarksToJsonFile(
ctx,
tempJsonPath,
jobId,
);
logger.info(
`[backup][${jobId}] Streamed ${bookmarkCount} bookmarks to JSON file`,
);
// Step 2: Compress the JSON file as zip
logger.info(`[backup][${jobId}] Compressing JSON file as zip ...`);
await createZipArchiveFromFile(tempJsonPath, timestamp, tempZipPath);
const fileStats = await stat(tempZipPath);
const compressedSize = fileStats.size;
const jsonStats = await stat(tempJsonPath);
logger.info(
`[backup][${jobId}] Compressed ${jsonStats.size} bytes to ${compressedSize} bytes`,
);
// Step 3: Check quota and store as asset
const quotaApproval = await QuotaService.checkStorageQuota(
db,
userId,
compressedSize,
);
const assetId = createId();
const fileName = `karakeep-backup-${timestamp}.zip`;
// Step 4: Create asset record
await db.insert(assets).values({
id: assetId,
assetType: AssetTypes.BACKUP,
size: compressedSize,
contentType: "application/zip",
fileName: fileName,
bookmarkId: null,
userId: userId,
});
await saveAssetFromFile({
userId,
assetId,
assetPath: tempZipPath,
metadata: {
contentType: "application/zip",
fileName,
},
quotaApproved: quotaApproval,
});
// Step 5: Update backup record
await backupInstance.update({
size: compressedSize,
bookmarkCount: bookmarkCount,
status: "success",
assetId,
});
logger.info(
`[backup][${jobId}] Successfully created backup for user ${userId} with ${bookmarkCount} bookmarks (${compressedSize} bytes)`,
);
// Step 6: Clean up old backups based on retention
await cleanupOldBackups(ctx, user.backupsRetentionDays, jobId);
} catch (error) {
if (backup) {
try {
await backup.update({
status: "failure",
errorMessage:
error instanceof Error ? error.message : "Unknown error",
});
} catch (updateError) {
logger.error(
`[backup][${jobId}] Failed to mark backup ${backup.id} as failed: ${updateError}`,
);
}
}
throw error;
} finally {
// Final cleanup of temporary files
try {
await unlink(tempJsonPath);
} catch {
// Ignore errors during cleanup
}
try {
await unlink(tempZipPath);
} catch {
// Ignore errors during cleanup
}
}
}
/**
* Streams bookmarks to a JSON file in batches to avoid loading everything into memory
* @returns The total number of bookmarks written
*/
async function streamBookmarksToJsonFile(
ctx: AuthedContext,
outputPath: string,
jobId: string,
): Promise<number> {
return new Promise((resolve, reject) => {
const writeStream = createWriteStream(outputPath, { encoding: "utf-8" });
let bookmarkCount = 0;
let isFirst = true;
writeStream.on("error", reject);
// Start JSON structure
writeStream.write('{"bookmarks":[');
(async () => {
try {
for await (const batch of fetchBookmarksInBatches(ctx, 1000)) {
for (const bookmark of batch) {
const exported = toExportFormat(bookmark);
if (exported.content !== null) {
// Add comma separator for all items except the first
if (!isFirst) {
writeStream.write(",");
}
writeStream.write(JSON.stringify(exported));
isFirst = false;
bookmarkCount++;
}
}
// Log progress every batch
if (bookmarkCount % 1000 === 0) {
logger.info(
`[backup][${jobId}] Streamed ${bookmarkCount} bookmarks so far...`,
);
}
}
// Close JSON structure
writeStream.write("]}");
writeStream.end();
writeStream.on("finish", () => {
resolve(bookmarkCount);
});
} catch (error) {
writeStream.destroy();
reject(error);
}
})();
});
}
/**
* Creates a zip archive from a JSON file (streaming from disk instead of memory)
*/
async function createZipArchiveFromFile(
jsonFilePath: string,
timestamp: string,
outputPath: string,
): Promise<void> {
return new Promise((resolve, reject) => {
const archive = archiver("zip", {
zlib: { level: 9 }, // Maximum compression
});
const output = createWriteStream(outputPath);
output.on("close", () => {
resolve();
});
output.on("error", reject);
archive.on("error", reject);
// Pipe archive data to the file
archive.pipe(output);
// Add the JSON file to the zip (streaming from disk)
const jsonFileName = `karakeep-backup-${timestamp}.json`;
archive.file(jsonFilePath, { name: jsonFileName });
archive.finalize();
});
}
/**
* Cleans up old backups based on retention policy
*/
async function cleanupOldBackups(
ctx: AuthedContext,
retentionDays: number,
jobId: string,
) {
try {
logger.info(
`[backup][${jobId}] Cleaning up backups older than ${retentionDays} days for user ${ctx.user.id} ...`,
);
const oldBackups = await Backup.findOldBackups(ctx, retentionDays);
if (oldBackups.length === 0) {
return;
}
logger.info(
`[backup][${jobId}] Found ${oldBackups.length} old backups to delete for user ${ctx.user.id}`,
);
// Delete each backup using the model's delete method
for (const backup of oldBackups) {
try {
await backup.delete();
logger.info(
`[backup][${jobId}] Deleted backup ${backup.id} for user ${ctx.user.id}`,
);
} catch (error) {
logger.warn(
`[backup][${jobId}] Failed to delete backup ${backup.id}: ${error}`,
);
}
}
logger.info(
`[backup][${jobId}] Successfully cleaned up ${oldBackups.length} old backups for user ${ctx.user.id}`,
);
} catch (error) {
logger.error(
`[backup][${jobId}] Error cleaning up old backups for user ${ctx.user.id}: ${error}`,
);
}
}

View File

@@ -0,0 +1,131 @@
import { asc, eq } from "drizzle-orm";
import type { ZBookmark } from "@karakeep/shared/types/bookmarks";
import type { ZCursor } from "@karakeep/shared/types/pagination";
import type { AuthedContext } from "@karakeep/trpc";
import { db } from "@karakeep/db";
import { bookmarks } from "@karakeep/db/schema";
import { BookmarkTypes } from "@karakeep/shared/types/bookmarks";
import { Bookmark } from "@karakeep/trpc/models/bookmarks";
/**
* Fetches all bookmarks for a user with all necessary relations for export
* @deprecated Use fetchBookmarksInBatches for memory-efficient iteration
*/
export async function fetchAllBookmarksForUser(
dbInstance: typeof db,
userId: string,
): Promise<ZBookmark[]> {
const allBookmarks = await dbInstance.query.bookmarks.findMany({
where: eq(bookmarks.userId, userId),
with: {
tagsOnBookmarks: {
with: {
tag: true,
},
},
link: true,
text: true,
asset: true,
assets: true,
},
orderBy: [asc(bookmarks.createdAt)],
});
// Transform to ZBookmark format
return allBookmarks.map((bookmark) => {
let content: ZBookmark["content"] | null = null;
switch (bookmark.type) {
case BookmarkTypes.LINK:
if (bookmark.link) {
content = {
type: BookmarkTypes.LINK,
url: bookmark.link.url,
title: bookmark.link.title || undefined,
description: bookmark.link.description || undefined,
imageUrl: bookmark.link.imageUrl || undefined,
favicon: bookmark.link.favicon || undefined,
};
}
break;
case BookmarkTypes.TEXT:
if (bookmark.text) {
content = {
type: BookmarkTypes.TEXT,
text: bookmark.text.text || "",
};
}
break;
case BookmarkTypes.ASSET:
if (bookmark.asset) {
content = {
type: BookmarkTypes.ASSET,
assetType: bookmark.asset.assetType,
assetId: bookmark.asset.assetId,
};
}
break;
}
return {
id: bookmark.id,
title: bookmark.title || null,
createdAt: bookmark.createdAt,
archived: bookmark.archived,
favourited: bookmark.favourited,
taggingStatus: bookmark.taggingStatus || "pending",
note: bookmark.note || null,
summary: bookmark.summary || null,
content,
tags: bookmark.tagsOnBookmarks.map((t) => ({
id: t.tag.id,
name: t.tag.name,
attachedBy: t.attachedBy,
})),
assets: bookmark.assets.map((a) => ({
id: a.id,
assetType: a.assetType,
})),
} as ZBookmark;
});
}
/**
* Fetches bookmarks in batches using cursor-based pagination from the Bookmark model
* This is memory-efficient for large datasets as it only loads one batch at a time
*/
export async function* fetchBookmarksInBatches(
ctx: AuthedContext,
batchSize = 1000,
): AsyncGenerator<ZBookmark[], number, undefined> {
let cursor: ZCursor | null = null;
let totalFetched = 0;
while (true) {
const result = await Bookmark.loadMulti(ctx, {
limit: batchSize,
cursor: cursor,
sortOrder: "asc",
includeContent: false, // We don't need full content for export
});
if (result.bookmarks.length === 0) {
break;
}
// Convert Bookmark instances to ZBookmark
const batch = result.bookmarks.map((b) => b.asZBookmark());
yield batch;
totalFetched += batch.length;
cursor = result.nextCursor;
// If there's no next cursor, we've reached the end
if (!cursor) {
break;
}
}
return totalFetched;
}

View File

@@ -10,6 +10,7 @@ import { Context } from "@karakeep/trpc";
import trpcAdapter from "./middlewares/trpcAdapter";
import admin from "./routes/admin";
import assets from "./routes/assets";
import backups from "./routes/backups";
import bookmarks from "./routes/bookmarks";
import health from "./routes/health";
import highlights from "./routes/highlights";
@@ -37,7 +38,8 @@ const v1 = new Hono<{
.route("/users", users)
.route("/assets", assets)
.route("/admin", admin)
.route("/rss", rss);
.route("/rss", rss)
.route("/backups", backups);
const app = new Hono<{
Variables: {

View File

@@ -0,0 +1,44 @@
import { Hono } from "hono";
import { authMiddleware } from "../middlewares/auth";
const app = new Hono()
.use(authMiddleware)
// GET /backups
.get("/", async (c) => {
const backups = await c.var.api.backups.list();
return c.json(backups, 200);
})
// POST /backups
.post("/", async (c) => {
const backup = await c.var.api.backups.triggerBackup();
return c.json(backup, 201);
})
// GET /backups/[backupId]
.get("/:backupId", async (c) => {
const backupId = c.req.param("backupId");
const backup = await c.var.api.backups.get({ backupId });
return c.json(backup, 200);
})
// GET /backups/[backupId]/download
.get("/:backupId/download", async (c) => {
const backupId = c.req.param("backupId");
const backup = await c.var.api.backups.get({ backupId });
if (!backup.assetId) {
return c.json({ error: "Backup not found" }, 404);
}
return c.redirect(`/api/assets/${backup.assetId}`);
})
// DELETE /backups/[backupId]
.delete("/:backupId", async (c) => {
const backupId = c.req.param("backupId");
await c.var.api.backups.delete({ backupId });
return c.body(null, 204);
});
export default app;

View File

@@ -0,0 +1,18 @@
CREATE TABLE `backups` (
`id` text PRIMARY KEY NOT NULL,
`userId` text NOT NULL,
`assetId` text,
`createdAt` integer NOT NULL,
`size` integer NOT NULL,
`bookmarkCount` integer NOT NULL,
`status` text DEFAULT 'pending' NOT NULL,
`errorMessage` text,
FOREIGN KEY (`userId`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade,
FOREIGN KEY (`assetId`) REFERENCES `assets`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE INDEX `backups_userId_idx` ON `backups` (`userId`);--> statement-breakpoint
CREATE INDEX `backups_createdAt_idx` ON `backups` (`createdAt`);--> statement-breakpoint
ALTER TABLE `user` ADD `backupsEnabled` integer DEFAULT false NOT NULL;--> statement-breakpoint
ALTER TABLE `user` ADD `backupsFrequency` text DEFAULT 'weekly' NOT NULL;--> statement-breakpoint
ALTER TABLE `user` ADD `backupsRetentionDays` integer DEFAULT 30 NOT NULL;

File diff suppressed because it is too large Load Diff

View File

@@ -470,6 +470,13 @@
"when": 1763854050669,
"tag": "0066_collaborative_lists_invites",
"breakpoints": true
},
{
"idx": 67,
"version": "6",
"when": 1764418020312,
"tag": "0067_add_backups_table",
"breakpoints": true
}
]
}

View File

@@ -58,6 +58,17 @@ export const users = sqliteTable("user", {
.notNull()
.default("show"),
timezone: text("timezone").default("UTC"),
// Backup Settings
backupsEnabled: integer("backupsEnabled", { mode: "boolean" })
.notNull()
.default(false),
backupsFrequency: text("backupsFrequency", {
enum: ["daily", "weekly"],
})
.notNull()
.default("weekly"),
backupsRetentionDays: integer("backupsRetentionDays").notNull().default(30),
});
export const accounts = sqliteTable(
@@ -229,6 +240,7 @@ export const enum AssetTypes {
LINK_HTML_CONTENT = "linkHtmlContent",
BOOKMARK_ASSET = "bookmarkAsset",
USER_UPLOADED = "userUploaded",
BACKUP = "backup",
UNKNOWN = "unknown",
}
@@ -248,6 +260,7 @@ export const assets = sqliteTable(
AssetTypes.LINK_HTML_CONTENT,
AssetTypes.BOOKMARK_ASSET,
AssetTypes.USER_UPLOADED,
AssetTypes.BACKUP,
AssetTypes.UNKNOWN,
],
}).notNull(),
@@ -574,6 +587,35 @@ export const rssFeedImportsTable = sqliteTable(
],
);
export const backupsTable = sqliteTable(
"backups",
{
id: text("id")
.notNull()
.primaryKey()
.$defaultFn(() => createId()),
userId: text("userId")
.notNull()
.references(() => users.id, { onDelete: "cascade" }),
assetId: text("assetId").references(() => assets.id, {
onDelete: "cascade",
}),
createdAt: createdAtField(),
size: integer("size").notNull(),
bookmarkCount: integer("bookmarkCount").notNull(),
status: text("status", {
enum: ["pending", "success", "failure"],
})
.notNull()
.default("pending"),
errorMessage: text("errorMessage"),
},
(b) => [
index("backups_userId_idx").on(b.userId),
index("backups_createdAt_idx").on(b.createdAt),
],
);
export const config = sqliteTable("config", {
key: text("key").notNull().primaryKey(),
value: text("value").notNull(),
@@ -766,6 +808,7 @@ export const userRelations = relations(users, ({ many, one }) => ({
subscription: one(subscriptions),
importSessions: many(importSessions),
listCollaborations: many(listCollaborators),
backups: many(backupsTable),
listInvitations: many(listInvitations),
}));
@@ -989,3 +1032,14 @@ export const importSessionBookmarksRelations = relations(
}),
}),
);
export const backupsRelations = relations(backupsTable, ({ one }) => ({
user: one(users, {
fields: [backupsTable.userId],
references: [users.id],
}),
asset: one(assets, {
fields: [backupsTable.assetId],
references: [assets.id],
}),
}));

View File

@@ -26,6 +26,8 @@
"devDependencies": {
"@karakeep/prettier-config": "workspace:^0.1.0",
"@karakeep/tsconfig": "workspace:^0.1.0",
"@types/adm-zip": "^0.5.7",
"adm-zip": "^0.5.16",
"vite-tsconfig-paths": "^4.3.1",
"vitest": "^3.2.4"
},

View File

@@ -0,0 +1,285 @@
import AdmZip from "adm-zip";
import { beforeEach, describe, expect, inject, it } from "vitest";
import { createKarakeepClient } from "@karakeep/sdk";
import { createTestUser } from "../../utils/api";
describe("Backups API", () => {
const port = inject("karakeepPort");
if (!port) {
throw new Error("Missing required environment variables");
}
let client: ReturnType<typeof createKarakeepClient>;
let apiKey: string;
beforeEach(async () => {
apiKey = await createTestUser();
client = createKarakeepClient({
baseUrl: `http://localhost:${port}/api/v1/`,
headers: {
"Content-Type": "application/json",
authorization: `Bearer ${apiKey}`,
},
});
});
it("should list backups", async () => {
const { data: backupsData, response } = await client.GET("/backups");
expect(response.status).toBe(200);
expect(backupsData).toBeDefined();
expect(backupsData!.backups).toBeDefined();
expect(Array.isArray(backupsData!.backups)).toBe(true);
});
it("should trigger a backup and return the backup record", async () => {
const { data: backup, response } = await client.POST("/backups");
expect(response.status).toBe(201);
expect(backup).toBeDefined();
expect(backup!.id).toBeDefined();
expect(backup!.userId).toBeDefined();
expect(backup!.assetId).toBeDefined();
expect(backup!.status).toBe("pending");
expect(backup!.size).toBe(0);
expect(backup!.bookmarkCount).toBe(0);
// Verify the backup appears in the list
const { data: backupsData } = await client.GET("/backups");
expect(backupsData).toBeDefined();
expect(backupsData!.backups).toBeDefined();
expect(backupsData!.backups.some((b) => b.id === backup!.id)).toBe(true);
});
it("should get and delete a backup", async () => {
// First trigger a backup
const { data: createdBackup } = await client.POST("/backups");
expect(createdBackup).toBeDefined();
const backupId = createdBackup!.id;
// Get the specific backup
const { data: backup, response: getResponse } = await client.GET(
"/backups/{backupId}",
{
params: {
path: {
backupId,
},
},
},
);
expect(getResponse.status).toBe(200);
expect(backup).toBeDefined();
expect(backup!.id).toBe(backupId);
expect(backup!.userId).toBeDefined();
expect(backup!.assetId).toBeDefined();
expect(backup!.status).toBe("pending");
// Delete the backup
const { response: deleteResponse } = await client.DELETE(
"/backups/{backupId}",
{
params: {
path: {
backupId,
},
},
},
);
expect(deleteResponse.status).toBe(204);
// Verify it's deleted
const { response: getDeletedResponse } = await client.GET(
"/backups/{backupId}",
{
params: {
path: {
backupId,
},
},
},
);
expect(getDeletedResponse.status).toBe(404);
});
it("should return 404 for non-existent backup", async () => {
const { response } = await client.GET("/backups/{backupId}", {
params: {
path: {
backupId: "non-existent-backup-id",
},
},
});
expect(response.status).toBe(404);
});
it("should return 404 when deleting non-existent backup", async () => {
const { response } = await client.DELETE("/backups/{backupId}", {
params: {
path: {
backupId: "non-existent-backup-id",
},
},
});
expect(response.status).toBe(404);
});
it("should handle multiple backups", async () => {
// Trigger multiple backups
const { data: backup1 } = await client.POST("/backups");
const { data: backup2 } = await client.POST("/backups");
expect(backup1).toBeDefined();
expect(backup2).toBeDefined();
expect(backup1!.id).not.toBe(backup2!.id);
// Get all backups
const { data: backupsData, response } = await client.GET("/backups");
expect(response.status).toBe(200);
expect(backupsData).toBeDefined();
expect(backupsData!.backups).toBeDefined();
expect(Array.isArray(backupsData!.backups)).toBe(true);
expect(backupsData!.backups.length).toBeGreaterThanOrEqual(2);
expect(backupsData!.backups.some((b) => b.id === backup1!.id)).toBe(true);
expect(backupsData!.backups.some((b) => b.id === backup2!.id)).toBe(true);
});
it("should validate full backup lifecycle", async () => {
// Step 1: Create some test bookmarks
const bookmarks = [];
for (let i = 0; i < 3; i++) {
const { data: bookmark } = await client.POST("/bookmarks", {
body: {
type: "text",
title: `Test Bookmark ${i + 1}`,
text: `This is test bookmark number ${i + 1}`,
},
});
expect(bookmark).toBeDefined();
bookmarks.push(bookmark!);
}
// Step 2: Trigger a backup
const { data: createdBackup, response: createResponse } =
await client.POST("/backups");
expect(createResponse.status).toBe(201);
expect(createdBackup).toBeDefined();
expect(createdBackup!.id).toBeDefined();
expect(createdBackup!.status).toBe("pending");
expect(createdBackup!.bookmarkCount).toBe(0);
expect(createdBackup!.size).toBe(0);
const backupId = createdBackup!.id;
// Step 3: Poll until backup is completed or failed
let backup;
let attempts = 0;
const maxAttempts = 60; // Wait up to 60 seconds
const pollInterval = 1000; // Poll every second
while (attempts < maxAttempts) {
const { data: currentBackup } = await client.GET("/backups/{backupId}", {
params: {
path: {
backupId,
},
},
});
backup = currentBackup;
if (backup!.status === "success" || backup!.status === "failure") {
break;
}
await new Promise((resolve) => setTimeout(resolve, pollInterval));
attempts++;
}
// Step 4: Verify backup completed successfully
expect(backup).toBeDefined();
expect(backup!.status).toBe("success");
expect(backup!.bookmarkCount).toBeGreaterThanOrEqual(3);
expect(backup!.size).toBeGreaterThan(0);
expect(backup!.errorMessage).toBeNull();
// Step 5: Download the backup
const downloadResponse = await fetch(
`http://localhost:${port}/api/v1/backups/${backupId}/download`,
{
headers: {
authorization: `Bearer ${apiKey}`,
},
},
);
expect(downloadResponse.status).toBe(200);
expect(downloadResponse.headers.get("content-type")).toContain(
"application/zip",
);
const backupBlob = await downloadResponse.blob();
expect(backupBlob.size).toBeGreaterThan(0);
expect(backupBlob.size).toBe(backup!.size);
// Step 6: Unzip and validate the backup contents
const arrayBuffer = await backupBlob.arrayBuffer();
const buffer = Buffer.from(arrayBuffer);
// Verify it's a valid ZIP file (starts with PK signature)
expect(buffer[0]).toBe(0x50); // 'P'
expect(buffer[1]).toBe(0x4b); // 'K'
// Unzip the backup file
const zip = new AdmZip(buffer);
const zipEntries = zip.getEntries();
// Should contain exactly one JSON file
expect(zipEntries.length).toBe(1);
const jsonEntry = zipEntries[0];
expect(jsonEntry.entryName).toMatch(/^karakeep-backup-.*\.json$/);
// Extract and parse the JSON content
const jsonContent = jsonEntry.getData().toString("utf8");
const backupData = JSON.parse(jsonContent);
// Validate the backup structure
expect(backupData).toBeDefined();
expect(backupData.bookmarks).toBeDefined();
expect(Array.isArray(backupData.bookmarks)).toBe(true);
expect(backupData.bookmarks.length).toBeGreaterThanOrEqual(3);
// Validate that our test bookmarks are in the backup
const backupTitles = backupData.bookmarks.map(
(b: { title: string }) => b.title,
);
expect(backupTitles).toContain("Test Bookmark 1");
expect(backupTitles).toContain("Test Bookmark 2");
expect(backupTitles).toContain("Test Bookmark 3");
// Validate bookmark structure
const firstBookmark = backupData.bookmarks[0];
expect(firstBookmark).toHaveProperty("content");
expect(firstBookmark.content).toHaveProperty("type");
// Step 7: Verify the backup appears in the list with updated status
const { data: backupsData } = await client.GET("/backups");
const listedBackup = backupsData!.backups.find((b) => b.id === backupId);
expect(listedBackup).toBeDefined();
expect(listedBackup!.status).toBe("success");
expect(listedBackup!.bookmarkCount).toBe(backup!.bookmarkCount);
expect(listedBackup!.size).toBe(backup!.size);
});
});

View File

@@ -7,6 +7,7 @@ import {
import { registry as adminRegistry } from "./lib/admin";
import { registry as assetsRegistry } from "./lib/assets";
import { registry as backupsRegistry } from "./lib/backups";
import { registry as bookmarksRegistry } from "./lib/bookmarks";
import { registry as commonRegistry } from "./lib/common";
import { registry as highlightsRegistry } from "./lib/highlights";
@@ -24,6 +25,7 @@ function getOpenApiDocumentation() {
userRegistry,
assetsRegistry,
adminRegistry,
backupsRegistry,
]);
const generator = new OpenApiGeneratorV3(registry.definitions);

View File

@@ -45,6 +45,10 @@
"type": "string",
"example": "ieidlxygmwj87oxz5hxttoc8"
},
"BackupId": {
"type": "string",
"example": "ieidlxygmwj87oxz5hxttoc8"
},
"Bookmark": {
"type": "object",
"properties": {
@@ -596,6 +600,14 @@
"required": true,
"name": "assetId",
"in": "path"
},
"BackupId": {
"schema": {
"$ref": "#/components/schemas/BackupId"
},
"required": true,
"name": "backupId",
"in": "path"
}
}
},
@@ -3703,6 +3715,341 @@
}
}
}
},
"/backups": {
"get": {
"description": "Get all backups",
"summary": "Get all backups",
"tags": [
"Backups"
],
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Object with all backups data.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"backups": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"userId": {
"type": "string"
},
"assetId": {
"type": "string",
"nullable": true
},
"createdAt": {
"type": "string"
},
"size": {
"type": "number"
},
"bookmarkCount": {
"type": "number"
},
"status": {
"type": "string",
"enum": [
"pending",
"success",
"failure"
]
},
"errorMessage": {
"type": "string",
"nullable": true
}
},
"required": [
"id",
"userId",
"assetId",
"createdAt",
"size",
"bookmarkCount",
"status"
]
}
}
},
"required": [
"backups"
]
}
}
}
}
}
},
"post": {
"description": "Trigger a new backup",
"summary": "Trigger a new backup",
"tags": [
"Backups"
],
"security": [
{
"bearerAuth": []
}
],
"responses": {
"201": {
"description": "Backup created successfully",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"userId": {
"type": "string"
},
"assetId": {
"type": "string",
"nullable": true
},
"createdAt": {
"type": "string"
},
"size": {
"type": "number"
},
"bookmarkCount": {
"type": "number"
},
"status": {
"type": "string",
"enum": [
"pending",
"success",
"failure"
]
},
"errorMessage": {
"type": "string",
"nullable": true
}
},
"required": [
"id",
"userId",
"assetId",
"createdAt",
"size",
"bookmarkCount",
"status"
]
}
}
}
}
}
}
},
"/backups/{backupId}": {
"get": {
"description": "Get backup by its id",
"summary": "Get a single backup",
"tags": [
"Backups"
],
"security": [
{
"bearerAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/BackupId"
}
],
"responses": {
"200": {
"description": "Object with backup data.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"userId": {
"type": "string"
},
"assetId": {
"type": "string",
"nullable": true
},
"createdAt": {
"type": "string"
},
"size": {
"type": "number"
},
"bookmarkCount": {
"type": "number"
},
"status": {
"type": "string",
"enum": [
"pending",
"success",
"failure"
]
},
"errorMessage": {
"type": "string",
"nullable": true
}
},
"required": [
"id",
"userId",
"assetId",
"createdAt",
"size",
"bookmarkCount",
"status"
]
}
}
}
},
"404": {
"description": "Backup not found",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"code": {
"type": "string"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
]
}
}
}
}
}
},
"delete": {
"description": "Delete backup by its id",
"summary": "Delete a backup",
"tags": [
"Backups"
],
"security": [
{
"bearerAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/BackupId"
}
],
"responses": {
"204": {
"description": "No content - the backup was deleted"
},
"404": {
"description": "Backup not found",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"code": {
"type": "string"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
]
}
}
}
}
}
}
},
"/backups/{backupId}/download": {
"get": {
"description": "Download backup file",
"summary": "Download a backup",
"tags": [
"Backups"
],
"security": [
{
"bearerAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/BackupId"
}
],
"responses": {
"200": {
"description": "Backup file (zip archive)",
"content": {
"application/zip": {
"schema": {}
}
}
},
"404": {
"description": "Backup not found",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"code": {
"type": "string"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
]
}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,149 @@
import {
extendZodWithOpenApi,
OpenAPIRegistry,
} from "@asteasolutions/zod-to-openapi";
import { z } from "zod";
import { zBackupSchema } from "@karakeep/shared/types/backups";
import { BearerAuth } from "./common";
import { ErrorSchema } from "./errors";
export const registry = new OpenAPIRegistry();
extendZodWithOpenApi(z);
export const BackupIdSchema = registry.registerParameter(
"BackupId",
z.string().openapi({
param: {
name: "backupId",
in: "path",
},
example: "ieidlxygmwj87oxz5hxttoc8",
}),
);
registry.registerPath({
method: "get",
path: "/backups",
description: "Get all backups",
summary: "Get all backups",
tags: ["Backups"],
security: [{ [BearerAuth.name]: [] }],
responses: {
200: {
description: "Object with all backups data.",
content: {
"application/json": {
schema: z.object({
backups: z.array(zBackupSchema),
}),
},
},
},
},
});
registry.registerPath({
method: "post",
path: "/backups",
description: "Trigger a new backup",
summary: "Trigger a new backup",
tags: ["Backups"],
security: [{ [BearerAuth.name]: [] }],
responses: {
201: {
description: "Backup created successfully",
content: {
"application/json": {
schema: zBackupSchema,
},
},
},
},
});
registry.registerPath({
method: "get",
path: "/backups/{backupId}",
description: "Get backup by its id",
summary: "Get a single backup",
tags: ["Backups"],
security: [{ [BearerAuth.name]: [] }],
request: {
params: z.object({ backupId: BackupIdSchema }),
},
responses: {
200: {
description: "Object with backup data.",
content: {
"application/json": {
schema: zBackupSchema,
},
},
},
404: {
description: "Backup not found",
content: {
"application/json": {
schema: ErrorSchema,
},
},
},
},
});
registry.registerPath({
method: "get",
path: "/backups/{backupId}/download",
description: "Download backup file",
summary: "Download a backup",
tags: ["Backups"],
security: [{ [BearerAuth.name]: [] }],
request: {
params: z.object({ backupId: BackupIdSchema }),
},
responses: {
200: {
description: "Backup file (zip archive)",
content: {
"application/zip": {
schema: z.instanceof(Blob),
},
},
},
404: {
description: "Backup not found",
content: {
"application/json": {
schema: ErrorSchema,
},
},
},
},
});
registry.registerPath({
method: "delete",
path: "/backups/{backupId}",
description: "Delete backup by its id",
summary: "Delete a backup",
tags: ["Backups"],
security: [{ [BearerAuth.name]: [] }],
request: {
params: z.object({ backupId: BackupIdSchema }),
},
responses: {
204: {
description: "No content - the backup was deleted",
},
404: {
description: "Backup not found",
content: {
"application/json": {
schema: ErrorSchema,
},
},
},
},
});

View File

@@ -68,6 +68,16 @@ export interface paths {
/** @enum {string} */
crawlPriority?: "low" | "normal";
importSessionId?: string;
/** @enum {string} */
source?:
| "api"
| "web"
| "cli"
| "mobile"
| "extension"
| "singlefile"
| "rss"
| "import";
} & (
| {
/** @enum {string} */
@@ -322,6 +332,18 @@ export interface paths {
summarizationStatus: "success" | "failure" | "pending" | null;
note?: string | null;
summary?: string | null;
/** @enum {string|null} */
source?:
| "api"
| "web"
| "cli"
| "mobile"
| "extension"
| "singlefile"
| "rss"
| "import"
| null;
userId: string;
};
};
};
@@ -384,6 +406,18 @@ export interface paths {
summarizationStatus: "success" | "failure" | "pending" | null;
note?: string | null;
summary?: string | null;
/** @enum {string|null} */
source?:
| "api"
| "web"
| "cli"
| "mobile"
| "extension"
| "singlefile"
| "rss"
| "import"
| null;
userId: string;
};
};
};
@@ -668,6 +702,7 @@ export interface paths {
| "video"
| "bookmarkAsset"
| "precrawledArchive"
| "userUploaded"
| "unknown";
};
};
@@ -691,7 +726,9 @@ export interface paths {
| "video"
| "bookmarkAsset"
| "precrawledArchive"
| "userUploaded"
| "unknown";
fileName?: string | null;
};
};
};
@@ -1684,6 +1721,7 @@ export interface paths {
"application/json": {
/** @enum {string} */
color?: "yellow" | "red" | "green" | "blue";
note?: string | null;
};
};
};
@@ -1822,6 +1860,20 @@ export interface paths {
name: string;
count: number;
}[];
bookmarksBySource: {
/** @enum {string|null} */
source:
| "api"
| "web"
| "cli"
| "mobile"
| "extension"
| "singlefile"
| "rss"
| "import"
| null;
count: number;
}[];
};
};
};
@@ -2017,6 +2069,241 @@ export interface paths {
patch?: never;
trace?: never;
};
"/backups": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Get all backups
* @description Get all backups
*/
get: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Object with all backups data. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
backups: {
id: string;
userId: string;
assetId: string;
createdAt: string;
size: number;
bookmarkCount: number;
/** @enum {string} */
status: "pending" | "success" | "failure";
errorMessage?: string | null;
}[];
};
};
};
};
};
put?: never;
/**
* Trigger a new backup
* @description Trigger a new backup
*/
post: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Backup created successfully */
201: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
id: string;
userId: string;
assetId: string;
createdAt: string;
size: number;
bookmarkCount: number;
/** @enum {string} */
status: "pending" | "success" | "failure";
errorMessage?: string | null;
};
};
};
};
};
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/backups/{backupId}": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Get a single backup
* @description Get backup by its id
*/
get: {
parameters: {
query?: never;
header?: never;
path: {
backupId: components["parameters"]["BackupId"];
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Object with backup data. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
id: string;
userId: string;
assetId: string;
createdAt: string;
size: number;
bookmarkCount: number;
/** @enum {string} */
status: "pending" | "success" | "failure";
errorMessage?: string | null;
};
};
};
/** @description Backup not found */
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
code: string;
message: string;
};
};
};
};
};
put?: never;
post?: never;
/**
* Delete a backup
* @description Delete backup by its id
*/
delete: {
parameters: {
query?: never;
header?: never;
path: {
backupId: components["parameters"]["BackupId"];
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description No content - the backup was deleted */
204: {
headers: {
[name: string]: unknown;
};
content?: never;
};
/** @description Backup not found */
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
code: string;
message: string;
};
};
};
};
};
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/backups/{backupId}/download": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Download a backup
* @description Download backup file
*/
get: {
parameters: {
query?: never;
header?: never;
path: {
backupId: components["parameters"]["BackupId"];
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Backup file (zip archive) */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/zip": unknown;
};
};
/** @description Backup not found */
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
code: string;
message: string;
};
};
};
};
};
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
}
export type webhooks = Record<string, never>;
export interface components {
@@ -2031,6 +2318,8 @@ export interface components {
HighlightId: string;
/** @example ieidlxygmwj87oxz5hxttoc8 */
AssetId: string;
/** @example ieidlxygmwj87oxz5hxttoc8 */
BackupId: string;
Bookmark: {
id: string;
createdAt: string;
@@ -2044,6 +2333,18 @@ export interface components {
summarizationStatus: "success" | "failure" | "pending" | null;
note?: string | null;
summary?: string | null;
/** @enum {string|null} */
source?:
| "api"
| "web"
| "cli"
| "mobile"
| "extension"
| "singlefile"
| "rss"
| "import"
| null;
userId: string;
tags: {
id: string;
name: string;
@@ -2105,7 +2406,9 @@ export interface components {
| "video"
| "bookmarkAsset"
| "precrawledArchive"
| "userUploaded"
| "unknown";
fileName?: string | null;
}[];
};
PaginatedBookmarks: {
@@ -2126,6 +2429,9 @@ export interface components {
type: "manual" | "smart";
query?: string | null;
public: boolean;
hasCollaborators: boolean;
/** @enum {string} */
userRole: "owner" | "editor" | "viewer" | "public";
};
Highlight: {
bookmarkId: string;
@@ -2170,6 +2476,7 @@ export interface components {
TagId: components["schemas"]["TagId"];
HighlightId: components["schemas"]["HighlightId"];
AssetId: components["schemas"]["AssetId"];
BackupId: components["schemas"]["BackupId"];
};
requestBodies: never;
headers: never;

View File

@@ -234,3 +234,19 @@ export async function triggerRuleEngineOnEvent(
opts,
);
}
// Backup worker
export const zBackupRequestSchema = z.object({
userId: z.string(),
backupId: z.string().optional(),
});
export type ZBackupRequest = z.infer<typeof zBackupRequestSchema>;
export const BackupQueue = QUEUE_CLIENT.createQueue<ZBackupRequest>(
"backup_queue",
{
defaultJobArgs: {
numRetries: 2,
},
keepFailedJobs: false,
},
);

View File

@@ -26,6 +26,7 @@ export const enum ASSET_TYPES {
IMAGE_PNG = "image/png",
IMAGE_WEBP = "image/webp",
APPLICATION_PDF = "application/pdf",
APPLICATION_ZIP = "application/zip",
TEXT_HTML = "text/html",
VIDEO_MP4 = "video/mp4",
@@ -65,6 +66,7 @@ export const SUPPORTED_ASSET_TYPES: Set<string> = new Set<string>([
...SUPPORTED_UPLOAD_ASSET_TYPES,
ASSET_TYPES.TEXT_HTML,
ASSET_TYPES.VIDEO_MP4,
ASSET_TYPES.APPLICATION_ZIP,
]);
export const zAssetMetadataSchema = z.object({

View File

@@ -0,0 +1,14 @@
import { z } from "zod";
export const zBackupSchema = z.object({
id: z.string(),
userId: z.string(),
assetId: z.string().nullable(),
createdAt: z.date(),
size: z.number(),
bookmarkCount: z.number(),
status: z.enum(["pending", "success", "failure"]),
errorMessage: z.string().nullable().optional(),
});
export type ZBackup = z.infer<typeof zBackupSchema>;

View File

@@ -108,6 +108,9 @@ export const zUserSettingsSchema = z.object({
]),
archiveDisplayBehaviour: z.enum(["show", "hide"]),
timezone: z.string(),
backupsEnabled: z.boolean(),
backupsFrequency: z.enum(["daily", "weekly"]),
backupsRetentionDays: z.number().int().min(1).max(365),
});
export type ZUserSettings = z.infer<typeof zUserSettingsSchema>;
@@ -116,4 +119,13 @@ export const zUpdateUserSettingsSchema = zUserSettingsSchema.partial().pick({
bookmarkClickAction: true,
archiveDisplayBehaviour: true,
timezone: true,
backupsEnabled: true,
backupsFrequency: true,
backupsRetentionDays: true,
});
export const zUpdateBackupSettingsSchema = zUpdateUserSettingsSchema.pick({
backupsEnabled: true,
backupsFrequency: true,
backupsRetentionDays: true,
});

View File

@@ -17,6 +17,7 @@ export function mapDBAssetTypeToUserType(assetType: AssetTypes): ZAssetType {
[AssetTypes.LINK_HTML_CONTENT]: "linkHtmlContent",
[AssetTypes.BOOKMARK_ASSET]: "bookmarkAsset",
[AssetTypes.USER_UPLOADED]: "userUploaded",
[AssetTypes.BACKUP]: "unknown", // Backups are not displayed as regular assets
[AssetTypes.UNKNOWN]: "bannerImage",
};
return map[assetType];

View File

@@ -0,0 +1,172 @@
import { TRPCError } from "@trpc/server";
import { and, desc, eq, lt } from "drizzle-orm";
import { z } from "zod";
import { assets, backupsTable } from "@karakeep/db/schema";
import { BackupQueue } from "@karakeep/shared-server";
import { deleteAsset } from "@karakeep/shared/assetdb";
import { zBackupSchema } from "@karakeep/shared/types/backups";
import { AuthedContext } from "..";
export class Backup {
private constructor(
private ctx: AuthedContext,
private backup: z.infer<typeof zBackupSchema>,
) {}
static async fromId(ctx: AuthedContext, backupId: string): Promise<Backup> {
const backup = await ctx.db.query.backupsTable.findFirst({
where: and(
eq(backupsTable.id, backupId),
eq(backupsTable.userId, ctx.user.id),
),
});
if (!backup) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Backup not found",
});
}
return new Backup(ctx, backup);
}
private static fromData(
ctx: AuthedContext,
backup: z.infer<typeof zBackupSchema>,
): Backup {
return new Backup(ctx, backup);
}
static async getAll(ctx: AuthedContext): Promise<Backup[]> {
const backups = await ctx.db.query.backupsTable.findMany({
where: eq(backupsTable.userId, ctx.user.id),
orderBy: [desc(backupsTable.createdAt)],
});
return backups.map((b) => new Backup(ctx, b));
}
static async create(ctx: AuthedContext): Promise<Backup> {
const [backup] = await ctx.db
.insert(backupsTable)
.values({
userId: ctx.user.id,
size: 0,
bookmarkCount: 0,
status: "pending",
})
.returning();
return new Backup(ctx, backup!);
}
async triggerBackgroundJob({
delayMs,
idempotencyKey,
}: { delayMs?: number; idempotencyKey?: string } = {}): Promise<void> {
await BackupQueue.enqueue(
{
userId: this.ctx.user.id,
backupId: this.backup.id,
},
{
delayMs,
idempotencyKey,
},
);
}
/**
* Generic update method for backup records
*/
async update(
data: Partial<{
size: number;
bookmarkCount: number;
status: "pending" | "success" | "failure";
assetId: string | null;
errorMessage: string | null;
}>,
): Promise<void> {
await this.ctx.db
.update(backupsTable)
.set(data)
.where(
and(
eq(backupsTable.id, this.backup.id),
eq(backupsTable.userId, this.ctx.user.id),
),
);
// Update local state
this.backup = { ...this.backup, ...data };
}
async delete(): Promise<void> {
if (this.backup.assetId) {
// Delete asset
await deleteAsset({
userId: this.ctx.user.id,
assetId: this.backup.assetId,
});
}
await this.ctx.db.transaction(async (db) => {
// Delete asset first
if (this.backup.assetId) {
await db
.delete(assets)
.where(
and(
eq(assets.id, this.backup.assetId),
eq(assets.userId, this.ctx.user.id),
),
);
}
// Delete backup record
await db
.delete(backupsTable)
.where(
and(
eq(backupsTable.id, this.backup.id),
eq(backupsTable.userId, this.ctx.user.id),
),
);
});
}
/**
* Finds backups older than the retention period
*/
static async findOldBackups(
ctx: AuthedContext,
retentionDays: number,
): Promise<Backup[]> {
const cutoffDate = new Date();
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
const oldBackups = await ctx.db.query.backupsTable.findMany({
where: and(
eq(backupsTable.userId, ctx.user.id),
lt(backupsTable.createdAt, cutoffDate),
),
});
return oldBackups.map((backup) => Backup.fromData(ctx, backup));
}
asPublic(): z.infer<typeof zBackupSchema> {
return this.backup;
}
get id() {
return this.backup.id;
}
get assetId() {
return this.backup.assetId;
}
}

View File

@@ -430,6 +430,9 @@ export class User {
bookmarkClickAction: true,
archiveDisplayBehaviour: true,
timezone: true,
backupsEnabled: true,
backupsFrequency: true,
backupsRetentionDays: true,
},
});
@@ -444,6 +447,9 @@ export class User {
bookmarkClickAction: settings.bookmarkClickAction,
archiveDisplayBehaviour: settings.archiveDisplayBehaviour,
timezone: settings.timezone || "UTC",
backupsEnabled: settings.backupsEnabled,
backupsFrequency: settings.backupsFrequency,
backupsRetentionDays: settings.backupsRetentionDays,
};
}
@@ -463,6 +469,9 @@ export class User {
bookmarkClickAction: input.bookmarkClickAction,
archiveDisplayBehaviour: input.archiveDisplayBehaviour,
timezone: input.timezone,
backupsEnabled: input.backupsEnabled,
backupsFrequency: input.backupsFrequency,
backupsRetentionDays: input.backupsRetentionDays,
})
.where(eq(users.id, this.user.id));
}

View File

@@ -2,6 +2,7 @@ import { router } from "../index";
import { adminAppRouter } from "./admin";
import { apiKeysAppRouter } from "./apiKeys";
import { assetsAppRouter } from "./assets";
import { backupsAppRouter } from "./backups";
import { bookmarksAppRouter } from "./bookmarks";
import { feedsAppRouter } from "./feeds";
import { highlightsAppRouter } from "./highlights";
@@ -25,6 +26,7 @@ export const appRouter = router({
prompts: promptsAppRouter,
admin: adminAppRouter,
feeds: feedsAppRouter,
backups: backupsAppRouter,
highlights: highlightsAppRouter,
importSessions: importSessionsRouter,
webhooks: webhooksAppRouter,

View File

@@ -0,0 +1,54 @@
import { z } from "zod";
import { zBackupSchema } from "@karakeep/shared/types/backups";
import { authedProcedure, createRateLimitMiddleware, router } from "../index";
import { Backup } from "../models/backups";
export const backupsAppRouter = router({
list: authedProcedure
.output(z.object({ backups: z.array(zBackupSchema) }))
.query(async ({ ctx }) => {
const backups = await Backup.getAll(ctx);
return { backups: backups.map((b) => b.asPublic()) };
}),
get: authedProcedure
.input(
z.object({
backupId: z.string(),
}),
)
.output(zBackupSchema)
.query(async ({ ctx, input }) => {
const backup = await Backup.fromId(ctx, input.backupId);
return backup.asPublic();
}),
delete: authedProcedure
.input(
z.object({
backupId: z.string(),
}),
)
.mutation(async ({ input, ctx }) => {
const backup = await Backup.fromId(ctx, input.backupId);
await backup.delete();
}),
triggerBackup: authedProcedure
.use(
createRateLimitMiddleware({
name: "backups.triggerBackup",
windowMs: 60 * 60 * 1000, // 1 hour window
maxRequests: 5, // Max 5 backup triggers per hour
}),
)
.output(zBackupSchema)
.mutation(async ({ ctx }) => {
const backup = await Backup.create(ctx);
await backup.triggerBackgroundJob();
return backup.asPublic();
}),
});

View File

@@ -155,11 +155,17 @@ describe("User Routes", () => {
bookmarkClickAction: "open_original_link",
archiveDisplayBehaviour: "show",
timezone: "UTC",
backupsEnabled: false,
backupsFrequency: "weekly",
backupsRetentionDays: 30,
});
// Update settings
await caller.users.updateSettings({
bookmarkClickAction: "expand_bookmark_preview",
backupsEnabled: true,
backupsFrequency: "daily",
backupsRetentionDays: 7,
});
// Verify updated settings
@@ -168,6 +174,9 @@ describe("User Routes", () => {
bookmarkClickAction: "expand_bookmark_preview",
archiveDisplayBehaviour: "show",
timezone: "UTC",
backupsEnabled: true,
backupsFrequency: "daily",
backupsRetentionDays: 7,
});
// Test invalid update (e.g., empty input, if schema enforces it)

217
pnpm-lock.yaml generated
View File

@@ -811,6 +811,9 @@ importers:
'@tsconfig/node22':
specifier: ^22.0.0
version: 22.0.2
archiver:
specifier: ^7.0.1
version: 7.0.1
async-mutex:
specifier: ^0.4.1
version: 0.4.1
@@ -932,6 +935,9 @@ importers:
'@karakeep/prettier-config':
specifier: workspace:^0.1.0
version: link:../../tooling/prettier
'@types/archiver':
specifier: ^7.0.0
version: 7.0.0
'@types/jsdom':
specifier: ^21.1.6
version: 21.1.7
@@ -1125,6 +1131,12 @@ importers:
'@karakeep/tsconfig':
specifier: workspace:^0.1.0
version: link:../../tooling/typescript
'@types/adm-zip':
specifier: ^0.5.7
version: 0.5.7
adm-zip:
specifier: ^0.5.16
version: 0.5.16
vite-tsconfig-paths:
specifier: ^4.3.1
version: 4.3.2(typescript@5.9.3)(vite@7.0.6(@types/node@22.15.30)(jiti@2.4.2)(lightningcss@1.30.1)(sass@1.89.1)(terser@5.41.0)(tsx@4.20.3)(yaml@2.8.0))
@@ -5702,6 +5714,12 @@ packages:
'@tybys/wasm-util@0.10.0':
resolution: {integrity: sha512-VyyPYFlOMNylG45GoAe0xDoLwWuowvf92F9kySqzYh8vmYm7D2u4iUJKa1tOUpS70Ku13ASrOkS4ScXFsTaCNQ==}
'@types/adm-zip@0.5.7':
resolution: {integrity: sha512-DNEs/QvmyRLurdQPChqq0Md4zGvPwHerAJYWk9l2jCbD1VPpnzRJorOdiq4zsw09NFbYnhfsoEhWtxIzXpn2yw==}
'@types/archiver@7.0.0':
resolution: {integrity: sha512-/3vwGwx9n+mCQdYZ2IKGGHEFL30I96UgBlk8EtRDDFQ9uxM1l4O5Ci6r00EMAkiDaTqD9DQ6nVrWRICnBPtzzg==}
'@types/argparse@1.0.38':
resolution: {integrity: sha512-ebDJ9b0e702Yr7pWgB0jzm+CX4Srzz8RcXtLJDJB+BSccqMa36uyH/zUsSYao5+BD1ytv3k3rPYCq4mAE1hsXA==}
@@ -5944,12 +5962,12 @@ packages:
'@types/react@19.1.8':
resolution: {integrity: sha512-AwAfQ2Wa5bCx9WP8nZL2uMZWod7J7/JSplxbTmBQ5ms6QpqNYm672H0Vu9ZVKVngQ+ii4R/byguVEUZQyeg44g==}
'@types/react@19.2.2':
resolution: {integrity: sha512-6mDvHUFSjyT2B2yeNx2nUgMxh9LtOWvkhIU3uePn2I2oyNymUAX1NIsdgviM4CH+JSrp2D2hsMvJOkxY+0wNRA==}
'@types/react@19.2.5':
resolution: {integrity: sha512-keKxkZMqnDicuvFoJbzrhbtdLSPhj/rZThDlKWCDbgXmUg0rEUFtRssDXKYmtXluZlIqiC5VqkCgRwzuyLHKHw==}
'@types/readdir-glob@1.1.5':
resolution: {integrity: sha512-raiuEPUYqXu+nvtY2Pe8s8FEmZ3x5yAH4VkLdihcPdalvsHltomrRC9BzuStrJ9yk06470hS0Crw0f1pXqD+Hg==}
'@types/request-ip@0.0.41':
resolution: {integrity: sha512-Qzz0PM2nSZej4lsLzzNfADIORZhhxO7PED0fXpg4FjXiHuJ/lMyUg+YFF5q8x9HPZH3Gl6N+NOM8QZjItNgGKg==}
@@ -6205,6 +6223,10 @@ packages:
resolution: {integrity: sha512-4B/qKCfeE/ODUaAUpSwfzazo5x29WD4r3vXiWsB7I2mSDAihwEqKO+g8GELZUQSSAo5e1XTYh3ZVfLyxBc12nA==}
engines: {node: '>= 10.0.0'}
adm-zip@0.5.16:
resolution: {integrity: sha512-TGw5yVi4saajsSEgz25grObGHEUaDrniwvA2qwSC060KfqGPdglhvPMA2lPIoxs3PQIItj2iag35fONcQqgUaQ==}
engines: {node: '>=12.0'}
agent-base@7.1.3:
resolution: {integrity: sha512-jRR5wdylq8CkOe6hei19GGZnxM6rBGwFl3Bg0YItGDimvjGtAvdZk4Pu6Cl4u4Igsws4a1fd1Vq3ezrhn4KmFw==}
engines: {node: '>= 14'}
@@ -6339,6 +6361,14 @@ packages:
resolution: {integrity: sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw==}
engines: {node: '>= 8'}
archiver-utils@5.0.2:
resolution: {integrity: sha512-wuLJMmIBQYCsGZgYLTy5FIB2pF6Lfb6cXMSF8Qywwk3t20zWnAi7zLcQFdKQmIB8wyZpY5ER38x08GbwtR2cLA==}
engines: {node: '>= 14'}
archiver@7.0.1:
resolution: {integrity: sha512-ZcbTaIqJOfCc03QwD468Unz/5Ir8ATtvAHsK+FdXbDIbGfihqh9mrvdcYunQzqn4HrvWWaFyaxJhGZagaJJpPQ==}
engines: {node: '>= 14'}
arg@5.0.2:
resolution: {integrity: sha512-PYjyFOLKQ9y57JvQ6QLo8dAgNqswh8M1RMJYdQduT6xbWSgK36P/Z/v+p888pM69jMMfS8Xd8F6I1kQ/I9HUGg==}
@@ -6456,6 +6486,14 @@ packages:
resolution: {integrity: sha512-wvUjBtSGN7+7SjNpq/9M2Tg350UZD3q62IFZLbRAR1bSMlCo1ZaeW+BJ+D090e4hIIZLBcTDWe4Mh4jvUDajzQ==}
engines: {node: '>= 0.4'}
b4a@1.7.3:
resolution: {integrity: sha512-5Q2mfq2WfGuFp3uS//0s6baOJLMoVduPYVeNmDYxu5OUA1/cBfvr2RIS7vi62LdNj/urk1hfmj867I3qt6uZ7Q==}
peerDependencies:
react-native-b4a: '*'
peerDependenciesMeta:
react-native-b4a:
optional: true
babel-jest@29.7.0:
resolution: {integrity: sha512-BrvGY3xZSwEcCzKvKsCi2GgHqDqsYkOP4/by5xCgIwGXQxIEh+8ew3gmrE1y7XRR6LHZIj6yLYnUi/mm2KXKBg==}
engines: {node: ^14.15.0 || ^16.10.0 || >=18.0.0}
@@ -6540,6 +6578,14 @@ packages:
balanced-match@1.0.2:
resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==}
bare-events@2.8.2:
resolution: {integrity: sha512-riJjyv1/mHLIPX4RwiK+oW9/4c3TEUeORHKefKAKnZ5kyslbN+HXowtbaVEqt4IMUB7OXlfixcs6gsFeo/jhiQ==}
peerDependencies:
bare-abort-controller: '*'
peerDependenciesMeta:
bare-abort-controller:
optional: true
base-64@0.1.0:
resolution: {integrity: sha512-Y5gU45svrR5tI2Vt/X9GPd3L0HNIKzGu202EjxrXMpuc2V2CiKgemAbUUsqYmZJvPtCXoUKjNZwBJzsNScUbXA==}
@@ -6642,6 +6688,10 @@ packages:
bser@2.1.1:
resolution: {integrity: sha512-gQxTNE/GAfIIrmHLUE3oJyp5FO6HRBfhjnw4/wMmA63ZGDJnWBmgY/lyQBpnDUkGmAhbSe39tx2d/iTOAfglwQ==}
buffer-crc32@1.0.0:
resolution: {integrity: sha512-Db1SbgBS/fg/392AblrMJk97KggmvYhr4pB5ZIMTWtaivCPMWLkmb7m21cJvpvgK+J3nsU2CmmixNBZx4vFj/w==}
engines: {node: '>=8.0.0'}
buffer-from@1.1.2:
resolution: {integrity: sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==}
@@ -7035,6 +7085,10 @@ packages:
compare-versions@6.1.1:
resolution: {integrity: sha512-4hm4VPpIecmlg59CHXnRDnqGplJFrbLG4aFEl5vl6cK1u76ws3LLvX7ikFnTDl5vo39sjWD6AaDPYodJp/NNHg==}
compress-commons@6.0.2:
resolution: {integrity: sha512-6FqVXeETqWPoGcfzrXb37E50NP0LXT8kAMu5ooZayhWWdgEY4lBEEcbQNXtkuKQsGduxiIcI4gOTsxTmuq/bSg==}
engines: {node: '>= 14'}
compressible@2.0.18:
resolution: {integrity: sha512-AF3r7P5dWxL8MxyITRMlORQNaOA2IkAFaTr4k7BUumjPtRpGDTZpl0Pb1XCO6JeDCBdp126Cgs9sMxqSjgYyRg==}
engines: {node: '>= 0.6'}
@@ -7173,6 +7227,15 @@ packages:
typescript:
optional: true
crc-32@1.2.2:
resolution: {integrity: sha512-ROmzCKrTnOwybPcJApAA6WBWij23HVfGVNKqqrZpuyZOHqK2CwHSvpGuyt/UNNvaIjEd8X5IFGp4Mh+Ie1IHJQ==}
engines: {node: '>=0.8'}
hasBin: true
crc32-stream@6.0.0:
resolution: {integrity: sha512-piICUB6ei4IlTv1+653yq5+KoqfBYmj9bw6LqXoOneTMDXk5nM1qt12mFW1caG3LlJXEKW1Bp0WggEmIfQB34g==}
engines: {node: '>= 14'}
cross-spawn@6.0.6:
resolution: {integrity: sha512-VqCUuhcd1iB+dsv8gxPttb5iZh/D0iubSP21g36KXdEuf6I5JiioesUVjpCdHV9MZRUfVFlvwtIUyPfxo5trtw==}
engines: {node: '>=4.8'}
@@ -8127,6 +8190,9 @@ packages:
eventemitter3@4.0.7:
resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==}
events-universal@1.0.1:
resolution: {integrity: sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==}
events@3.3.0:
resolution: {integrity: sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==}
engines: {node: '>=0.8.x'}
@@ -8407,6 +8473,9 @@ packages:
fast-deep-equal@3.1.3:
resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==}
fast-fifo@1.3.2:
resolution: {integrity: sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==}
fast-glob@3.3.3:
resolution: {integrity: sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg==}
engines: {node: '>=8.6.0'}
@@ -9789,6 +9858,10 @@ packages:
resolution: {integrity: sha512-RE2g0b5VGZsOCFOCgP7omTRYFqydmZkBwl5oNnQ1lDYC57uyO9KqNnNVxT7COSHTxrRCWVcAVOcbjk+tvh/rgQ==}
engines: {node: '>=0.10.0'}
lazystream@1.0.1:
resolution: {integrity: sha512-b94GiNHQNy6JNTrt5w6zNyffMrNkXZb3KTkCZJb2V1xaEGCk093vkZ2jk3tpaeP33/OiXC+WvK9AxUebnf5nbw==}
engines: {node: '>= 0.6.3'}
leac@0.6.0:
resolution: {integrity: sha512-y+SqErxb8h7nE/fiEX07jsbuhrpO9lL8eca7/Y1nuWV2moNlXhyd59iDGcRf6moVyDMbmTNzL40SUyrFU/yDpg==}
@@ -12645,6 +12718,13 @@ packages:
resolution: {integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==}
engines: {node: '>= 6'}
readable-stream@4.7.0:
resolution: {integrity: sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==}
engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0}
readdir-glob@1.1.3:
resolution: {integrity: sha512-v05I2k7xN8zXvPD9N+z/uhXPaj0sUFCe2rcWZIpBsqxfP7xXFQ0tipAd/wjj1YxWyWtUS5IDJpOG82JKt2EAVA==}
readdirp@3.6.0:
resolution: {integrity: sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==}
engines: {node: '>=8.10.0'}
@@ -13451,6 +13531,9 @@ packages:
resolution: {integrity: sha512-Mcc5wHehp9aXz1ax6bZUyY5afg9u2rv5cqQI3mRrYkGC8rW2hM02jWuwjtL++LS5qinSyhj2QfLyNsuc+VsExg==}
engines: {node: '>=10.0.0'}
streamx@2.23.0:
resolution: {integrity: sha512-kn+e44esVfn2Fa/O0CPFcex27fjIL6MkVae0Mm6q+E6f0hWv578YCERbv+4m02cjxvDsPKLnmxral/rR6lBMAg==}
strict-uri-encode@2.0.0:
resolution: {integrity: sha512-QwiXZgpRcKkhTj2Scnn++4PKtWsH0kpzZ62L2R6c/LUVYv7hVnZqcg2+sMuT6R7Jusu1vviK/MFsu6kNJfWlEQ==}
engines: {node: '>=4'}
@@ -13666,6 +13749,9 @@ packages:
resolution: {integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==}
engines: {node: '>=6'}
tar-stream@3.1.7:
resolution: {integrity: sha512-qJj60CXt7IU1Ffyc3NJMjh6EkuCFej46zUqJ4J7pqYlThyd9bO0XBTmcOIhSzZJVWfsLks0+nle/j538YAW9RQ==}
tar@7.4.3:
resolution: {integrity: sha512-5S7Va8hKfV7W5U6g3aYxXmlPoZVAwUMy9AOKyF2fVuZa2UD3qZjg578OrLRt8PcNN1PleVaL/5/yYATNL0ICUw==}
engines: {node: '>=18'}
@@ -13716,6 +13802,9 @@ packages:
resolution: {integrity: sha512-cAGWPIyOHU6zlmg88jwm7VRyXnMN7iV68OGAbYDk/Mh/xC/pzVPlQtY6ngoIH/5/tciuhGfvESU8GrHrcxD56w==}
engines: {node: '>=8'}
text-decoder@1.2.3:
resolution: {integrity: sha512-3/o9z3X0X0fTupwsYvR03pJ/DjWuqqrfwBgTQzdWDiQSm9KitAyz/9WqsT2JQW7KV2m+bC2ol/zqpW37NHxLaA==}
text-hex@1.0.0:
resolution: {integrity: sha512-uuVGNWzgJ4yhRaNSiubPY7OjISw4sw4E5Uv0wbjp+OzcbmVU/rsT8ujgcXJhn9ypzsgr5vlzpPqP+MBBKcGvbg==}
@@ -14829,6 +14918,10 @@ packages:
resolution: {integrity: sha512-GQHQqAopRhwU8Kt1DDM8NjibDXHC8eoh1erhGAJPEyveY9qqVeXvVikNKrDz69sHowPMorbPUrH/mx8c50eiBQ==}
engines: {node: '>=18'}
zip-stream@6.0.1:
resolution: {integrity: sha512-zK7YHHz4ZXpW89AHXUPbQVGKI7uvkd3hzusTdotCg1UxyaVtg0zFJSTfW/Dq5f7OBBVnq6cZIaC8Ti4hb6dtCA==}
engines: {node: '>= 14'}
zlibjs@0.3.1:
resolution: {integrity: sha512-+J9RrgTKOmlxFSDHo0pI1xM6BLVUv+o0ZT9ANtCxGkjIVCCUdx9alUF8Gm+dGLKbkkkidWIHFDZHDMpfITt4+w==}
@@ -20834,6 +20927,14 @@ snapshots:
tslib: 2.8.1
optional: true
'@types/adm-zip@0.5.7':
dependencies:
'@types/node': 22.15.30
'@types/archiver@7.0.0':
dependencies:
'@types/readdir-glob': 1.1.5
'@types/argparse@1.0.38': {}
'@types/babel__core@7.20.5':
@@ -21109,7 +21210,7 @@ snapshots:
'@types/react-router@5.1.20':
dependencies:
'@types/history': 4.7.11
'@types/react': 19.2.2
'@types/react': 19.2.5
'@types/react-syntax-highlighter@15.5.13':
dependencies:
@@ -21123,14 +21224,14 @@ snapshots:
dependencies:
csstype: 3.1.3
'@types/react@19.2.2':
dependencies:
csstype: 3.1.3
'@types/react@19.2.5':
dependencies:
csstype: 3.1.3
'@types/readdir-glob@1.1.5':
dependencies:
'@types/node': 22.15.30
'@types/request-ip@0.0.41':
dependencies:
'@types/node': 22.15.30
@@ -21440,6 +21541,8 @@ snapshots:
address@1.2.2: {}
adm-zip@0.5.16: {}
agent-base@7.1.3: {}
agentkeepalive@4.6.0:
@@ -21583,6 +21686,29 @@ snapshots:
normalize-path: 3.0.0
picomatch: 2.3.1
archiver-utils@5.0.2:
dependencies:
glob: 10.4.5
graceful-fs: 4.2.11
is-stream: 2.0.1
lazystream: 1.0.1
lodash: 4.17.21
normalize-path: 3.0.0
readable-stream: 4.7.0
archiver@7.0.1:
dependencies:
archiver-utils: 5.0.2
async: 3.2.6
buffer-crc32: 1.0.0
readable-stream: 4.7.0
readdir-glob: 1.1.3
tar-stream: 3.1.7
zip-stream: 6.0.1
transitivePeerDependencies:
- bare-abort-controller
- react-native-b4a
arg@5.0.2: {}
argparse@1.0.10:
@@ -21691,6 +21817,8 @@ snapshots:
dependencies:
possible-typed-array-names: 1.1.0
b4a@1.7.3: {}
babel-jest@29.7.0(@babel/core@7.26.0):
dependencies:
'@babel/core': 7.26.0
@@ -21877,6 +22005,8 @@ snapshots:
balanced-match@1.0.2: {}
bare-events@2.8.2: {}
base-64@0.1.0: {}
base64-js@1.5.1: {}
@@ -22018,6 +22148,8 @@ snapshots:
dependencies:
node-int64: 0.4.0
buffer-crc32@1.0.0: {}
buffer-from@1.1.2: {}
buffer@5.7.1:
@@ -22437,6 +22569,14 @@ snapshots:
compare-versions@6.1.1: {}
compress-commons@6.0.2:
dependencies:
crc-32: 1.2.2
crc32-stream: 6.0.0
is-stream: 2.0.1
normalize-path: 3.0.0
readable-stream: 4.7.0
compressible@2.0.18:
dependencies:
mime-db: 1.54.0
@@ -22583,6 +22723,13 @@ snapshots:
optionalDependencies:
typescript: 5.9.3
crc-32@1.2.2: {}
crc32-stream@6.0.0:
dependencies:
crc-32: 1.2.2
readable-stream: 4.7.0
cross-spawn@6.0.6:
dependencies:
nice-try: 1.0.5
@@ -23563,6 +23710,12 @@ snapshots:
eventemitter3@4.0.7: {}
events-universal@1.0.1:
dependencies:
bare-events: 2.8.2
transitivePeerDependencies:
- bare-abort-controller
events@3.3.0: {}
eventsource-parser@3.0.2: {}
@@ -23957,6 +24110,8 @@ snapshots:
fast-deep-equal@3.1.3: {}
fast-fifo@1.3.2: {}
fast-glob@3.3.3:
dependencies:
'@nodelib/fs.stat': 2.0.5
@@ -25536,6 +25691,10 @@ snapshots:
lazy-cache@1.0.4: {}
lazystream@1.0.1:
dependencies:
readable-stream: 2.3.8
leac@0.6.0: {}
leven@3.1.0: {}
@@ -29296,6 +29455,18 @@ snapshots:
string_decoder: 1.3.0
util-deprecate: 1.0.2
readable-stream@4.7.0:
dependencies:
abort-controller: 3.0.0
buffer: 6.0.3
events: 3.3.0
process: 0.11.10
string_decoder: 1.3.0
readdir-glob@1.1.3:
dependencies:
minimatch: 5.1.6
readdirp@3.6.0:
dependencies:
picomatch: 2.3.1
@@ -30325,6 +30496,15 @@ snapshots:
streamsearch@1.1.0: {}
streamx@2.23.0:
dependencies:
events-universal: 1.0.1
fast-fifo: 1.3.2
text-decoder: 1.2.3
transitivePeerDependencies:
- bare-abort-controller
- react-native-b4a
strict-uri-encode@2.0.0: {}
string-argv@0.3.2: {}
@@ -30597,6 +30777,15 @@ snapshots:
inherits: 2.0.4
readable-stream: 3.6.2
tar-stream@3.1.7:
dependencies:
b4a: 1.7.3
fast-fifo: 1.3.2
streamx: 2.23.0
transitivePeerDependencies:
- bare-abort-controller
- react-native-b4a
tar@7.4.3:
dependencies:
'@isaacs/fs-minipass': 4.0.1
@@ -30663,6 +30852,12 @@ snapshots:
glob: 7.2.3
minimatch: 3.1.2
text-decoder@1.2.3:
dependencies:
b4a: 1.7.3
transitivePeerDependencies:
- react-native-b4a
text-hex@1.0.0: {}
thenify-all@1.6.0:
@@ -31940,6 +32135,12 @@ snapshots:
yoctocolors@2.1.1: {}
zip-stream@6.0.1:
dependencies:
archiver-utils: 5.0.2
compress-commons: 6.0.2
readable-stream: 4.7.0
zlibjs@0.3.1: {}
zod-to-json-schema@3.24.5(zod@3.24.2):