* feat: add OpenTelemetry tracing infrastructure
Introduce distributed tracing capabilities using OpenTelemetry:
- Add @opentelemetry packages to shared-server for tracing
- Create tracing utility module with span helpers (withSpan, addSpanEvent, etc.)
- Add tRPC middleware for automatic span creation on API calls
- Initialize tracing in API and workers entry points
- Add demo instrumentation to bookmark creation and crawler worker
- Add configuration options (OTEL_TRACING_ENABLED, OTEL_EXPORTER_OTLP_ENDPOINT, etc.)
- Document tracing configuration in environment variables docs
When enabled, traces are collected for tRPC calls, bookmark creation flow,
and crawler operations, with support for any OTLP-compatible backend (Jaeger, Tempo, etc.)
* refactor: remove tracing from workers for now
Keep tracing infrastructure but remove worker instrumentation:
- Remove tracing initialization from workers entry point
- Remove tracing instrumentation from crawler worker
- Fix formatting in tracing files
The tracing infrastructure remains available for future use.
* add hono and next tracing
* remove extra span logging
* more fixes
* update config
* some fixes
* upgrade packages
* remove unneeded packages
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: Add automated bookmark backup system
Implements a comprehensive automated backup feature for user bookmarks with the following capabilities:
Database Schema:
- Add backupSettings table to store user backup preferences (enabled, frequency, retention)
- Add backups table to track backup records with status and metadata
- Add BACKUP asset type for storing compressed backup files
- Add migration 0066_add_backup_tables.sql
Background Workers:
- Implement BackupSchedulingWorker cron job (runs daily at midnight UTC)
- Create BackupWorker to process individual backup jobs
- Deterministic scheduling spreads backup jobs across 24 hours based on user ID hash
- Support for daily and weekly backup frequencies
- Automated retention cleanup to delete old backups based on user settings
Export & Compression:
- Reuse existing export functionality for bookmark data
- Compress exports using Node.js built-in zlib (gzip level 9)
- Store compressed backups as assets with proper metadata
- Track backup size and bookmark count for statistics
tRPC API:
- backups.getSettings - Retrieve user backup configuration
- backups.updateSettings - Update backup preferences
- backups.list - List all user backups with metadata
- backups.get - Get specific backup details
- backups.delete - Delete a backup
- backups.download - Download backup file (base64 encoded)
- backups.triggerBackup - Manually trigger backup creation
UI Components:
- BackupSettings component with configuration form
- Enable/disable automatic backups toggle
- Frequency selection (daily/weekly)
- Retention period configuration (1-365 days)
- Backup list table with download and delete actions
- Manual backup trigger button
- Display backup stats (size, bookmark count, status)
- Added backups page to settings navigation
Technical Details:
- Uses Restate queue system for distributed job processing
- Implements idempotency keys to prevent duplicate backups
- Background worker concurrency: 2 jobs at a time
- 10-minute timeout for large backup exports
- Proper error handling and logging throughout
- Type-safe implementation with Zod schemas
* refactor: simplify backup settings and asset handling
- Move backup settings from separate table to user table columns
- Update BackupSettings model to use static methods with users table
- Remove download mutation in favor of direct asset links
- Implement proper quota checks using QuotaService.checkStorageQuota
- Update UI to use new property names and direct asset downloads
- Update shared types to match new schema
Key changes:
- backupSettingsTable removed, settings now in users table
- Backup downloads use direct /api/assets/{id} links
- Quota properly validated before creating backup assets
- Cleaner separation of concerns in tRPC models
* migration
* use zip instead of gzip
* fix drizzle
* fix settings
* streaming json
* remove more dead code
* add e2e tests
* return backup
* poll for backups
* more fixes
* more fixes
* fix test
* fix UI
* fix delete asset
* fix ui
* redirect for backup download
* cleanups
* fix idempotency
* fix tests
* add ratelimit
* add error handling for background backups
* i18n
* model changes
---------
Co-authored-by: Claude <noreply@anthropic.com>
Implements a new API endpoint that returns the server version from serverConfig.
This will be used for Home Assistant integration to check for updates.
Fixes#2148
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* feat: Add collaborative lists backend implementation
This commit implements the core backend functionality for collaborative
lists, allowing multiple users to share and interact with bookmark lists.
Database changes:
- Add listCollaborators table to track users with access to lists and
their roles (viewer/editor)
- Add addedBy field to bookmarksInLists to track who added bookmarks
- Add relations for collaborative list functionality
Access control updates:
- Update List model to support role-based access (owner/editor/viewer)
- Add methods to check and enforce permissions for list operations
- Update Bookmark model to allow access through collaborative lists
- Modify bookmark queries to include bookmarks from collaborative lists
List collaboration features:
- Add/remove/update collaborators
- Get list of collaborators
- Get lists shared with current user
- Only manual lists can have collaborators
tRPC procedures:
- addCollaborator: Add a user as a collaborator to a list
- removeCollaborator: Remove a collaborator from a list
- updateCollaboratorRole: Change a collaborator's role
- getCollaborators: Get all collaborators for a list
- getSharedWithMe: Get all lists shared with the current user
- cloneBookmark: Clone a bookmark to the current user's collection
Implementation notes:
- Editors can add/remove bookmarks from the list (must own the bookmark)
- Viewers can only view bookmarks in the list
- Only the list owner can manage collaborators and list metadata
- Smart lists cannot have collaborators (only manual lists)
- Users cannot edit bookmarks they don't own, even in shared lists
* feat: Add collaborative lists frontend UI
This commit implements the frontend user interface for collaborative lists,
allowing users to view shared bookmarks and manage list collaborators.
New pages:
- /dashboard/shared: Shows bookmarks from lists shared with the user
- Displays bookmarks from all collaborative lists
- Uses SharedBookmarks component
- Shows empty state when no lists are shared
Navigation:
- Added "Shared with you" link to sidebar with Users icon
- Positioned after "Home" in main navigation
- Available in both desktop and mobile sidebar
Collaborator management:
- ManageCollaboratorsModal component for managing list collaborators
- Add collaborators by user ID with viewer/editor role
- View current collaborators with their roles
- Update collaborator roles inline
- Remove collaborators
- Shows empty state when no collaborators
- Integrated into ListOptions dropdown menu
- Accessible via "Manage Collaborators" menu item
Components created:
- SharedBookmarks.tsx: Server component fetching shared lists/bookmarks
- ManageCollaboratorsModal.tsx: Client component with tRPC mutations
- /dashboard/shared/page.tsx: Route for shared bookmarks page
UI features:
- Role selector for viewer/editor permissions
- Real-time collaborator list updates
- Toast notifications for success/error states
- Loading states for async operations
- Responsive design matching existing UI patterns
Implementation notes:
- Uses existing tRPC endpoints (getSharedWithMe, getCollaborators, etc.)
- Follows established modal patterns from ShareListModal
- Integrates seamlessly with existing list UI
- Currently uses user ID for adding collaborators (email lookup TBD)
* fix typecheck
* add collaborator by email
* add shared list in the sidebar
* fix perm issue
* hide UI components from non list owners
* list leaving
* fix shared bookmarks showing up in homepage
* fix getBookmark access check
* e2e tests
* hide user specific fields from shared lists
* simplify bookmark perm checks
* disable editable fields in bookmark preview
* hide lists if they don't have options
* fix list ownership
* fix highlights
* move tests to trpc
* fix alignment of leave list
* make tag lists unclickable
* allow editors to remove from list
* add a badge for shared lists
* remove bookmarks of user when they're removed from a list
* fix tests
* show owner in the manage collab modal
* fix hasCollab
* drop shared with you
* i18n
* beta badge
* correctly invalidate caches on collab change
* reduce unnecessary changes
* Add ratelimits
* stop manually removing bookmarks on remove
* some fixes
* fixes
* remove unused function
* improve tests
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: Add source field to track bookmark creation sources
Add a new 'source' field to the bookmarks table to track where bookmarks
were created from. Possible values: api, web, cli, mobile, singlefile, rss.
Changes:
- Add source field to bookmarks table schema
- Update Zod schemas to include source field
- Update tRPC createBookmark procedure to store source
- Update all callsites to pass appropriate source value:
- api: Default to "api" if not provided
- singlefile: Set to "singlefile"
- rss: Set to "rss" in feedWorker
- cli: Set to "cli"
- mobile: Set to "mobile" in all mobile app bookmark creation
- browser-extension: Set to "web"
- web: Set to "web" in all web app bookmark creation
- Create migration file for database schema change
Fixes#2036🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* feat: Add extension source type for browser extension
- Add 'extension' to bookmark source enum
- Update browser extension to use 'extension' instead of 'web'
Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* fix CI
* fix CI
* fix the migration file
* add import source
* make source nullish
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* feat: Add tag search and use in the homepage
* use paginated query in the all tags view
* wire the load more buttons
* add skeleton to all tags page
* fix attachedby aggregation
* fix loading states
* fix hasNextPage
* use action buttons for load more buttons
* migrate the tags auto complete to the search api
* Migrate the tags editor to the new search API
* Replace tag merging dialog with tag auto completion
* Merge both search and list APIs
* fix tags.list
* add some tests for the endpoint
* add relevance based sorting
* change cursor
* update the REST API
* fix review comments
* more fixes
* fix lockfile
* i18n
* fix visible tags
* WIP: public lists
* Drop viewing modes
* Add the public endpoint for assets
* regen the openapi spec
* proper handling for different asset types
* Add num bookmarks and a no bookmark banner
* Correctly set page title
* Add a not-found page
* merge the RSS and public list endpoints
* Add e2e tests for the public endpoints
* Redesign the share list modal
* Make NEXTAUTH_SECRET not required
* propery render text bookmarks
* rebase migration
* fix public token tests
* Add more tests
* refactor: Move bookmark utils from shared-react to shared
* Expose RSS feeds for lists
* Add e2e tests
* Slightly improve the look of the share dialog
* allow specifying a limit in the rss endpoint
* Setup Hono and migrate the highlights API there
* Implement the tags and lists endpoint
* Implement the bookmarks and users endpoints
* Add the trpc error code adapter
* Remove the old nextjs handlers
* fix api key not found handling
* Fix trpc error handling
* Fix 204 handling
* Fix search ordering
* Implement the singlefile endpoint
* Implement the asset serving endpoints
* Implement webauth
* Add hono as a catch all route under api
* fix tests
