1387 Commits

Author SHA1 Message Date
Jamie Pine
0506f9a45a Update version to 2.0.0-pre.1 2025-12-01 12:45:10 -08:00
Jamie Pine
01ec19d352 Bump version to 2.0.0-pre 2025-12-01 12:41:57 -08:00
Jamie Pine
142008dead Add Active Jobs API and Path Checks Plus UI Prefs 2025-12-01 10:07:31 -08:00
Jamie Pine
9acf86a9ea Add location export/import and schema changes 2025-11-30 17:27:31 -08:00
Jamie Pine
3730776fd4 Remove redundant StateSyncHandler and send peer device in state requests
to get only their state resources (device owned data)
2025-11-28 10:36:00 -08:00
Jamie Pine
1e4992f454 Generate watermark during snapshot-only sync 2025-11-28 09:52:43 -08:00
Jamie Pine
00eebd6c23 Add per peer watermark store for shared sync
- Introduce PeerWatermarkStore to track max HLC received per peer for
shared resource incremental sync - Initialize the
peer_received_watermarks table at startup - Persist received watermarks
during backfill and after backfill completion - Query the max watermark
across peers via PeerWatermarkStore - Expose the new module and export
PeerWatermarkStore from sync
2025-11-28 09:35:40 -08:00
Jamie Pine
3791372f57 Apply current_state snapshot before backfill
Process the pre-sync current_state data in dependency order before
applying HLC-ordered log entries. This ensures base records exist for FK
constraints, and then resolves dependent changes after inserting
snapshot records. Extend FK error handling to recognize foreign key
failures and extract missing UUIDs for buffering and retry.
2025-11-28 08:58:13 -08:00
Jamie Pine
7c864e9588 Buffer shared changes when FK dep missing 2025-11-28 08:37:27 -08:00
Jamie Pine
e4c60c58ca Guard against self-watermarks and apply backfill
Skip self-watermarks when peer is the local device; log a warning. Apply
current_state in dependency order by computing registry order. Fall back
to unordered if the computation fails. On automatic backfill failure,
reset device state to Uninitialized. This triggers a retry and clears
backfill_attempted.
2025-11-28 08:20:28 -08:00
Jamie Pine
0e5e9827d1 Add native drag and drop and update submodules 2025-11-28 06:57:00 -08:00
Jamie Pine
ace39839a2 Improve subscription handling and RPC reliability
- Flush RPC writer after sending responses to ensure delivery - Add
cancellation guard in useNormalizedQuery to avoid leaks - Make
SubscriptionManager concurrency-safe: deduplicate in-flight subscribes -
Track pending subscriptions to avoid races and add centralized cleanup
logic - Introduce createSubscription helper and centralized cleanup
logic - Update docs with Sync State Machine, HLC update algorithm, retry
queue, metrics, and protocol messages - Reflect new config defaults for
timeouts and intervals
2025-11-27 12:48:50 -08:00
Jamie Pine
97cc3853b5 Batch load tags for entries and refine path filter 2025-11-27 10:05:11 -08:00
Jamie Pine
83138f798e Emit direct virtual resources and apply tags 2025-11-27 10:05:11 -08:00
Jamie Pine
41fa50433c Add fullscreen quick preview via portal layer 2025-11-27 10:05:11 -08:00
Jamie Pine
c7eff9589d refactor: update main.rs and backfill.rs for improved error handling 2025-11-26 11:09:00 -08:00
Jamie Pine
4c35d6d7f7 refactor: update Cargo.toml and FK handling 2025-11-26 11:08:59 -08:00
Jamie Pine
db1dbf6655 Add Node ID field for cross-network pairing 2025-11-26 08:43:49 -08:00
Jamie Pine
35af15aefc Add deterministic UUIDs for library defaults
- Introduce deterministic UUID generation for library defaults (spaces,
groups, and items) - Add post-backfill rebuild support in the sync
registry (with_rebuild) - Export BatchFkMapResult type via the sync API
- Track per-peer RTT latency via SyncMetricsCollector integration - Use
deterministic IDs when creating default space and related items in
LibraryManager
2025-11-26 08:19:24 -08:00
Jamie Pine
35263791cf Revert to November 16 baseline for realtime sync
- Revert to the November 16 baseline for realtime sync - Use a single
state watermark and drop per-resource watermarks, counts, and hashes
from watermark messages - Revert DataAvailableNotification handling and
the updated FK filtering approach in favor of NULL-based handling - Add
registry helper get_model_type_by_table to map tables to models for FK
resolution - Update mocks and tests to reflect the baseline (linkage
stats in summary) - Fix test infrastructure: remove block_in_place usage
and minor imports - Remove obsolete test fixture file
2025-11-25 15:42:25 -08:00
Jamie Pine
400f923772 Register syncable models via inventory system
Introduce inventory-based registration for Syncable models. Add
SyncableInventoryEntry and macros to register models to register models
at link time. Refactor registry and FK mapper to use the registry for FK
resolution and lookups. Add post-backfill hooks and per-model FK
callbacks. Register all entities via the new macros.
2025-11-25 14:45:01 -08:00
Jamie Pine
73da2bf1e9 Migrate pairing to pkarr-based remote discovery
- Remove relay-URL based pairing flow and rely on pkarr/DNS for remote
discovery - Extend pairing code with optional node_id and adopt version
2 QR format - Generate pairing codes using node_id and drop relay_url in
non-QR flows - Enable combined mDNS, pkarr, and DNS discovery in the
endpoint - Cache relay_url in persisted device info for reconnection
optimization - Update protocol handlers to propagate node_id and
pkarr-based flow - Adjust tests and logging to reflect the new
cross-network pairing
2025-11-25 13:57:41 -08:00
Jamie Pine
524b7e7507 Enable FK to UUID conversion for space entities 2025-11-25 10:11:32 -08:00
Jamie Pine
2cb1611cb1 Switch to shared sync for spaces and related entities
- Add apply_shared_change handlers for Space, SpaceGroup, and SpaceItem
to upsert by UUID and delete by UUID

- Update registry to use shared_with_query and call apply_shared_change;
remove deletion hooks

- Enhance SyncMonitorPopover to display a colored state badge in a
header bar
2025-11-25 10:03:17 -08:00
Jamie Pine
8b8bade25c Add device-owned space models to sync registry 2025-11-25 09:15:20 -08:00
Jamie Pine
48e5c6665a Improve sync activity UI and registry sync order 2025-11-25 09:09:15 -08:00
Jamie Pine
7a71718eda Introduce sync activity API and UI monitor
- Add SyncActivityType enum and GetSyncActivity input/output - Implement
SyncActivityAggregator to emit state, activity, and errors - Wire
aggregator into SyncService startup to run in background - Build
frontend SyncMonitor UI: popover, peer list, activity feed, hooks -
Extend TS client types with sync.activity endpoints
2025-11-25 07:00:26 -08:00
Jamie Pine
d2d94e3b72 Merge DB-tracked volumes and emit UI events
- Wire library manager into VolumeManager to enable merging tracked
volumes from the database into runtime state - Validate visibility
before tracking: reject non-user-visible volumes - Merge DB metadata for
tracked volumes (library_id, display_name) into detected volumes and
emit ResourceChanged for user-visible ones - Emit ResourceChanged on
adds and updates; emit ResourceDeleted on removals for user-visible
volumes to support UI reactivity - Generate stable UUIDs for new volumes
from fingerprints (APFS and macOS)
2025-11-25 05:16:40 -08:00
Jamie Pine
69e825fe06 Revert sync to November 14th - Add new sync tests
1. Watermark Messages: - Nov 16: Simple - single my_state_watermark,
single state_watermark in response - Main: Complex - per-resource
my_resource_watermarks HashMap, resource counts, content hashes 2.
DataAvailableNotification: - Nov 16: Doesn't exist - Main: New message
type that triggers watermark exchanges 3. FK Mapper: - Nov 16: Sets
missing FKs to NULL, applies all records - Main: Marks records with
missing FKs as failed, filters them out 4. Watermark Storage (internal,
not in messages): - Nov 16: Single last_watermark per resource - Main:
Dual watermarks - cursor_watermark and
2025-11-24 14:27:32 -08:00
Jamie Pine
742e9f32bf Add Empty type and centralized icon utilities 2025-11-24 07:05:47 -08:00
Jamie Pine
8808e85f4e Add paired devices support across CLI, core, and UI
- Add show_paired flag to device listing (CLI arg and input) - Extend
LibraryDeviceInfo with is_paired and is_connected; include Paired
devices in library listing when requested - Add Devices group to UI
(DevicesGroup) and hook into SpaceGroup - Extend device queries/types to
support show_paired and paired devices - Refactor Dockerfile to
multi-stage Bun + Rust builds; reuse web assets - Remove obsolete
core/ops/entries/mod.rs
2025-11-24 05:22:54 -08:00
Jamie Pine
bcab31462e Add Spacedrive server with embedded daemon
- Introduce an Axum-based HTTP server with an embedded daemon and a
JSON-RPC proxy to the daemon via a Unix socket - Bundle web UI assets
into the server with an assets feature and a build.rs that builds the
frontend using pnpm - Add multi-stage Dockerfile, docker-compose.yml,
and a Distroless runtime image - Provide TrueNAS deployment support with
a build script and setup guide - Add a new web UI (apps/web) with a
Vite-based dev/build flow and a web platform shim for the frontend -
Implement server logic (apps/server/src/main.rs): health, auth, /rpc
proxy and data-dir/socket-path wiring - Include server-specific
Cargo.toml and a comprehensive server README - Add architecture and
memory-focused docs to guide usage and design - Minor core tweak:
simplify location/resource event emission in
core/src/location/manager.rs to align with new flow - Tauri app: adjust
menus to add an Edit submenu and remove unused items
2025-11-23 11:01:01 -08:00
Jamie Pine
8a3387ca69 Add memory system with archive and UI integration 2025-11-23 03:33:19 -08:00
Jamie Pine
d2163d842d Enable ephemeral indexing and update docs 2025-11-22 05:57:51 -08:00
Jamie Pine
bd504a721e Add tagging UI and tag system components 2025-11-22 05:12:55 -08:00
Jamie Pine
672d3d32ef Seed ancestor cache and remove ghost-folder test 2025-11-21 09:20:49 -08:00
Jamie Pine
08f2d551ff Fix ghost folder bug by scoping indexing to path
- Scope the closure query to the indexing path by resolving its entry_id
from directory_paths and use that for descendants; fall back to the
location root if not found. - Seed all ancestors between the location
root and the indexing path into the in-memory cache during the
processing phase to ensure correct parent links. - Seed ancestor
directories into the create path cache to prevent ghost folders on
subpath indexing. - Add test core/tests/indexing_move_folder_bug_test.rs
reproducing the ghost-folder bug when moving a folder into a managed
location and asserting correct parent linkage.
2025-11-21 09:18:03 -08:00
Jamie Pine
008d05414a Refactor file handling in Explorer component
- Updated file type checks from `file.kind.type` to `file.kind` for consistency across various components in the Explorer views.
- Enhanced the `Thumb` component to conditionally hide the icon based on thumbnail loading status.
- Adjusted the `HeroStats` component for improved readability and structure.
- Added a new `iconScale` prop to the `FileInspector` component's thumbnail for better visual scaling.
2025-11-21 08:01:37 -08:00
Jamie Pine
553fadd2d4 Switch to ts-client types from root package 2025-11-20 21:43:48 -08:00
Jamie Pine
8c14803eae identify srt 2025-11-20 12:25:18 -08:00
Jamie Pine
86570a65ed Add job count hook and refactor JobManager UI
- Introduce useJobCount to track active and running jobs via query and
event subscriptions. - Refactor JobManagerPopover to render a separate
content component and rely on the new hook for counts. - Add
JobManagerPopoverContent to render filtered JobList with pause/resume
callbacks and dynamic height. - Update useJobManager to subscribe to job
events via a single filtered subscription, replacing per-event handlers.
- Extend client types to support event_type filtering for subscriptions.
- Tidy up UI logs and minor rendering tweaks in QuickPreview and
ContentRenderer, including folder icon handling. - Adjust storage
overview to use real volume fields and drop dummy data paths
2025-11-20 08:57:10 -08:00
Jamie Pine
d31ba54080 chore: update subproject commit and add blank lines for consistency
- Updated the subproject commit reference to indicate a dirty state.
- Added blank lines to various files for improved readability and consistency across the codebase.
2025-11-20 04:38:22 -08:00
Jamie Pine
8d751b0713 feat: enhance event filtering and subscription management
- Added `include_descendants` option to event filters, allowing recursive path matching for resource events.
- Updated `affects_path` method to support descendant matching, improving event handling accuracy.
- Refactored subscription logic to utilize the new filtering capabilities, ensuring only relevant events are processed.
- Introduced tests for event filtering to validate exact vs. recursive matching, enhancing reliability of event-driven updates.
- Updated related components to leverage the new filtering options, improving overall performance and user experience.
2025-11-20 04:38:01 -08:00
Jamie Pine
08912e141e feat: implement library opening functionality and filesystem watcher
- Added a new "Open Library" menu item to the application, allowing users to select a library folder.
- Implemented the backend logic to handle library opening, including JSON-RPC requests to the daemon.
- Introduced a filesystem watcher in the LibraryManager to monitor library directory changes, enabling automatic opening and closing of libraries based on filesystem events.
- Enhanced error handling for library operations and added relevant logging for better traceability.
- Updated the LibraryError enum to include filesystem watcher errors, improving error reporting.
2025-11-18 18:40:56 -08:00
Jamie Pine
28988eed1b feat: add HLC constructor and enhance indexing tests
- Introduced a new constructor for the HLC struct to allow explicit initialization with timestamp, counter, and device ID.
- Updated indexing tests to improve error handling for entry IDs and added a comprehensive test for reindexing after offline changes, ensuring all original and new files are detected correctly.
2025-11-18 18:23:59 -08:00
Jamie Pine
85012ac165 feat: add proxy generation policy to job configurations
- Introduced a new ProxyPolicy struct to manage video scrubbing settings, including options for enabling and regenerating proxy files.
- Updated JobPolicies to include the new proxy policy with default values.
- Enhanced FileInspector to filter batch events by resource ID for improved event handling.
- Modified useNormalizedCache to skip path validation checks when resourceId is provided, optimizing resource handling.
2025-11-18 05:48:23 -08:00
Jamie Pine
2f6c8a985d feat: enhance location update process and event emission
- Updated the LocationUpdateAction to clone the location before building the ActiveModel, ensuring data integrity during updates.
- Implemented a new event emission for ResourceChanged, providing real-time updates to the UI upon location changes.
- Enhanced error handling for missing entry IDs and directory paths, improving robustness in the update process.
- Refactored the LocationInspector component to include new quick actions for reindexing locations, enhancing user interaction.
- Removed redundant isGlobalList flags from various components to streamline the codebase.
2025-11-18 05:38:37 -08:00
Jamie Pine
e103265e20 feat: enhance location path validation and event routing logic
- Added critical safety checks to validate that indexing paths are within specified location boundaries, preventing potential data loss from incorrect event routing.
- Improved event routing logic in the LocationWatcher to match events to the longest valid location path, ensuring accurate event handling in scenarios with multiple locations.
- Enhanced logging for successful path validation and event routing, providing better insights into the processing flow.
- Introduced unit tests to verify path matching functionality across multiple locations, ensuring robustness in event handling.
2025-11-18 03:39:42 -08:00
Jamie Pine
5a6c3eb5a0 feat: add database migration step to ensure schema consistency
- Implemented a migration step in the LibraryManager to ensure the database schema is up to date before proceeding with operations.
- This change enhances the reliability of the application by preventing potential schema-related issues during runtime.
2025-11-18 02:36:31 -08:00
Jamie Pine
ad63cba189 chore: update subproject commit and add blank lines for consistency
- Updated the subproject commit reference to indicate a dirty state.
- Added blank lines to various files for improved readability and consistency across the codebase.
2025-11-18 02:31:20 -08:00