- Introduce deterministic UUID generation for library defaults (spaces,
groups, and items) - Add post-backfill rebuild support in the sync
registry (with_rebuild) - Export BatchFkMapResult type via the sync API
- Track per-peer RTT latency via SyncMetricsCollector integration - Use
deterministic IDs when creating default space and related items in
LibraryManager
- Revert to the November 16 baseline for realtime sync - Use a single
state watermark and drop per-resource watermarks, counts, and hashes
from watermark messages - Revert DataAvailableNotification handling and
the updated FK filtering approach in favor of NULL-based handling - Add
registry helper get_model_type_by_table to map tables to models for FK
resolution - Update mocks and tests to reflect the baseline (linkage
stats in summary) - Fix test infrastructure: remove block_in_place usage
and minor imports - Remove obsolete test fixture file
- Introduce an Axum-based HTTP server with an embedded daemon and a
JSON-RPC proxy to the daemon via a Unix socket - Bundle web UI assets
into the server with an assets feature and a build.rs that builds the
frontend using pnpm - Add multi-stage Dockerfile, docker-compose.yml,
and a Distroless runtime image - Provide TrueNAS deployment support with
a build script and setup guide - Add a new web UI (apps/web) with a
Vite-based dev/build flow and a web platform shim for the frontend -
Implement server logic (apps/server/src/main.rs): health, auth, /rpc
proxy and data-dir/socket-path wiring - Include server-specific
Cargo.toml and a comprehensive server README - Add architecture and
memory-focused docs to guide usage and design - Minor core tweak:
simplify location/resource event emission in
core/src/location/manager.rs to align with new flow - Tauri app: adjust
menus to add an Edit submenu and remove unused items
- Introduced a new `unique_bytes` field in the volume model to track deduplicated content size.
- Implemented a volume refresh action to recalculate unique bytes for all volumes owned by the device, emitting a cache refresh event to invalidate frontend caches.
- Updated the storage overview component to display unique bytes and other volume statistics.
- Enhanced the event system to handle cache invalidation events, ensuring real-time updates across the application.
- Added necessary migrations and database updates to support the new volume tracking features.
- Deleted the Agent Manager Design document to streamline documentation and focus on the new extension-based agent architecture.
- Updated the whitepaper to reflect the transition to an extension-based agent architecture, detailing the capabilities of specialized AI agents implemented as WASM extensions.
- Revised sections to emphasize the event-driven processing, memory systems, and safety mechanisms of the new agent architecture.
- Enhanced clarity on the integration of agents within the VDFS and their roles in proactive file management and user assistance.
- Marked the `leader` argument in `SetupArgs` as deprecated, clarifying its usage.
- Introduced a `backfill_manager` to the `SyncService`, enabling automatic orchestration of initial sync processes.
- Enhanced the `run_sync_loop` method to manage backfill attempts and periodic maintenance tasks, improving overall sync reliability.
- Updated the `MockTransportPeer` to support request/response handling for backfill operations, ensuring seamless data retrieval during synchronization.
- Refactored the entry model to extract fields from JSON instead of direct deserialization, allowing for better error handling and validation of incoming data.
- Introduced a helper function to streamline field extraction, ensuring all required fields are present before processing.
- Updated the tag model to similarly extract fields from JSON, enhancing its robustness during synchronization.
- Improved the handling of optional fields in both models, ensuring that missing data is managed gracefully.
- Removed the obsolete ENTRY_DIRECTORY_PATH_SUMMARY.md and ENTRY_PATH_SYNC_ANALYSIS.md files, consolidating documentation for clarity.
- Added a new `foreign_key_mappings` method to the Syncable trait, allowing models to declare their foreign key relationships for automatic UUID conversion during synchronization.
- Implemented the `to_sync_json` method to utilize the new FK mappings, enabling seamless conversion of local integer IDs to UUIDs before syncing.
- Updated the entry and location models to include foreign key mappings, enhancing their synchronization capabilities.
- Enhanced the `apply_state_change` method to leverage the generic mapping logic, simplifying the implementation for models with foreign keys.
- Introduced comprehensive documentation on the new FK mapping system and its usage across models.
- Added `sync_enabled` and `last_sync_at` fields to the device model to support synchronization capabilities.
- Created a migration script to update the devices table, consolidating sync-related data.
- Updated various components to ensure sync is enabled by default for new devices.
- Removed the obsolete sync partners table, streamlining the sync architecture.
- Introduced comprehensive sync integration tests to validate the new functionality and ensure robust performance.
- Added `sync_depends_on` method to the Syncable trait, allowing models to declare their dependencies for synchronization.
- Updated device, location, entry, and tag models to specify their dependencies, facilitating automatic computation of sync order.
- Enhanced backfill process to respect model dependencies, preventing foreign key violations during synchronization.
- Improved documentation to reflect the new dependency graph and its benefits for model synchronization.
- Implemented a complete WASM extension framework, enabling secure, sandboxed plugins.
- Added core components including `PluginManager`, `host_functions`, and `permissions` for managing the lifecycle and security of extensions.
- Integrated Wasmer runtime for executing WASM modules, enhancing the platform's extensibility.
- Developed a demo extension showcasing the new API, significantly reducing boilerplate code and improving developer experience.
- Updated documentation to reflect the new architecture and provide guidance for extension development.
- Prepared for testing and validation of the extension system, marking a significant step towards a robust plugin ecosystem.
- Finalized the core synchronization infrastructure, enabling device-independent writes without leader checks.
- Implemented critical components including `TransactionManager`, `SyncProtocolHandler`, and `PeerSync` with enhanced broadcast capabilities.
- Achieved full message routing for state and shared changes, ensuring robust error handling and logging.
- Registered new models in the async-safe registry, facilitating state-based and log-based synchronization.
- Documented implementation progress and architecture changes, highlighting the transition to a leaderless model.
- Prepared for integration testing with remaining tasks outlined for backfill and retry queue implementation.
- Added `apply_state_change` method to the `location` model for idempotent state-based replication, allowing device-owned data to be synchronized without conflicts.
- Introduced `apply_shared_change` method in the `tag` model to handle shared resources with union merge conflict resolution, preserving tags with the same name in different contexts.
- Enhanced documentation to clarify the synchronization strategies for device-owned and shared models, including error handling and usage examples.
- Updated tests to validate the new functionality and ensure correct behavior during synchronization processes.
- Removed leadership-related components from the sync infrastructure, including `LeadershipManager` and `sync_leadership` fields across various models.
- Implemented a new peer-to-peer sync model utilizing Hybrid Logical Clocks (HLC) for shared resources and state-based sync for device-owned data.
- Updated the `Syncable` trait and related modules to reflect the new architecture, ensuring seamless integration of state and log-based synchronization.
- Introduced `PeerLog` for managing device-specific changes and `PeerSync` for handling synchronization in the leaderless environment.
- Revised documentation to outline the new sync architecture and its implications for device synchronization, emphasizing the benefits of a leaderless approach.
- Added a comprehensive checklist for removing leader-related code in the transition to a leaderless hybrid sync model.
- Documented necessary changes across core infrastructure, service layer, library management, and network protocols to facilitate the removal of leadership components.
- Updated sync documentation to reflect the new architecture, emphasizing the leaderless approach and its implications for device synchronization.
- Included migration strategies and terminology updates to ensure clarity in the transition process.
- Updated the library sync system to utilize a leaderless hybrid model, enhancing real-time synchronization of shared resources.
- Implemented state-based sync for device-owned data and log-based sync with Hybrid Logical Clocks (HLC) for shared resources.
- Revised documentation to reflect architectural changes, including the removal of leader election and the introduction of per-device sync logs.
- Enhanced conflict resolution strategies using HLC for shared metadata, ensuring deterministic merging and consistency across devices.
- Updated implementation tasks and acceptance criteria for various components to align with the new architecture.
- Introduced a comprehensive sync infrastructure, including a `Syncable` trait for models, enabling automatic logging of changes in a dedicated sync log database.
- Added `LeadershipManager` for managing leader election and lease tracking, ensuring reliable synchronization across devices.
- Implemented `TransactionManager` to handle atomic writes and sync logging, enhancing data integrity during state changes.
- Created a new `sync` module with necessary components such as `SyncLogDb`, `SyncLogEntry`, and sync protocol handlers for efficient communication between leader and follower devices.
- Developed integration tests to validate the sync functionality and ensure robust performance across various scenarios.
- Introduced `index verify` command to perform comprehensive integrity checks on the Spacedrive index.
- Implemented functionality to compare filesystem state with database entries, reporting discrepancies such as missing files, stale entries, and metadata mismatches.
- Added detailed output options for verification results, including a summary report of findings.
- Created associated input and output types for verification actions, enhancing the overall integrity management of the indexing system.
- Documented usage and examples in the CLI documentation for user guidance.
- Enhanced `MacOSHandler` to query the database for inode information when handling rename events, allowing detection of renamed files even when the old path no longer exists.
- Implemented `get_inode_from_db()` method to facilitate database lookups for inode retrieval.
- Updated `handle_single_rename_event()` to utilize the new database query logic, ensuring proper event emission and entry identity preservation.
- Registered and unregistered database connections for locations in `LocationWatcher`, improving rename detection accuracy.
- Added integration tests to validate rename functionality and ensure no duplicate entries are created during renames.
- Updated `ApiDispatcher` to delegate action and query execution to `ActionManager` and `QueryManager`, respectively, improving separation of concerns.
- Refactored permission checks and action dispatching to ensure consistent handling across API calls.
- Introduced `QueryContext` for tracking query metadata, enhancing query execution with contextual information.
- Improved error handling by integrating new `QueryError` types for better clarity in query-related failures.
- Cleaned up obsolete code and updated import paths to reflect the new structure, ensuring maintainability and clarity.
- Added QR code generation for remote pairing, allowing users to scan codes for easier device connection.
- Updated CLI to include QR code display and manual entry options for pairing.
- Enhanced PairingCoordinator to handle QR code JSON, including NodeId and relay URL for improved connectivity.
- Implemented camera permission handling in iOS for QR code scanning functionality.
- Refactored networking service to support relay-based pairing, ensuring reliable connections across different networks.
- Added tests for relay-only pairing scenarios to validate functionality and reliability.
Add comprehensive design document for reorganizing API infrastructure:
- Move cqrs.rs to infra/query/ for better symmetry with actions
- Move registry, type_extraction, api_types to infra/wire/
- Improve code organization and discoverability
- Clear separation of infrastructure vs business logic
This is Option A: Full consolidation with wire/ directory for all
wire protocol and type system concerns.
- Introduced three new markdown files: CLI_LIBRARY_SYNC_COMPLETE.md, IMPLEMENTATION_COMPLETE.md, and LIBRARY_SYNC_SETUP_IMPLEMENTATION.md for comprehensive documentation.
- Updated various CLI domain modules to improve argument handling and output formatting.
- Enhanced device, index, job, library, location, network, and search modules for better integration and user experience.
- Refactored code across multiple domains to improve maintainability and clarity.
- Implemented pauseJob and resumeJob methods in DaemonConnector to manage job states.
- Updated SharedAppState to handle new pause and resume actions.
- Introduced JobActionButton in JobRowView for user interaction to pause or resume jobs.
- Enhanced launch configuration in VSCode for debugging Spacedrive applications.
- Added a warning suppression directive in the Rust CLI main file to reduce noise during compilation.
- Deleted several Swift files related to job management and UI components, streamlining the codebase and focusing on essential functionality.
- Updated the main application structure to enhance window management and user experience.
- Improved the organization of action context handling in the Rust core, facilitating better job dispatching with action context awareness.
- Updated `CopyMethod` enum to include `Atomic` and `Streaming` variants, replacing the previous `AtomicMove` and `StreamingCopy` options for clarity.
- Refactored the `select_strategy` method to respect user preferences for copy methods, improving the logic for same-volume operations.
- Added CLI support for the new copy methods in `args.rs`, allowing users to specify their preferred method during file copy operations.
- Updated relevant tests to reflect changes in copy method naming and functionality.
- Enhanced documentation to include new copy method options and their usage.
Co-authored-by: ijamespine <ijamespine@me.com>
- Deleted `ARCHITECTURE.md`, `Cargo.toml.bak`, and various integration summary files to streamline the codebase.
- Removed outdated demo files and testing guides that are no longer relevant.
- Cleaned up the repository by eliminating unnecessary documentation and binary files, ensuring a more maintainable project structure.
Co-authored-by: ijamespine <ijamespine@me.com>
- Introduced `LibraryInfoArgs` struct for querying library information, allowing optional library ID input.
- Implemented `to_query` method for `LibraryInfoArgs` to facilitate query creation.
- Updated `LibraryCmd` enum to include `Info` variant for handling library information requests.
- Refactored argument handling in `args.rs` for improved readability and consistency.
- Enhanced `run` function in `mod.rs` to process the new `Info` command and display detailed library information.
Co-authored-by: ijamespine <ijamespine@me.com>
- Added a comprehensive testing guide for the FS Event Pipeline, detailing metrics collection, logging, and troubleshooting steps.
- Updated the CLI package name from "spacedrive-cli" to "sd-cli" in Cargo.toml for consistency.
- Modified various files to reflect the new package structure and improve logging and metrics handling.
Co-authored-by: ijamespine <ijamespine@me.com>
- Renamed the `user_metadata_manager` module to `manager` for consistency and clarity.
- Updated all references to `UserMetadataManager` across the codebase to reflect the new module name.
- Adjusted documentation to point to the new module location.
This refactor enhances the organization of the metadata management system, improving maintainability and readability.
- Renamed and restructured the tagging modules, replacing `semantic_tag` with `tag` for consistency across the codebase.
- Updated all references in the domain, operations, and database layers to reflect the new naming conventions.
- Removed deprecated files and entities related to the old semantic tagging system, including `semantic_tag.rs` and `metadata_tag.rs`.
- Enhanced the organization of the tagging system, improving clarity and maintainability for future development.
This commit streamlines the tagging architecture, paving the way for advanced features and improved usability.
