237 Commits

Author SHA1 Message Date
Jamie Pine
ace39839a2 Improve subscription handling and RPC reliability
- Flush RPC writer after sending responses to ensure delivery - Add
cancellation guard in useNormalizedQuery to avoid leaks - Make
SubscriptionManager concurrency-safe: deduplicate in-flight subscribes -
Track pending subscriptions to avoid races and add centralized cleanup
logic - Introduce createSubscription helper and centralized cleanup
logic - Update docs with Sync State Machine, HLC update algorithm, retry
queue, metrics, and protocol messages - Reflect new config defaults for
timeouts and intervals
2025-11-27 12:48:50 -08:00
Jamie Pine
c7eff9589d refactor: update main.rs and backfill.rs for improved error handling 2025-11-26 11:09:00 -08:00
Jamie Pine
35af15aefc Add deterministic UUIDs for library defaults
- Introduce deterministic UUID generation for library defaults (spaces,
groups, and items) - Add post-backfill rebuild support in the sync
registry (with_rebuild) - Export BatchFkMapResult type via the sync API
- Track per-peer RTT latency via SyncMetricsCollector integration - Use
deterministic IDs when creating default space and related items in
LibraryManager
2025-11-26 08:19:24 -08:00
Jamie Pine
35263791cf Revert to November 16 baseline for realtime sync
- Revert to the November 16 baseline for realtime sync - Use a single
state watermark and drop per-resource watermarks, counts, and hashes
from watermark messages - Revert DataAvailableNotification handling and
the updated FK filtering approach in favor of NULL-based handling - Add
registry helper get_model_type_by_table to map tables to models for FK
resolution - Update mocks and tests to reflect the baseline (linkage
stats in summary) - Fix test infrastructure: remove block_in_place usage
and minor imports - Remove obsolete test fixture file
2025-11-25 15:42:25 -08:00
Jamie Pine
bcab31462e Add Spacedrive server with embedded daemon
- Introduce an Axum-based HTTP server with an embedded daemon and a
JSON-RPC proxy to the daemon via a Unix socket - Bundle web UI assets
into the server with an assets feature and a build.rs that builds the
frontend using pnpm - Add multi-stage Dockerfile, docker-compose.yml,
and a Distroless runtime image - Provide TrueNAS deployment support with
a build script and setup guide - Add a new web UI (apps/web) with a
Vite-based dev/build flow and a web platform shim for the frontend -
Implement server logic (apps/server/src/main.rs): health, auth, /rpc
proxy and data-dir/socket-path wiring - Include server-specific
Cargo.toml and a comprehensive server README - Add architecture and
memory-focused docs to guide usage and design - Minor core tweak:
simplify location/resource event emission in
core/src/location/manager.rs to align with new flow - Tauri app: adjust
menus to add an Edit submenu and remove unused items
2025-11-23 11:01:01 -08:00
Jamie Pine
bd504a721e Add tagging UI and tag system components 2025-11-22 05:12:55 -08:00
Jamie Pine
8d751b0713 feat: enhance event filtering and subscription management
- Added `include_descendants` option to event filters, allowing recursive path matching for resource events.
- Updated `affects_path` method to support descendant matching, improving event handling accuracy.
- Refactored subscription logic to utilize the new filtering capabilities, ensuring only relevant events are processed.
- Introduced tests for event filtering to validate exact vs. recursive matching, enhancing reliability of event-driven updates.
- Updated related components to leverage the new filtering options, improving overall performance and user experience.
2025-11-20 04:38:01 -08:00
Jamie Pine
ef25390441 feat: add unique bytes tracking and cache refresh functionality for volumes
- Introduced a new `unique_bytes` field in the volume model to track deduplicated content size.
- Implemented a volume refresh action to recalculate unique bytes for all volumes owned by the device, emitting a cache refresh event to invalidate frontend caches.
- Updated the storage overview component to display unique bytes and other volume statistics.
- Enhanced the event system to handle cache invalidation events, ensuring real-time updates across the application.
- Added necessary migrations and database updates to support the new volume tracking features.
2025-11-17 23:37:48 -08:00
Jamie Pine
8305d7d235 feat: working sync test, sick log analyzer, cleanup 2025-11-16 01:53:35 -08:00
Jamie Pine
ddcefe2495 docs 2025-11-14 21:40:49 -08:00
James Pine
f7d7468bce remove submodules 2025-11-14 21:31:21 -08:00
Jamie Pine
9182b1f1ff fix: improve sync partner retrieval 2025-10-20 15:34:59 -07:00
Jamie Pine
dcc4dc4fc1 pairing 2025-10-18 10:33:20 -07:00
Jamie Pine
e07c28c484 (chore): update tasks 2025-10-14 20:39:56 -07:00
Jamie Pine
83dbf88c47 refactor: improve sidebar functionality and code organization 2025-10-14 03:36:48 -07:00
Jamie Pine
566d9c6a6a feat: add VolumeEvent enum and implement ApfsContainer 2025-10-14 03:00:50 -07:00
Jamie Pine
c8e598b602 Add cloud variant to SdPath 2025-10-13 03:07:13 -07:00
Jamie Pine
d4700e4c43 docs 2025-10-12 23:10:05 -07:00
Jamie Pine
0cd4dad7e8 docs 2025-10-12 03:30:22 -07:00
Jamie Pine
fa32c67906 docs 2025-10-11 23:57:48 -07:00
Jamie Pine
a33ee63361 cleanup 2025-10-11 19:28:52 -07:00
Jamie Pine
92af932058 design docs moved to submodule 2025-10-11 11:13:37 -07:00
Jamie Pine
35ac1f214f sorting docs 2025-10-11 11:11:25 -07:00
Jamie Pine
efef06af63 add dates to docs 2025-10-11 11:07:42 -07:00
Jamie Pine
1a3c644092 move docs 2025-10-11 10:53:27 -07:00
Jamie Pine
f5e3465b75 move docs 2025-10-11 10:53:17 -07:00
Jamie Pine
63397de4f5 yes 2025-10-11 10:49:32 -07:00
Jamie Pine
284f709932 cleanup 2025-10-11 09:13:17 -07:00
Jamie Pine
68afdb9c51 cleanup 2025-10-11 09:00:58 -07:00
Jamie Pine
94a52dd2df cleanup 2025-10-11 08:48:11 -07:00
Jamie Pine
c77121aff8 The Spacedrive Collection 2025-10-11 07:16:16 -07:00
Jamie Pine
a5c9717dcb modularize photos extension and fix all errors 2025-10-11 05:32:47 -07:00
Jamie Pine
37772b7e2c docs: Remove Agent Manager Design Document and Update Whitepaper
- Deleted the Agent Manager Design document to streamline documentation and focus on the new extension-based agent architecture.
- Updated the whitepaper to reflect the transition to an extension-based agent architecture, detailing the capabilities of specialized AI agents implemented as WASM extensions.
- Revised sections to emphasize the event-driven processing, memory systems, and safety mechanisms of the new agent architecture.
- Enhanced clarity on the integration of agents within the VDFS and their roles in proactive file management and user assistance.
2025-10-11 01:21:36 -07:00
Jamie Pine
eb06c0a5cb feat: Introduce Spacedrive SDK API Reference and core modules
- Added a comprehensive API reference document for the Spacedrive SDK, detailing all APIs as type-checked stubs for type-checking purposes.
- Introduced new core modules including `actions`, `agent`, `ai`, `job_context`, `models`, `query`, `tasks`, and `vdfs`, each with defined structures and functionalities.
- Updated `Cargo.toml` to include new dependencies and ensure compatibility with the SDK.
- Enhanced `types.rs` to include additional error types and result types for better error handling across the SDK.
- Established a clear structure for extension development, allowing for future implementations and runtime testing.
2025-10-11 00:58:24 -07:00
Jamie Pine
8eefce6bf0 docs: Clean up whitespace in SDK specification
- Removed unnecessary blank lines in the SDK specification document to improve readability and maintain consistency in formatting.
2025-10-10 21:22:09 -07:00
Jamie Pine
9096eefca9 docs 2025-10-10 21:22:02 -07:00
Jamie Pine
492a3124d4 docs: Update SDK specification with new primitives and memory query API
- Expanded the SDK specification to include detailed sections on new primitives: `#[app]`, `#[model]`, and `#[agent]`.
- Introduced a comprehensive Memory Query API reference, detailing capabilities for temporal and associative memory types.
- Added custom query methods for `ChronicleMind`, enhancing functionality for retrieving related papers and summarizing recent activities.
- Removed outdated SDK_SYNTAX_FINAL.md to streamline documentation and focus on the latest design updates.
2025-10-10 19:03:58 -07:00
Jamie Pine
df716ad0b3 docs: Update MEMO2.md with latest insights on Spacedrive V2 launch and extension strategy
- Revised MEMO2.md to incorporate recent developments and strategic insights for the upcoming Spacedrive V2 launch.
- Clarified the integration of new extensions and their roles within the platform, emphasizing the business model focused on open-source and paid offerings.
- Enhanced projections and validation metrics to align with the November launch timeline, including user acquisition and revenue expectations.
- Streamlined descriptions to improve clarity and engagement for potential investors and stakeholders.
2025-10-10 18:16:21 -07:00
Jamie Pine
a1ec44cff2 docs: Refine MEMO2.md for Spacedrive V2 with updated business model and extension details
- Clarified the business model, emphasizing the free and open-source core with paid extensions launching in November.
- Streamlined descriptions of core VDFS primitives and their applications in various data-intensive scenarios.
- Updated projections and validation metrics for the November launch, including user acquisition targets and revenue expectations.
- Enhanced clarity on the mix of open and closed source extensions, maintaining trust through sandboxed execution and transparent permissions.
2025-10-10 06:31:07 -07:00
Jamie Pine
e3f81c976b docs: Revise extension lineup and investor memo for Spacedrive V2 launch
- Updated the extension lineup document to reflect new names and features for the November 2025 launch, including Chronicle (formerly Research) and Cipher (formerly Vault).
- Enhanced descriptions of each extension's capabilities, pricing, and strategic advantages, emphasizing their roles as subsystems of a data OS.
- Revised the investor memo to clarify the transition from V1 to V2, highlighting the complete VDFS architecture and the business model centered around open source and paid extensions.
- Included insights on the platform's unique value proposition and market strategy, projecting significant growth and customer acquisition plans.
2025-10-10 06:27:17 -07:00
Jamie Pine
44fd259f12 docs: Add Extension Lineup Ideas for November 2025 Launch
- Introduced a new document outlining proposed extensions for the November 2025 launch, detailing core principles and confirmed must-have extensions.
- Included descriptions, pricing models, and market analysis for each extension: Research, CRM, Vault, and Finance.
- Provided alternative extension ideas and strategic considerations for the launch lineup.
- Recommended a final launch lineup of four extensions, emphasizing their roles as subsystems of a data OS.
2025-10-10 04:00:02 -07:00
Jamie Pine
50d4ac1199 refactor: Remove entries module and update investor memo for clarity
- Deleted the `entries` module from the project structure to streamline operations.
- Revised the investor memo to enhance clarity and conciseness, consolidating information about Spacedrive V2's launch and its business model.
- Updated supporting materials and risk mitigation strategies to better reflect the current development and market approach.
2025-10-10 03:12:20 -07:00
Jamie Pine
1302a4099b docs: Update MEMO2.md with Spacedrive V2 launch insights and platform advantages
- Revised the memo to clarify the transition from V1 to V2, emphasizing the execution failures of V1 and the improvements made in V2.
- Highlighted the development timeline and architectural refinements that enabled a production-ready launch in four months.
- Detailed the platform's unique value proposition, focusing on local-first privacy and the advantages of the Extension SDK.
- Enhanced the go-to-market strategy, including targeted customer acquisition plans and compliance certifications.
- Included validation metrics and a call to action for potential investors.
2025-10-10 01:30:35 -07:00
Jamie Pine
9a74e0bbf6 docs: Revise investor memo for Spacedrive V2 launch and platform enhancements
- Updated the investor memo to reflect the upcoming launch of Spacedrive V2 in November 2025, highlighting the transition from V1 to V2 and the lessons learned.
- Detailed the new architecture and features, including the introduction of three paid extensions: Finance, Notes, and CRM, along with their pricing models.
- Clarified the platform's unique value proposition, emphasizing local-first privacy and the advantages of the Extension SDK.
- Enhanced the go-to-market strategy and unit economics, projecting significant growth and outlining customer acquisition plans.
- Included supporting materials and a call to action for potential investors.
2025-10-10 01:19:32 -07:00
Jamie Pine
a6d32bf2c7 docs: Update MEMO.md with Spacedrive V2 launch details and SDK enhancements
- Revised the introduction to clarify the failures of Spacedrive V1 and the responsibilities taken.
- Expanded on the Extension SDK, detailing the reduction in development complexity and providing examples of traditional vs. Spacedrive extensions.
- Updated the business model section to reflect changes in pricing and compliance certifications.
- Adjusted projections and market strategies to align with the new product offerings and compliance roadmap.
- Enhanced customer acquisition strategies and clarified the first extension launch details.
2025-10-10 00:06:44 -07:00
Jamie Pine
6266c5f081 feat: Deprecate leader device argument and enhance sync service with backfill manager
- Marked the `leader` argument in `SetupArgs` as deprecated, clarifying its usage.
- Introduced a `backfill_manager` to the `SyncService`, enabling automatic orchestration of initial sync processes.
- Enhanced the `run_sync_loop` method to manage backfill attempts and periodic maintenance tasks, improving overall sync reliability.
- Updated the `MockTransportPeer` to support request/response handling for backfill operations, ensuring seamless data retrieval during synchronization.
2025-10-10 00:06:36 -07:00
Jamie Pine
8c868b41c7 refactor: Enhance entry and tag models for improved JSON deserialization and sync capabilities
- Refactored the entry model to extract fields from JSON instead of direct deserialization, allowing for better error handling and validation of incoming data.
- Introduced a helper function to streamline field extraction, ensuring all required fields are present before processing.
- Updated the tag model to similarly extract fields from JSON, enhancing its robustness during synchronization.
- Improved the handling of optional fields in both models, ensuring that missing data is managed gracefully.
- Removed the obsolete ENTRY_DIRECTORY_PATH_SUMMARY.md and ENTRY_PATH_SYNC_ANALYSIS.md files, consolidating documentation for clarity.
2025-10-09 16:42:17 -07:00
Jamie Pine
8b1476af5d feat: Introduce automatic foreign key mapping for syncable models
- Added a new `foreign_key_mappings` method to the Syncable trait, allowing models to declare their foreign key relationships for automatic UUID conversion during synchronization.
- Implemented the `to_sync_json` method to utilize the new FK mappings, enabling seamless conversion of local integer IDs to UUIDs before syncing.
- Updated the entry and location models to include foreign key mappings, enhancing their synchronization capabilities.
- Enhanced the `apply_state_change` method to leverage the generic mapping logic, simplifying the implementation for models with foreign keys.
- Introduced comprehensive documentation on the new FK mapping system and its usage across models.
2025-10-09 15:01:31 -07:00
Jamie Pine
28e3ee443d feat: Enhance sync functionality by adding sync fields to devices and implementing migration
- Added `sync_enabled` and `last_sync_at` fields to the device model to support synchronization capabilities.
- Created a migration script to update the devices table, consolidating sync-related data.
- Updated various components to ensure sync is enabled by default for new devices.
- Removed the obsolete sync partners table, streamlining the sync architecture.
- Introduced comprehensive sync integration tests to validate the new functionality and ensure robust performance.
2025-10-09 08:48:23 -07:00
Jamie Pine
10110fd710 feat: Implement automatic dependency resolution for syncable models
- Added `sync_depends_on` method to the Syncable trait, allowing models to declare their dependencies for synchronization.
- Updated device, location, entry, and tag models to specify their dependencies, facilitating automatic computation of sync order.
- Enhanced backfill process to respect model dependencies, preventing foreign key violations during synchronization.
- Improved documentation to reflect the new dependency graph and its benefits for model synchronization.
2025-10-09 07:28:45 -07:00