- Introduce PeerWatermarkStore to track max HLC received per peer for shared resource incremental sync - Initialize the peer_received_watermarks table at startup - Persist received watermarks during backfill and after backfill completion - Query the max watermark across peers via PeerWatermarkStore - Expose the new module and export PeerWatermarkStore from sync
Spacedrive
A file manager built on a virtual distributed filesystem
spacedrive.com
·
Discord
·
Whitepaper
Spacedrive is an open source cross-platform file manager, powered by a virtual distributed filesystem (VDFS) written in Rust.
Organize files across multiple devices, clouds, and platforms from a single interface. Tag once, access everywhere. Never lose track of where your files are.
Note: This is Spacedrive v2 (2025)—a complete architectural rewrite. After three years of learning from v1's challenges, the entire system was rebuilt from scratch to deliver on the original vision: true cross-device file management with a working sync engine, P2P networking, and a sustainable architecture. If you're looking for the previous version, see the v1 branch.
The Problem
Your data lives everywhere—laptop, phone, NAS, external drives, Dropbox, Google Drive. Each location is its own silo. Finding files requires remembering which device you used last. Cloud services lock you into their ecosystems. Traditional file managers can't see beyond the local filesystem.
The Solution
Spacedrive creates a unified index of all your data, regardless of where it physically lives. A file on your Mac's SSD and the same file on your NAS backup are recognized as one piece of content. Tag it once, access it from anywhere. The files stay where they are—Spacedrive just makes them universally addressable.
Architecture
Spacedrive is built on four core principles:
1. Virtual Distributed Filesystem (VDFS)
Files and folders become first-class objects with rich metadata, independent of their physical location. Every file gets a universal address (SdPath) that works across devices. Content-aware addressing means you can reference files by what they contain, not just where they live.
2. Content Identity System
Adaptive hashing (BLAKE3 with strategic sampling for large files) creates a unique fingerprint for every piece of content. This enables:
- Deduplication: Recognize identical files across devices
- Redundancy tracking: Know where your backups are
- Content-based operations: "Copy this file from wherever it's available"
3. Transactional Actions
Every file operation can be previewed before execution. See exactly what will happen—space savings, conflicts, estimated time—then approve or cancel. Operations become durable jobs that survive network interruptions and device restarts.
4. Leaderless Sync
Peer-to-peer synchronization without central coordinators. Device-specific data (your filesystem index) uses state replication. Shared metadata (tags, ratings) uses a lightweight HLC-ordered log with deterministic conflict resolution. No leader election, no single point of failure.
Core Features
| Feature | Description |
|---|---|
| Cross-Platform | macOS, Windows, Linux, iOS, Android |
| Multi-Device Index | Unified view of files across all your devices |
| Content Addressing | Find optimal file copies automatically (local-first, then LAN, then cloud) |
| Smart Deduplication | Identify identical files regardless of name or location |
| Cloud Integration | Index S3, Google Drive, Dropbox as first-class volumes |
| P2P Networking | Direct device connections with automatic NAT traversal (Iroh + QUIC) |
| Semantic Tags | Graph-based tagging with hierarchies, aliases, and contextual disambiguation |
| Action Preview | Simulate any operation before execution |
| Offline-First | Full functionality without internet, syncs when devices reconnect |
| Local Backup | P2P backup between your own devices (iOS photo backup available now) |
| Extension System | WASM-based plugins for domain-specific functionality |
Tech Stack
Core
- Rust - Entire VDFS implementation (~183k lines)
- SQLite + SeaORM - Local-first database with type-safe queries
- Iroh - P2P networking with QUIC transport and hole-punching
- BLAKE3 - Fast cryptographic hashing for content identity
- WASM - Sandboxed extension runtime
Apps
- Swift - Native iOS/macOS with embedded Rust core via FFI
- React - Cross-platform desktop (Tauri) and web interface
Architecture Patterns
- Event-driven design with centralized EventBus
- CQRS: Actions (mutations) and Queries (reads) with preview-commit-verify
- Durable jobs with MessagePack serialization
- Domain-separated sync with clear data ownership boundaries
Project Structure
spacedrive/
├── core/ # Rust VDFS implementation
│ ├── src/
│ │ ├── domain/ # Core models (Entry, Library, Device, Tag)
│ │ ├── ops/ # CQRS operations (actions & queries)
│ │ ├── infra/ # Infrastructure (DB, events, jobs, sync)
│ │ ├── service/ # High-level services (network, file sharing)
│ │ ├── location/ # Location management and indexing
│ │ ├── library/ # Library lifecycle and operations
│ │ └── volume/ # Volume detection and fingerprinting
├── apps/
│ ├── cli/ # Rust CLI
│ ├── ios/ # Native Swift app
│ ├── macos/ # Native Swift app
│ └── desktop/ # Tauri desktop app
├── extensions/ # WASM extensions
├── crates/ # Shared Rust utilities
└── docs/ # Architecture documentation
Extensions
Spacedrive's WASM-based extension system enables specialized functionality while maintaining security and portability.
Professional Extensions
| Extension | Purpose | Key Features | Status |
|---|---|---|---|
| Photos | AI-powered photo management | Face recognition, place identification, moments, scene classification | In Progress |
| Chronicle | Research & knowledge management | Document analysis, knowledge graphs, AI summaries | In Progress |
| Atlas | Dynamic CRM & team knowledge | Runtime schemas, contact tracking, deal pipelines | In Progress |
| Studio | Digital asset management | Scene detection, transcription, proxy generation | Planned |
| Ledger | Financial intelligence | Receipt OCR, expense tracking, tax preparation | Planned |
| Guardian | Backup & redundancy monitoring | Content identity tracking, zero-redundancy alerts, smart backup suggestions | Planned |
| Cipher | Security & encryption | Password manager, file encryption, breach alerts | Planned |
Open Source Archive Extensions
| Extension | Purpose | Provides Data For | Status |
|---|---|---|---|
| Email Archive | Gmail/Outlook backup | Atlas, Ledger, Chronicle | Planned |
| Chrome History | Browsing history backup | Chronicle | Planned |
| Spotify Archive | Listening history | Analytics | Planned |
| GPS Tracker | Location timeline | Photos, Analytics | Planned |
| Tweet Archive | Twitter backup | Chronicle, Analytics | Planned |
| GitHub Tracker | Repository tracking | Chronicle | Planned |
Getting Started
Prerequisites
- Rust 1.81+ (rustup)
- Xcode (for iOS/macOS development)
Quick Start with CLI
# Clone the repository
git clone https://github.com/spacedriveapp/spacedrive
cd spacedrive
# Build and run the CLI
cargo run -p sd-cli -- --help
# Create a library
cargo run -p sd-cli -- library create "My Library"
# Add a location to index
cargo run -p sd-cli -- location add ~/Documents
# Search indexed files
cargo run -p sd-cli -- search .
iOS/macOS Development
# Open the iOS project
open apps/ios/Spacedrive.xcodeproj
# Or macOS project
open apps/macos/Spacedrive.xcodeproj
# Build from Xcode or command line
xcodebuild -project apps/ios/Spacedrive.xcodeproj -scheme Spacedrive
The Rust core is automatically compiled during Xcode builds.
Running Examples
# Run the indexing demo
cargo run --example indexing_demo
# Run file type detection demo
cargo run --example file_type_demo
# See all examples
ls core/examples/
Development Commands
# Run all tests
cargo test
# Run tests for specific package
cargo test -p sd-core
# Build CLI in release mode
cargo build -p sd-cli --release
# Format code
cargo fmt
# Run lints
cargo clippy
Privacy & Security
Spacedrive is local-first. Your data stays on your devices.
- End-to-End Encryption: All P2P traffic encrypted via QUIC/TLS
- At-Rest Encryption: Libraries can be encrypted on disk (SQLCipher)
- No Telemetry: Zero tracking or analytics in the open source version
- Self-Hostable: Run your own relay servers and cloud cores
- Data Sovereignty: You control where your data lives
Optional cloud integration (Spacedrive Cloud) is available for backup and remote access, but it's never required. The cloud service runs unmodified Spacedrive core as a standard P2P device—no special privileges, no custom APIs.
Documentation
- Whitepaper - Complete technical architecture
- Contributing Guide - How to contribute
- Architecture Docs - Detailed system design
- Extension SDK - Build your own extensions
Get Involved
- Star the repo to support the project
- Join Discord to chat with developers and community
- Read the Whitepaper for the full technical vision
- Build an Extension - Check out the SDK docs
Your files, unified. Your data, sovereign.
