379 Commits

Author SHA1 Message Date
Jamie Pine
553fadd2d4 Switch to ts-client types from root package 2025-11-20 21:43:48 -08:00
Jamie Pine
63aa0f73cc Add KeyboardHandler component that wires into the Explorer keyboard -
Integrate KeyboardHandler into ExplorerLayout to enable global keys -
Memoize Explorer state flow (useCallback/useMemo) to reduce re-renders -
Propagate selection state through GridView/FileCard via new
SelectionContext props - Memoize TopBar using React.memo to prevent
unnecessary updates
2025-11-20 11:44:06 -08:00
Jamie Pine
d31ba54080 chore: update subproject commit and add blank lines for consistency
- Updated the subproject commit reference to indicate a dirty state.
- Added blank lines to various files for improved readability and consistency across the codebase.
2025-11-20 04:38:22 -08:00
Jamie Pine
ad63cba189 chore: update subproject commit and add blank lines for consistency
- Updated the subproject commit reference to indicate a dirty state.
- Added blank lines to various files for improved readability and consistency across the codebase.
2025-11-18 02:31:20 -08:00
Jamie Pine
c6cda350be feat: media proxy and thumbstrip functionality
- Introduced new modules and actions for generating media proxies and thumbstrips, improving media handling capabilities.
- Added configuration, error handling, and processing logic for both proxy and thumbstrip generation.
- Updated existing media operations to integrate new functionalities, ensuring seamless media processing and user experience.
- Enhanced tests to cover new features and ensure reliability in media operations.
2025-11-17 04:24:05 -08:00
Jamie Pine
91b6808c5e refactor: remove unused StateSyncHandler dead code
The StateSyncHandler was a leftover from an earlier architecture where
protocol handlers were meant to be separate. The current implementation
bypasses it entirely by having the network protocol handler directly
call PeerSync::get_device_state() which properly uses the registry.

Backfill has been working fine because it completely avoids this code path.

Changes:
- Remove StateSyncHandler struct and all its methods from protocol_handler.rs
- Remove state_handler field from BackfillManager
- Update exports in mod.rs
- Clean up imports in backfill.rs

This eliminates the confusing TODO comments about serialization that
were already implemented elsewhere.
2025-11-17 00:42:37 -08:00
Jamie Pine
e5c87ffda5 refactor: optimize entry filtering in database queries
- Replaced complex subquery logic with raw SQL for filtering entries based on device ownership, improving query performance and readability.
- Enhanced error handling for UUID synchronization to ensure data consistency.
- Streamlined code by consolidating delete and count operations in the entry closure rebuild process.
- Improved logging in the backfill manager to provide better insights into state changes and record processing.
2025-11-16 11:58:13 -08:00
Jamie Pine
8305d7d235 feat: working sync test, sick log analyzer, cleanup 2025-11-16 01:53:35 -08:00
Jamie Pine
c1680d866d feat: add support for image and audio metadata extraction, enhance media listing query with new sorting options, and update related types in TypeScript definitions 2025-11-13 01:22:34 -08:00
Jamie Pine
f43d2456da (chore): CI fix 2025-10-14 23:02:13 -07:00
Jamie Pine
68afdb9c51 cleanup 2025-10-11 09:00:58 -07:00
Jamie Pine
94a52dd2df cleanup 2025-10-11 08:48:11 -07:00
Jamie Pine
c77121aff8 The Spacedrive Collection 2025-10-11 07:16:16 -07:00
Jamie Pine
a5c9717dcb modularize photos extension and fix all errors 2025-10-11 05:32:47 -07:00
Jamie Pine
eb06c0a5cb feat: Introduce Spacedrive SDK API Reference and core modules
- Added a comprehensive API reference document for the Spacedrive SDK, detailing all APIs as type-checked stubs for type-checking purposes.
- Introduced new core modules including `actions`, `agent`, `ai`, `job_context`, `models`, `query`, `tasks`, and `vdfs`, each with defined structures and functionalities.
- Updated `Cargo.toml` to include new dependencies and ensure compatibility with the SDK.
- Enhanced `types.rs` to include additional error types and result types for better error handling across the SDK.
- Established a clear structure for extension development, allowing for future implementations and runtime testing.
2025-10-11 00:58:24 -07:00
Jamie Pine
63627cd777 feat: Enhance job macro argument parsing for improved flexibility
- Introduced a new `JobArgs` struct to handle parsing of job macro arguments more effectively.
- Updated the `job_impl` function to utilize the new parsing logic, allowing for both named arguments and direct string literals.
- Improved error handling and clarity in argument parsing, enhancing the overall robustness of the job macro implementation.
2025-10-09 05:48:00 -07:00
Jamie Pine
4ef5d59ebe feat: Add library validation and selection logic to Context
- Implemented `validate_and_fix_library` method in the Context struct to ensure the current library selection is valid.
- If the stored library ID is not found, the method selects the first available library.
- Updated the main execution flow to call this new method, enhancing library management during command execution.
2025-10-09 05:45:47 -07:00
Jamie Pine
705249019c feat: Implement job registration system for WASM extensions
- Introduced a new `ExtensionJobRegistry` to manage custom job types for extensions at runtime.
- Added `host_register_job` function to facilitate job registration from WASM extensions.
- Updated `PluginEnv` and `PluginManager` to include job registry functionality.
- Enhanced the `job` macro to support automatic job registration during plugin initialization.
- Updated documentation and tests to reflect the new job registration capabilities.
2025-10-09 05:18:35 -07:00
Jamie Pine
741bb7ce0d refactor: Rename job macros and update documentation
- Renamed `spacedrive_job` macro to `job` for clarity and consistency across the SDK.
- Updated references in the SDK and test extension to reflect the new macro name.
- Enhanced comments in the test extension to better describe functionality and usage.
- Added "checkpointing" to the cspell project words for improved spell checking.
2025-10-09 04:07:12 -07:00
Jamie Pine
49c3ff9f5e feat: Introduce Spacedrive SDK for building WASM extensions
- Added a new SDK for developing Spacedrive WASM extensions, featuring a type-safe API and comprehensive documentation.
- Implemented core components including job context management, FFI bindings, and error handling mechanisms.
- Created macros for simplifying extension and job definitions, enhancing developer experience.
- Included detailed README and status documentation to guide users in utilizing the SDK effectively.
- Updated Cargo configurations for the SDK and its macros, ensuring proper dependency management and build processes.
2025-10-09 04:00:12 -07:00
Jamie Pine
4ade103d38 feat: Complete implementation of leaderless hybrid sync system
- Finalized the core synchronization infrastructure, enabling device-independent writes without leader checks.
- Implemented critical components including `TransactionManager`, `SyncProtocolHandler`, and `PeerSync` with enhanced broadcast capabilities.
- Achieved full message routing for state and shared changes, ensuring robust error handling and logging.
- Registered new models in the async-safe registry, facilitating state-based and log-based synchronization.
- Documented implementation progress and architecture changes, highlighting the transition to a leaderless model.
- Prepared for integration testing with remaining tasks outlined for backfill and retry queue implementation.
2025-10-09 00:03:27 -07:00
Jamie Pine
95295b8303 refactor: Transition to leaderless hybrid sync architecture
- Removed leadership-related components from the sync infrastructure, including `LeadershipManager` and `sync_leadership` fields across various models.
- Implemented a new peer-to-peer sync model utilizing Hybrid Logical Clocks (HLC) for shared resources and state-based sync for device-owned data.
- Updated the `Syncable` trait and related modules to reflect the new architecture, ensuring seamless integration of state and log-based synchronization.
- Introduced `PeerLog` for managing device-specific changes and `PeerSync` for handling synchronization in the leaderless environment.
- Revised documentation to outline the new sync architecture and its implications for device synchronization, emphasizing the benefits of a leaderless approach.
2025-10-08 19:18:23 -07:00
Jamie Pine
910dce67f5 feat: Add new documentation and enhance CLI functionality
- Introduced three new markdown files: CLI_LIBRARY_SYNC_COMPLETE.md, IMPLEMENTATION_COMPLETE.md, and LIBRARY_SYNC_SETUP_IMPLEMENTATION.md for comprehensive documentation.
- Updated various CLI domain modules to improve argument handling and output formatting.
- Enhanced device, index, job, library, location, network, and search modules for better integration and user experience.
- Refactored code across multiple domains to improve maintainability and clarity.
2025-10-04 21:31:47 -07:00
Jamie Pine
b4dd9a9454 refactor: Update event handling and type definitions in Swift client
- Removed manual type definitions in favor of generated types from SpacedriveClient, enhancing type safety.
- Refactored event handling in DaemonConnector to utilize a new type-safe Event enum for better clarity and maintainability.
- Updated job management logic to align with the new event structure, improving real-time job tracking and state management.
- Enhanced serialization and deserialization processes for various data structures to ensure compatibility with the Rust daemon output.
- Removed obsolete schema generation binary and updated Cargo.toml to reflect changes in dependencies and project structure.
2025-09-22 00:06:06 -07:00
Jamie Pine
ae19c0cd97 refactor: Update type handling and schema generation for Swift client
- Removed the `codegen` module and its associated files to streamline the codebase.
- Introduced a new binary for generating Swift types using Specta, enhancing type safety and integration.
- Updated various data structures to utilize the `specta` crate for type generation.
- Refactored existing types to remove `JsonSchema` derives in favor of `specta::Type`, improving compatibility with the new generation system.
- Enhanced the `Cargo.toml` to exclude unnecessary crates and improve workspace organization.
2025-09-21 19:13:38 -07:00
Jamie Pine
6e25fbfe41 feat: Introduce persistence completion handling in job management
- Added a new `persistence_complete_tx` channel to facilitate signaling when job state persistence is complete.
- Updated `JobExecutor` and `JobManager` to manage the persistence completion channel, ensuring proper signaling during job execution and state management.
- Enhanced the `ChangeDetector` to include a cache for file existence checks, improving performance and accuracy in detecting changes.
- Refactored related methods to streamline job state handling and improve logging for persistence operations.
2025-09-20 17:40:29 -07:00
Jamie Pine
f83ac9f829 refactor: Clean up emoji handling in Rust files and improve output formatting
- Refactored `remove_emojis.sh` to enhance the emoji removal process by using a more efficient Python script for detecting and removing 3D/colorful emojis followed by spaces.
- Updated various Rust files to remove emoji characters from log messages and print statements, resulting in a cleaner output.
- Improved user experience with color-coded progress and results during the emoji removal process.
- Ensured that the script counts processed files and reports modifications accurately.
2025-09-19 18:40:19 -07:00
Jamie Pine
ecdcf04066 Refactor logging output to remove emojis and improve consistency
- Updated various print statements across the codebase to remove emoji indicators, ensuring a more uniform logging style.
- Adjusted output messages in CLI, job handling, and event processing to enhance clarity and maintainability.
- Refactored job-related logging to provide clearer status updates without emoji clutter.

Co-authored-by: ijamespine <ijamespine@me.com>
2025-09-18 22:44:10 -07:00
Jamie Pine
469fc7cbe2 refactor: complete migration to modular action system and remove ActionOutput enum
- Eliminated the centralized `ActionOutput` enum, achieving true modularity in action handling.
- Introduced the `ActionType` trait, allowing each action to define its own output type directly.
- Updated multiple actions to utilize native output types, enhancing type safety and reducing complexity.
- Enhanced the `ActionManager` with a generic `dispatch_action` method for improved action execution.
- Preserved existing infrastructure for validation, audit logging, and error handling throughout the migration.

These changes significantly improve the modularity and maintainability of the action system while ensuring a consistent API surface.
2025-09-09 02:03:17 -04:00
Jamie Pine
d90b486865 refactor: update project dependencies and remove unused components
- Removed the `prisma-client-rust` and `prisma-client-rust-sdk` dependencies from `Cargo.toml` to streamline the project.
- Deleted the `spacedrive-jobs-derive` crate, consolidating job registration functionality into the main job system.
- Updated the `.gitattributes` file to reflect changes in generated code tracking.
- Adjusted the `package.json` scripts for improved build processes and removed unnecessary commands.
- Enhanced the `combine.sh` script to correctly reference documentation paths.

These changes aim to simplify the project structure and improve maintainability.
2025-09-08 17:26:31 -04:00
Jamie Pine
a8a2c732ef more refactor 2025-09-07 01:43:57 -04:00
Jamie Pine
29a87a91ce huge refactor 2025-09-06 21:00:37 -04:00
Arnab Chakraborty
dcb78f079e Working Sync & Thumbnail Sync (#2875)
* First draft on new p2p design

* Test sync

* Sync works

* Update version

* Fix merge conflicts

* More stuff for thumbnail sync

* Get the thumbnail

* Update mod.rs

* Working thumbnail sync

* Format

* Update thumbnails.rs

* Update runner.rs

* Update runner.rs

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2025-03-20 07:34:59 +03:00
Julian Braha
d51f4eb14d Optimize Option<Flash>::from(FlashValue) using bitwise AND (#2843) 2025-02-12 01:25:08 +03:00
Arnab Chakraborty
58e7151e19 Move Auth Tokens storage from Local Storage to custom encrypted .sdks file (#2831)
* Initial encryption functions

* Storage of encrypted tokens & working login

* Add customized `tauri-plugin-cors-fetch`

We need to have it in our crates folder due to the dependency of `sd-crypto` for the handling of cookies.

* Lint
2024-12-14 19:49:50 -05:00
Ericson Soares
dd56820898 Rustfmt 2024-12-10 12:37:14 -03:00
0xBA5E64
d4f40e5cf0 Update CONTRIBUTING.md (#2820)
* docs: Remove extranious `clean` commands, `pnpm clean` already runs all three.

* docs: Update recommended Rust version to reflect current `rust-toolchain.toml`

* docs: Refactor the Desktop clients developer setup guides

* docs: Refactor the Troubleshooting section for developer setup

* docs: `dev:web` is deprecated, according to @Rocky43007

* docs: Specify that the Tauri app doesn't play with `sd-server`

* docs: Nevermind! `dev:web` is still on the menu!

* docs: GitHub.md "Alerts" don't like indentation.

* fix: "You're just like the team now, just committing random fixes to whatever branch" - @rocky43007

* Apply suggestions from code review

Thanks Lynx!

Co-authored-by: Lynx <141365347+iLynxcat@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Lynx <141365347+iLynxcat@users.noreply.github.com>

* docs: Completely rewritten Mobile App section in `CONTRIBUTING.md`

* docs: Polish polish polish

* docs: Poleish?

* docs: Trimming & Trimming

* docs: Add final step clarification for running Android app

* docs: Try fixing the tip line embed

* Update CONTRIBUTING.md

Co-authored-by: Philipp <34287258+Phippe@users.noreply.github.com>

* Fix minor grammar & spelling mistakes

* Fix Github Docs Contributing link

---------

Co-authored-by: Lynx <141365347+iLynxcat@users.noreply.github.com>
Co-authored-by: Philipp <34287258+Phippe@users.noreply.github.com>
Co-authored-by: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
2024-11-30 16:38:37 -05:00
Ericson Soares
31f954f38f Optimize and fix attempt at cloud ingester 2024-10-21 22:27:09 -03:00
Vítor Vasconcellos
eb098ebec2 Update iroh and sd-cloud-schema
- Fix usage of deprecated functions
 - Implement clippy beta suggestions
2024-10-21 19:44:08 -03:00
Vítor Vasconcellos
7e7dd8fbde Merge remote-tracking branch 'origin/main' into eng-1828-migration-to-new-cloud-api-system 2024-10-21 14:40:59 -03:00
Vítor Vasconcellos
cc7c9d5793 Improve file thumbnails and Quick Preview (+ some code clean-up and rust deps update) (#2758)
* Update rspc, prisma-client-rust, axum and tanstack-query
 - Deleted some unused examples and fully commented out frontend code
 - Implement many changes required due to the updates
 - Update most rust dependencies

* Re-enable p2p

* Fix server

* Auto format

* Fix injected script format
 - Update some github actions
 - Update pnpm lock file

* Fix devtools showing up when app opens
 - Fix million complaining about Sparkles component

* Fix sd-server

* Fix and improve thumbnails rendering
 - Fix core always saying a new thumbnail was generated even for files that it skiped thumbnail generation
 - Rewrite FileThumb and improve related components

* Ignore tmp files when running prettier

* Improve FileThumb component performance
 - Rework useExplorerDraggable and useExplorerItemData hooks due to reduce unecessary re-renders

* More fixes for thumb component
 - A couple of minor performance improvements to frontend code

* auto format

* Fix Thumbnail and QuickPreview

* Fix logic for when to show 'fail to load original' error message in QuickPreview
 - Updated prisma-client-rust, libp2p, tauri, tauri-specta, rspc and hyper

* Fix type checking
 - Format scripts

* Add script prettier config

* Fix serde missing feature
 - Use rust-libp2p spacedrive fork again
 - Update rspc

* Autoformat + fix pnpm lock

* Fix thumbnail first load again

* Autoformat

* autoformat

* Fix rust-libp2p fork url again?

* Remove usePathsInfiniteQuery hook

* Update tauri 2.0.6
2024-10-21 15:47:40 +00:00
Ericson Soares
f414e549f8 Some tweaks on sync messages send and compression 2024-10-20 22:26:06 -03:00
Ericson Soares
6a69f5f8ef Update sync messages push and pull
Also fix scalability issues on sync design
2024-10-19 02:48:27 -03:00
Ericson Soares
2bb06d6317 Many tweaks to improve sync responsiveness and observability 2024-10-14 23:39:52 -03:00
James Pine
0ac05f46b3 increase max file size for thubm generation 2024-10-12 00:59:27 -07:00
Ericson Soares
375c111305 Some tweaks on sync messages encryption and decryption 2024-10-09 22:04:49 -03:00
Ericson Soares
3208ef5dd2 Change database to use device id on everything instead of pub_id 2024-10-07 22:38:45 -03:00
Vítor Vasconcellos
a223fa0a8b Merge remote-tracking branch 'origin/main' into eng-1828-migration-to-new-cloud-api-system 2024-10-06 23:36:06 -03:00
Vítor Vasconcellos
cdc50140b0 [ENG-1928] Update to tauri 2.0.1 stable (#2752)
* Update to tauri 2.0.0 stable
 - Update a some othe dependencies
 - Autoformat rust and toml files

* Fix clippy complain

* Update tauri 2.0.1
 - Update rspc
 - Update some frontend files due changes in rspc
 - Update some frontend build dependencies

* Revert babel update

* Fix code typing typing and style + a couple of bugs

* More type fixes
2024-10-05 21:17:30 +00:00
Arnab Chakraborty
2c0d94abc4 Merge branch 'main' into eng-1828-migration-to-new-cloud-api-system 2024-10-03 22:23:31 -04:00