397 Commits

Author SHA1 Message Date
Oscar Beaumont
9719940552 Typesafe stream handling (#603)
* typesafe stream handling
Now it's impossible for your to respond to a broadcast stream. Previously this would just fail due to the TCP connection having been closed by the sender.

* remove connected_peers state from `SpaceTime`

* use `OutEvent` instead of `tokio::spawn` for `SpaceTime` events

* Polling for events > tokio::mpsc

* extend max length of sync messages to ~4GB + broadcast done message

* Fix "actOS" operating system decode error
2023-03-13 16:58:35 +08:00
Ericson "Fogo" Soares
12a437ffb6 [ENG-371] Subpath shallow re indexing (#599)
* Introducing sub path indexing for IndexerJob

* Introducing shallow indexer job

* Some feature flags to avoid warnings at pnpm prep

* Many fixes at shallow indexer job

* Changing how we implement the bookkeeping for file_path ids
Now we're account for independent locations, and also integrating
better with the LibraryContext, instead of using a static global

* Making indexer job reentrant

* Introducing shallow file identifier job
And a bunch of minor refactors

* Rust fmt

* Removing light scan from getExplorerData query
Light scan is a mutation, so we can call it on useEffect function from
the Explorer component in the frontend, when a location_id or the explorer
path changes

* Handling job early finish on init for identifier

* Only invalidate query if we have orphan paths

* Introducing ShalowThumbnailerJob

* Clippy warnings about `into_iter()`

* Naming scheme for Prisma's selects and includes

* Invalidating getExplorerData at thumbnailer

* Small mistakes lol

* Some nitpicks with pnpm prep

* Rust fmt

* Changing indexer's walk log to `trace!`

* Not deleting all file_paths on location fullRescan

* TS typecheck

* Removing `file_path` selection just with id
2023-03-10 16:25:58 +00:00
Oscar Beaumont
7588442d5d Basic Spacedrop working with rspc! (#601) 2023-03-09 06:23:17 -08:00
Jamie Pine
93fecd6492 [ENG-408] Dynamic Object icons (#600)
* improved object icons

* remove redundant control flow

* added support for base icon overrides by extension
+ deleted old icons folder and moved official icons into there

* fixed folder icon error

* removed temp nodes example

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-03-09 11:08:10 +00:00
Oscar Beaumont
2518e98ad5 [ENG-407] Spacedrop backend (#598)
* format Rust

* Spacedrop a string

* Praise thee Clippy, lord of the Rust

* add protobuf to Mac and Linux CI

* plz GH Actions have Chocolatey
2023-03-09 11:27:45 +08:00
Oscar Beaumont
e696db186b P2P Core (#597)
* implement space time

* I love it when my code works
2023-03-09 03:18:33 +08:00
Brendan Allan
2a09c7214b generate + use ModelSyncData enum (#592)
* generate + use ModelSyncData enum

* cache prisma clients on sync-generator
2023-03-04 22:12:37 -08:00
Brendan Allan
2393a1f594 restrict all swift stuff to macos only (#590)
restrict all swift stuff to macos only
2023-03-03 21:29:52 -08:00
Brendan Allan
9397aa617d Colocate native code + reorganise (#589)
colocate native code + all rust in crates/
2023-03-03 09:22:22 +00:00
Ericson "Fogo" Soares
ae41b1a0f4 Fix unique constraint on file path table (#585)
* Complying with a pedantic Clippy

* Some task runners for vscode

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Damn rustfmt
2023-03-02 02:36:49 +00:00
Jamie Pine
fab84b7f7a [ENG-363] Spacedrop UI + Misc Improvements (#568)
* begin spacedrop ui + misc ui improvements

* better 404 xox

* Update extensions.rs

I think I prefer Container

* added DragRegion component,  ot tested cuz im on my fone

* Update DragRegion.tsx

fix import

* added dummy drop items

* better dummy data

* added clouds & search bar

* added action buttons to spacedrop items

* customize subtle button

* added support for apng, thanks luka big pants

* use relative path in sidebar

* use BYTES const

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-24 08:12:21 +00:00
jake
d99e586766 [ENG-134] OSS Licenses and Projects (#575)
* add `deps-generator` crate

* move `clap` types to separate file

* idiomatic filtering

* experimentally update ci

* add *really* basic dependency page

* remove `license_id` from `License`

* add bare minimum JSON files

* Revert "experimentally update ci"

This reverts commit c04897d902c9a55baf14d532f4590bc711f08c16.

* re-insert comments

* Using reqwest blocking feature to avoid asyncness

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-20 05:58:55 +00:00
jake
c21743f44e [ENG-361] Crypto crate docs and tests (#572)
* add hashing tests

* add encryption/decryption tests

* remove `reason`

* add file header tests (preview media deserialization is broken)

* fix keyslot reading bug

* add sd-crypto testing to ci

* add tests/constants for all hashing algorthms and param levels

* add blake3-kdf tests

* use `const` arrays for storing expected output

* test for `5MiB` `encrypt_streams` and `decrypt_streams`

* add invalid/mismatched nonce tests

* update `primitives` docs

* remove erroneous `,`

* grammar tweaks

* add errors to `#[should_panic]`

* cleanup `stream` tests

* cleanup hashing tests a little

* function docs
2023-02-16 11:42:30 +00:00
Utku
adb322b9f3 Tailwind Prettier Plugin (#557)
* add tailwind prettier plugin + run prettier

* add eslint tailwind plugin and eslint fix to turbo

* Ran eslint fix + fixed every other lint issues

* twStyle and lint stuff

* remove auto lint as it's buggy

* update eslint & add twStyle to tailwind regex

* Fix auto lint + config inconsistencies

* Add tailwind config to prettier config

* run lint:fix

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-16 07:04:19 +00:00
Oscar Beaumont
60d5bc4afe improvements to spacedrive:// custom URI protocol (#550)
* fix `spacedrive://` custom protocol on Windows
(hopefully)

* custom protocol using `http::Response` + fix broken web

* import patches before App on web

* use `http::Request` for input to `handle_custom_uri`

* break  into dedicated file + error handling

* serving files via custom protocol

* cargo fmt because vscode did cringe

* lru cache to reduce video chunk request time

* add  helper to JS

* clippy be like

* remove duplicate Open buttons in context menu

* fix Linux 🙏

* no shot

* fix Windows custom URI passing (hopefully)

* better fix for custom uri on Linux

* upgrade Tauri for  feature

* switch url replacement order

* prevent React dev tools script being added in prod to desktop

* remove React devtools from html

* upgrade Tauri; required upgrading rspc, Axum, PCR

* pass typecheck + less cringe bigint

* clippy is love, clippy is life

* Typecheck plz

* fix bigint to number conversion

* use httpz + localhost server for Linux

* clippy be right

* Remove console.log

* [wip] proper auth

* fix Linux sidebar padding

* Secure Axum server with random

* Extracting app setup specific to linux to a different file

* remove outdated comment

* Some tweaks on cursom_uri.rs

* file_path_with_location doesn't need to be a named include

* fix typo

* factually wrong comment

* Change `unwrap` to `expect`

* bruh

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-14 13:27:11 +08:00
Jamie Pine
c07a42b7c8 [ENG 239] Onboarding Flow & Location Settings (#529)
* begin better onboarding

* added input and altered text

* better router & text + database icon

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* work on privacy screen + radio buttons

* fix video extension bug and alter screens

* add pending schema and location manager helper

* functional onboarding

* added secure temp store and started creating library loading screen

* fix secure temp keystore + api

* better onboarding

* added location settings and some overview concept, all WIP

* fix switch

* prep

* fix location router

* added backend settings

* attempted to fix form

* begin indexer rules editor, plus tweaks

* indexer rules coming soon

* fix onboarding img size

* cleanup

* clone is needed here, but clippy no like

* sike

* whole bunch of fixes

* clippy + ts

* Removing some TODOs from api/libraries.rs and fixing db size calculation

* moved object kind to client, added half functionality for appearance settings

* fix RadioGroup helper

* fix type issues

* cargo fmt

* fix creating library error handling + invalidate location list on update

* forgot to switch back to onError

* Invalidating getStatistics query on library creation and introducing the concept of waiting for a job on FileCopierJob

* F* cargo fmt

* fix RadioGroup interactivity

* wipe all migrations

* put back COLLATE NOCASE on extension columns

* update core.ts

* remove unused device component

* fix typeerror in mobile

---------

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-02-10 14:08:13 -08:00
jake
08059ae700 [ENG-358] Malicious client protection for the KM (#562)
* add `queue_check`, `mount_check` and `unlock_check`

* make `unlock_check` marginally more readable

* clippy

* clean up & rename functions

* remove unlock check on `keys.list`

* use american spelling of `favorite`

* use `then_some` (thanks clippy)

* more american spelling
2023-02-09 09:30:26 +00:00
jake
e5b45d8f49 [ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
Brendan Allan
174c8e3ef7 Merge pull request #553 from spacedriveapp/non-unique-objects
Store CAS ids on file paths
2023-02-01 12:11:03 +08:00
jake
d97ebc4a67 [ENG-341] OS Keychains (#554)
* super barebones keychain start

* working+unfinished linux keychain

* add `copy` to `Identifier` and add `delete()`

* add generic errors + code cleanup

* cleanup code & add support for apple keychains

* remove `users` dep

* use uppercase UUID and tweak apple `account` variable

* revert uppercase change and clean up linux labels

* code cleanup & add a readme to the crypto crate

* remove useless `map`

* correctly handle keyring not supported errors

* add `Send` to `Box<dyn Keyring>`

* remove useless `cfg`s

* only return `Ok()` if we support keychains (pls work CI)

* clippy

* Minor fixes

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-02-01 00:11:03 +00:00
Ericson Fogo Soares
c1944005b7 A bunch of minor fixes 2023-01-31 20:02:23 -03:00
Brendan Allan
d91bb9aa1d Merge branch 'main' into non-unique-objects 2023-01-30 12:11:35 +08:00
Brendan Allan
224ed772ac store cas ids on file paths 2023-01-28 11:42:24 +08:00
brxken128
c26f2edd92 make requested changes 2023-01-26 10:39:18 +00:00
brxken128
b86ec247c8 update things to use workspace tokio 2023-01-25 22:03:10 +00:00
brxken128
253739c83e Merge branch 'main' into eng-348-async-crypto 2023-01-25 22:02:23 +00:00
brxken128
9fbc9f83ed use a tokio::sync::Mutex just to be safe 2023-01-25 18:20:29 +00:00
brxken128
f6845d7927 tell everything to use workspace tokio + add tokio git patch 2023-01-25 14:36:29 +00:00
brxken128
5bd86f5d60 change fs and macros tokio features to dev-deps 2023-01-25 14:03:00 +00:00
brxken128
db9b999f94 add correct features to tokio dep 2023-01-25 12:47:26 +00:00
Brendan Allan
6a71936252 update shared sync functions to use new system 2023-01-25 13:17:01 +08:00
brxken128
c641b03b2f fix stream reading due to read()
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-01-24 23:28:16 +00:00
brxken128
ec6fbed75e bring in Salt, Key and EncryptedKey type aliases 2023-01-24 17:01:50 +00:00
brxken128
d41a759058 add ProtectedVec type alias 2023-01-24 16:47:41 +00:00
brxken128
74f1a5268b clippy 2023-01-24 16:35:31 +00:00
brxken128
ab75e426a7 disable benchmarks as they need async support 2023-01-24 14:25:42 +00:00
brxken128
04b17a6e54 update examples 2023-01-24 14:25:19 +00:00
brxken128
73dd4df26b async crypto! 2023-01-24 14:19:27 +00:00
brxken128
7f5396249a return to match/for statements for async iterators (until rust gets better support) 2023-01-24 14:04:18 +00:00
brxken128
fe2538a603 more async crypto 2023-01-24 13:50:46 +00:00
brxken128
d4b9dcaada mostly async keymanager 2023-01-24 13:16:47 +00:00
brxken128
48622104f5 async stream encryption and decryption 2023-01-24 12:49:54 +00:00
brxken128
766564afaa Merge branch 'main' into eng-345-cut-copy-duplicate-actions 2023-01-23 11:03:27 +00:00
Ericson Fogo Soares
7aa644c7a2 Removing some unnecessary clones and other small fixes 2023-01-20 18:22:17 -03:00
Brendan Allan
87286dba13 Refactor routes (#535)
* refactor routes

* use default export for screens

* use "route" relative routing

* don't use absolute import for root screens

* no absolute importing folders!!!

* fine we can absolute import folders
2023-01-20 12:40:36 -08:00
Brendan Allan
9c0c870964 Fix webkit reload (#534)
* wrap WKWebView in UnsafePointer

* formatting

* more formatting
2023-01-20 00:25:12 -08:00
maxichrome
1b42826056 cleanup(don't hate me): prettier format everything 2023-01-19 19:32:35 -08:00
jake
aa02720460 [ENG-257] File erasure and deletion (#524)
* add helper functions for getting context-menu related filesystem info

* update comments and such

* add a barebones delete job

* add file deletion route

* add (empty but working) delete file dialog

* add super experimental file erase function

* update function doc

* add erase files route

* add erase files job

* tweak file erasing in crypto crate

* bindings

* add erase files dialog

* add missing `?`

* fix erase file form trickery

* remove unused imports

* terrible attempt at recursion for file erase

* IT WORKS

* code cleanup + clippy

* async just dropped (not for erase yet)

* async erase just dropped (+async crypto crate)

* add "nice data" for jobs
2023-01-18 19:55:14 -08:00
jake
92a7e66755 [ENG-328] Keymanager mounting queue (#514)
* add `get_queue` and `is_queued` to KM and router

* mark keys as being mounted/finished mounting, use a `DashSet` instead of vec

also includes temporary `dbg!()`s for manual valdiation that it works

* remove dbg and fix build (works great)

* mark key manager as unlocking correctly

* cleanup keys api

* add `KeyAlreadyQueued` error

* use react mounting queue

* fully prevent against multiple mount jobs of the same key with the rust queue

* show keymanager as unlocking in the UI

* handle query invalidation for `isKeyManagerUnlocking` with a closure (semi-temporary)

* revert changing text (doesn't work where the keymanager is unlocked from, only the other unlock screen)

* clippy

* add `useRef` for mounting queue within react

* pass `ref.current` to `Key` instead
2023-01-17 06:19:54 -08:00
Brendan Allan
80f85d6f7c Eng 332 auto generate sync ids (#523)
* generate sync ids and use model name from ModelActions

* cleanup AddLocationDialog imports

* rustfmt
2023-01-15 07:25:11 -08:00