58 Commits

Author SHA1 Message Date
Arnab Chakraborty
57750a96ee Update setup instructions to require Bun over pnpm
Replaces pnpm with Bun as the required JavaScript package manager in setup.sh and updates CONTRIBUTING.md to reflect Bun as a prerequisite for Tauri desktop app development. Adds detailed setup steps and troubleshooting for the Tauri app, clarifies dependency installation, and updates migration workflow documentation.
2025-12-01 19:53:32 -05:00
Jamie Pine
2ef179d4df chore: update Zustand dependency to version 5.0.8, modify Cargo.toml for sd-core features, and enhance sidecar manager initialization in library management 2025-11-11 15:44:00 -08:00
Jamie Pine
c8e598b602 Add cloud variant to SdPath 2025-10-13 03:07:13 -07:00
Jamie Pine
68afdb9c51 cleanup 2025-10-11 09:00:58 -07:00
Jamie Pine
520703a3a0 chore: Update Cargo.toml and remove obsolete files
- Added "xtask" to the Cargo.toml members for improved task management.
- Removed outdated documentation files: CLI_LIBRARY_SYNC_COMPLETE.md, DEVICE_DISCONNECTION_FIX.md, FINAL_SUMMARY.md, IMPLEMENTATION_COMPLETE.md, INFINITE_LOOP_FIX.md, LIBRARY_SYNC_SETUP_IMPLEMENTATION.md, PAIRING_CONNECTION_ISSUE_SUMMARY.md, PAIRING_IMPLEMENTATION.md, PERSISTENT_CONNECTION_FIX.md.
- Cleaned up the project structure to enhance maintainability and clarity.
2025-10-06 20:56:54 -07:00
Jamie Pine
3925dac4f6 refactor: Remove Deprecated Build Scripts and Enhance Core Initialization 2025-09-29 14:36:13 -07:00
Jamie Pine
43c77e52d5 chore: Update iOS Core Dependencies and Build Process 2025-09-29 14:36:09 -07:00
Jamie Pine
54d36ab3f7 refactor: Update Embedded Core Management and Library Handling 2025-09-29 14:36:03 -07:00
Jamie Pine
902eca3f26 feat: Introduce iOS Core and Enhance Backup Functionality
- Added a new iOS core module with Rust integration for improved performance and background processing capabilities.
- Implemented essential components including `BackupManager`, `EmbeddedCoreManager`, and `PhotoManager` to facilitate automatic photo backup and management.
- Developed a comprehensive architecture design document outlining the new iOS implementation's advantages over the previous React Native version.
- Created various Swift files for UI components and services, enhancing the overall structure and maintainability of the iOS app.
- Included build scripts for easier compilation and integration of the iOS core into the project.
2025-09-27 22:24:58 -07:00
Jamie Pine
79728fdb10 feat: Enhance Library Management and UI Components
- Introduced a new `LibraryCard` and `EmptyLibraryCard` components for improved library selection UI in the macOS app.
- Updated `LibrarySelector` to display current library information and allow users to switch libraries through a new button.
- Enhanced `MenuBarManager` to dynamically update the libraries menu based on available libraries and current selection.
- Refactored `DaemonConnector` to handle library-related events and refresh library statistics accordingly.
- Improved `LibraryInfo` structure to include optional statistics, enhancing data representation for libraries.
- Updated various UI components to utilize new library statistics, providing users with better insights into their libraries.
2025-09-25 05:29:09 -07:00
Jamie Pine
ebb5b59c88 chore: Remove emoji handling script and update documentation
- Deleted `remove_emojis.sh` as it is no longer needed.
- Updated `AGENTS.md` to clarify that emojis should not be used at all in logs, enhancing code style guidelines.
2025-09-19 18:47:30 -07:00
Arnab Chakraborty
35633ab5c3 Change spacedrive.desktop to Spacedrive.desktop 2025-03-03 05:03:27 +03:00
Arnab Chakraborty
17467f1a2b Fix script docs 2025-01-01 14:25:26 +03:00
Arnab Chakraborty
bcf82614e6 Scripts switching prod and dev service endpoints
A script that switches the prod and dev endpoints for the auth & relay services.
2025-01-01 13:57:01 +03:00
Lynx
90db32b1c7 add node-gyp dependency to scripts
this fixes "ELIFECYCLE Command failed." sometimes appearing on pnpm install if node-gyp isn't globally installed
2024-11-05 02:58:19 -08:00
Leora
be975996b0 Add Solus Build Support (#2781)
* Add Solus Build Support

* New Solus Linux Build Support

* Add Solus experimental warning
2024-10-27 23:44:26 +00:00
Vítor Vasconcellos
cc7c9d5793 Improve file thumbnails and Quick Preview (+ some code clean-up and rust deps update) (#2758)
* Update rspc, prisma-client-rust, axum and tanstack-query
 - Deleted some unused examples and fully commented out frontend code
 - Implement many changes required due to the updates
 - Update most rust dependencies

* Re-enable p2p

* Fix server

* Auto format

* Fix injected script format
 - Update some github actions
 - Update pnpm lock file

* Fix devtools showing up when app opens
 - Fix million complaining about Sparkles component

* Fix sd-server

* Fix and improve thumbnails rendering
 - Fix core always saying a new thumbnail was generated even for files that it skiped thumbnail generation
 - Rewrite FileThumb and improve related components

* Ignore tmp files when running prettier

* Improve FileThumb component performance
 - Rework useExplorerDraggable and useExplorerItemData hooks due to reduce unecessary re-renders

* More fixes for thumb component
 - A couple of minor performance improvements to frontend code

* auto format

* Fix Thumbnail and QuickPreview

* Fix logic for when to show 'fail to load original' error message in QuickPreview
 - Updated prisma-client-rust, libp2p, tauri, tauri-specta, rspc and hyper

* Fix type checking
 - Format scripts

* Add script prettier config

* Fix serde missing feature
 - Use rust-libp2p spacedrive fork again
 - Update rspc

* Autoformat + fix pnpm lock

* Fix thumbnail first load again

* Autoformat

* autoformat

* Fix rust-libp2p fork url again?

* Remove usePathsInfiniteQuery hook

* Update tauri 2.0.6
2024-10-21 15:47:40 +00:00
Vítor Vasconcellos
cdc50140b0 [ENG-1928] Update to tauri 2.0.1 stable (#2752)
* Update to tauri 2.0.0 stable
 - Update a some othe dependencies
 - Autoformat rust and toml files

* Fix clippy complain

* Update tauri 2.0.1
 - Update rspc
 - Update some frontend files due changes in rspc
 - Update some frontend build dependencies

* Revert babel update

* Fix code typing typing and style + a couple of bugs

* More type fixes
2024-10-05 21:17:30 +00:00
Vítor Vasconcellos
b4d1a295ff [ENG-1840, ENG-1842] Add native dependencies for iOS (#2693)
* Implement dowload of mobile native deps
 - Add a spinner animation to `pnpm prep`
 - Change abandoned dependency @iarna/toml with smol-toml
 - Validate cargo config.toml after generating it from mustache template
 - Disabled HTTP2 downloads with udici, it is very broken

* Initial ios native deps xcframework logic

* Remove logic for handling dynamic iOS libs, using static libs now

* Fix PATH in build-rust.sh
 - Remove app.json

* Restore crates/images/src/pdf.rs

* minor fix .editorconfig

* Finally ios successfully compiles with ffmpeg enabled
 - Change SDCore.podspec to add extra libraries required by ffmpeg
 - Fix heif linking for ios in config.toml template
 - Add symlink logic for extra libraries required to compile ios in build-rust.sh
2024-09-17 02:27:42 +00:00
Vítor Vasconcellos
287a809de0 Fix release workflow (#2668)
* Fix frontend bundle upload
 - Update pnpm

* Revert incorrect removal of fontenf packaging job

* Fix tauri not building updater bundles

* Ensure release action is executed when PRs modify the publish-artifacts action

* Fix unused argument for patchTauri function

* Couple for format fixes

* tauri requires building the app bundle to build the updater
2024-08-17 08:29:32 +00:00
Vítor Vasconcellos
3675675d31 Fix prisma generate & Windows compilation (#2663)
* Ensure prisma generation works before continuing CI
 - Add script that list all dependencies with multiple version in Cargo.lock

* Ensure CI runs when custom actions are modified

* Rever windows change in hopes that it fixes prisma generation

* Fix prisma generation check in CI

* Update deps

* Replace prisma-client-rust-cli with oficial prisma cli to fix windows model generation
 - Revert back to windows-rs implementation of get_inode in file-path-helper
 - Adjust Github CI to use oficial prisma cli
 - Update scripts dependencies
 - Fix some new rust warnings due to it trying to compile OS specific code on all plataforms

* Fix windows-rs impl for get_inode_windows in file-path-helper

* More fixes to windows-rs impl for get_inode_windows in file-path-helper

* More fixes to windows-rs impl for get_inode_windows in file-path-helper

* More fixes to windows-rs impl for get_inode_windows in file-path-helper

* More fixes to windows-rs impl for get_inode_windows in file-path-helper

* More fixes to windows-rs impl for get_inode_windows in file-path-helper

* Import windows constant intead of hardcoding its value

* Fix check for prisma files in setup-rust

* Autoformat

* Fix iOS and Android targets in watcher

---------

Co-authored-by: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
2024-08-16 10:55:38 +00:00
Arnab Chakraborty
8cfbd374e3 [ENG-1854] Migrate to Tauri v2 Release Candidate (#2654)
* Migrate to Tauri v2 Release Canidate

* Formatting

* Update pnpm

* Update all tauri deps (js and rust) to Release Candidate 2
 - A lot of misc fixes due to changes in the new versions of tauri, rspc and specta

* Recreate pnpm-lock

* Fix pnpm-lock

* Formatting

* Use 10 chars SHA for git deps
 - Update a couple of rust dependencies
 - Add taplo for toml auto formatting
 - Adjust some formatting configurations
 - Updated prettier and its plugins

* Fix formatter settings

* Minor improvements to Cargo.toml format

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-08-14 18:22:46 +00:00
Vítor Vasconcellos
06aa7181d8 [ENG-1838] Fix Debian executable name (#2648)
Fix .desktop incorrect executable name
2024-07-30 23:32:08 +00:00
Vítor Vasconcellos
1813bef30c Fix inconsistencies and update project's gitignore (#2642)
* Improve gitignore

* Update pnpm
2024-07-28 11:25:35 +00:00
leo
da837d450f Change openssl-dev dependency to openssh-devel (#2630)
fix: changes openssl-dev to openssl-devel, the former isn't available

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-07-23 20:06:52 +00:00
Vítor Vasconcellos
8a9d690aab Fix autoformat CI & format codebase (#2581)
* Update pnpm version

* Fix autoformat

* Improve autoformat msg

* Attempt to fix autoformat 2

* Fix autoformat

* Ignore deleted files in auto-format

* Fix diff filter

* Autoformat whole codebase

* Improve error message for autoformat CI

* Test autoformat CI

* Revert "Test autoformat CI"

This reverts commit 0bf2f46d1aa19bec92d73d4d62c8156473455040.
2024-07-04 08:57:43 +00:00
Vítor Vasconcellos
0e07e068fc Style auto format (#2557)
* Autoformat script

* Executable + package.json

* Fix pnpm format not exiting

* Fix exit handling

* Allow to only run fontend autoformat

* Rename script to autoformat.sh
 - Fix exit handling

* Add style check to CI
 - Replace deprecated actions-rs-plus/clippy-check@v2 with giraffate/clippy-action@v1

* Use HEAD as ancestor when running in CI

* Improve uncommmited erro message

* Increate timeout for Type and style check action

* Run Clippy when ci.yml file changes

* increase cypress timeout
2024-06-20 00:04:43 +00:00
Vítor Vasconcellos
01e346f92a Fix nvidia on linux segfault (#2542)
* Workaround nvidia on linux segfault

* Fixes

* Don't vendor dbus

* Make dbus and opener linux exclusive dependencies

* opener is a generic dependency

* Add dbus as a dependency for deb and setup.sh

* Clippy fixes

* Replace gl and glfw with glutin for querying nvidia card

* Replace glutin with wgpu

* Update icrate to fix 'multiple applicable items in scope'
2024-06-07 19:14:50 +00:00
Vítor Vasconcellos
ae06459eea Package & distribute frontend bundle when doing releases for Linux (#2500)
* Package & distribute frontend bundle when doing releases for Linux

* Only check for tar.xz frontend bundle

* debug

* Fix sd-desktop
2024-06-04 23:19:41 +00:00
Vítor Vasconcellos
c516a3a45b Improve prod bundle size (#2447)
Greatly improve production build size, now prod builds are 1/3 of the size
 - Main spacedrive binary went from ~120M to ~40M
 - Add more optimize options for rust prod build
 - Improve vite settings for front-end prod build
 - Improve lossless compression of PNGs
 - Don't include videos in bundles when they are not used
 - Don't generate/bundle sourcemap for prod builds
2024-05-07 19:32:34 +00:00
Oscar Beaumont
60cdeafea2 [ENG-1690] Upgrade to Tauri 2.0 (#2433)
* Most of it

* Cleanup menu

* fixes

* Fixed linux compilation, still not working though, as the app panic on start

* Add dialog permissions

* fix search keybind

* Fix linux
∙  - Disable linux-ipc-protocol feature, as that is causing panics due to trying to access some WebKit internal structures outside the main thread

* Update to Ubuntu 22.04, new tauri doesn't support ubuntu 20.04
>  - Add note to download page about deb's distro support

* Remove gen/schemas

* Update linux dependencies in setup.sh

* Fix linux deps again
 - Enable rt-tokio-crypto-rust feature in secret-service so clippy stops complaining

* Clippy + auto fmt

* Fix cache-factory
 - Create a devtools feature for desktop app, which is enabled by default on dev builds

* Fix minor error in patchTauri.mjs

* Fix some envvars that have been renamed on tauri v2

* Dont change the secrets

* Add missing linux dependency

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-05-07 08:36:50 +00:00
Vítor Vasconcellos
29054e0f79 Remove AppImage logic and build system (#2446) 2024-05-04 22:27:14 +00:00
Vítor Vasconcellos
84c50ddd86 Fix cargo test & improve pnpm prep native deps download (#2393)
Couple of fixes
 - Increase `pnpm prep` connection timeout to 5min, to better handle downloading native deps under flaky network conditions
 - Fix `cargo test` and cache-factory CI
 - Clippy and fmt
2024-04-25 17:58:50 +00:00
Vítor Vasconcellos
0b5dd045f0 Add support for OS level network proxy in pnpm prep (#2304) 2024-04-10 00:13:05 +00:00
Vítor Vasconcellos
02c0d37514 Fix pnpm prep fails to download native-deps on some specific network env (#2275)
* Fix pnpm prep failing to download native-deps on some specific network configurations

* Eslint/Prettier auto-format
2024-04-04 14:19:49 +00:00
Vítor Vasconcellos
dcb4ace969 New test: Add location (#2222)
* New test: Add location
 - Add script to download some test files to allows tests to have a stable path for creating locations
 - Improve onboarding test
 - Add custom Cypress commands for fastOnboarding and deleteLibrary
 - Add some documentation related to tests to CONTRIBUTING.md
 - Update some deps

* Download small test-data for Cypress CI

* Replace globstar with find to make it compatible with bash 3.x

* Fix react too many re-renders
 - add-location test should now pass successfully

* Make job model task text match less flaky

* Check if we were redirected to onboarding after Library was deleted
 - Check if sidebar entry was created after adding a location
 - Put route regex's into a separate file
 - Make onboarding test ensure that no Library exists before running

* Enable test retries
 - Log cypress default config during test run

* Run tests under webkit
 - Pass CI envvar to tests to ensure correct cypress config

* Attempt fix CI config again

* Remove single use checkUrlIsLibrary function
2024-03-23 21:24:16 +00:00
Vítor Vasconcellos
d04dbcbbfc [ENG-1624] E2E Tests (#2192)
* Basic cypress test

* Add e2e test command

* Add full e2e test for creating and deleting a library

* Add cypress to CI

* Fix cypress CI
 - Use arm macos for running cypress
 - Setup rust on cypress job
 - Increase cypress CI timeout

* Remove unused APPLE_SIGNING_IDENTITY envvar

* Update deps again + Try to fix Cypress CI setup

* Update deps again

* Only test default location if it exists

* Use latest v18 NodeJS in CI

* Increase minimum node version due to updated dependency
 - @typescript-eslint/eslint-plugin now requires node >=18.18

* Fix testing failing for optional default locations

* Enable video recording
 - Fix screenshot and video artifact uploading

* Fix location test
 - Use correct location name, instead of trying to derive it from the internal id
2024-03-19 20:22:17 +00:00
Arnab Chakraborty
ca8049eeed [MOB-23] Mobile Hardware Information for Overview Page (#2106)
* wip for iDevices

* Working HardwareModel Info for iOS

* wip

* Merge 'main' into 'mob-hw-info-overview'

* Half-Working `get_volume()`

* Objective c bridge to talk to FS

* Working objc bridge

The bridge works now, and we can now access the iOS file system using the native objective-c APIs instead for proper values, including on the simulator.

* Isolate `icrate` for `ios` deployments only

* Working Stats for Android

* Clean Up + `pnpm format`

* Fix to FSInfoResult Type

Due to the RNFS fork change, I had to change the types to make it so it doesn't fail building and CI.

* iOS Device Name Fix
2024-03-06 06:46:22 +00:00
Vítor Vasconcellos
b9c190654d [ENG-1472] Frontend bug fixes and dependencies update (#2137)
* Fix onboarding sometimes not redirecting to Explorer on prod builds
 - Fix failure trying to compile landing prod builds outside Vercel
 - Fix Server docker failing to restart due to some incorrecting logic for creating the unprivileged users erroring out
 - Fix Storybook failing to build due to updates to Vite
 - Update Storybook dependencies
 - Remove unused Inter font dependency
 - Fix some incorrect references to NodeJS types inside web code
 - Fix $libraryId index using the incorrect redirect function
 - Improve error handling for the $libraryId index route
 - Fix Prism not being correctly loaded, and failing to register its plugins
 - Improve Prism loading error handling
 - Small improvement to the text highlight logic
 - Fix SCSS deprecation for untyped hsl values
 - Fix library query cache incorrectly saving empty values, which lead to the onboarding redirect bug
 - Patch contentLayer to fix an error during the final part of it's build process
 - Update most dev dependencies
 - Update publish-artifact to be compatible with new @actions/artifact
 - Fix issue with new vite-plugin-solid failing to build our .ts files due to the removal of the typescript plugin
 - Fix pnpm overrides not applying due to incorrect placement in package.json
 - Replace deprecated react-tsparticles and updated three used by the Bubbles background in the landing page
 - Rework Bubbles background to be compatible with new @tsparticles/react
 - Update @sd/config dependencies
 - Update @sd/scripts dependencies

* Implement suggestions
 - Replace mobile JS node setup with custom setup-pnpm action
 - Handle GITHUB_SECRET default value in code and throw a warning when it is not set
 - Fix pnpm now resolving the correct node version when building Spacedrive server docker
 - Add missing getent command to spacedrive server docker
 - Fix typo in entrypoint.sh
 - Implement a more robust check if the user is already in a group
 - Fix adduser failing due to missing default group
 - Disconnect IntersectionObserver on component unmount
 - Improve prism import comment

* Implement more suggestions
 - Pin genent version to latest stable release of UClibc
 - Add checksum checks for all ADD clauses in Spacedrive server Dockerfile

* Increase Maestro timeout to reduce CI failures due to slow simulator startup

* Dowgrade maestro to workaround CI timeout

* Improvements to the script that run maestro mobile tests
 - Increase the amount of retries for a maestro test run to 6
 - Increase Maestro driver startup timeout to 2 minutes

* Let run-maestro-tests.sh decide how to run itself

* ¯\_(ツ)_/¯
2024-02-28 23:52:24 +00:00
Vítor Vasconcellos
c1e98dcb0d Update github actions due to nodejs 16 deprecation (#2107)
* Update github actions due to nodejs 16 deprecation
 - Replace archived actions-rs/clippy-check with maintained fork actions-rs-plus/clippy-check
 - Replace redhat-actions/push-to-registry with updated fork Eusebiotrigo/push-to-registry
 - Point redhat-actions/buildah-build and softprops/action-gh-release to current master to fix nodejs deprecation

* Build the correct ios core rust arch for CI runs

* Build ios app for the same arch as the host in Mobile CI

* Some changes to try and make cache-factory faster and avoid failing so much

* Add trigger to run cache-factory on pull requests when there are changes to itself

* Attempt to fix sed usage on macOS

* Don't treat warning as errors

* Fix windows

* Fix windows 2

* Use target ad cache key for rust to differentiate between macOS x86_64 and arm64

* Use faster/better linkers to compile for macOS, Linux and Windows

* Fix missing shell in action

* Fix typo

* Fix missing shell in action 2

* Fix mold download
 - Replace bsdtar with plain tar

* Fix permission denied when extracting mold

* Remove zld

* Don't restore cache for rustfmt
 - Remove target symlink to C:/ in an attempt to speed-up windows CI

* Fix typo

* Restore target symlink on windows
 - Removing it didn't make CI faster

* Run Mobile on macos-14
2024-02-21 09:27:40 +00:00
Arnab Chakraborty
ce0203ff74 New Android Build Script (#2096)
* New Android Build Script

* Clean up + Works for CI now

* Simplify android build.sh
 - Fix /var/home/vitor fallback for Linux systems
 - Run a single cargo ndk for all targets (not parallel build, but a bit faster)
 - Fix android target s/x86/x86_64/
 - Format setup.sh
 - Minor improvements to rust mobile targets installation step in setup.sh

* Add notice to CONTRIBUTING that only Java <= 17 is supported for building android
 - Make prettier ignore some mobile build artifacts

* When in CI, Fix build android core for host architecture

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-02-20 06:33:52 +00:00
Vítor Vasconcellos
6e6e5f7674 Fix Linux release (#2081)
* Fix Linux release
 - Pin appimage-builder to latest master
 - Add required cmp option to AppImageBuilder.yml
 - Disable intro video on Linux (it breaks onboarding for some reason)

* Auto format

* Fix typecheck
2024-02-13 18:26:19 +00:00
Vítor Vasconcellos
46a4b788e9 Port AppImage build to use appimage-builder (#1785)
* Initial port to appimage-builder

* Almost

* Fix appimage build on arm64

* Custom patch for external binaries run under appimage
 - Disable bubblewrap sandbox when running under appimage
 - Change cwd to APPDIR when running under appimage

* AppImage Works (for the first lunch, then it crashes with Stack Smash 😭)

* Fix stack smashing, now AppImage almos fully works \o/ (gstreamer is still broken)
 - Temporarily disable the volume watcher when running under appimage (Workaround for the stack smash error)
 - Wrap gnu lic version check for appimage under conditional compile for glibc targets
 - Add error handling for the justUpdatedCheck
 - Fix VITE_LANDING_ORIGIN being undefined

* On non glibc systems default to runtime/compat

* Use glibc version 2.8 instead of 0 for non-gnu systems

* Fix video playback not working due to broken GstRegistry

* Build and publish new AppImage release artifact
 - Fix model location when building deb
 - Improve model path resolution logic
 - Remove patchelf dependency from setup script
 - Fix incorrectly ignore gstreamer dependency in AppImage recipe

* Fix clippy complaining about `get_path_relative_to_exe`
 - Read GLIBC_FAKE_VERSION or use 2.8 for musl in appimage (while the code is there, this is not really supported for now)

* Remove appimage tauri target from release CI

* Remove setup-buildx-action, not relly needed

* typo fix

* Fix git describe command running on cwd instead of the repo root dir

* Attempt fix weird git permissions errors in CI+docker

* Pass CI env to docker appimage

* Only use git after installing it

* Pass target to appimage build script

* Fix permission after creating appimage

* -_-

* Swap envvar with github ci var

* Format

* Add instruction on how to manually build an AppImage

* Fix typos

* docs: add note about running podman with `--privileged` if there's a permission denied error

* docs: fix typo and link directly to appimage-building `README.md`

* refactor: streamline code and make it a bit cleaner

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2024-01-22 10:52:26 +00:00
Vítor Vasconcellos
6e00d17449 Fix pnpm prep on Linux (#1904)
* Fix pnpm-prep on linux

* update archive-wasm to 1.6.1
2023-12-20 09:12:49 +00:00
Ericson "Fogo" Soares
8af3a5f286 [ENG-1479] AI Prototype (#1845)
* First draft on image labeling

* Fixing execution providers for other OSs

* Better error handling and shutdown

* Working with shallow media processor

* bruh

* Fix warnings

* Now hooked to media processor job

* Link desktop app with libonnxruntime to avoid TLS error during startup

* Be able to change models on runtime
Revert to use labels table instead of tags

* A bug on a model-less inference

* Show AI labels on Inspector
 - Change yolo inference to use half precision
 - Add labels api to core

* Remove LD_PRELOAD

* Fix race condition on model executor shutdown

* Don't load all images in memory moron

* Embeed yolo model in prod build
 - Change yolo model path to new one relative to executable

* Disable volume watcher on linux, it was crashing the app
 - Invalidate labels when they are updated

* Rust fmt

* Minor changes

* Gate onnxruntime linking to the ai-models feature

* Add build script to sd-server to handle onnxruntime linking workaround

* Move AI stuff to its own crate and normalize deps

* Rust fmt

* Don't regenerate labels unless asked to

* Now blazingly fast

* Bad merge

* Fix

* Fix

* Add backend logic to download extra yolo models

* Add models api route
 - Add api call to get available model version
 - Add api call to change the model version

* Improve new model download logic
 - Add frontend to change image labeler model

* Fix new model downloader

* Fix model select width

* invalidate labels count after media_processor generates a new output

* Rename AI crate and first draft on download notifications

* fix types

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
Vítor Vasconcellos
4e555232c8 Pin rust to 1.73 (Workaround Internal Compiler Error) (#1796)
* Pin rust to 1.73
- Clippy fixes

* Don't install cargo-watch on CI
2023-11-17 22:47:06 +00:00
Vítor Vasconcellos
48afea5a4b [ENG-1184, ENG-1286, ENG-1330] Rework native dependencies (+ deb fixes) (#1685)
* Almost working

* Downgrade libplacebo
 - FFMpeg 6.0 uses some now removed deprecated functions

* Use -Oz for zimg

* Fix CI script to run the new ffmpeg build script

* Fix heif step name + Ignore docker cache while building in CI

* Fix Opencl build on linux

* Fix adding incorrect -target argument to linker
 - Update zig for windows target

* Disable opengl for ffmpeg, it only uses it as an outdev, not for processing
 - Disable opengl and directx for libplacebo, ffmpeg only supports vulkan when using it
 - Add WIN32_LEAN_AND_MEAN to global cflags to optimize windows api usage
 - Fix 99-heif.sh incorrect bsdtar flag

* Remove WIN32_LEAN_AND_MEAN from global CFLAGS as that was breaking OpenCL build
 - Fix Dockerfile step for cleaning up the out dir
 - Improve licensing handling

* x86_64 windows and linux builds are working

* Fix aarch64 build for windows and linux

* Fix symbol visibility in linux builds
 - Fix soxr failing to download due to sourcefourge
 - Only patch zimg on windows targets
 - Tell cmake to hide libheif symbols

* Fix Linux .so rpath
 - Add lzo dependency
 - Publish source for the built libs
 - Add warning for missing nasm in tauri.mjs
 - Remove ffmpeg install from setup.sh
 - Add download logic for our linux ffmpeg bundle in preprep.mjs

* Remove jobs, docker doesn't support this

* Fix typing

* Change ffmpeg references to native deps
 - Rename FFMpeg.framework to Spacedrive.framework
 - Centralize the macOS native deps build with the windows and linux one
 - Change the preprep script to only download our native deps
 - Remove old macOS ffmpeg build scripts

* Compress native deps before creating github artifact
 - The zip implementation for github artifact does not mantain symlinks and permissions
 - Remove conditional protoc, it is now always included

* Don't strip dylibs, it was breaking them
 - Only download macOS Framework for darwin targets
 - Fix preprep script
 - Improve README.md for native-deps
 - Fix not finding native-deps src

* Attempt to fix macOS dylib

* Fix macOS dylibs
 - Replace lld.ld64 with apple's own linker
 - Add stages for building apple's compiler tools to use instead of LLVM ones

* Ensure sourced file exists

* All targets should build now
 - Fix environment sourcing in build.sh
 - Some minor improvements to cc.sh
 - Fix incorrect flag in zlib.sh
 - Improve how -f[...] flags are passed to compiler and linker
 - Add more stack hardening flags

* We now can support macOS 11.0 on arm64

* Improve macOS Framework generation
 - Remove installed unused deps
 - Improve cleanup and organization logic in Dockerfile last step
 - Move libav* .dll.a to .lib to fix missing files in windows target
 - Remove apple tools from /srv folder after installation to prevent their files from being copied by other stage steps
 - Create all the necessary symlinks for the macOS targets while building
 - Remove symlink logic for macOS target from preprep.mjs

* Remove native-deps from spacedrive repo
 - It now resides in https://github.com/spacedriveapp/native-deps
 - Modify preprep script to dowload native-deps from new location
 - Remove Github API code from scripts (not needed anymore)
 - Add flock.mjs to allow running tauri.mjs cleanup as soon as cargo finishes building in linux

* Handle flock not present in system
 - Allow macOS to try using flock

* Fix preprep on macOS

* Add script that patch deb to fix errors and warnings raised by lintian

* Fix ctrl+c/ctrl+v typo

* Remove gstreamer1.0-gtk3 from deb dependencies

* eval is evil

* Handle tauri build release with an explicit target in fix-deb.sh

* Preserve environment variables when re-executing fix-deb with sudo

* Only execute fix-deb.sh when building a deb bundle

* Improvements fix-deb.sh

* Improve setup.sh (Add experiemental alpine support)
2023-11-17 19:20:14 +00:00
Ericson "Fogo" Soares
685ad90790 Remove < 21 node restriction (#1728) 2023-11-04 03:44:07 +00:00
Vítor Vasconcellos
609e7e1228 Fix Windows CI (#1695)
* Add Nasm to setup.ps1
 - Nasm is now required to compile the `openssl-sys` crate

* Run Clippy CI when setup or prep scripts change

* Ensure nasm is also installed in Windows CI
 - Add nasm to setup.sh
 - Remove openssl-dev from setup.sh

* Ensure nasm is installed in macOS too

* Dumb ctrl+c/ctrl+v mistake

* 's/==/=/'
2023-10-29 00:25:26 +00:00
Julian Braha
0f2512a567 Add installation of libvips for arch systems to setup script (#1688)
Add libvips package for arch systems to setup script
2023-10-27 20:31:30 +00:00