* Copy ephemeral files by chunks
* Improving buffered writer
* Report progress
* Copy and emit progress as two separated tasks
* Simplify for-if-return with `Iterator::any`
* Docs and use structured tracing
* Simplify code using paths
* wip
* wip
* wip
* wip
* Add report of messages while copying files
* Add info field to job report
* Group paths from OldFileCopierJobStep into a single struct
* Improve progress information
* Remove the need for synchronization
* Error handling for copy
* Clean up frontend
* Make watcher only consider new files
* Fix concurrent renaming of multiple files that have the same name
* Add documentation for file strategist
* Remove editions to ephemeral file copies
* Remove experimental/nightly features from Cargo.toml
* Fix no-case-declarations
* Remove dead comments
* Format code
* Use "Duplicate" instead of "Copy" in the frontend messages
* Remove inline always to make clippy happy
* icons for deleter and copier
* Fix JobManager for copy jobs
- Fix some types definitions
* Fix Job icon choosing logic
- Fix Copier job showing two progress bars
* Log which files are not formatted in the Type and style check CI
* fmt
* Forgot an import
* autoformat
---------
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* Ensure prisma generation works before continuing CI
- Add script that list all dependencies with multiple version in Cargo.lock
* Ensure CI runs when custom actions are modified
* Rever windows change in hopes that it fixes prisma generation
* Fix prisma generation check in CI
* Update deps
* Replace prisma-client-rust-cli with oficial prisma cli to fix windows model generation
- Revert back to windows-rs implementation of get_inode in file-path-helper
- Adjust Github CI to use oficial prisma cli
- Update scripts dependencies
- Fix some new rust warnings due to it trying to compile OS specific code on all plataforms
* Fix windows-rs impl for get_inode_windows in file-path-helper
* More fixes to windows-rs impl for get_inode_windows in file-path-helper
* More fixes to windows-rs impl for get_inode_windows in file-path-helper
* More fixes to windows-rs impl for get_inode_windows in file-path-helper
* More fixes to windows-rs impl for get_inode_windows in file-path-helper
* More fixes to windows-rs impl for get_inode_windows in file-path-helper
* Import windows constant intead of hardcoding its value
* Fix check for prisma files in setup-rust
* Autoformat
* Fix iOS and Android targets in watcher
---------
Co-authored-by: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
* Remove deps-generator app
- It is broken for quite a while and not being used
* Remove unused code and unused deps
* auto format
* Restore incorrectly removed files
- Remove aws deps and test code using it
* Replace winapi-util with windows-rs
- Adjust some deps versions
* Autoformat
* Update rand, remove unused dep
* Fix image crate due to changes in pdf-renderer
- Dont use default features in reqwest
- Use native-tls tauri
* Remove incorrect notice about blake3
* Migrate to Tauri v2 Release Canidate
* Formatting
* Update pnpm
* Update all tauri deps (js and rust) to Release Candidate 2
- A lot of misc fixes due to changes in the new versions of tauri, rspc and specta
* Recreate pnpm-lock
* Fix pnpm-lock
* Formatting
* Use 10 chars SHA for git deps
- Update a couple of rust dependencies
- Add taplo for toml auto formatting
- Adjust some formatting configurations
- Updated prettier and its plugins
* Fix formatter settings
* Minor improvements to Cargo.toml format
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
All cloud routes are now migrated to the new system in the backend.
The old routes are still available, as the frontend has not been modified to use the new ones yet.
* Error handling for sd-core-sync crate
Also a bunch of tweaks and fixes
* Update core.ts
* Auto format
* Comment out background_processing_percentage on frontend too
- Update rust version in contributing
* Trying to avoid data loss on actor stop
* Stronger linter on sync generator
* Stronger lint on sync subcrate
* Clean back tasks.json file
* TS typechecking
* Autoformat
* Add more verbose errors for Cloud REST API
* Removing some comments
Copilot is fun sometimes lol
* Properly stopping actors
* Fix ingest stop
* Racing on stop for actors
* Error conversion from merge with main
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
* Catching panics
* Re-enable unwind on release profile
* Reverting deps due to bad assertion
* Comment why the deps where reverted
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* First draft on task system usage, still missing job system
* Scan location roughly working, a ton of stuff to fix yet
* Updating some deps due to crashes and bugs
* Exposing non critical errors to frontend
* Getting active job reports from job system
* Using boxed opaque type to avoid a downcast issue with generics
* Task system issues discovered on race conditions
* Enable debug
* Fix job report in the job manager
* Fix race condition on steal tasks
* Fixed race condition on task suspend
* Some fixes on job progress reporting and save
* Fixed many race conditions and a hard deadlock
Also some progress report polishing
* Ignore .ts and .mts video files for now
* Some better logs
* bruh
* Internal deadlocks and excess of communication in the task system
- Also better logs
* Bunch of fixes and optimizations
* WIP at fixing file identifier
* Fixed file identifier job
- still need to work on its progress report frontend
* A bunch of polishing
* Fixed some bugs and did more polishing
* Cleanup
* Bridging old and new job systems
* A ton of fixes
* A bunch of bugs related to shutdown and resume
* Indexer and watcher bugs
* Log normalizing
* Fixing CI
* Change error! to warn! on non critical errors log
* Fix redirect to new location
* Type annotation
* Bogus merge resolution on cargo lock
* Add title to rspc
* wip
* rspc docs title cleanup
* a
* todo
* `p2p/crates`
* remove normalised cache docs
* wip
* words words and more words
* a
* wip
* a
* cleanup settings
* wip p2p docs
* wip
* Moving stuff around
* Media data extraction task
* New thumbnailer task
* Taking more metrics on thumbnailer
* First drafts on media processor job
* Better interruption latency on thumbnailer task
* Also solve the latency on interrupt at media data extraction
* Media processor job
* Shallow media processor
* Better prioritize files identification
* Integrate ffmpeg media data extraction
- Thumbnail quality improvements parameters
* initial ffprobe commit
* Working slim down version ffprobe
* Auto format ffprobe and deps source
* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning
* Remove show_* and print_format options and their respective logic
* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata
* Reduce the amount of ffmpeg log messages while generating thumbnails
* Fix completly wrong comments
* mend
* Start modeling ffmpeg extracted metadata on schema
- Start porting ffprobe code to rust
- Rename some references to media_data to exif_data
* Finish modeling media info data
- Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
- Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl
* Add relation between MediaInfo and FilePath
- Remove shared properties from MediaInfo and related structs
- Implement Iterator for FFmpegDict
* Fix and update schema
* Data models and start populating MediaInfo in rust
* Finish populating media info, chapters and program
* Improve FFmpegFormatContext data raw pointer access
- Implement stream data gathering
* Impl FFmpegCodecContext, retrieve codec information
- Improve some unsafe pointer uses
- Impl from FFmpegFormatContext to MediaInfo conversion
* Fix FFmpegDict Drop
* Fix some crago warnings
* Impl retrieval of video props
- Fix C char* to Rust String convertion
* Impl retrieval of audio and subtitle props
- Fill props for MediaCodec
* Remove simple_ffprobe now that the Rust impl is done
* Fix schema to match actually retrieved media info
- Fix import some FFmpeg constants instead of directly using values
* Rework movie_decoder
- Re-implement create_scale_string and add support anamorphic video
- Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
- Use newer FFmpeg abstractions in movie_decoder
* Fix incorrect props when initializing MovieDecoder
* Remove unecessary lifetimes
* Added more native wrappers for some FFmpeg native objects used in movie_decoder
* Remove FFmpegPacket
- Some more improvements to movie_decoder
* WIP
* Some small fixes
* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips
* fmt
* Fix duplicate migration for job error changes
* fix rebase
* Solving segfaults, fuck C lang
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Update rust to version 1.77
- Pin rust version with rust-toolchain.toml
- Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
- Remove unused function and imports
- Replace most CString uses with new c literal string
* More segfault solving and other minor fixes
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
- Plus some other misc fixes
* Auto format
* Retrieve video/audio metadata on frontend
* Auto format
* First draft on ffmpeg data save on db
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix some incorrect changes to prisma schema
* Some fixes for the FFmpegData schema
- Expand logic to save FFmpegData to db
* A ton of things
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Integrating ffmpeg media data in jobs and API
* Rspc can't BigInt
* 🙄
* Add initial ffmpeg metadata entries to Inspector
- Fix ephemeral metadata api to match the files metadata api call
* Fix Inspector not showing ffmpeg metadata
* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils
* Solving some issues
* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
- Fix humanizeSize breaking when receiving floating numbers
- Fix incorrect equality in StatCard
- Fix unhandled error in Dialog when trying to remove an unknown dialog
* fmt
* small improvements
- Remove some unecessary recursion_limit directive
- Remove unused app_image releated functions
- Fix metadata query enabled flag
* Add migration for ffmpeg media data
* Fix cypress test
* Requested changes
* Implement feedback
- Update locale keys for all languages
- Add pnpm command to update all language keys
* Fix thumb reactivity in non indexed locations
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Most of it
* Cleanup menu
* fixes
* Fixed linux compilation, still not working though, as the app panic on start
* Add dialog permissions
* fix search keybind
* Fix linux
∙ - Disable linux-ipc-protocol feature, as that is causing panics due to trying to access some WebKit internal structures outside the main thread
* Update to Ubuntu 22.04, new tauri doesn't support ubuntu 20.04
> - Add note to download page about deb's distro support
* Remove gen/schemas
* Update linux dependencies in setup.sh
* Fix linux deps again
- Enable rt-tokio-crypto-rust feature in secret-service so clippy stops complaining
* Clippy + auto fmt
* Fix cache-factory
- Create a devtools feature for desktop app, which is enabled by default on dev builds
* Fix minor error in patchTauri.mjs
* Fix some envvars that have been renamed on tauri v2
* Dont change the secrets
* Add missing linux dependency
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Couple of fixes
- Increase `pnpm prep` connection timeout to 5min, to better handle downloading native deps under flaky network conditions
- Fix `cargo test` and cache-factory CI
- Clippy and fmt
* Introduce deep vs shallow for indexer tasks with different priorities
* Make job wait to dispatch if it's paused
* Extract file metadata task on file identifier job
* Some initial drafts on object processor task
* Object processor task for file identifier
* File Identifier job and shallow
