Not directly related to the post (which is focused on which database to host for your app), but I'm writing desktop apps (think DAWs) in C++ and Rust (not JS), and want to synchronize settings through a Dropbox or Google Drive (so I don't have to host my own cloud sync servers). What's a good library or schema to achieve this?
Personally I usually don't have multiple instances of the same app open on multiple machines, but other people might open the same or different files on their desktop and laptop.
- Not all settings should be synchronized (don't include machine-specific "recent files" paths).
- How should settings be stored locally (if I may have multiple instances of my app open on a single machine)? Registry (Windows-only)? INI with atomic saving (requires care and locking to prevent multiple instances from trampling or racing with each other)? SQLite?
IMO Stylus is a pretty good implementation of offline-first cloud settings sync over Dropbox/etc. It's currently based around one JSON file per CSS file (Dropbox/Apps/Stylus - Userstyles Manager/docs/uuid.json), and what appears to be a transaction log (Dropbox/Apps/Stylus - Userstyles Manager/changes/number.json). Cloud sync has been 100% reliable in my experience, though I do notice temporary file lock errors when switching between different machines in my dual-boot setup (but sync seems to be eventually consistent nonetheless).
uBlock Origin is worse. Instead of merging settings, it expects the user to upload and download the entire settings blob at once (and pulling an old blob can erase changes you've made locally). And in the past it's entirely failed to sync because the blob was too big to upload to Mozilla's servers. (Right now it "works" but takes several minutes for one computer to see a config uploaded from another computer.)
I've done something pretty ridiculous to solve this problem and I'm not sure I'd recommend it, but here it is:
- A directory is synced with the server with no conflict resolution
- The application creates its config file in that directory, named by a random UUID, which is stored outside the synced folder
- The config file stores the setting overrides (defaults were compiled-in) in any format (I used YAML)
- Each setting override includes a "locked" and "lastModifed"
- On startup (sync was external) all files in the directory are read and merged starting with the local one, then skipping any settings that are locked (locally or remotely), last modified wins
Some deployments used a daily rsync cronjob, some had a mounted network share (with hilarious broken file locking) and of course it worked with direct bind mounts as well.
I also briefly experimented turning the "locked" field into a "group" field to enable multiple "sync groups" with some keys shared globally and some only with other group members (even different groups for different settings), but it ended up not being useful for my use case, although it did work.
Sounds interesting. I suppose it would break if a computer's clock was set in the future (its lastModified would always win), but IDK what wouldn't break in that scenario.
What does the locked field do?
Do you have a link to your implementation, or is this proprietary?
Personally I usually don't have multiple instances of the same app open on multiple machines, but other people might open the same or different files on their desktop and laptop.
- Not all settings should be synchronized (don't include machine-specific "recent files" paths).
- How should settings be stored locally (if I may have multiple instances of my app open on a single machine)? Registry (Windows-only)? INI with atomic saving (requires care and locking to prevent multiple instances from trampling or racing with each other)? SQLite?
IMO Stylus is a pretty good implementation of offline-first cloud settings sync over Dropbox/etc. It's currently based around one JSON file per CSS file (Dropbox/Apps/Stylus - Userstyles Manager/docs/uuid.json), and what appears to be a transaction log (Dropbox/Apps/Stylus - Userstyles Manager/changes/number.json). Cloud sync has been 100% reliable in my experience, though I do notice temporary file lock errors when switching between different machines in my dual-boot setup (but sync seems to be eventually consistent nonetheless).
uBlock Origin is worse. Instead of merging settings, it expects the user to upload and download the entire settings blob at once (and pulling an old blob can erase changes you've made locally). And in the past it's entirely failed to sync because the blob was too big to upload to Mozilla's servers. (Right now it "works" but takes several minutes for one computer to see a config uploaded from another computer.)