For zsh users who like this idea and don't want to have to worry about anything leaving the device (not to insinuate that Atuin is doing anything below board, I don't think they are) I built my own version of this for myself called raven:
Raven has no sync component, stores your history in a sqlite, and never tries to move data off the device. It has a similar / large overlap in features with Atuin.
Sharing here in case anyone else could benefit from it, it only supports zsh for the moment since that's what I use.
I am happy Atuin user now, but I was initially worried that it would sync my data unless I explicitly disabled that feature. The fact that it's opt-in becomes clear once you read the docs and understand how it works, but it might be worth emphasizing that on the landing page. Currently it says:
Shell history sync
Sync your shell history to all of your machines, wherever they are
this is my only quip with atuin, despite my endorsement in a sibling comment.
this should absolutely be opted-out BY DEFAULT. not defaulted to opt-in.
if you have someone outright cloning your entire project just to skip this opted-in privacy gap feature, it's a good indicator it's an issue to be mindful of!
Opt-in means that the user needs to actively decide to use the feature, it's OFF by default. That's currently the case for atuin (happy user without sync here).
Opt-out would mean the feature would be ON by default and users would need to actively decide NOT to use it.
Understood. I'm actually thinking about a product design for an aarch64-based industrial controller.
I'd want to run all of the machine interfaces directly on Linux, but am interested in using Android for everything user-facing. Could I get away with running Cuttlefish in KVM (on an aarch64 SOC) and get OK performance? I'm thinking that it could be a good way to keep the important stuff isolated.
AOSP platform dev here. (Filesystem) Opinions my own, I don't speak for Google.
Disclaimer: I don't use nextcloud, and have not looked at their app specifically, this is just a surface level observation from my relatively informed perspective.
My take: SAF would work for this use case, as others have already mentioned.
Google Drive does not have the permissions that next cloud claims Google is giving preferential treatment to, and is delivered via the Play store in the same way nextcloud's app is.
As others have also observed, permissions such as MANAGE_EXTERNAL_STORAGE have been rampantly abused in the past, often in horrific ways.
SAF is not an option because it is HORRIBLY slow[1][2][3][4][5], which makes it an absolute no-go for any decent cloud synchronization app.
Excerpt from [1]:
> SAF is slow. Every SAF file IO operation takes like 20-30ms because it uses an IPC call. And sometimes you may want to check whether a lot of files exist on the disk and if they do not then create them (or something similar that requires a lot of file operations). It's so slow that even in google example they use hacks to make it faster.
Excerpt from [3]:
> Just to add a new sample for the performance of SAF vs standard File operations:
> […]
> 15 seconds with SAF, 6 milliseconds with native ls ! And there's only 128 files LOL
Wow the resolution to your link [3] is infuriating:
> We assure you that we are doing our best to address the issue reported, however our product team has shifted work priority that doesn't include this issue. For now, we will be closing the issue as won't fix obsolete. If this issue currently still exists, we request that you log a new issue (...)
It's all possible excuses all at once: "Thank you for your issue, we are working on it. Also we have no intention to work on it and we are closing it, the problem probably doesn't exist anymore anyways, and if you think it does make sure to open a new issue for us to treat it with the same amount of respect".
Reminds me of some bug reports I was looking at for Chromium regarding SVG rendering the other day.
Some of them are almost a decade old and require constant bumps to keep them alive, or filling new issues. Somehow their bot system considers anything more than a year old "probably patched". Their messages read much the same way as that.
This may be true, may have workarounds, and may be solved on later Android versions, but it also is not why Nextcloud says they are avoiding the framework. And Google Drive provides the same functionality using SAF, so I'm not sure it's a problem for this use case.
There are no good workarounds unfortunately and it's definitely not solved on later Android versions. I agree though that the reason that Nextcloud gives is factually incorrect (they say that SAF is for providing files rather than accessing them, which is plain wrong).
From what I understand, you can resolve most issues by making ContentResolver queries yourself once you have URI permissions for a folder, including data like filename, mime type and file size in the query projection. This reduces an NxM number of queries to N, where N is the number of nested directories in the tree and M is the number of files in the directory.
Obviously this should be solved with a better API at the SAF layer.
> As others have also observed, permissions such as MANAGE_EXTERNAL_STORAGE have been rampantly abused in the past, often in horrific ways.
The lack of consideration for this point in this thread scares me. The amount of data that can be taken from a device through a permission like this is likely huge and it’s not just about “protecting users from themselves”. I wouldn’t feel safe enabling it for any app, and while syncing all data on the device sounds very useful, it’s a damned if they do, damned if they don’t scenario for Google.
So scary that I don't know how billions of people in the world manage to have the right to use whatever software they want to access "all" the files in the file system of their computers (PC/mac) and that the civilisations have still not yet collapsed...
Google simply needs to add "I'm an adult" functionality to their phones. I know the author of this app and trust them, I know the functionality I want and I accept the risk because I'm a grown adult and can make my own choices.
But why? Just for the odd app that can't be bothered to use the new API?
Even if you trust the app, if there is a vulnerability in there, the Android sandbox provides an additional line of defense. Most apps don't have defenses of their own, the only apps that self-sandbox are web browsers.
Then don't enable it, no need to take away my ability to do so. Granular permissions are good (especially when the app can't reliably know they were refused), providing I have the ultimate control.
> it’s a damned if they do, damned if they don’t scenario for Google.
Did they consider my scenario above - where the app doesn't know it was not granted a permission?
> especially when the app can't reliably know they were refused
That's the problem. Android didn't do this even though it was obviously what is needed. Android apps can easily tell what permissions they have.
I think Google prioritised UX over power and security here. They were presumably scared about people accidentally clicking the "Silently deny" button and then getting confused when the app didn't work. Big missed opportunity.
Let me guess, loan shark apps, or is there something even worse that I wasn't aware of?
(Essentially ransomware that people "voluntarily" install in order to get a predatory loan, often without knowing how it actually operates. If they fail to pay, not only is their phone locked, but the data from it is used to threaten/extort them, from threatening to send their nudes to their contacts to death threats to relatives identified from the data - https://www.welivesecurity.com/en/eset-research/beware-preda...)
Pretty much every game and every social media app asked for full storage access under the guise of "downloading game files" or "allowing our photo picker to work".
Then they got free access to all your photos and their location data and all your documents and downloads. Yes, including those banking statements downloaded as PDFs. Forever. In background. Whenever.
That's the access Nextcloud demands instead of using the API where YOU choose what it can read.
> As others have also observed, permissions such as MANAGE_EXTERNAL_STORAGE have been rampantly abused in the past, often in horrific ways.
I understand that great care should be taken when this permission is granted to new apps, but NextCloud is well known and on top of that it is a file management app. If anything, apps like that should have this permission.
If you plan to phase it out completely, then the alternatives have to be good enough, which judging from some of the child comments, they are not. I have never developed for Android (and likely never will, because of stuff like this), so I cannot judge properly.
It's also my understanding that the Google Drive App is just some UI over cloud storage. All the interesting bits like Backup are not handled through it and Google Drive gets preferential treatment for this. Additional permissions are required to emulate such functionality.
SAF is a terrible solution for anyone trying to make a cross-platform app (and if you're not then why are you targeting Android as your one platform?), because it doesn't work with native code.
Totally unrelated question but since we're talking filesystems on Android here, I have been incredibly annoyed that AOSP devs disabled "special" characters ":<>?|* in file names in Android 11[0], probably to achieve compatibility with NTFS/FAT. Unfortunately, not only does this make some (especially older) applications accidentally crash when I want to use, say, a question mark in a file name, I can also no longer sync all my files from my Linux machine to my phone.
Now, there is probably little chance for this to get rolled back entirely. But maybe one could make it an option in the developer settings to allow ext4 filenames like before? Do you see any chance your team could be convinced of such a move? And where would be the appropriate place for me to raise this issue and/or contribute a patch?
AOSP engineer here; I don't have a deep knowledge of the issue at hand as I don't work on the bluetooth part of the OS, but I did just want to call out that GrapheneOS (while a really cool project!) supports ~11 total device types, and changes in AOSP have to keep a much larger context in mind.
I'm always looking for ways to implement cool stuff and make things in android better, but it's a little myopic to ignore the larger OEM ecosystem when complaining about specific feature roll outs.
I highly doubt the reason this isn't enabled is the 3% memory/cache usage, and there's some other consideration that's informing the decision.
> AOSP engineer here; I don't have a deep knowledge of the issue at hand as I don't work on the bluetooth part of the OS, but I did just want to call out that GrapheneOS (while a really cool project!) supports ~11 total device types, and changes in AOSP have to keep a much larger context in mind.
AOSP also only directly supports the phones/tablets we support. We were only talking about 2 of the devices supported by AOSP. Pixels enabling MTE is not connected to other devices. App compatibility is also not an issue because it can be enabled for the base OS with user installed apps excluded until they opt into it. It can be made opt-out in a future target API level (Android 15) and then the opt-out can be removed in the next target API level. This is entirely doable.
> I'm always looking for ways to implement cool stuff and make things in android better, but it's a little myopic to ignore the larger OEM ecosystem when complaining about specific feature roll outs.
We're not ignoring it. We're talking about the Pixel 8, Pixel 8 Pro and future Pixels. It's a request for Pixels to enable it, not AOSP to enable it for all devices.
> I highly doubt the reason this isn't enabled is the 3% memory/cache usage, and there's some other consideration that's informing the decision.
Performance and memory usage are definitely the only blocker to enabling it for the base OS. It would already be enabled if the security team was in charge of these decisions. It's not enabled because they likely have to prove it's fast enough. Our specific recommendation was enabling it in asynchronous mode for all cores for the base OS and user installed apps explicitly opting into it. We do not think it's a good idea to start automatically opting apps into using it until more Android devices support it. Google could require that ARMv9 devices support MTE as a developer option for Android 15 onwards in the CDD in order to make sure developers have it available for testing. App developers can't be expected to have a Pixel so it's unrealistic to have it forced on app developers via target API level until more devs have devices available to test it. We understand all that and what we are pushing for takes that into account.
https://github.com/tylersaunders/raven
Raven has no sync component, stores your history in a sqlite, and never tries to move data off the device. It has a similar / large overlap in features with Atuin.
Sharing here in case anyone else could benefit from it, it only supports zsh for the moment since that's what I use.
Atuin is great though!