If only the system were open source so you wouldn't have to wonder about that...
But we do still have to trust that they are actually running the code they posted.
Unless that code somehow contains some way to verify itself?
I wonder if there is some way to do that? Have the code include a hash of itself and some way to query the running service that guarantees that the running service must be running the code you are looking at?
At first glance it seems any response could always be faked, but maybe there is some cryptography trick where you submit something, like an encrypted copy of the public code maybe, and it crunches and returns something, and that somehow proves that the running code you can't see must be the same as the code you can see.
Depending on how the protocol for the challenge works, that could still be faked. The challenge has to somehow not be seperable from ordinary traffic so that you can't have one piece of code handle the challenge and another piece of code handle other traffic.
- Multi-party computation. Too much overhead for something like this.
- Remote attestation, as seen in e.g. Intel SGX. Usually provided by the CPU vendor. Not a cryptographic guarantee, more of a "it'd be very hard to defeat this if you're not Intel". Probably not that warrant-resistant.
I think the normal solution to this is all the prove you are the software you say you are calls is proxied to that software and all the normal services calls you want to log and duplicate or otherwise violate the contract are sent to the modified code.
You can talk about hypothetically doing anything. You're not really making a point.
Hypothetically, you can rewrite the firmware for the IPMI with a backdoor and extract data.
Hypothetically, you could kidnap the family of a developer and force them to add a surreptitious side channel for exfiltrating data.
Hypothetically, you could run the universe in a simulator and use the simulator's controls to read data from RAM in the simulated Mullvad servers on the simulated Earth.