Very interesting, this will really change the dynamics of bug bounty and webapp security more broadly

I work on oss-fuzz.

I don't think it's plausible OSS-Fuzz could have found this. The backdoor required a build configuration that was not used in OSS-Fuzz.

I'm guessing "Jia Tan" knew this and made changes to XZ's use of OSS-Fuzz for the purposes of cementing their position as the new maintainer of XZ, rather than out of worry OSS-Fuzz would find the backdoor as people have speculated.

How many oss-fuzz packages have a Dockerfile that runs apt-get install liblzma-dev first?

Had this not been discovered, the backdoored version of xz could have eventually ended up in the ubuntu version oss-fuzz uses for its docker image - and linked into all those packages being tested as well.

Except now there's an explanation if fuzzing starts to fail - honggfuzz uses -fsanitize which is incompatible with xz's use of ifunc, so any package that depends on it should rebuild xz from source with --disable-ifunc instead of using the binary package.

This is interesting, but do you think this would have aroused enough suspicion to find the backdoor (after every Ubuntu user was owned by it)? I don't see why this is the case. It wasn't a secret that ifuncs were being used in XZ.

And if that's the case, it was sloppy of "Jia" to disable it in OSS-Fuzz and not do this:

``` __attribute__((__used__,__no_sanitize_address__)) ```

to the XZ source code to fix the false positive and turn off the compilation warning, no attention would have been drawn to this at all since no one would have to change their build script.

With or without this PR, it's very unlikely OSS-Fuzz would have found the bug. OSS-Fuzz also happens to be on Ubuntu 20. I'm not very familiar with Ubuntu release cycles, but I think it would have been a very long time before backdoored packages made their way into Ubuntu 20.

>Woha, is this legit or some sort of scam on Google in some way?:

I work on OSS-Fuzz.

As far as I can tell, the author's PRs do not compromise OSS-Fuzz in any way.

OSS-Fuzz doesn't trust user code for this very reason.

It looks more like they disabled a feature of oss-fuzz that would've caught the exploit, no?

That's what people are saying though I haven't had the chance to look into this myself.

Fuzzing isn't really the best tool for catching bugs the maintainer intentionally inserted though.

It's more likely that fuzzing would blow up on new code and they wanted an excuse to remove it.

After all, if it hadn't had a performance regression (someone could submit a PR fixing whatever slowed it down, heh) it still wouldn't be known.

No projects yet, but I bet we'll have some by next week.

I don't think we have plans to build this for now.

I find it a really cool idea, but for now, running fuzzers natively on Google Cloud with ClusterFuzz (https://github.com/google/clusterfuzz) suits our needs.

One challenge for the WASM approach is it will always be at least as hard to build a project for WASM as it is for native.

Right I think WASM offers some nice advantages over native for distributed fuzzing.

It's also worth pointing out that Mozilla made a (non-WASM) distributed fuzzing project, virgo: https://github.com/MozillaSecurity/virgo but it appears to be inactive.

I haven't done a comprehensive study of this but in general I find that fuzzing programs in different environments (e.g. CPU architectures, OSes) tends to find some bugs that won't be found by fuzzing in just one environment.

But in general, I agree a lot of the bugs in WASM apps could be found by fuzzing their native versions.

Great post Guido!

Guido's bignum fuzzer which tests the correctness of math operations in crypto libraries is one of the most interesting fuzzers we run on ClusterFuzz.

There are tools for fuzzing go: https://github.com/dvyukov/go-fuzz

But I think the kinds of bugs found by fuzzing aren't generally security issues in go (I don't know much about go) as they are in C/C++.

EDIT: See guidovranken's excellent sibling comment for how fuzzing can still be useful for go.

Awesome, I will check that out. Thank you!

btw, ClusterFuzz, the infrastructure behind OSS-Fuzz was open sourced today: https://news.ycombinator.com/item?id=19106771