We're seeing a small cottage industry of state-machine verification techniques presenting themselves over the past year. K-framework has its accessibility proofs (ranging over the state space of possible program paths), Manticore has its symbolic state analysis, and a few others do roughly the same thing. None of these frameworks are focusing on improvement of the language or supplementing its features - they're just trying to make up for how easy it is to write bug-ridden EVM contracts! Good on them, but it points to how fundamentally flawed the EVM is by design.
FWIW I agree with the intent of this statement. The odd lesson we've learned is that extreme testability can overcome broken tools to create secure code. It's not the outcome you would expect since the orthodox in security engineering is that it depends on secure programming languages, frameworks, and compilers, which we have none of in Ethereum. But somehow people are carrying around hundreds of millions of dollars in smart contracts secured with high-powered testing and verification techniques. It's very weird.
I will also note that our long-term goal for Slither is to directly address some of the problems in Solidity (https://github.com/crytic/slither). It's like 2/3rds of a compiler already. It just needs a little extra push and we can generate EVM bytecode, then start ripping features out of the language that just aren't safe to use. It's amazing how far Ethereum has come with insecure tools but extreme testability. It begs the question what it would look like with both? I know Kadena is going for the clean slate approach (and we're keeping an eye on you all) but our investment at the moment is in adjustments to the current ecosystem.
The problem is likely that the platform won, it has a big ecosystem of tooling developing around it, and targeting it is best route. Maybe they can make a better language that compiles to it. Meanwhile, tool vendors are making tools that correct for flaws in the language rather than a little-known one. Also, given the tools out there, using the platform language with the tools might be safer than the other language.
The language I encouraged people to use for smart contracts was SPARK Ada. It's a mature language used in many industrial projects. Verification is easier with it. Building on what it already had would've cost them less and reduced risk. A SPARK Ada to EVM compiler is all they needed.
The EVM is just a virtual machine. People are working on languages with less opportunity for bugs than Solidity, like Flint: https://github.com/flintlang/flint
The EVM itself isn't the end of the road either. Over the next couple years, Ethereum is migrating to a more scalable sharded version using proof of stake, and it looks like that will allow pluggable execution engines.
It's in the sense that it's extremely difficult to write a program and fully understand what it's doing - that's why higher order representations and correct-by-construction languages were invented. Rather than putting bandaids on it with secondary and tertiary frameworks, languages like Haskell were built to be correct by construction.
But then there's the question of whether any of those flaws are even reasonable to take upon your contract when you consider their use-case within a blockchain context. If you're only ever running terminating programs, why expose the entire language to the attack surface of non-determinism and re-entrancy bugs when there are very reasonable and expressive CbyC languages available?
EVM is not a language. EVM is deterministic otherwise the whole Ethereum would not function at all. Your criticism is directed more towards ie. Solidity. However alternatives are not availble as far as I know. There are some research projects but everything in this area is in alpha-research state.
The problem is with the structure of the VM and its bytecode language. There are additional problems with Solidity, but the VM itself is the problem. Solidity only amplifies a poor architecture: https://medium.com/kadena-io/the-evm-is-fundamentally-unsafe...
If you're looking for push-button vulnerability detection for Ethereum smart contracts, then you want to use Slither (https://github.com/crytic/slither), not Manticore. Manticore's detector features were shoe-horned in because it kept getting compared to Mythril, not because they are a core feature. Manticore's core feature is its ability to reliably explore the state space of a program and generate inputs to reach them.
If you want to compare the effectiveness of symbolic execution engines, then you should compare how many unique codepaths they can reach. ETH Zurich recently completed such an analysis in their paper on VerX (https://files.sri.inf.ethz.ch/website/papers/sp20-verx.pdf), an abstract interpretation-based verifier, and found that Manticore had the most complete model of the EVM compared to other symbolic execution engines by far, and we have continued to improve its accuracy in the releases since then. That said, ETH Zurich did underestimate a few of the capabilities of Manticore that we hope to better describe in an upcoming academic paper.
If you're looking to start using Manticore, then browse through the manticore-examples repository (https://github.com/trailofbits/manticore-examples) where we have complete working solutions to CTF challenges, crackmes, logic bombs, and other programs that Manticore excels at exploring.
The SWC benchmark is not an effective way to test bug detection tools. The SWC only contains contrived examples that are not representative of vulnerabilities in real code. As such, it fails to capture important outcomes like false positives, which quickly lead to developers uninstalling the tool, and rewards "noisey" tools that simply alert on everything.
It would be more effective to test bug detection tools against a database of real smart contract code with precisely identified vulnerabilities and example patches. Any benchmark must contain more than simple tests: it must approximate real software with enough complexity to properly stress tools for automated bug discovery.
You'll note that ETH Zurich and ChainSecurity came up with their own benchmark that meets the description above when they evaluated VerX in the paper I linked above. I'm eagerly awaiting the release of those test cases since they'll help us improve the functionality of many of our tools.
Manticore is tested extensively against real contracts from our clients, a database of purposefully challenging code samples and mutants we've developed, and the Ethereum VMTests, a version of the state tests you linked. It has found many real bugs in our client's code.
