You read your chips' datasheets that go in detail on pinouts and protocols. You use logic analyzers that capture signals and programmatically decode a multitude of protocols from those signals. When you don't know what are the pinouts or protocols, you compare against similar enough known ones, or bruteforce them.
Some examples:
* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.
* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.
* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.
* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.
* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.
So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.
When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.
I agree with but extend your last point about breadcrumbs - it's not just that write-ups are hard to find, it's that even several great write-ups for adjacent projects may not be sufficient for a beginner to synthesize a path for his specific project. I spent a week watching YouTube videos and reading in preparation to dump the router firmware, and it still took me 3 days before I committed to actually soldering the wires to the UART contacts - the videos all had header pins! (Sounds dumb, but I'd never permanently altered hardware before.)
In case people are curious, the router automatically gave me a root shell when I accidentally pressed a key during startup once I was connected to UART (using a [programmer board](https://a.co/d/0aVp22Mq)), and I was able to dump firmware using `dd` on one of the device partitions labeled "firmware", then `netcat`ing it out.
I also tried dumping the SPI flash chip directly using tiny alligator clips and flashrom, but I think I literally didn't know to build the latest version of flashrom from source, and so couldn't connect. Maybe same issue with JTAG - old software couldn't find my new programmer board. But I wouldn't have known what to do with gdb access even if I had connected to JTAG! Oh, I also didn't know that I needed to solder together the 2 pins that were specified by the datasheet to activate JTAG - thought I could just hold a wire between them like stealing a car in the movies lol.
Final example of "no breadcrumbs": I wanted to modify the firmware, then flash it back, but I couldn't for several days and the life of me figure out how to repack the firmware after using binwalk to unpack it. Ultimately I flashed something badly formatted out of desperation (maybe wrong size??) and bricked the router. Every guide mentioned "flashing back", but none showed how to do it - I am actually still curious haha, what was I missing?
It looks like the compilation database is only used for discovering source files, without any additional cross translation unit analysis [1]. Even if you don't plan on implementing data flow tracking or other passes, it can still be interesting to know on which translation unit a function declaration is defined on.
Also, it seems we always have to recompile each unit when switching between source files. Consider pre-compiling and caching so it's only done once. That should also enable users to search across files/ASTs.
> Fraud detection in transaction data is mostly SQL. Not machine learning, not graph databases, not whatever Gartner is hyping this year. SQL, run against the right tables, with the right joins, looking for the right shapes.
It's also not all program-integrity, which is the only work that could justify such blanket statements. Worse is better as long as it addresses the problem domain.
Fintech clients are generally interested in knowing whether a transaction happening _right now_ is fraud. They want to know that in a few milliseconds, for high-dimensional data. It's work done at a scale where relational databases cannot meet these real-time constraints, and instead find other uses like historical data loading. That's how you end up with in-memory databases, stream-processing engines, and yes, even machine learning.
Having said that, some of the author's points are valid, and I'm looking forward for their next writings, in particular dealing with noisy alerts is a general problem beyond performance engineering.
In my experience, what you're describing would more specifically be called Fraud Prevention rather than Fraud Detection. Both tend to coexist and are complementary in a mature setup.
For Prevention, you're always going to be constrained by latency requirements, available data and an incomplete picture of user behaviour. You make a quick decision using ML and rules that deals with the majority of cases. But those constraints make it impossible to precisely prevent all fraud.
Detection deals with the downstream consequences of this. A team of analysts will typically analyse the accepted transactions for signs of fraud. This is particularly important for fraud types where you don't get an external signal like a chargeback or customer complaint. Platform integrity is one such example. But Fintechs will also see this building anti-money laundering systems - you need to go looking for the fraud. This is the process the article is describing.
I say they're complementary because the detected transactions become the labels for training and evaluating the next iteration of prevention models.
The author is referring to limitations in analysing banking:
> Ghidra supports the 8051 architecture but not code banking.
Usually in these ISAs an I/O port or a register sets the bank number, so any processor module should be able to resolve concrete banked references. But you still need to know what that register holds in various code paths, which are likely dynamically computing those values.
No tooling can give this out-of-the-box, as it relies on knowing the concrete initial state of the system (i.e. memory and register contents), and knowing what to return when hooking into I/O accesses.
Once these are known, we can leverage the built-in pcode emulator and run it with this state. It seems nowadays Ghidra has some built-in support for Z3, but I personally never used it, so I'm not sure how viable it is for symbolic execution. Regardless, with either approach, we would now have concrete banked code references being resolved, and could script some auto annotation of the disassembly with these references. These would be equivalent to what the author gathered from the logic analyzer trace.
A pure static analysis approach seems to suggest one would manually brute-force through all possible bank numbers at any given code path, which I guess is only viable if you have the time for that.
Which drives and parameters for the READ BUF SCSI command yielded the expected 2366 bytes per sector? I imagine that it was combined with seeks to each sector before reading from the buffer (as it would be harder to isolate multiple sectors data in cache?).
Nice work Dmitry, looking forward to read your next article.
The later model Pixter Multimedia had the full memory space accessible via JTAG, which is how some carts and even boot ROM got dumped a while ago [1], is it the same deal with Pixter Color?
That OpenOCD script was a bit flaky, and sometimes the boot ROM would be already unloaded before reading, maybe you have some insights in how to make it more robust.
btw, have you looked into the original Pixter? The cart connector seems to have a very narrow bus, so it doesn't look like those carts have code, and probably can only be dumped with a decap.
That only dumps the data. That’s the easy part. None of that dumps the melodies.
The pin outs that page links to are also not quite accurate. I need to finish editing my other article on this.
I have indeed looked into the original Pixter. Deeply: I have decoded the bus, documented the device, dumped games, and produced a working emulator.
The cartridges do contain memory. Most of them are about 1 MB in size, split between code (the maximum for which is 32 kB) and audio effects + images which occupy the rest of the space. If you are very, very curious and don’t want to wait for me to finish my editing, email me and I can explain how it works.
For other architectures, it feels like a missed opportunity to not have an independent WASM build of MAME's debugger, as the whole project could already be built in WASM (although I think the latest versions were broken, as that target isn't actively maintained): https://docs.mamedev.org/initialsetup/compilingmame.html#ems...
I found their ELF format specification to have a decent coverage, even if not completely exhaustive (e.g. some debug info isn't breakdown after a certain point, but it just might be incomplete rather than limitations).
