This paper is horribly written. It's like the authors are trying to sell me on them as researchers, instead of helping me understand their research (y'know, the entire reason journals got started??). An entire section for "stream batching" was just too much, and none of their ideas were innovative or unique. It was incredibly dense, simply because it's obfuscated, which makes me believe the authors themselves don't really understand what they're doing.
The results aren't even very good. They claim 60x speedup, but compared to what? HuggingFace's Diffusers Autopipeline... a company notorious for buggy code and inefficient pipelines. And that's for naively running the pipeline on every image. Give me a break.
> instead of helping me understand their research (y'know, the entire reason journals got started??)
ML is crazy right now and people don't see papers as means of researchers communicating to other researchers. You write papers to reviewers. But your reviewers are stochastic so it's hard to write to them because they may or may not be in your niche.
I'll add though that this isn't why journals were created and that CS/ML doesn't typically use journals ({T,J}MLR, PAMI, and a few exist, sure) and instead write to conferences. Fixed dates, zero sum, 1.5 shot setting (1 rebuttal, zero revisions). Journals were created for dissemination of papers, indirectly about communicating to one another, but you know... now we got Arxiv and blogs and websites are sometimes way better just like how papers got better with pictures with computer graphics.
Somehow just hacking together code to create something is considered publishable these days. The code works but it really is just pasted together stuff from the last few weeks of research.
I don't think I have a problem with this tbh. Though this specifically looks more engineering and product oriented. What I do have a problem with is comparing papers across vastly different TRLs and comparing works done with 100 GPU years of compute to works with 1 GPU year (or less). Just completely different class of works and comparing is void of context, you know?
The reason I don't have a problem is I see papers as how we researchers communicate with other researchers. But I feel that's not how everyone sees them and there's the aspect that this is how we're judged so incentives get misaligned with actual goal. Idk if the reward hacking is ironic or makes sense because our job is to optimize. But don't let anyone try to convince you that reward (or any cost function) is enough.
The parameters and algorithms can be inferred from the code. Perhaps what’s unnecessary is tradition to wrap physical reality in human language semantics.
The complexity is in the hardware. Programming has only ever been templating desired machine state. Programmers fell into a religious like state of seeing their more ornate efforts as essential to making a purpose built counting machine count.
The results aren't even very good. They claim 60x speedup, but compared to what? HuggingFace's Diffusers Autopipeline... a company notorious for buggy code and inefficient pipelines. And that's for naively running the pipeline on every image. Give me a break.