Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
WithinReason
on Nov 4, 2022
|
parent
|
context
|
favorite
| on:
Tinygrad: A simple and powerful neural network fra...
But then SUB, DIV and RELU could be an HLOP as well, no?
georgehotz
on Nov 4, 2022
[–]
We could have NEG instead of SUB, but with the constant folding it's a wash. DIV is already an HLOP with reciprocal (used to use POW, but that was slower. And what would you implement RELU in terms of?
WithinReason
on Nov 4, 2022
|
parent
[–]
max(0,x)
georgehotz
on Nov 4, 2022
|
root
|
parent
[–]
That's a ReduceOp right now, more annoying to reason about than a UnaryOp. But in the limit, yea. Or add an elementwise BinaryOp for max.
Submit a PR if you can improve something!
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
