You can use sin as activation function, but that would require careful initializ...

blackcat201 on April 30, 2022 | parent | context | favorite | on: Why train when you can optimize?

You can use sin as activation function, but that would require careful initialization to avoid gradient explosion as you would ended up with a lot of points where gradient is simply zero. You can refer to Implicit Neural Representations with Periodic Activation Functions for more details.

zarzavat on April 30, 2022 [–]

It works perfectly if you don’t have any parameters.