From what little I've gathered about the subject, I think so. Spiking neural networks would be orders of magnitude more energy efficient than deep neural networks for image and audio processing. The main hurdle lies in designing network structure and weights; the cost function of a classical neural network can be differentiated using basic calculus, but optimizing a spiking neural network is not as easy.
Once you describe the specific modulation and noise properties you could recover the equivalent real valued continuous time neutral network from spiking neutral network. Then you can similarly map the training algorithms from gradient networks to "spike gradients".
The next necessary step is to use (potentially quantum) interference in addition to integration or summation to train the interconnections.
You will likely end up with quite complicated Hessians in both time and value.
Study: Ultralow power artificial synapses using nanotextured magnetic Josephson junctions
Citation: Michael L. Schneider, Christine A. Donnelly, Stephen E. Russek, Burm Baek, Matthew R. Pufall, Peter F. Hopkins, Paul D. Dresselhaus, Samuel P. Benz, William H. Rippard. Science Advances 26 Jan 2018. Vol. 4, no. 1, e1701329.
Abstract: Neuromorphic computing promises to markedly improve the efficiency of certain computational tasks, such as perception and decision-making. Although software and specialized hardware implementations of neural networks have made tremendous accomplishments, both implementations are still many orders of magnitude less energy efficient than the human brain. We demonstrate a new form of artificial synapse based on dynamically reconfigurable superconducting Josephson junctions with magnetic nanoclusters in the barrier. The spiking energy per pulse varies with the magnetic configuration, but in our demonstration devices, the spiking energy is always less than 1 aJ. This compares very favorably with the roughly 10 fJ per synaptic event in the human brain. Each artificial synapse is composed of a Si barrier containing Mn nanoclusters with superconducting Nb electrodes. The critical current of each synapse junction, which is analogous to the synaptic weight, can be tuned using input voltage spikes that change the spin alignment of Mn nanoclusters. We demonstrate synaptic weight training with electrical pulses as small as 3 aJ. Further, the Josephson plasma frequencies of the devices, which determine the dynamical time scales, all exceed 100 GHz. These new artificial synapses provide a significant step toward a neuromorphic platform that is faster, more energy-efficient, and thus can attain far greater complexity than has been demonstrated with other technologies.
Unfortunately, they didn't mention noise properties in the abstract. Is it present in the paper?
Edit: it is. The device is actually pretty difficult to implement as it requires liquid hydrogen cooling. The noise is shown to be temperature dependent but a full error analysis is not present. Looking forward to further research and an attempt to convert this quantum device to higher temperatures.
I meant 'randomdrake posting relevant DOIs and abstracts all over HN threads discussing things that discuss scientific papers. But thanks for the clarification on the content of this paper :).
Yes, let's compare a natural system using low energy and fuelled by food with a hungry silicon system fuelled by electricity. It's going to be a great comparison. Only problem is that biological brains don't just compute, they also construct themselves, unlike the silicon chips. But we can ignore that as an insignificant detail. Let's also ignore that the brain has no clock or clearly separated layers. Doesn't matter that brain can't do backprop and we have no idea how it works, either. We can just compare neuronal speeds as if it's apples to apples. /s
Can you please not post snarky dismissals like this? It isn't that you're wrong, it's that the point is a shallow one. No one here likes linkbait titles, but ranting in response to them doesn't make for curious discussion either. (We've edited the title above to use a better phrase from the article itself.)
The following site guideline, applied at the article level, covers this situation: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize." https://news.ycombinator.com/newsguidelines.html
> The synapses can fire up to one billion times per second — several orders of magnitude faster than human neurons — and use one ten-thousandth of the amount of energy used by a biological synapse.
So if this is true, and these chips function as expected, we should be able to simulate a human brain orders of magnitude faster than realtime using with less than 2mW.
That requires about 200W of liquid hydrogen cooling right now. Brain, some 20-40W of above room temperature water cooling. (Also evaporative.)
Total efficiency equation is not yet favourable.
Random Conjecture: the most efficient way to build a new intelligent entity is through the process that we’ve been familiar with all this time — natural birth.
Intuition: to truly achieve all of the functions of the human brain and neurons, the most efficient structure is one that is biological or simulates our biological composition. There is likely no process (energy and material-wise) to produce human cells at the same fidelity as simply having sex and giving birth.
Random Conjecture: the most efficient way to build a new flying entity is through the process that we’ve been familiar with all this time — natural birth.
