HN2new | past | comments | ask | show | jobs | submitlogin

As an ML practitioner, you certainly know that the ANNs you use have not much in common with biological (real) NNs. First, ANNs tend to be mostly feedforward, while rNNs are highly recurrent. ANNs are therefore not very good in tasks where memory is needed (I know about the developments involving LSTM neurons, but they are not analogous to the implicit memory of recurrent neural networks.) Second, ANNs usually don't have a time dimension, the firing is essentially a floating point value instead of action potential. Third, recurrent neural network structures do not scale: an efficient/reliable/reduntant system of 100B neurons will probably have extremely different structures than another one with 100M neurons -- because of recurrency and other stuff a rNN is a chaotic process (in the sense of sensitivity to parameters) that should be stabilized by the structure.

And there are many-many other differences. Note that our task is not to solve problems but to figure out how the brain works.

Also, recurrent neural network simulation cannot really be scaled right now, we don't have the hardware. It is not parallelizable with our current tools because of the huge number of connections.

(Disclaimer: I was involved in a project trying to model real neural networks. It wasn't a huge success, but we learned a lot.)



This is a good informed summary, thank you. I asked the same question of my Alzheimer researcher friend, and he gave a pretty similar response including aspects like the huge computational requirements and the basic non-similarity of rNNs and ANNs (albeit with a disclaimer that he wasn't in the field).

Nonetheless, I look forward to seeing more simple rNNs being created over time (besides the C. elegans one that was modeled recently). Who knows what strange organizational rules or structures we will discover from this strand of research?


Many scientists (including my former advisor DeLiang Wang) are simulating dynamic oscillatory neural networks:

http://web.cse.ohio-state.edu/~dwang/pnl/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: