Our research in blue neural networks
The BlueNN group have laid the ground (connected papers) for sparse training in deep learning (training sparse artificial neural networks from scratch), while introducing both static (static sparse training) and dynamic sparsity (dynamic sparse training). Besides the expected computational benefits, sparse training achieves in many cases better generalisation than dense training. For more details, please see:
- Static sparsity in Decebal Constantin Mocanu, Elena Mocanu, Phuong H. Nguyen, Madeleine Gibescu, Antonio Liotta, A topological insight into restricted Boltzmann machines, Machine Learning (2016), preprint https://arxiv.org/abs/1604.05978 (2016);
- Dynamic sparsity in Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H. Nguyen, Madeleine Gibescu, Antonio Liotta, Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science, Nature Communications (2018), preprint https://arxiv.org/abs/1707.04780 (2017);
- Short survey/position paper: Decebal Constantin Mocanu, Elena Mocanu, Tiago Pinto, Selima Curci, Phuong H. Nguyen, Madeleine Gibescu, Damien Ernst, Zita A. Vale, Sparse Training Theory for Scalable and Efficient Agents, AAMAS (2021), preprint https://arxiv.org/abs/2103.01636 (2021).
The BlueNN group short-term research interest is to conceive scalable deep artificial neural network models and their corresponding learning algorithms using principles from network science, evolutionary computing, optimization and neuroscience. Such models shall have sparse and evolutionary connectivity, make use of previous knowledge, and have strong generalization capabilities to be able to learn, and to reason, using few examples in a continuous and adaptive manner.
Most science carried out throughout human evolution uses the traditional reductionism paradigm, which even if it is very successful, still has some limitations. Aristotle wrote in Metaphysics “the totality is not, as it were, a mere heap, but the whole is something beside the parts”. Inspired by this quote, in long term, the BlueNN group would like to follow the alternative complex systems paradigm and study the synergy between artificial intelligence, neuroscience, and network science for the benefits of science and society.