Novel interpretable mechanism of neural networks based on network decoupling method

Abstract

The lack of interpretability of the neural network algorithm has become the bottleneck of its wide application. We propose a general mathematical framework, which couples the complex structure of the system with the nonlinear activation function to explore the decoupled dimension reduction method of high-dimensional system and reveal the calculation mechanism of the neural network. We apply our framework to some network models and a real system of the whole neuron map of Caenorhabditis elegans. Result shows that a simple linear mapping relationship exists between network structure and network behavior in the neural network with high-dimensional and nonlinear characteristics. Our simulation and theoretical results fully demonstrate this interesting phenomenon. Our new interpretation mechanism provides not only the potential mathematical calculation principle of neural network but also an effective way to accurately match and predict human brain or animal activities, which can further expand and enrich the interpretable mechanism of artificial neural network in the future.

Graphical abstract

Keywords

neural networks / interpretability / dynamical behavior / network decouple

References

[1] Arjovsky M, Chintala S, Bottou L (2017). Wasserstein generative adversarial networks. In: 34th International Conference on Machine Learning. Sydney, 214–223

[2] Bach S, Binder A, Montavon G, Klauschen F, Müller K R, Samek W (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10(7): e0130140

[3] Bumbarger D J, Riebesell M, Rödelsperger C, Sommer R J (2013). System-wide rewiring underlies behavioral differences in predatory and bacterial-feeding nematodes. Cell, 152(1–2): 109–119

[4] Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016). InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In: 30th Conference on Neural Information Processing System (NIPS). Barcelona, 2180–2188

[5] Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, 1724–1734

[6] Cook S J, Jarrell T A, Brittin C A, Wang Y, Bloniarz A E, Yakovlev M A, Nguyen K C Q, Tang L T, Bayer E A, Duerr J S, Bülow H E, Hobert O, Hall D H, Emmons S W (2019). Whole-animal connectomes of both Caenorhabditis elegans sexes. Nature, 571(7763): 63–71

[7] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014). Generative adversarial nets. In: 27th International Conference on Neural Information Processing Systems (NIPS). Montreal, Quebec, 2672–2680

[8] Hendricks L A, Akata Z, Rohrbach M, Donahue J, Schiele B, Darrell T (2016). Generating visual explanations. In: European Conference on Computer Vision. Amsterdam, 3–19

[9] Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780

[10] Kádár Á, Chrupała G, Alishahi A (2017). Representation of linguistic form and function in recurrent neural networks. Computational Linguistics, 43(4): 761–780

[11] Kindermans P J, Schütt K T, Alber M, Müller K R, Erhan D, Kim B, Dähne S (2017). Learning how to explain neural networks: PatternNet and PatternAttribution. arXiv preprint, arXiv: 1705.05598

[12] Li J, Chen X, Hovy E, Jurafsky D (2015). Visualizing and understanding neural models in NLP. In: 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, CA, 681–691

[13] Mirza M, Osindero S (2014). Conditional generative adversarial nets. arXiv preprint, arXiv: 1411.1784

[14] Nguyen A, Clune J, Bengio Y, Dosovitskiy A, Yosinski J (2017). Plug & Play Generative Networks: Conditional iterative generation of images in latent space. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, 3510–3520

[15] Park D H, Hendricks L A, Akata Z, Rohrbach A, Schiele B, Darrell T, Rohrbach M (2018). Multimodal explanations: Justifying decisions and pointing to the evidence. In: IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, 8779–8788

[16] Radford A, Metz L, Chintala S (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint, arXiv: 1511.06434

[17] Samek W, Binder A, Montavon G, Lapuschkin S, Müller K R (2017). Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems, 28(11): 2660–2673

[18] Schuster M, Paliwal K K (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11): 2673–2681

[19] Strobelt H, Gehrmann S, Pfister H, Rush A M (2018). LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Transactions on Visualization and Computer Graphics, 24(1): 667–676

[20] Tang Z, Shi Y, Wang D, Feng Y, Zhang S (2017). Memory visualization for gated recurrent neural networks in speech recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New Orleans, LA, 2736–2740

[21] Wu M, Hughes M C, Parbhoo S, Zazzi M, Roth V, Doshi-Velez F (2017). Beyond sparsity: Tree regularization of deep models for interpretability. In: 32nd AAAI Conference on Artificial Intelligence. New Orleans, LA, 1670–1678

[22] Yu L, Zhang W, Wang J, Yu Y (2016). SeqGAN: Sequence generative adversarial nets with policy gradient. In: 31st AAAI Conference on Artificial Intelligence. San Francisco, CA, 2852–2858

[23] Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X, Metaxas D N (2017). StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In: IEEE International Conference on Computer Vision (ICCV). Venice, 5908–5916

[24] Zhang Y, Xiao Y, Hwang S W, Wang H, Wang X S, Wang W (2017). Entity suggestion with conceptual explanation. In: 26th International Joint Conference on Artificial Intelligence (IJCAI). Melbourne, 4244–4250

[25] Zharov Y, Korzhenkov D, Shvechikov P, Tuzhilin A (2018). YASENN: Explaining neural networks via partitioning activation sequences. arXiv preprint, arXiv: 1811.02783

[26] Zhou B, Bau D, Oliva A, Torralba A (2019). Interpreting deep visual representations via network dissection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9): 2131–2145

RIGHTS & PERMISSIONS

2021 Higher Education Press