What are the best books to study Neural Networks from a purely mathematical perspective?












5












$begingroup$


I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).



Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?






matrices book-recommendation mathematical-modeling neural-networks







share|cite|improve this question







New contributor




Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$

















    5












    $begingroup$


    I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).



    Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?






    matrices book-recommendation mathematical-modeling neural-networks







    share|cite|improve this question







    New contributor




    Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.







    $endgroup$















      5












      5








      5


      3



      $begingroup$


      I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).



      Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?






      matrices book-recommendation mathematical-modeling neural-networks







      share|cite|improve this question







      New contributor




      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.







      $endgroup$




      I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).



      Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?






      matrices book-recommendation mathematical-modeling neural-networks




      matrices book-recommendation mathematical-modeling neural-networks






      share|cite|improve this question







      New contributor




      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|cite|improve this question







      New contributor




      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|cite|improve this question




      share|cite|improve this question






      New contributor




      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 2 hours ago









      EliEli

      341




      341




      New contributor




      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          2 Answers
          2






          active

          oldest

          votes


















          3












          $begingroup$

          For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.



          This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf



          I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.






          share|cite|improve this answer











          $endgroup$





















            2












            $begingroup$

            I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.






            share|cite|improve this answer









            $endgroup$









            • 1




              $begingroup$
              Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
              $endgroup$
              – Eli
              2 hours ago











            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "69"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            noCode: true, onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });






            Eli is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            3












            $begingroup$

            For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.



            This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf



            I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.






            share|cite|improve this answer











            $endgroup$


















              3












              $begingroup$

              For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.



              This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf



              I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.






              share|cite|improve this answer











              $endgroup$
















                3












                3








                3





                $begingroup$

                For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.



                This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf



                I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.






                share|cite|improve this answer











                $endgroup$



                For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.



                This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf



                I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited 1 hour ago

























                answered 1 hour ago









                Shamisen ExpertShamisen Expert

                2,81821945




                2,81821945























                    2












                    $begingroup$

                    I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.






                    share|cite|improve this answer









                    $endgroup$









                    • 1




                      $begingroup$
                      Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                      $endgroup$
                      – Eli
                      2 hours ago
















                    2












                    $begingroup$

                    I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.






                    share|cite|improve this answer









                    $endgroup$









                    • 1




                      $begingroup$
                      Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                      $endgroup$
                      – Eli
                      2 hours ago














                    2












                    2








                    2





                    $begingroup$

                    I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.






                    share|cite|improve this answer









                    $endgroup$



                    I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.







                    share|cite|improve this answer












                    share|cite|improve this answer



                    share|cite|improve this answer










                    answered 2 hours ago









                    Jair TaylorJair Taylor

                    9,05432144




                    9,05432144








                    • 1




                      $begingroup$
                      Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                      $endgroup$
                      – Eli
                      2 hours ago














                    • 1




                      $begingroup$
                      Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                      $endgroup$
                      – Eli
                      2 hours ago








                    1




                    1




                    $begingroup$
                    Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                    $endgroup$
                    – Eli
                    2 hours ago




                    $begingroup$
                    Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
                    $endgroup$
                    – Eli
                    2 hours ago










                    Eli is a new contributor. Be nice, and check out our Code of Conduct.










                    draft saved

                    draft discarded


















                    Eli is a new contributor. Be nice, and check out our Code of Conduct.













                    Eli is a new contributor. Be nice, and check out our Code of Conduct.












                    Eli is a new contributor. Be nice, and check out our Code of Conduct.
















                    Thanks for contributing an answer to Mathematics Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Loup dans la culture

                    Image
                    Loups sur une enluminure du Bestiaire d'Aberdeen Le loup est l'animal le plus emblématique de l'histoire de l'Eurasie, il était à l'honneur durant l'Antiquité chez la totalité des anciens peuples européens [ 1 ] . Sommaire 1 Le loup dans la mythologie 2 Les loups dans le folklore 2.1 L'évolution des mentalités 2.2 Le loup dans la psychanalyse 3 Le loup dans les œuvres culturelles 3.1 Le loup dans la littérature 3.1.1 Fables 3.1.2 Poésie 3.1.3 Romans 3.1.4 Littérature jeunesse 3.1.4.1 Quelques titres d'albums contemporains 3.1.5 Bande dessinée 3.2 Le loup à l'écran 3.2.1 Le cinéma et la télévision 3.2.2 Documentaires 3.2.3 Dessins animés 3.2.4 Jeux vidéo 3.3 Le loup en peinture et en sculpture 3.4 Le loup dans les œuvres musicales 4 Le loup et les noms propres 5 Bibliographie 6 Notes et références 7 Voir aussi 7.1 Articles connexes
                    Read more

                    How to solve the problem of ntp “Unable to contact time server” from KDE?

                    Image
                    1 0 I want to set date and time automatically but this error appears when I try to .. I'm using openSUSE, I tried different servers but it didn't work either, help please! It doesn't seem to be a network problem: $ ping pool.ntp.org 64 bytes from 41.78.128.17: icmp_seq=1 ttl=41 time=260 ms To the question of whether ntpd is installed and running: $ ps -C ntpd PID TTY TIME CMD i.e. there's no ntpd process running. $ rpm -qa | grep ntp yast2-ntp-client-3.1.12-1.7.noarch ntp-4.2.6p5-25.2.1.i586 Further information from the comments: $ sudo ntpdate pool.ntp.org 25 Sep 19:41:51 ntpdate[3162]: no server suitable for synchronization found networking kde opensuse ntp ntpd sh
                    Read more

                    ASUS Zenbook UX433/UX333 — Configure Touchpad-embedded numpad on Linux

                    Image
                    0 1 I have an ASUS Zenbook UX433,on which ASUS decided to innovate by embedding a Numpad inside the touchpad (see picture 1). Toggling between classic touchpad mode and numpad (also turns the touchpad LEDs on) can be done through the golden button in the top-right corner of the touchpad. However, on my Manjaro arch Linux config, this behavior does not work by default and I can only use the touchpad, no numpad. Has anyone been able to configure the numpad to work on Linux, or have any idea how this could be achieved? EDIT1: I have had a look at the official Windows drivers for the UX433FA TouchPad, but was not able to make anything of it. I tried installing them on Linux using tools like NDISWrapper, but they do not detect the TouchPad/Numpad hardware, and my understanding of drivers is too limited.
                    Read more