A neural network tutorial (with no math):
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
A slightly more formal introduction:
http://ieee.uow.edu.au/~daniel/software/libneural/BPN_tutorial/BPN_English/BPN_English/BPN_English.html
The original article on back-propagation.
http://www.iro.umontreal.ca/~pift6266/A06/refs/backprop_old.pdf
Hinton, G.E. Preface to the special issue on connectionist symbol processing, Artificial Intelligence 46, 1-4
http://www.cs.toronto.edu/~hinton/absps/connectionist.pdf
How backprop for feed-forward and recurrent nets can be derived cleanly from a Lagrangian formalism:
Y.
LeCun?, "A theoretical framework for Back-Propagation," in Proceedings of the 1988 Connectionist Models Summer School, (D. Touretzky, G. Hinton, and T. Sejnowski, eds.), (CMU, Pittsburgh, Pa), pp. 21-28, 1988.
http://yann.lecun.com/exdb/publis/pdf/lecun-88.pdf
The bias-variance dilemma for neural networks, by Geman et al, Neural Computation 1991 paper.
http://www.iro.umontreal.ca/~pift6266/A06/refs/neural_nets_bias_variance.pdf
Explores the issue of overfitting and underfitting with non-parametric models in general.
Michael Jordan's encyclopedic paper on neural networks, graphical models...
Neural networks. M. I. Jordan and C. Bishop. In Tucker, A. B. (Ed.), CRC Handbook of Computer Science, Boca Raton, FL: CRC Press, 1997.
http://www.cs.berkeley.edu/~jordan/papers/crc.ps.Z
Lecture notes from Rich Zemel's classes on machine learning and datamining:
http://www.cs.toronto.edu/~zemel/Courses/CS411/lects.html
Lecture notes from Geoffrey Hinton's classes on neural networks:
http://www.cs.toronto.edu/~hinton/csc2535/lectures.html
http://www.cs.toronto.edu/~hinton/csc321/lectures.html
Lecture notes from Sam Roweis undergraduate class in machine learning:
http://www.cs.toronto.edu/~roweis/csc2515/lectures.html
Statistical data-mining tutorials:
http://www.autonlab.org/tutorials/
I suggest the following: probability for data miners, probability density functions, Gaussians,
maximum likelihood estimation, cross-validation, neural networks, AI class introduction
If you really want to implement multi-layer neural networks back-prop, this paper is for you:
all the tricks and the theory behind them to efficiently train neural networks with backpropagation, including how to compute the optimal learning rate, how to back-propagate second derivatives, and other sundries.
Y.
LeCun?, L. Bottou, G. Orr, and K. Muller, "Efficient
BackProp?," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.), 1998.
http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf
Y. Le Cun and Y. Bengio. Pattern Recognition and Neural Networks.
In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 711-714. MIT Press, 1995.
http://www.iro.umontreal.ca/%7Elisa/pointeurs/handbook-patrec.ps
Dietterich, T. G. (2002). Machine Learning for Sequential Data: A Review. In T. Caelli (Ed.) Lecture Notes
in Computer Science. Springer-Verlag.
http://www.cs.orst.edu/~tgd/publications/mlsd-ssspr.pdf
Neal, R. M. (1990). Learning stochastic feedforward networks.
Technical Report CRG-TR-90-7, Dept. of Computer Science, University of Toronto
http://www.cs.toronto.edu/~hinton/absps/sff.pdf
Hinton, G. E., Dayan, P., Frey, B. J. and Neal, R. The wake-sleep algorithm for unsupervised Neural Networks. Science, 268, 1158-1161.
http://www.cs.toronto.edu/~hinton/absps/ws.pdf
Y. Le Cun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-Based Learning Applied to Document Recognition.
Proceedings of the IEEE, 86(11):2278-2324, 1998.
http://www.iro.umontreal.ca/~lisa/pointeurs/lecun-98.ps.gz
--
YoshuaBengio - 30 May 2006