Simon Fraser University

Spring 2004

CMPT 882

Instructor: Oliver Schulte

Due: 4:30 pm. Friday, January 30, 2004. Please put your assignment in my mailbox in the departmental office. I believe the office closes at 4:30 pm.

Paper and Pencil: Exercise 3.2, 3.4, 4.7. Bonus question: 4.9. Note: 4.9 is a “thinking question”, which can be treated at various levels of precision, from some relevant informal observations to a full-blown formal proof. I will grade it as a bonus question. I’d like you to give some thought to it because it’s quite instructive about neural networks, but if you find it difficult to get your head around it, don’t sink too much time into it (e.g. > 2 hours).

The purpose of this assignment is for you to get a sense of various standard machine learning technology, by looking at some implementations of the algorithms rather than a general definition. To pass, I'd like to see some evidence that you've managed to run an algorithm and that you had some understanding of the output you saw. The evidence should be in the form of a print-out of screen displays or program outputs, and usually a brief discussion of the output (things you noticed or that strike you.) Specifically, this week I'd like you to run an implementation of one or more neural nets. What follows are some specific instructions and suggestions for experimentation. Feel free to explore and share with us if you find something interesting.

Check Out the Web

  1. Enter in google, "neural net software", "neural net companies". Take a look at what comes up, check out a few links. You'll notice that there are quite a few companies selling some kind of neural net software - perhaps a future employer? http://www.emsl.pnl.

Neural Net Software

  1. There are a number of companies that offer evaluation copies of their neural net software. One that I've tried is NeuroDimensions. You can get an evaluation copy of their software and run their demos (or more runs if you like). They have something for the PC - I'm not sure about other platforms.
  2. A number of research groups offer Java demos. One nice site is http://neuron.eng.wayne.edu/software.html which has a number of neural net architectures for various problems. The function approximation is nice to explore. Which functions seem to be easy for the network, and which are hard (e.g., compare linear with sin). Let the network train a number of times, and see what happens - does the error always go down with training? How quickly? - A downside to this site is that they don't seem to offer ways of inspecting the network, but some of the documentation has nice pictures.
  3. Aaron Hunter recommends two applets for handwriting recognition by neural nets, namely http://members.aol.com/Trane64/java/JRec.html, which allows you to experiment with your own handwriting (rather, "mouse writing"), and http://www.geocities.com/SiliconValley/2548/ochre.html. Julia Birke has tried out http://www-dsi.ing.unifi.it/neural/software.html which has a number of machine learning software, including neural nets, and www.geocities.com/CapeCanaveral/1624/ for a number of illustrative tasks solved by neural nets. Kimberley Voll worked with EasyNN, a fairly small-scale commercial package.
  4. The text book has neural net code (in C, to be run on a Unix system) and face recognition training data. Instructions and questions are posted as an assignment. If you use this, YOU DO NOT HAVE TO WORK THROUGH ALL THE QUESTIONS. Just get it to run on the input files; basically, you would look at Part I, Questions 1-3 and 5-6. As always, you are of course welcome to play around. Some familiarity with this C code might be useful if you are thinking of doing your own programming with neural nets.
  5. Gabor Melli has used the NNinExcel tool which allows you to run a neural net with in an Excel spreadsheet. As a neural net package, it’s a bit slow, but it has a nice user interface and you can use the Excel graphing tools to visualize your findings.
  6. The Stuttgart Neural Network Simulator is a neural network package for the research community.
  7. Zhongmin Shi recommends the commercial package Alyuda NeuroIntelligence.
  8. There are other network simulators and packages on the web. Glendon Holst recommends Lutz Prechelt's website which has frequently asked questions, pointers to literature and an annotated list of free software.