But What is a Neural Network? Deep learning, chapter 1

Below are questions for the video But what is a Neural Network? | Deep learning, chapter 1. NN is short for “neural network”.

  • What problem is the NN discussed throughout the video trying to solve?

  • According to the video, what kind of NN is best for:

    1. image recognition?
    2. speech recognition?

  • What is the name of the kind of neural network discussed in the video?

  • In simple mathematical terms, what does a neuron do?

  • What is the number in a neuron called?

  • What are the range of possible activations for a neuron?

  • In the NN example in the video:

    • What are the activations of the neurons in the first layer?
    • How many neurons are in the last layer of the NN? Why that many? What do the activations in neurons of the last layer mean?
    • What are the layers between the input layer and last layer called? How many such layers are there? Why? How many neurons are in these layers? Why?

  • Describe how neurons in one layer connect to other neurons.

  • What are the possible values for weights on th edges between neurons?

  • What is the sigmoid function?

  • Why is the sigmoid function function used at all? What is it’s purpose?

  • What is the purpose of a bias in a neuron?

  • In a NN, what is learning?

  • Explain each part of the following formula from the video:

    \[a^{(1)} = \sigma (Wa^{(0)} + b)\]

  • What squishing function do modern NNs often used instead of the sigmoid function? Why?

  • Near the end of the video, it is stated that a NN can be thought of as a kind of function. If you treat the NN in this video as a function, what is it’s input and output?

Practice Question

Suppose you created a multi-layer NN, the same kind as in the video, to recognize both hand-written digits and alphabetic letters. Each symbol is in a 200 x 200 pixel grayscale image, and the letters are both lowercase a-z and uppercase A-Z (there are no punctuation or other symbols). Suppose there are two hidden layers in the network, both with 25 neurons.

  1. How many neurons are in the:
    1. input layer?
    2. output layer?
    3. two hidden layers?
    4. entire network?
  2. How many edge weights are there from:
    1. The input layer to the first hidden layer?
    2. The first hidden layer to the second hidden layer?
    3. The second hidden layer to the output layer?
    4. In total, how many edge weights does this neural network have?
  3. How many biases does this neural network have?