M.E. Irizarry-Gelpí

M.E. Irizarry-Gelpí

Physics impostor. Mathematics interloper. Husband. Father.

Deep Learning Fundamentals with Keras 1

Deep learning is hot stuff. There is a lot of excitement around it. Some applications are color restoration, speech reenactment, automatic handwriting generation, automatic machine translation, object classification, and self-driving cars.

Deep learning uses artificial neural networks, which are inspired by biological neural networks. Some important parts of a neuron are

  • soma: nucleus of the neuron and main body
  • dendrites: network of arms sticking out of the body
  • axon: long arm stick out of the soma in a distinct direction
  • synapses: whiskers at the end of the axon

The dendrites receive information from electrical pulses from sensors or synapses from adjoining neurons. They then carry the data to the soma. The soma processes and combines the signal and passes it onward to the axon. The axon carries the data to the synapses, and this output can become the input to other neurons.

An artificial neuron has similar components and behavior: soma, dendrites, and axons. A network of neurons is divided into different layers. The first layer is called the input layer. The last layer is called the output layer. The intermediate layers are called hidden layers.

Forward propagation is the process for which data passes through from the input layer to the output layer. The data flows through the network via connections to dendrites. Every connection has a specific weight. Consider a neuron with two dendrites and one axon. That is, two inputs \(x_{1}\) and \(x_{2}\), and one result \(z\). As the inputs pass through the connections, they are adjusted by the connection weights \(w_{1}\) and \(w_{2}\). The neuron processes the inputs by calculating a weighted sum for the result:

\begin{align*} z = w_{1} x_{1} + w_{2} x_{2} + b_{1} \end{align*}

Here \(b_{1}\) is a bias term. In practice, this linear transformation does not get you very far, so the result is applied a non-linear transformation to obtain the output:

\begin{align*} a = f(z) \end{align*}

One popular transformation is the logistic function:

\begin{align*} f(z) = \frac{1}{1 + \exp(-z)} \end{align*}

These non-linear transformations are called activation functions. These functions are important because they determine whether a neuron is activated or not.