CHAROTAR UNIVERSITY OF SCIENCE AND TECHNOLOGY

FACULTY OF TECHNOLOGY AND ENGINEERING

MACHINE LEARNING

ARTIFICIAL NEURAL NETWORKS

1. What is the biological motivation behind using the artificial neural network? Define artificial
neural networks and list down the applications in which ANN is use.
 Brain consists of 200 billion of neurons and neuron is formed from 4 basic parts as Dendrites,
Cell Body, Axon, and Synapses. The neuron collect signals from Dendrites, and the cell body
cells sums up all the signals collected, and when the summation reaches the threshold the
signal pass through the axon to the other neurons. The Synapses indicate the strength of the
interconnection in between the neurons. Similar to the brain, the Artificial Neural Network,
imitates this biological Neural Network of human body.

 ANN: A computing system made up of a number of simple, highly interconnected processing

elements(neurons), which process information by their dynamic state response to external
inputs simultaneously to solve a specific problem.

 Applications:
Handwriting Recognition
Forecasting
Credit Card fraud detection
Optimization problems
Speech recognition, etc

2. Define perceptron with a well labelled diagram. Why the perceptron cannot be used to
implement the EXCLUSIVE-OR function?
 A perceptron is a neural network unit (an artificial neuron) that does certain computations to
detect features or business intelligence in the input data.
 A "single-layer" perceptron can't implement XOR. The reason is because the classes in XOR
are not linearly separable. You cannot draw a straight line to separate the points (0,0),(1,1)
from the points (0,1),(1,0).

3. List and explain the perceptron training rules.




4. What is linear separability? Give example of linearly separable and inseparable problem?
Linear separability refers to the fact that classes of patterns with n-dimensional vector x = (x1,
x2, ... , xn) can be separated with a single decision surface.
Linearly Separable Problem: AND, OR, NOT Problem
Non-Linearly Separable Problem: XOR Problem
5. Explain gradient descent algorithm and also derive Gradient Descent rule
 Gradient descent is an optimization algorithm used to minimize some function by iteratively
moving in the direction of steepest descent as defined by the negative of the gradient. In
machine learning, we use gradient descent to update the parameters of our model.
Parameters refer to coefficients in Linear Regression and weights in neural networks.
 https://mccormickml.com/2014/03/04/gradient-descent-derivation/

6. What is perceptron? Write the differences between Single Layer Perceptron(SLP) and
Multilayer Perceptron(MLP).
 Perceptron:
REFER Q2
Difference
Single Multilayer
Layers- input ,output Layers-input,hidden,output
Can solve lineraly separable problems only Introduced to solve non-linearly separable
problems.
Input nodes are connected to output Input nodes are connected to hidden layer
nodes directly. nodes and then ,hidden nodes are
connected to output nodes.

7. Explain the Limitation with Back Propagation Algorithm.

 Local minima problem,scaling and slow convergenc.
Backpropagation can be quite sensitive to noisy data
You need to use the matrix-based approach for backpropagation instead of mini-batch.
8. Explain back propagation algorithm and derive expressions for weight update relations .
9. Explain Hebbian Learning rule.

10. Explain Neural Network architecture based on Learning Method.

11. Explain various activation functions of neural network.
Linear or Identity Activation Function – f(x)=x.

Non linear Activation Function

1.Sigmoid -f(x)= 1/1+ e-x

2. Tanhx - f(x)= 2/1+ e-2x - 1

3. ReLU( Rectified Linear Unit) f(x)= 0 for x<0, x for x >=0

4. Leaky RELU f(x)= 0.01x for 0 x<0 , x for x >=0

5. ArcTanh – f(x) = tan-1(x)

6. Softmax

12. Mention the characteristics of problems suitable for ANNs.

13. List some applications of ANNs.
Refer Q1
14. What are the design parameters of ANN?
 Number of hidden layers
 Learning rate
 Activation Function
 Epochs
 Mini Batch Size
 Dropouts
(Reference:https://towardsdatascience.com/neural-networks-parameters-
hyperparameters-and-optimization-strategies-3f0842fac0a5)
15. Mention the linear and nonlinear activation functions used in Artificial Neural Networks.
Refer Q11
16. What are the design steps to be followed for using ANN for your problem?

17. Explain the three classifications of ANNs based on their functions. Explain them in brief.
Feedback ANN – In these type of ANN, the output goes back into the network to achieve the
best-evolved results internally. The feedback network feeds information back into itself and is
well suited to solve optimization problems. Feedback ANNs are used by the Internal system
error corrections.
Feed Forward ANN – A feed-forward network is a simple neural network consisting of an input
layer, an output layer and one or more layers of neurons.Through evaluation of its output by
reviewing its input, the power of the network can be noticed base on group behavior of the
 connected neurons and the output is decided. The main advantage of this network is that it
learns to evaluate and recognize input patterns.
Classification-Prediction ANN –It is the subset of feed-forward ANN and the classification-
prediction ANN is applied to data-mining scenarios. The network is trained to identify
particular patterns and classify them into specific groups and then further classify them into
“novel patterns” which are new to the network.

18. Develop simple ANNs to implement the three input AND, OR and XOR functions.
19. Draw the architecture of a Multilayer perceptron (MLP) and explain its operation. Mention its
advantages and disadvantages.
A Multi Layer Perceptron (MLP) contains one or more hidden layers (apart from one input and
one output layer). While a single layer perceptron can only learn linear functions, a multi layer
perceptron can also learn non – linear functions.

Figure 4 shows a multi layer perceptron with a single hidden layer. Note that all connections
have weights associated with them, but only three weights (w0, w1, w2) are shown in the
figure.

Input Layer: The Input layer has three nodes. The Bias node has a value of 1. The other two
nodes take X1 and X2 as external inputs (which are numerical values depending upon the input
dataset). No computation is performed in the Input layer, so the outputs from nodes in the
Input layer are 1, X1 and X2 respectively, which are fed into the Hidden Layer.

Hidden Layer: The Hidden layer also has three nodes with the Bias node having an output of
1. The output of the other two nodes in the Hidden layer depends on the outputs from the
Input layer (1, X1, X2) as well as the weights associated with the connections (edges). Figure
4 shows the output calculation for one of the hidden nodes (highlighted). Similarly, the output
from other hidden node can be calculated. Remember that f refers to the activation function.
These outputs are then fed to the nodes in the Output layer.
 Output Layer: The Output layer has two nodes which take inputs from the Hidden layer and
perform similar computations as shown for the highlighted hidden node. The values
calculated (Y1 and Y2) as a result of these computations act as outputs of the Multi Layer
Perceptron.

Given a set of features X = (x1, x2, …) and a target y, a Multi Layer Perceptron can learn the
relationship between the features and the target, for either classification or regression.

20. A two-layer network is to have four inputs and six outputs. The range of the outputs is to be
continuous between 0 and 1. What can you tell about the network architecture?
 Specifically,

(a)How many neurons are required in each layer?

-Six neurons, one for each output, are required.

(b)What are the dimensions of the first-layer and second layer weight matrices? (Hidden
layer neurons are 5)
(c) What kinds of transfer functions can be used in each layer?

- logsig

21. Assume we have a set of data from patients who have visited UPMC hospital during the year
2011. A set of features (e.g., temperature, height) have been also extracted for each patient.
Our goal is to decide whether a new visiting patient has any of diabetes, heart disease, or
Alzheimer (a patient can have one or more of these diseases).
(a) We have decided to use a neural network to solve this problem. We have two
choices: either to train a separate neural network for each of the diseases or to train
a single neural network with one output neuron for each disease, but with a shared
hidden layer. Which method do you prefer? Justify your answer.

For preferring shared: Neural network with a shared hidden layer can capture dependencies
between diseases. It can be shown that in some cases, when there is a dependency between
the output nodes, having a shared node in the hidden layer can improve the accuracy.

For preferring separate: If there is no dependency between diseases (output neurons), then
we would prefer to have a separate neural network for each disease.