artifical neural network

brijmohan

Download

Post to :

URL :

Related Presentations :

Add to Flag Embed

Views: 22818

Category: Education

Presentation Description

its a neural network slide

Comments

By: manimoney (1 month(s) ago) Private Comment

sir, send me this ppt on mks2405@gmail.com please its urgent

By: ganji (2 month(s) ago) Private Comment

sir i want this ppt ,please sent it to my email-id ravali.raga@gmail.com

By: sst1 (4 month(s) ago) Private Comment

sir, i need this ppt. can u send me on my email id? talele.shraddha@yahoo.in

By: sreejesh.ck (4 month(s) ago) Private Comment

i liked this ppt.please add more

By: biocys (6 month(s) ago) Private Comment

dear sir, could you please send me this ppt to my mail"gouthamigangula6@gmail.com" your presentation is relally quit intresting and hope that you will mail me....... after visiting your presentation it impressed me to give seminar on this particular topic.

See all

Presentation Transcript

Artificial Neural Networks:

Artificial Neural Networks

Slide2:

 Seeing the Billions of interconnections in the human brain, and the way the human brain recognizes different patterns, it was felt that there was a need to simulate the human brain.  How did Artificial Neural Networks develop?

Model of a Biological Neuron:

Model of a Biological Neuron

Three major components of biological neuron are::

Three major components of biological neuron are: Axon Cell Body Dendrites

Slide5:

At one end of the neuron there are a multitude of tiny filaments called Dendrites

Slide6:

Dendrites join together to form larger larger branches and trunks where they attach to the cell body

Slide7:

is a At the other end of the neuron single filament leading out of the cell body called the axon

Slide8:

Axon has extensive branching end links in its far end called axon terminals

Slide9:

Dendrites represented as inputs to the neuron Axon Neuron's output Each neuron has many inputs through its multiple Dendrites Only one output through its Single Axon

Slide10:

Synapse Each branch of the axon meeting exactly one dendrite of another cell Synaptic gap Gap between the axon terminals and dendrites of another cell. Distance 50 and 200 Angstroms

Connections between neurons are formed at synapses:

Connections between neurons are formed at synapses Axon of a neuron Synaptic gap Dendrite of another Neuron Neurons Information processors

Slide12:

Communication between neurons How do they take place ? Communication takes place with the help of electrical signals

Slide13:

Signals are sent through the axon of one neuron to the dendrites of other neurons

Slide14:

 The processing tasks in the brain are distributed among about 1011 - 1012 elementary nerve cells called neurons. Even then the brain has very less difficulty in correctly and immediately recognizing patterns or objects.  The crucial difference therefore lies not in the essential speed of processing but in the organization of processing.  The key is the notion of massive parallelism or connectionism.

Biological and Artificial Neural Systems:

Biological and Artificial Neural Systems

Slide16:

 Artificial Neural Networks mimic the brain in several ways. The storage of information and control of the system is done in a manner quite similar to that in the brain  The learning phase of artificial Neural Networks is analogous to the development phase of mental faculties of humans. Biological neural systems is made up of basic elements known as neurons. Artificial Neural Networks is made up of Neurons

Slide17:

The brain is capable of handling complex tasks such as sensory processing, recognition, classification, discrimination etc. Likewise Artificial Neural Networks is also capable of performing all these tasks.

Slide18:

In addition, biological systems are able to learn adaptively from experience and from representations of knowledge. This is due to their massively parallel processing architecture. This feature also provides them with a high degree of fault tolerance even in cases of considerable damage. Similarly Artificial Neural Networks can learn adaptively

Slide19:

Despite their essentially biological bases, many developments in Artificial Neural Networks have stemmed from ideas in fields such as Statistics,Computer Science,Cognitive learning, Mathematics, Engineering viz. learning rules such as least mean squares and generalized Delta rule are used to train multi-layer feedforward networks. Contributions from other fields  Learning algorithms such as Probabilistic Neural Networks are based on Bayesian theory.

NEURAL MODELLING:

NEURAL MODELLING McCullock and Pitts suggested the first synthetic neuron. In the McCullock-Pitts model the artificial neuron produces a binary output whose value depends on the weighted sum of its inputs. However, this model consists of only one output

Classification Models:

Classification Models Single Layer Perceptrons A single Layer Perceptron consists of an input and an output Layer. The activation function employed is a hard limiting function. An output unit will assume the value 1 if the sum of the weighted inputs is greater than its threshold.

Slide23:

In terms of classification, an object will be classified as class A if Wij Xi >  j where Wij is the weight from unit i to unit j and  j is the threshold on unit j. Otherwise, the object will be classified as class B. The equation Wij Xi =  j forms a hyperplane in the n dimensional space, dividing the space into two halves.

Slide24:

 When n is 2, it becomes a line.Linear separability refers to the case when a linear hyperplane exists to place the instances of one class on one side and those of the other class on the other side of the plane  Unfortunately, many classification problems are not linearly separable.  The Exclusive OR problem is a good example.

Slide25:

A single layer perceptron cannot simulate an exclusive OR function. The function accepts two inputs ( 0 or 1) and produces an output of one only if either input is one (i.e one of the inputs one) but not both. Inputs Output (1,1) 0 (1,0) 1 (0,1) 1 (0,0) 0

Slide26:

If we plot the above four points in the two dimensional space, it is impossible to draw a line so that (1,1) and (0,0) are on one side and (1,0) and (0,1) are on the other side. To cope up with a problem which is not linearly separable, a multilayer perceptron is required.

Weight Training:

Weight Training 1. Adjust weights by Wji(t+1) = Wji(t) + Wji where Wji(t) is the weight from unit i to unit j at time t (or the tth iteration) and Wji is the weight adjustment. 2. The weight change may be computed by the delta rule: Wji = jXi (1) where  is the trial-independent learning rate (0 <  < 1, e.g., 0.3) and j is the error at unit j:

Slide28:

j = Tj - Oj where Tj is the desired (target) output activation and Oj is the actual output activation at output unit j. 3. Repeat iterations until convergence.

Slide29:

According to the perceptron convergence theorem, if the data points are linearly separable, the perceptron learning rule will converge to some solution in a finite number of steps for any initial choice of weights. The delta rule is a simple generalization of the perceptron learning rule. The learning rate  sets the step size. If  is too small, the convergence is unnecessarily slow, whereas if  is too large, the learning process may diverge.

Drawbacks of the Perceptron Model:

Drawbacks of the Perceptron Model It cannot handle problems which are not linearly separable like the Exclusive OR problem Thus there is need for Multi layer feedforward networks

Multilayered feedfoward Networks:

Multilayered feedfoward Networks m no. of neurons in the input layer n  no. of neurons in the hidden layer ,wij  input to hidden weights k  no. of neurons in the output layer vjk  hidden to output weights

Backpropagation network:

Backpropagation network The backpropagation network is probably the most well known and widely used among the current types of neural network systems available. In contrast to earlier work on perceptrons, the backpropagation network is a multilayer feedfoward network with a different transfer function in the artificial neuron and a more powerful learning rule.  The learning rule is known as backpropagation, which is a kind of gradient descent technique with the backward error (gradient) propagation, as depicted in the figure.

Backpropagation algorithm:

Backpropagation algorithm Supervised training algorithm for multilayered feedfoward Networks Sum of squares of errors between the computed value and the target value is minimized E =½ Σ[Tr  OUTr]² Tr -----> Target value OUTr ----> Computed value

Slide37:

Backpropagation Algorithm 1. Apply input vector X (x1, x2, …, xm) to the input units. 2. Find net input values to hidden layer units : m N j = Σ Wij Xi i=1 3. Apply activation function: Oj = F(Nj) = 1/(1+ e-Nj)

Slide38:

Backpropagation Algorithm(contd.) 4. Move to the output layer. Determine the net input values to each unit: n NETr =ΣVjr Oj j =1 5. Again apply activation function: OUTr = F(NETr)

Gradient Descent Rule is employed to change the weights.:

Gradient Descent Rule is employed to change the weights. Changes in weights from hidden to output layer Vjr = - ∂E/∂Vjr ∂E/∂Vjr = ∂E/∂(OUTr) * ∂(OUTr)/∂Vjr ∂E/∂(OUTr) = -2(Tr-OUTr) OUTr = F(NETr) OUTr = 1/(1+e - NETr ) OUTr = 1/(1+e - ΣOj Vjr) ∂(OUTr)/∂Vjr = F'(NETr)Oj

Vjr = > Change in weights from the jth hidden unit to the rth output unit:

Vjr = > Change in weights from the jth hidden unit to the rth output unit Vjr = 2(Tr-OUTr) F'(NETr)Oj = 2(Tr-OUTr)F(NET)(1-NET)Oj where r = (Tr  OUTr)F'(NETr) Vjr = rOj Oj => Output at the hidden layer  => Learning rate 0 <  < 1

Slide41:

Adjustments of weights : Vjr(n+1) = Vjr(n) + Vjr Vjr(n) => Weight from the jth unit in the hidden layer to the rth unit in the Output layer at step n.

Updating weights from Input to Hidden layer:

Updating weights from Input to Hidden layer In this case Target Output is not known E = ½ Σ[Tr  OUTr]² = ½ Σ[Tr  F(NETr)]² E = ½ Σ[Tr  (FΣVjr Oj)]² E = ½ Σ[Tr  OUTr]² ∂E/∂wij = - Σ(Tr  OUTr)∂OUTr∂wij = -Σ(Tr  OUTr)∂OUTr∂NETr*∂NETr∂Oj*∂Oj∂Nj*∂Nj∂wij ∂OUTr/∂NETr = F'(NETr) = OUTr(1-OUTr)

Slide43:

∂NETr/∂Oj = ∂(ΣOjVjr)/ ∂Oj=Vjr ∂Oj/∂Nj = F'(Nj) ∂Nj/∂Wij = ∂(ΣwijxI)/ ∂wij = xi ∂E/∂wij = -k Σ(Tr-OUTr)OUTr(1-OUTr)VjrxiF'(Nj) wij =  * ∂E/∂wij wij =  F'(Nj) xi ΣrVjr wij(n+1) = wij(n) + wij = wij(n) + jxi

Slide44:

Slide45:

Backpropagation Algorithm carried out on Transformed Data SQRT(X) for Classification of IRIS Data Backprop(10,100,50) Epoch Mean Square Error 0 2.01221 500 0.0627589 1000 0.047611 1500 0.0436433 2000 0.0402018 2500 0.0369779 3000 0.033533 3500 0.0296887 4000 0.026169 4500 0.0231882 5000 0.0206999

Slide46:

Result Ist 2nd 3rd 4th 5th 6th 7th 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 0 Number correctly classified out of 50 = 47

Slide47:

Columns 1 through 12 2 3 3 1 3 1 2 1 2 1 1 1 2 3 3 1 3 1 2 1 1 1 1 1 Columns 13 through 24 2 2 3 3 1 1 2 3 2 3 3 2 2 2 3 3 1 1 3 3 2 3 3 2 Columns 25 thru 36 2 2 3 1 2 1 2 2 3 2 1 2 2 2 3 1 2 1 2 2 3 2 1 2

Slide48:

Columns 37 through 48 3 2 3 1 3 1 2 1 1 3 2 2 3 2 3 1 3 1 3 1 1 3 2 3 Columns 49 through 50 3 3 3 3 Classification Efficiency = 94%

Slide50:

Classes No. of Patterns No. Correctly No. Classified Misclassified I 15 15 0 II 19 16 3 2 3 2 3 2 3 III 16 16 0 50 47 3

Slide51:

In Artificial Neural Networks, different activation functions are used. NNs with the identity function only support linear models. The sigmoid function lets you model higher order functions

Breast Cancer Data:

Breast Cancer Data Attribute Information 

Slide53:

Slide54:

1365328, 1, 1, 2, 1, 2, 1, 2, 1, 1, 0,benign. 242970, 5, 7, 7, 1, 5, 8, 3, 4, 1, 0,benign. 1133041, 5, 3, 1, 2, 2, 1, 2, 1, 1, 0,benign. 183936, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1168278, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1059552, 1, 1, 1, 1, 2, 1, 3, 1, 1,0, benign. 1185610, 1, 1, 1, 1, 3, 2, 2, 1, 1, 0,benign. 1158247, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0,benign. 1206841, 10, 5, 6, 10, 6, 10, 7, 7, 10, 1,malignant; 1166654, 10, 3, 5, 1, 10, 5, 3, 10, 2, 1 ,malignant; 1100524, 6, 10, 10, 2, 8, 10, 7, 3, 3, 1 ,malignant; 1253955, 8, 7, 4, 4, 5, 3, 5, 10, 1, 1 ,malignant; 1344121, 8, 10, 4, 4, 8, 10, 8, 2, 1, 1 ,malignant; 760239, 10, 4, 6, 4, 5, 10, 7, 1, 1, 1 ,malignant; 1257470, 10, 6, 5, 8, 5, 10, 8, 6, 1, 1 ,malignant; 1241559, 10, 8, 8, 2, 8, 10, 4, 8, 10, 1 ,malignant; 1173216, 10, 10, 10, 3, 10, 8, 8, 1, 1, 1 ,malignant; 859350, 8, 10, 10, 7, 10, 10, 7, 3, 8, 1 ,malignant;

Slide55:

breast_ann_all siz = 699 9 TRAINGD, Epoch 0/150, MSE 0.576234/0.01, Gradient 2.57122/0 TRAINGD, Epoch 30/150, MSE 0.0321848/0.01, Gradient 0.0521112/0 TRAINGD, Epoch 60/150, MSE 0.0275306/0.01, Gradient 0.0303056/0 TRAINGD, Epoch 90/150, MSE 0.0254584/0.01, Gradient 0.0228935/0 TRAINGD, Epoch 120/150, MSE 0.0241553/0.01, Gradient 0.0190191/0 TRAINGD, Epoch 150/150, MSE 0.0232051/0.01, Gradient 0.0166925/0 . errors = 17

Slide56:

INPUT TO HIDDEN LAYER WEIGHT MATRIX » net.IW{1,1} No. of Input Neurons 9 No. of hidden Neurons 15 ans =Columns 1 through 7 0.4169 -0.0381 -0.4370 -0.0062 -0.0005 0.1794 -0.3079 -0.2043 0.3573 0.2168 0.1998 0.3585 0.0655 -0.3382 0.0354 0.3249 -0.1094 -0.1222 0.2336 -0.0926 0.3026 -0.0204 -0.0829 0.4098 -0.2141 0.1220 0.1731 -0.3123 0.5259 0.4910 0.1407 -0.0679 0.4966 0.3078 0.0568 0.3003 -0.3291 0.0342 -0.1282 0.2238 0.3262 0.2048 -0.1439 -0.2100 0.1936 0.0538 -0.2308 0.3183 -0.2627 -0.4914 0.1737 0.0401 -0.2056 -0.1890 -0.0102 0.0309 0.2800 -0.4105 -0.2244 0.0524 -0.1155 0.3221 -0.3484 -0.1093 -0.2717 0.0618 -0.3655 -0.0672 -0.3446 0.2642 0.1527 -0.1752 0.3118 0.1934 0.2152 0.4706 0.0520 0.3059 -0.2650 -0.4344 -0.0916 -0.1567 -0.1991 -0.0530 0.3573 0.0431 0.1460 0.2996 0.2877 -0.2312 0.0220 0.2374 -0.1740 -0.1277 0.3391 0.0669 0.3750 -0.1687 -0.2967 -0.2844 0.3282 0.1017 -0.1146 0.2248 -0.0591

Slide57:

Columns 8 through 9 -0.2425 -0.0959 0.0793 -0.1498 0.1961 0.3405 0.0165 -0.4758 0.2383 0.3602 -0.1741 0.4633 -0.1372 0.4131 0.2768 0.3363 0.1582 -0.0368 -0.0414 -0.0669 0.0669 -0.3128 0.2926 0.1437 -0.3925 -0.1476 0.0986 0.4142 -0.4222 0.2207

Slide58:

HIDDEN TO OUTPUT LAYER MATRIX 15 hidden neurons 1 output neuron » net.LW{2,1} ans = Columns 1 through 7 -0.2053 0.3585 -0.4380 -0.2438 0.9043 0.4490 -0.5363 Columns 8 through 14 0.5594 0.1198 -0.7061 -0.4253 0.0276 0.2084 -0.0951 Column 15 0.1637

Slide59:

Various types of Neural Network Models Supervised Training Unsupervised Training Backpropagation Network General Regression Neural Network Radial Basis Function Neural Network Probabilistic Neural Network Functional Link Neural Network Adaptive resonance Theory model Kohonen’s Self Organizing Map Neocognitron Model

Slide60:

Neural Networks Toolbox .Making new feedforward object Examples Let P denote the inputs and T denote the targets P = [0 1 2 3 4 5 6 7 8 9 10]; T = [0 1 2 3 4 3 2 1 2 3 4];

Slide61:

P = 1 2 3 4 5 6 7 8 9 10 11 12 » T=[ 1 0 1] T = 1 0 1 Example : If there are 3 patterns with 4 features then P will be of the form of 4X3 and if the output is a 1 and 0 simply then the T matrix will be of the 1X3 form. » P=[1,2,3;4,5,6;7,8,9;10,11,12]

Slide62:

Here a two-layer feed-forward network is created. The network's input ranges from [0 to 12]. The first layer has five TANSIG neurons, the second layer has one PURELIN neuron.

Slide63:

Other functions are LOGSIG & DLOGSIG. The TRAINLM network training function is to be used. The others which can be used are traingd, traingda, traingdx, trainlm, traingdm etc.

Slide64:

Errors in Heart Data

Slide65:

Error with 6 features of Heart Data

Slide66:

Breast Data with all features

Slide67:

Breast Data with 3 features

Slide68:

Breast data with four features

Slide69:

IRIS Data ANN Training with all features