logging in or signing up artifical neural network brijmohan Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 22818 Category: Education License: All Rights Reserved Like it (27) Dislike it (1) Added: January 26, 2008 This Presentation is Public Favorites: 13 Presentation Description its a neural network slide Comments Posting comment... By: manimoney (1 month(s) ago) sir, send me this ppt on mks2405@gmail.com please its urgent Saving..... Post Reply Close Saving..... Edit Comment Close By: ganji (2 month(s) ago) sir i want this ppt ,please sent it to my email-id ravali.raga@gmail.com Saving..... Post Reply Close Saving..... Edit Comment Close By: sst1 (4 month(s) ago) sir, i need this ppt. can u send me on my email id? talele.shraddha@yahoo.in Saving..... Post Reply Close Saving..... Edit Comment Close By: sreejesh.ck (4 month(s) ago) i liked this ppt.please add more Saving..... Post Reply Close Saving..... Edit Comment Close By: biocys (6 month(s) ago) dear sir, could you please send me this ppt to my mail"gouthamigangula6@gmail.com" your presentation is relally quit intresting and hope that you will mail me....... after visiting your presentation it impressed me to give seminar on this particular topic. Saving..... Post Reply Close Saving..... Edit Comment Close loading.... See all Premium member Presentation Transcript Artificial Neural Networks: Artificial Neural NetworksSlide2: Seeing the Billions of interconnections in the human brain, and the way the human brain recognizes different patterns, it was felt that there was a need to simulate the human brain. How did Artificial Neural Networks develop?Model of a Biological Neuron: Model of a Biological NeuronThree major components of biological neuron are:: Three major components of biological neuron are: Axon Cell Body DendritesSlide5: At one end of the neuron there are a multitude of tiny filaments called DendritesSlide6: Dendrites join together to form larger larger branches and trunks where they attach to the cell bodySlide7: is a At the other end of the neuron single filament leading out of the cell body called the axonSlide8: Axon has extensive branching end links in its far end called axon terminalsSlide9: Dendrites represented as inputs to the neuron Axon Neuron's output Each neuron has many inputs through its multiple Dendrites Only one output through its Single AxonSlide10: Synapse Each branch of the axon meeting exactly one dendrite of another cell Synaptic gap Gap between the axon terminals and dendrites of another cell. Distance 50 and 200 AngstromsConnections between neurons are formed at synapses: Connections between neurons are formed at synapses Axon of a neuron Synaptic gap Dendrite of another Neuron Neurons Information processorsSlide12: Communication between neurons How do they take place ? Communication takes place with the help of electrical signals Slide13: Signals are sent through the axon of one neuron to the dendrites of other neuronsSlide14: The processing tasks in the brain are distributed among about 1011 - 1012 elementary nerve cells called neurons. Even then the brain has very less difficulty in correctly and immediately recognizing patterns or objects. The crucial difference therefore lies not in the essential speed of processing but in the organization of processing. The key is the notion of massive parallelism or connectionism.Biological and Artificial Neural Systems: Biological and Artificial Neural SystemsSlide16: Artificial Neural Networks mimic the brain in several ways. The storage of information and control of the system is done in a manner quite similar to that in the brain The learning phase of artificial Neural Networks is analogous to the development phase of mental faculties of humans. Biological neural systems is made up of basic elements known as neurons. Artificial Neural Networks is made up of NeuronsSlide17: The brain is capable of handling complex tasks such as sensory processing, recognition, classification, discrimination etc. Likewise Artificial Neural Networks is also capable of performing all these tasks.Slide18: In addition, biological systems are able to learn adaptively from experience and from representations of knowledge. This is due to their massively parallel processing architecture. This feature also provides them with a high degree of fault tolerance even in cases of considerable damage. Similarly Artificial Neural Networks can learn adaptivelySlide19: Despite their essentially biological bases, many developments in Artificial Neural Networks have stemmed from ideas in fields such as Statistics,Computer Science,Cognitive learning, Mathematics, Engineering viz. learning rules such as least mean squares and generalized Delta rule are used to train multi-layer feedforward networks. Contributions from other fields Learning algorithms such as Probabilistic Neural Networks are based on Bayesian theory.NEURAL MODELLING: NEURAL MODELLING McCullock and Pitts suggested the first synthetic neuron. In the McCullock-Pitts model the artificial neuron produces a binary output whose value depends on the weighted sum of its inputs. However, this model consists of only one outputClassification Models: Classification Models Single Layer Perceptrons A single Layer Perceptron consists of an input and an output Layer. The activation function employed is a hard limiting function. An output unit will assume the value 1 if the sum of the weighted inputs is greater than its threshold. Slide23: In terms of classification, an object will be classified as class A if Wij Xi > j where Wij is the weight from unit i to unit j and j is the threshold on unit j. Otherwise, the object will be classified as class B. The equation Wij Xi = j forms a hyperplane in the n dimensional space, dividing the space into two halves. Slide24: When n is 2, it becomes a line.Linear separability refers to the case when a linear hyperplane exists to place the instances of one class on one side and those of the other class on the other side of the plane Unfortunately, many classification problems are not linearly separable. The Exclusive OR problem is a good example. Slide25: A single layer perceptron cannot simulate an exclusive OR function. The function accepts two inputs ( 0 or 1) and produces an output of one only if either input is one (i.e one of the inputs one) but not both. Inputs Output (1,1) 0 (1,0) 1 (0,1) 1 (0,0) 0 Slide26: If we plot the above four points in the two dimensional space, it is impossible to draw a line so that (1,1) and (0,0) are on one side and (1,0) and (0,1) are on the other side. To cope up with a problem which is not linearly separable, a multilayer perceptron is required. Weight Training: Weight Training 1. Adjust weights by Wji(t+1) = Wji(t) + Wji where Wji(t) is the weight from unit i to unit j at time t (or the tth iteration) and Wji is the weight adjustment. 2. The weight change may be computed by the delta rule: Wji = jXi (1) where is the trial-independent learning rate (0 < < 1, e.g., 0.3) and j is the error at unit j:Slide28: j = Tj - Oj where Tj is the desired (target) output activation and Oj is the actual output activation at output unit j. 3. Repeat iterations until convergence.Slide29: According to the perceptron convergence theorem, if the data points are linearly separable, the perceptron learning rule will converge to some solution in a finite number of steps for any initial choice of weights. The delta rule is a simple generalization of the perceptron learning rule. The learning rate sets the step size. If is too small, the convergence is unnecessarily slow, whereas if is too large, the learning process may diverge. Drawbacks of the Perceptron Model: Drawbacks of the Perceptron Model It cannot handle problems which are not linearly separable like the Exclusive OR problem Thus there is need for Multi layer feedforward networksMultilayered feedfoward Networks: Multilayered feedfoward Networks m no. of neurons in the input layer n no. of neurons in the hidden layer ,wij input to hidden weights k no. of neurons in the output layer vjk hidden to output weightsBackpropagation network: Backpropagation network The backpropagation network is probably the most well known and widely used among the current types of neural network systems available. In contrast to earlier work on perceptrons, the backpropagation network is a multilayer feedfoward network with a different transfer function in the artificial neuron and a more powerful learning rule. The learning rule is known as backpropagation, which is a kind of gradient descent technique with the backward error (gradient) propagation, as depicted in the figure. Backpropagation algorithm: Backpropagation algorithm Supervised training algorithm for multilayered feedfoward Networks Sum of squares of errors between the computed value and the target value is minimized E =½ Σ[Tr OUTr]² Tr -----> Target value OUTr ----> Computed valueSlide37: Backpropagation Algorithm 1. Apply input vector X (x1, x2, …, xm) to the input units. 2. Find net input values to hidden layer units : m N j = Σ Wij Xi i=1 3. Apply activation function: Oj = F(Nj) = 1/(1+ e-Nj)Slide38: Backpropagation Algorithm(contd.) 4. Move to the output layer. Determine the net input values to each unit: n NETr =ΣVjr Oj j =1 5. Again apply activation function: OUTr = F(NETr) Gradient Descent Rule is employed to change the weights.: Gradient Descent Rule is employed to change the weights. Changes in weights from hidden to output layer Vjr = - ∂E/∂Vjr ∂E/∂Vjr = ∂E/∂(OUTr) * ∂(OUTr)/∂Vjr ∂E/∂(OUTr) = -2(Tr-OUTr) OUTr = F(NETr) OUTr = 1/(1+e - NETr ) OUTr = 1/(1+e - ΣOj Vjr) ∂(OUTr)/∂Vjr = F'(NETr)Oj Vjr = > Change in weights from the jth hidden unit to the rth output unit: Vjr = > Change in weights from the jth hidden unit to the rth output unit Vjr = 2(Tr-OUTr) F'(NETr)Oj = 2(Tr-OUTr)F(NET)(1-NET)Oj where r = (Tr OUTr)F'(NETr) Vjr = rOj Oj => Output at the hidden layer => Learning rate 0 < < 1 Slide41: Adjustments of weights : Vjr(n+1) = Vjr(n) + Vjr Vjr(n) => Weight from the jth unit in the hidden layer to the rth unit in the Output layer at step n.Updating weights from Input to Hidden layer: Updating weights from Input to Hidden layer In this case Target Output is not known E = ½ Σ[Tr OUTr]² = ½ Σ[Tr F(NETr)]² E = ½ Σ[Tr (FΣVjr Oj)]² E = ½ Σ[Tr OUTr]² ∂E/∂wij = - Σ(Tr OUTr)∂OUTr∂wij = -Σ(Tr OUTr)∂OUTr∂NETr*∂NETr∂Oj*∂Oj∂Nj*∂Nj∂wij ∂OUTr/∂NETr = F'(NETr) = OUTr(1-OUTr) Slide43: ∂NETr/∂Oj = ∂(ΣOjVjr)/ ∂Oj=Vjr ∂Oj/∂Nj = F'(Nj) ∂Nj/∂Wij = ∂(ΣwijxI)/ ∂wij = xi ∂E/∂wij = -k Σ(Tr-OUTr)OUTr(1-OUTr)VjrxiF'(Nj) wij = * ∂E/∂wij wij = F'(Nj) xi ΣrVjr wij(n+1) = wij(n) + wij = wij(n) + jxi Slide44: ∂NETr/∂Oj = ∂(ΣOjVjr)/ ∂Oj=Vjr ∂Oj/∂Nj = F'(Nj) ∂Nj/∂Wij = ∂(ΣwijxI)/ ∂wij = xi ∂E/∂wij = -k Σ(Tr-OUTr)OUTr(1-OUTr)VjrxiF'(Nj) wij = * ∂E/∂wij wij = F'(Nj) xi ΣrVjr wij(n+1) = wij(n) + wij = wij(n) + jxi Slide45: Backpropagation Algorithm carried out on Transformed Data SQRT(X) for Classification of IRIS Data Backprop(10,100,50) Epoch Mean Square Error 0 2.01221 500 0.0627589 1000 0.047611 1500 0.0436433 2000 0.0402018 2500 0.0369779 3000 0.033533 3500 0.0296887 4000 0.026169 4500 0.0231882 5000 0.0206999Slide46: Result Ist 2nd 3rd 4th 5th 6th 7th 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 0 Number correctly classified out of 50 = 47 Slide47: Columns 1 through 12 2 3 3 1 3 1 2 1 2 1 1 1 2 3 3 1 3 1 2 1 1 1 1 1 Columns 13 through 24 2 2 3 3 1 1 2 3 2 3 3 2 2 2 3 3 1 1 3 3 2 3 3 2 Columns 25 thru 36 2 2 3 1 2 1 2 2 3 2 1 2 2 2 3 1 2 1 2 2 3 2 1 2Slide48: Columns 37 through 48 3 2 3 1 3 1 2 1 1 3 2 2 3 2 3 1 3 1 3 1 1 3 2 3 Columns 49 through 50 3 3 3 3 Classification Efficiency = 94%Slide50: Classes No. of Patterns No. Correctly No. Classified Misclassified I 15 15 0 II 19 16 3 2 3 2 3 2 3 III 16 16 0 50 47 3Slide51: In Artificial Neural Networks, different activation functions are used. NNs with the identity function only support linear models. The sigmoid function lets you model higher order functions Breast Cancer Data: Breast Cancer Data Attribute Information Slide53: Attribute Information: (class attribute is in last column) | | # Attribute Domain | -- ----------------------------------------- | 1. Sample code number id number | 2. Clump Thickness 1 - 10 | 3. Uniformity of Cell Size 1 - 10 | 4. Uniformity of Cell Shape 1 - 10 | 5. Marginal Adhesion 1 - 10 | 6. Single Epithelial Cell Size 1 - 10 | 7. Bare Nuclei 1 - 10 | 8. Bland Chromatin 1 - 10 | 9. Normal Nucleoli 1 - 10 | 10. Mitoses 1 - 10 | 11. Class: (0 for benign, 1 for malignant) | | Slide54: 1365328, 1, 1, 2, 1, 2, 1, 2, 1, 1, 0,benign. 242970, 5, 7, 7, 1, 5, 8, 3, 4, 1, 0,benign. 1133041, 5, 3, 1, 2, 2, 1, 2, 1, 1, 0,benign. 183936, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1168278, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1059552, 1, 1, 1, 1, 2, 1, 3, 1, 1,0, benign. 1185610, 1, 1, 1, 1, 3, 2, 2, 1, 1, 0,benign. 1158247, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0,benign. 1206841, 10, 5, 6, 10, 6, 10, 7, 7, 10, 1,malignant; 1166654, 10, 3, 5, 1, 10, 5, 3, 10, 2, 1 ,malignant; 1100524, 6, 10, 10, 2, 8, 10, 7, 3, 3, 1 ,malignant; 1253955, 8, 7, 4, 4, 5, 3, 5, 10, 1, 1 ,malignant; 1344121, 8, 10, 4, 4, 8, 10, 8, 2, 1, 1 ,malignant; 760239, 10, 4, 6, 4, 5, 10, 7, 1, 1, 1 ,malignant; 1257470, 10, 6, 5, 8, 5, 10, 8, 6, 1, 1 ,malignant; 1241559, 10, 8, 8, 2, 8, 10, 4, 8, 10, 1 ,malignant; 1173216, 10, 10, 10, 3, 10, 8, 8, 1, 1, 1 ,malignant; 859350, 8, 10, 10, 7, 10, 10, 7, 3, 8, 1 ,malignant;Slide55: breast_ann_all siz = 699 9 TRAINGD, Epoch 0/150, MSE 0.576234/0.01, Gradient 2.57122/0 TRAINGD, Epoch 30/150, MSE 0.0321848/0.01, Gradient 0.0521112/0 TRAINGD, Epoch 60/150, MSE 0.0275306/0.01, Gradient 0.0303056/0 TRAINGD, Epoch 90/150, MSE 0.0254584/0.01, Gradient 0.0228935/0 TRAINGD, Epoch 120/150, MSE 0.0241553/0.01, Gradient 0.0190191/0 TRAINGD, Epoch 150/150, MSE 0.0232051/0.01, Gradient 0.0166925/0 . errors = 17Slide56: INPUT TO HIDDEN LAYER WEIGHT MATRIX » net.IW{1,1} No. of Input Neurons 9 No. of hidden Neurons 15 ans =Columns 1 through 7 0.4169 -0.0381 -0.4370 -0.0062 -0.0005 0.1794 -0.3079 -0.2043 0.3573 0.2168 0.1998 0.3585 0.0655 -0.3382 0.0354 0.3249 -0.1094 -0.1222 0.2336 -0.0926 0.3026 -0.0204 -0.0829 0.4098 -0.2141 0.1220 0.1731 -0.3123 0.5259 0.4910 0.1407 -0.0679 0.4966 0.3078 0.0568 0.3003 -0.3291 0.0342 -0.1282 0.2238 0.3262 0.2048 -0.1439 -0.2100 0.1936 0.0538 -0.2308 0.3183 -0.2627 -0.4914 0.1737 0.0401 -0.2056 -0.1890 -0.0102 0.0309 0.2800 -0.4105 -0.2244 0.0524 -0.1155 0.3221 -0.3484 -0.1093 -0.2717 0.0618 -0.3655 -0.0672 -0.3446 0.2642 0.1527 -0.1752 0.3118 0.1934 0.2152 0.4706 0.0520 0.3059 -0.2650 -0.4344 -0.0916 -0.1567 -0.1991 -0.0530 0.3573 0.0431 0.1460 0.2996 0.2877 -0.2312 0.0220 0.2374 -0.1740 -0.1277 0.3391 0.0669 0.3750 -0.1687 -0.2967 -0.2844 0.3282 0.1017 -0.1146 0.2248 -0.0591 Slide57: Columns 8 through 9 -0.2425 -0.0959 0.0793 -0.1498 0.1961 0.3405 0.0165 -0.4758 0.2383 0.3602 -0.1741 0.4633 -0.1372 0.4131 0.2768 0.3363 0.1582 -0.0368 -0.0414 -0.0669 0.0669 -0.3128 0.2926 0.1437 -0.3925 -0.1476 0.0986 0.4142 -0.4222 0.2207Slide58: HIDDEN TO OUTPUT LAYER MATRIX 15 hidden neurons 1 output neuron » net.LW{2,1} ans = Columns 1 through 7 -0.2053 0.3585 -0.4380 -0.2438 0.9043 0.4490 -0.5363 Columns 8 through 14 0.5594 0.1198 -0.7061 -0.4253 0.0276 0.2084 -0.0951 Column 15 0.1637 Slide59: Various types of Neural Network Models Supervised Training Unsupervised Training Backpropagation Network General Regression Neural Network Radial Basis Function Neural Network Probabilistic Neural Network Functional Link Neural Network Adaptive resonance Theory model Kohonen’s Self Organizing Map Neocognitron ModelSlide60: Neural Networks Toolbox .Making new feedforward object Examples Let P denote the inputs and T denote the targets P = [0 1 2 3 4 5 6 7 8 9 10]; T = [0 1 2 3 4 3 2 1 2 3 4]; Slide61: P = 1 2 3 4 5 6 7 8 9 10 11 12 » T=[ 1 0 1] T = 1 0 1 Example : If there are 3 patterns with 4 features then P will be of the form of 4X3 and if the output is a 1 and 0 simply then the T matrix will be of the 1X3 form. » P=[1,2,3;4,5,6;7,8,9;10,11,12] Slide62: Here a two-layer feed-forward network is created. The network's input ranges from [0 to 12]. The first layer has five TANSIG neurons, the second layer has one PURELIN neuron. Slide63: Other functions are LOGSIG & DLOGSIG. The TRAINLM network training function is to be used. The others which can be used are traingd, traingda, traingdx, trainlm, traingdm etc. Slide64: Errors in Heart DataSlide65: Error with 6 features of Heart DataSlide66: Breast Data with all featuresSlide67: Breast Data with 3 featuresSlide68: Breast data with four featuresSlide69: IRIS Data ANN Training with all features You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.
artifical neural network brijmohan Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 22818 Category: Education License: All Rights Reserved Like it (27) Dislike it (1) Added: January 26, 2008 This Presentation is Public Favorites: 13 Presentation Description its a neural network slide Comments Posting comment... By: manimoney (1 month(s) ago) sir, send me this ppt on mks2405@gmail.com please its urgent Saving..... Post Reply Close Saving..... Edit Comment Close By: ganji (2 month(s) ago) sir i want this ppt ,please sent it to my email-id ravali.raga@gmail.com Saving..... Post Reply Close Saving..... Edit Comment Close By: sst1 (4 month(s) ago) sir, i need this ppt. can u send me on my email id? talele.shraddha@yahoo.in Saving..... Post Reply Close Saving..... Edit Comment Close By: sreejesh.ck (4 month(s) ago) i liked this ppt.please add more Saving..... Post Reply Close Saving..... Edit Comment Close By: biocys (6 month(s) ago) dear sir, could you please send me this ppt to my mail"gouthamigangula6@gmail.com" your presentation is relally quit intresting and hope that you will mail me....... after visiting your presentation it impressed me to give seminar on this particular topic. Saving..... Post Reply Close Saving..... Edit Comment Close loading.... See all Premium member Presentation Transcript Artificial Neural Networks: Artificial Neural NetworksSlide2: Seeing the Billions of interconnections in the human brain, and the way the human brain recognizes different patterns, it was felt that there was a need to simulate the human brain. How did Artificial Neural Networks develop?Model of a Biological Neuron: Model of a Biological NeuronThree major components of biological neuron are:: Three major components of biological neuron are: Axon Cell Body DendritesSlide5: At one end of the neuron there are a multitude of tiny filaments called DendritesSlide6: Dendrites join together to form larger larger branches and trunks where they attach to the cell bodySlide7: is a At the other end of the neuron single filament leading out of the cell body called the axonSlide8: Axon has extensive branching end links in its far end called axon terminalsSlide9: Dendrites represented as inputs to the neuron Axon Neuron's output Each neuron has many inputs through its multiple Dendrites Only one output through its Single AxonSlide10: Synapse Each branch of the axon meeting exactly one dendrite of another cell Synaptic gap Gap between the axon terminals and dendrites of another cell. Distance 50 and 200 AngstromsConnections between neurons are formed at synapses: Connections between neurons are formed at synapses Axon of a neuron Synaptic gap Dendrite of another Neuron Neurons Information processorsSlide12: Communication between neurons How do they take place ? Communication takes place with the help of electrical signals Slide13: Signals are sent through the axon of one neuron to the dendrites of other neuronsSlide14: The processing tasks in the brain are distributed among about 1011 - 1012 elementary nerve cells called neurons. Even then the brain has very less difficulty in correctly and immediately recognizing patterns or objects. The crucial difference therefore lies not in the essential speed of processing but in the organization of processing. The key is the notion of massive parallelism or connectionism.Biological and Artificial Neural Systems: Biological and Artificial Neural SystemsSlide16: Artificial Neural Networks mimic the brain in several ways. The storage of information and control of the system is done in a manner quite similar to that in the brain The learning phase of artificial Neural Networks is analogous to the development phase of mental faculties of humans. Biological neural systems is made up of basic elements known as neurons. Artificial Neural Networks is made up of NeuronsSlide17: The brain is capable of handling complex tasks such as sensory processing, recognition, classification, discrimination etc. Likewise Artificial Neural Networks is also capable of performing all these tasks.Slide18: In addition, biological systems are able to learn adaptively from experience and from representations of knowledge. This is due to their massively parallel processing architecture. This feature also provides them with a high degree of fault tolerance even in cases of considerable damage. Similarly Artificial Neural Networks can learn adaptivelySlide19: Despite their essentially biological bases, many developments in Artificial Neural Networks have stemmed from ideas in fields such as Statistics,Computer Science,Cognitive learning, Mathematics, Engineering viz. learning rules such as least mean squares and generalized Delta rule are used to train multi-layer feedforward networks. Contributions from other fields Learning algorithms such as Probabilistic Neural Networks are based on Bayesian theory.NEURAL MODELLING: NEURAL MODELLING McCullock and Pitts suggested the first synthetic neuron. In the McCullock-Pitts model the artificial neuron produces a binary output whose value depends on the weighted sum of its inputs. However, this model consists of only one outputClassification Models: Classification Models Single Layer Perceptrons A single Layer Perceptron consists of an input and an output Layer. The activation function employed is a hard limiting function. An output unit will assume the value 1 if the sum of the weighted inputs is greater than its threshold. Slide23: In terms of classification, an object will be classified as class A if Wij Xi > j where Wij is the weight from unit i to unit j and j is the threshold on unit j. Otherwise, the object will be classified as class B. The equation Wij Xi = j forms a hyperplane in the n dimensional space, dividing the space into two halves. Slide24: When n is 2, it becomes a line.Linear separability refers to the case when a linear hyperplane exists to place the instances of one class on one side and those of the other class on the other side of the plane Unfortunately, many classification problems are not linearly separable. The Exclusive OR problem is a good example. Slide25: A single layer perceptron cannot simulate an exclusive OR function. The function accepts two inputs ( 0 or 1) and produces an output of one only if either input is one (i.e one of the inputs one) but not both. Inputs Output (1,1) 0 (1,0) 1 (0,1) 1 (0,0) 0 Slide26: If we plot the above four points in the two dimensional space, it is impossible to draw a line so that (1,1) and (0,0) are on one side and (1,0) and (0,1) are on the other side. To cope up with a problem which is not linearly separable, a multilayer perceptron is required. Weight Training: Weight Training 1. Adjust weights by Wji(t+1) = Wji(t) + Wji where Wji(t) is the weight from unit i to unit j at time t (or the tth iteration) and Wji is the weight adjustment. 2. The weight change may be computed by the delta rule: Wji = jXi (1) where is the trial-independent learning rate (0 < < 1, e.g., 0.3) and j is the error at unit j:Slide28: j = Tj - Oj where Tj is the desired (target) output activation and Oj is the actual output activation at output unit j. 3. Repeat iterations until convergence.Slide29: According to the perceptron convergence theorem, if the data points are linearly separable, the perceptron learning rule will converge to some solution in a finite number of steps for any initial choice of weights. The delta rule is a simple generalization of the perceptron learning rule. The learning rate sets the step size. If is too small, the convergence is unnecessarily slow, whereas if is too large, the learning process may diverge. Drawbacks of the Perceptron Model: Drawbacks of the Perceptron Model It cannot handle problems which are not linearly separable like the Exclusive OR problem Thus there is need for Multi layer feedforward networksMultilayered feedfoward Networks: Multilayered feedfoward Networks m no. of neurons in the input layer n no. of neurons in the hidden layer ,wij input to hidden weights k no. of neurons in the output layer vjk hidden to output weightsBackpropagation network: Backpropagation network The backpropagation network is probably the most well known and widely used among the current types of neural network systems available. In contrast to earlier work on perceptrons, the backpropagation network is a multilayer feedfoward network with a different transfer function in the artificial neuron and a more powerful learning rule. The learning rule is known as backpropagation, which is a kind of gradient descent technique with the backward error (gradient) propagation, as depicted in the figure. Backpropagation algorithm: Backpropagation algorithm Supervised training algorithm for multilayered feedfoward Networks Sum of squares of errors between the computed value and the target value is minimized E =½ Σ[Tr OUTr]² Tr -----> Target value OUTr ----> Computed valueSlide37: Backpropagation Algorithm 1. Apply input vector X (x1, x2, …, xm) to the input units. 2. Find net input values to hidden layer units : m N j = Σ Wij Xi i=1 3. Apply activation function: Oj = F(Nj) = 1/(1+ e-Nj)Slide38: Backpropagation Algorithm(contd.) 4. Move to the output layer. Determine the net input values to each unit: n NETr =ΣVjr Oj j =1 5. Again apply activation function: OUTr = F(NETr) Gradient Descent Rule is employed to change the weights.: Gradient Descent Rule is employed to change the weights. Changes in weights from hidden to output layer Vjr = - ∂E/∂Vjr ∂E/∂Vjr = ∂E/∂(OUTr) * ∂(OUTr)/∂Vjr ∂E/∂(OUTr) = -2(Tr-OUTr) OUTr = F(NETr) OUTr = 1/(1+e - NETr ) OUTr = 1/(1+e - ΣOj Vjr) ∂(OUTr)/∂Vjr = F'(NETr)Oj Vjr = > Change in weights from the jth hidden unit to the rth output unit: Vjr = > Change in weights from the jth hidden unit to the rth output unit Vjr = 2(Tr-OUTr) F'(NETr)Oj = 2(Tr-OUTr)F(NET)(1-NET)Oj where r = (Tr OUTr)F'(NETr) Vjr = rOj Oj => Output at the hidden layer => Learning rate 0 < < 1 Slide41: Adjustments of weights : Vjr(n+1) = Vjr(n) + Vjr Vjr(n) => Weight from the jth unit in the hidden layer to the rth unit in the Output layer at step n.Updating weights from Input to Hidden layer: Updating weights from Input to Hidden layer In this case Target Output is not known E = ½ Σ[Tr OUTr]² = ½ Σ[Tr F(NETr)]² E = ½ Σ[Tr (FΣVjr Oj)]² E = ½ Σ[Tr OUTr]² ∂E/∂wij = - Σ(Tr OUTr)∂OUTr∂wij = -Σ(Tr OUTr)∂OUTr∂NETr*∂NETr∂Oj*∂Oj∂Nj*∂Nj∂wij ∂OUTr/∂NETr = F'(NETr) = OUTr(1-OUTr) Slide43: ∂NETr/∂Oj = ∂(ΣOjVjr)/ ∂Oj=Vjr ∂Oj/∂Nj = F'(Nj) ∂Nj/∂Wij = ∂(ΣwijxI)/ ∂wij = xi ∂E/∂wij = -k Σ(Tr-OUTr)OUTr(1-OUTr)VjrxiF'(Nj) wij = * ∂E/∂wij wij = F'(Nj) xi ΣrVjr wij(n+1) = wij(n) + wij = wij(n) + jxi Slide44: ∂NETr/∂Oj = ∂(ΣOjVjr)/ ∂Oj=Vjr ∂Oj/∂Nj = F'(Nj) ∂Nj/∂Wij = ∂(ΣwijxI)/ ∂wij = xi ∂E/∂wij = -k Σ(Tr-OUTr)OUTr(1-OUTr)VjrxiF'(Nj) wij = * ∂E/∂wij wij = F'(Nj) xi ΣrVjr wij(n+1) = wij(n) + wij = wij(n) + jxi Slide45: Backpropagation Algorithm carried out on Transformed Data SQRT(X) for Classification of IRIS Data Backprop(10,100,50) Epoch Mean Square Error 0 2.01221 500 0.0627589 1000 0.047611 1500 0.0436433 2000 0.0402018 2500 0.0369779 3000 0.033533 3500 0.0296887 4000 0.026169 4500 0.0231882 5000 0.0206999Slide46: Result Ist 2nd 3rd 4th 5th 6th 7th 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 0 Number correctly classified out of 50 = 47 Slide47: Columns 1 through 12 2 3 3 1 3 1 2 1 2 1 1 1 2 3 3 1 3 1 2 1 1 1 1 1 Columns 13 through 24 2 2 3 3 1 1 2 3 2 3 3 2 2 2 3 3 1 1 3 3 2 3 3 2 Columns 25 thru 36 2 2 3 1 2 1 2 2 3 2 1 2 2 2 3 1 2 1 2 2 3 2 1 2Slide48: Columns 37 through 48 3 2 3 1 3 1 2 1 1 3 2 2 3 2 3 1 3 1 3 1 1 3 2 3 Columns 49 through 50 3 3 3 3 Classification Efficiency = 94%Slide50: Classes No. of Patterns No. Correctly No. Classified Misclassified I 15 15 0 II 19 16 3 2 3 2 3 2 3 III 16 16 0 50 47 3Slide51: In Artificial Neural Networks, different activation functions are used. NNs with the identity function only support linear models. The sigmoid function lets you model higher order functions Breast Cancer Data: Breast Cancer Data Attribute Information Slide53: Attribute Information: (class attribute is in last column) | | # Attribute Domain | -- ----------------------------------------- | 1. Sample code number id number | 2. Clump Thickness 1 - 10 | 3. Uniformity of Cell Size 1 - 10 | 4. Uniformity of Cell Shape 1 - 10 | 5. Marginal Adhesion 1 - 10 | 6. Single Epithelial Cell Size 1 - 10 | 7. Bare Nuclei 1 - 10 | 8. Bland Chromatin 1 - 10 | 9. Normal Nucleoli 1 - 10 | 10. Mitoses 1 - 10 | 11. Class: (0 for benign, 1 for malignant) | | Slide54: 1365328, 1, 1, 2, 1, 2, 1, 2, 1, 1, 0,benign. 242970, 5, 7, 7, 1, 5, 8, 3, 4, 1, 0,benign. 1133041, 5, 3, 1, 2, 2, 1, 2, 1, 1, 0,benign. 183936, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1168278, 3, 1, 1, 1, 2, 1, 2, 1, 1, 0,benign. 1059552, 1, 1, 1, 1, 2, 1, 3, 1, 1,0, benign. 1185610, 1, 1, 1, 1, 3, 2, 2, 1, 1, 0,benign. 1158247, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0,benign. 1206841, 10, 5, 6, 10, 6, 10, 7, 7, 10, 1,malignant; 1166654, 10, 3, 5, 1, 10, 5, 3, 10, 2, 1 ,malignant; 1100524, 6, 10, 10, 2, 8, 10, 7, 3, 3, 1 ,malignant; 1253955, 8, 7, 4, 4, 5, 3, 5, 10, 1, 1 ,malignant; 1344121, 8, 10, 4, 4, 8, 10, 8, 2, 1, 1 ,malignant; 760239, 10, 4, 6, 4, 5, 10, 7, 1, 1, 1 ,malignant; 1257470, 10, 6, 5, 8, 5, 10, 8, 6, 1, 1 ,malignant; 1241559, 10, 8, 8, 2, 8, 10, 4, 8, 10, 1 ,malignant; 1173216, 10, 10, 10, 3, 10, 8, 8, 1, 1, 1 ,malignant; 859350, 8, 10, 10, 7, 10, 10, 7, 3, 8, 1 ,malignant;Slide55: breast_ann_all siz = 699 9 TRAINGD, Epoch 0/150, MSE 0.576234/0.01, Gradient 2.57122/0 TRAINGD, Epoch 30/150, MSE 0.0321848/0.01, Gradient 0.0521112/0 TRAINGD, Epoch 60/150, MSE 0.0275306/0.01, Gradient 0.0303056/0 TRAINGD, Epoch 90/150, MSE 0.0254584/0.01, Gradient 0.0228935/0 TRAINGD, Epoch 120/150, MSE 0.0241553/0.01, Gradient 0.0190191/0 TRAINGD, Epoch 150/150, MSE 0.0232051/0.01, Gradient 0.0166925/0 . errors = 17Slide56: INPUT TO HIDDEN LAYER WEIGHT MATRIX » net.IW{1,1} No. of Input Neurons 9 No. of hidden Neurons 15 ans =Columns 1 through 7 0.4169 -0.0381 -0.4370 -0.0062 -0.0005 0.1794 -0.3079 -0.2043 0.3573 0.2168 0.1998 0.3585 0.0655 -0.3382 0.0354 0.3249 -0.1094 -0.1222 0.2336 -0.0926 0.3026 -0.0204 -0.0829 0.4098 -0.2141 0.1220 0.1731 -0.3123 0.5259 0.4910 0.1407 -0.0679 0.4966 0.3078 0.0568 0.3003 -0.3291 0.0342 -0.1282 0.2238 0.3262 0.2048 -0.1439 -0.2100 0.1936 0.0538 -0.2308 0.3183 -0.2627 -0.4914 0.1737 0.0401 -0.2056 -0.1890 -0.0102 0.0309 0.2800 -0.4105 -0.2244 0.0524 -0.1155 0.3221 -0.3484 -0.1093 -0.2717 0.0618 -0.3655 -0.0672 -0.3446 0.2642 0.1527 -0.1752 0.3118 0.1934 0.2152 0.4706 0.0520 0.3059 -0.2650 -0.4344 -0.0916 -0.1567 -0.1991 -0.0530 0.3573 0.0431 0.1460 0.2996 0.2877 -0.2312 0.0220 0.2374 -0.1740 -0.1277 0.3391 0.0669 0.3750 -0.1687 -0.2967 -0.2844 0.3282 0.1017 -0.1146 0.2248 -0.0591 Slide57: Columns 8 through 9 -0.2425 -0.0959 0.0793 -0.1498 0.1961 0.3405 0.0165 -0.4758 0.2383 0.3602 -0.1741 0.4633 -0.1372 0.4131 0.2768 0.3363 0.1582 -0.0368 -0.0414 -0.0669 0.0669 -0.3128 0.2926 0.1437 -0.3925 -0.1476 0.0986 0.4142 -0.4222 0.2207Slide58: HIDDEN TO OUTPUT LAYER MATRIX 15 hidden neurons 1 output neuron » net.LW{2,1} ans = Columns 1 through 7 -0.2053 0.3585 -0.4380 -0.2438 0.9043 0.4490 -0.5363 Columns 8 through 14 0.5594 0.1198 -0.7061 -0.4253 0.0276 0.2084 -0.0951 Column 15 0.1637 Slide59: Various types of Neural Network Models Supervised Training Unsupervised Training Backpropagation Network General Regression Neural Network Radial Basis Function Neural Network Probabilistic Neural Network Functional Link Neural Network Adaptive resonance Theory model Kohonen’s Self Organizing Map Neocognitron ModelSlide60: Neural Networks Toolbox .Making new feedforward object Examples Let P denote the inputs and T denote the targets P = [0 1 2 3 4 5 6 7 8 9 10]; T = [0 1 2 3 4 3 2 1 2 3 4]; Slide61: P = 1 2 3 4 5 6 7 8 9 10 11 12 » T=[ 1 0 1] T = 1 0 1 Example : If there are 3 patterns with 4 features then P will be of the form of 4X3 and if the output is a 1 and 0 simply then the T matrix will be of the 1X3 form. » P=[1,2,3;4,5,6;7,8,9;10,11,12] Slide62: Here a two-layer feed-forward network is created. The network's input ranges from [0 to 12]. The first layer has five TANSIG neurons, the second layer has one PURELIN neuron. Slide63: Other functions are LOGSIG & DLOGSIG. The TRAINLM network training function is to be used. The others which can be used are traingd, traingda, traingdx, trainlm, traingdm etc. Slide64: Errors in Heart DataSlide65: Error with 6 features of Heart DataSlide66: Breast Data with all featuresSlide67: Breast Data with 3 featuresSlide68: Breast data with four featuresSlide69: IRIS Data ANN Training with all features