Advanced Classification Techniques - Understanding Multiclass Classifier
Explore the world of multiclass classifiers and their different strategies such as one-vs-rest and one-vs-one. Dive into confusion matrices for multi-class classification and learn how to calculate FN, FP, TN, and TP values using real-world examples. Enhance your understanding of binary and multiclass classification techniques to make better data-driven decisions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Binary Classification and Multiclass Classification
C1-c2c3 c2- c1c3 c3- c1c2 C1 +1 -1 -1 c2 -1 +1 -1 C3 -1 -1 +1
C1-c2c3 c2- c3 C1 +1 0 c2 -1 +1 C3 -1 -1
Confusion Matrix for Multi-Class Classification How to calculate FN, FP, TN, TP : FN: The False-negative value for a class will be the sum of values of corresponding rows except for the TP value. FP: The False-positive value for a class will be the sum of values of the corresponding column except for the TP value.TN: The True Negative value for a class will be the sum of values of all columns and rows except the values of that class that we are calculating the values for. TP: The True positive value is where the actual value and predicted value are the same. The confusion matrix for the IRIS dataset is as below:
1.Let us calculate the TP, TN, FP, FN values for the class Setosa using the Above tricks: TP: The actual value and predicted value should be the same. So concerning Setosa class, the value of cell 1 is the TP value. FN: The sum of values of corresponding rows except the TP value FN = (cell 2 + cell3) = (0 + 0) = 0 FP : The sum of values of corresponding column except the TP value. FP = (cell 4 + cell 7) = (0 + 0) = 0 TN: The sum of values of all columns and row except the values of that class that we are calculating the values for. TN = (cell 5 + cell 6 + cell 8 + cell 9) = 17 + 1 +0 + 11 = 29 Setosa VS other 2 TP 16 FN 0 FP 0 TN 29
Similarly, for Versicolor class the values/ metrics are calculated as below: TP : 17 (cell 5) FN : 0 + 1 = 1 (cell 4 +cell 6) FP : 0 + 0 = 0 (cell 2 + cell 8) TN : 16 +0 +0 + 11 =27 (cell 1 + cell 3 + cell 7 + cell 9). I hope the concept is clear you can try for the Virginia class. Why Confusion matrix? Confusion Matrix allows us to measure Recall, Precision, Accuracy and AUC-ROC curve are the metrics to measure the performance of the model. Verisocolor vs other2 TP 17 FN 1 FP 0 TN 27 Virginica vs other2 TP 11 FN 0 FP 1 TN 33
Multiclass classification Confusion Matrix Total Predicted C1 C2 C3 C4 Cj C1 TP1 FN12 FN13 FN14 FN1j N1 C2 FN21 TP2 FN23 FN24 FN2j N2 Actual C3 FN31 FN32 TP3 FN34 FN3j N3 C4 FN41 FN42 FN43 TP4 FN4j N4 : : : : : : : Cj FNj1 FNj2 FNj3 FNj4 TPj Nj N1 N2 N3 N4 Nj N Total
FN TP Total Predicted C1 C2 C3 C4 Cj C1 TP1 FN12 FN13 FN14 FN1j N1 C2 FN21 TP2 FN23 FN24 FN2j N2 Actual C3 FN31 FN32 TP3 FN34 FN3j N3 C4 FN41 FN42 FN43 TP4 FN4j N4 : : : : : : : Cj FNj1 FNj2 FNj3 FNj4 TPj Nj N1 N2 N3 N4 Nj N Total FP TN
FN TP TN FP
TN1 = TP2 + FN23 + + FN2j+ +Fn3j+ .+Fn4j+ .+Tpn FP1 = FN21 + FN31 + FN41 + + FNj1 FN1 = FN12 + FN13 + FN14 + + FN1j Accuracy = TP1 + TP2 + + TPj N Error Rate = 1 Accuracy
Precision of class1= TP1/N1 Precision of class2= TP2/N2 Precision of class j= TPj/Nj Precision = (1/j) ji=1 precision of classifier i Recall of class1= TP1/N1 Recall of class2= TP2/N2 Recall of classj= TPj/Nj Recall = (1/j) ji=1 recall of classifier i (avg of Recall of all classes)
Calculate Accuracy, error rate, precision and recall for following confusion matrix with one vs rest scheme. Accuracy = TP1 + TP2 + + TPj N = (30+60+80)/300 =0.5667= 56.67% FN TP Predicted Total Error Rate = 1 Accuracy =(1-0.56667) =0.4333=43.33% Precision of class1=TP1/N1 =(30/60)=0.5 Precision of class2=TP2/N2 =(60/120)=0.5 Precision of class3=TP3/N3 =(80/120)=0.5=0.67 Precision of multiclass classifier= Avg of all precisions =(0.5+0.5+0.67)/3=0.556=0.56=56% Recall of class1=TP1/N1 =(30/100)=0.3 Recall of class2=TP2/N2 =(60/100)=0.6 Recall of class3=TP1/N1 =(80/100)=0.8 Recall of multiclass classifier= Avg of all Recalls =(0.3+0.6+0.8)/3=0.566=0.57=57% C1 C2 C3 C1 30 TP1 50 FN12 20 FN13 N1 100 Actual C2 20 FN21 60 TP2 20 FN23 N2 100 C3 10 FN31 10 FN32 80 TP3 N3 100 Total N1 60 N2 120 N3 120 300 TN FP
Calculate Accuracy, error rate, precision and recall for following confusion matrix with one vs rest scheme. Accuracy = TP1 + TP2 + + TPj=(20+30+60) /250=44% N Predicted Total Error Rate = 1 Accuracy =56% Precision of class1=TP1/N1 =20/60=0.33 Precision of class2=TP2/N2 =30/90=0.33 Precision of class3=TP3/N3 =60/100=0.6 Precision of multiclass classifier= Avg of all precisions =0.33+0.33+06/3=0.42 Recall of class1=TP1/N1 =20/80=0.25 Recall of class2=TP2/N2 =30/70=0.428 Recall of class3=TP1/N1 =60/100=0.6 Recall of multiclass classifier= Avg of all Recalls (0.25+0.43+0.6)/3=0.426 C1 C2 C3 C1 20 40 20 80 Actual C2 20 30 20 70 C3 20 20 60 100 Total 60 90 100 250
Calculate Accuracy, error rate, precision and recall for following confusion matrix with one vs rest scheme. Accuracy = TP1 + TP2 + + TPj=(10+12+8+6) /60=60% N Predicted Tot al Error Rate = 1 Accuracy =40% Precision of class1=TP1/N1 =10/17=0.58 Precision of class2=TP2/N2 =12/18=0.66 Precision of class3=TP3/N3 =8/14=0.57 Precision of class4=TP4/n4 =6/11=0.54 Precision of multiclass classifier= Avg of all precisions =0.59 Recall of class1=TP1/N1 =10/20=0.5 Recall of class2=TP2/N2 =12/15=0.8 Recall of class3=TP1/N1 =8/10=0.8 Recall of class4=TP4/n4=6/15=0.4 Recall of multiclass classifier= Avg of all Recalls 0.625 SA D HAPP Y ANGRY NE UT RAL SAD 10 4 2 4 20 Actual HAPPY 2 12 1 0 15 ANGR Y 1 0 8 1 10 NEUTR AL 4 2 3 6 15 17 18 14 11 60
Calculate 1. Sensitivity 2. Recall 3.Miss Rate 4.Fall out Predicted Tot al Happy Sad Actual Happy 8 10 18 Sad 4 8 12 Total 12 18 30
