Confusion Matrix in Data Science

As the name implies, these two words and the concept they define confuses most people. And when it succeeds to confuse the rarer, the smart and the intelligent engineers imagine my plight being a veterinary doctor amidst the tidal wave of numbers.

Hence I stab on it by keeping things very simple.

My father has always led a life of military discipline and also ensured for most of my educational life that his kids follow it too. For example, travelling. Travelling with him is no less than going to a battle ground.

One day, he and I were going to the railway station to catch a train to Bangalore and unfortunately we got stuck in traffic. As we neared the station he started predicting that we would miss the train. Assuming it is one of his trait of negative thoughts and situational panic, I didn’t give it much thought.

However, when we reached the station, we came to know that we had missed the train by one minute. My father predicted NEGATIVE and it has become TRUE. Let us name this event as TRUE NEGATIVE aka TN.

So see ….. we should never ever do negative thinking, because in reality that becomes always true and especially when we do not want that event to happen (“Murphy’s Law” J ) . Oh Yeah ! definitely I have pressed the Panic button inside me and hence was subsequently tensed up.

In contrast my father without panicking he went to enquiry booth to gather further information while I, belonging to the so called smartest generation on the planet was taking help of mobile to enquire about next day train assuming even if there is a train for that day we will not get reservation for the same day.

While I was still figuring out about train he came back and requested me to fetch a cup of hot tea for him and he told me that next train to Bangalore was in another 15 minutes , so I should hurry up to bring tea for him.

Astonished by this announcement from him, I said to him boarding the next train is no use as we will not get reserved tickets. He gave me “BIG SMILE” and questioned me should not we be always positive? Hmmm……My advice to him in cab boomeranged to me. Clever !! Is he not ? …….Father after all……And I love him a lot .

Exactly after 15 minutes ,Train came and we got 2 reserved seats for us after talking to Ticket checker .Now here my father’s prediction (“Remember that winning BIG SMILE “ ) of getting reserved tickets was POSITIVE and has come to TRUE again .Let us call this as TRUE POSITIVE i.e TP. (Not a time pass or laptop )

Now look at the above story from my perspective. All the time , while we were in cab to railway station, I was predicting that We will catch the train and as per my nature I was predicting POSITIVE but reality has bolded my POSITIVE prediction . In other word reality made my POSITIVE PREDICTION FALSE. Let us call this as FALSE POSITIVE or FP .

Also very next moment my NEGATIVE prediction of not having train on same day and even if it is there not getting the reserved tickets was again bolded superbly at duck by NOT ONLY having next train immediately in 15 minutes but also by getting 2 reserved tickets. Again Reality made my NEGATIVE PREDICTION FALSE. Let us call this as FALSE NEGATIVE i.e FN

From the above story till now, we got 4 terminologies of “CONFUSED MATRIX “and their subtle difference in following nutshell:-

TRUE POSITIVE :- When PREDICTED POSITIVE but in ACTUAL also happens that way

TRUE NEGATIVE:- When PREDICTED NEGATIVE but in ACTUAL also happens that way

FALSE POSITIVE:- When PREDICTED POSITIVE but in ACTUAL happens absolutely OPPOSITE

FALSE NEGATIVE:- When PREDICTED POSITIVE but in ACTUAL happens absolutely OPPOSITE

By now hope you have got hang of terminologies used in “CONFUSED MATRIX”.

Guys I am thinking positive please do not prove me wrong.

Let us try to understand in different way and this time will take help of symbols to register the same concept in our mind.

In my team of 10 people, 70% of them come on dot but 30% are late comers.My boss wanted to evaluate me on my correctness of this reading .Crazy style ! but who will bell the cat J .

So first 4 days of evaluation:-

7 people came on time and 3 people as usual late. ( Whoosh ,so proud of my whole team and for the first time proud of those 3 late comers as they have proved me TRUE in front of my boss J ) .

PREDICTED POSITIVE	7	ACTUAL POSITIVE	7
PREDICTED NEGATIVE	3	ACTUAL NEGATIVE	3

Hence my predicted model is 100% accurate . Yeaaahhh !!!!! And here is the formula to prove my accuracy .

Accuracy = (7+3) /10 = 10/10 = 1 (100%)

But On Fifth Day--------------------------------------------------------------------------------------

8 people came on time, good improvement in the team which I have always wanted but my reading for accuracy in front my boss was shaking J .

PREDICTED POSITIVE	7	ACTUAL POSITIVE	8
PREDICTED NEGATIVE	3	ACTUAL NEGATIVE	2

So one person who always comes late ,has come early.For this person I have predicted NEGATIVE but in reality he proved me FALSE.So let us name that person as FALSE NEGATIVE or FN And My True Negative was reduced down to 2.