Precision and Recall Calculation Examples
Learn how to calculate precision and recall for information retrieval systems using two detailed examples. Understand the concepts with step-by-step solutions and tabulated data, including insights on average precision calculation.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
EXAMPLE 2: Two retrieval systems, X and Y, are being compared. Both are given the same query, applied to a collection of 1500 documents. System X returns 400 documents, of which 40 are relevant to the query. System Y returns 30 documents, of which 15 are relevant to the query. Within the whole collection there are in fact 50 documents relevant to the query. Tabulate the results for each system, and compute the precision and recall for both X and Y. Show your working.
EXAMPLE 2:Solution: Tabulate" means to exhibit in a table, so this question requires a table showing the results for each system. X Relevant Notrelevant Total Retrieved 40 360 400 Notretrieved 10 1090 1100 Total 50 1450 1500 Y Relevant Not relevant Total Retrieved 15 15 30 Notretrieved 35 1435 1470 Total 50 1450 1500
EXAMPLE 2:Solution: System X Precision P =40/ 400= 0.1 Recall R =40 / 50= 0.8 System Y Precision P =15 / 30= 0.5 Recall R =15 / 50= 0.3
EXAMPLE 3: Assume the following: A database contains 80 records on a particular topic A search was conducted on that topic and 60 records were retrieved. Of the 60 records retrieved, 45 were relevant. Calculate the precision and recall scores for the search.
EXAMPLE 3:Solution: Using the designations above: TP = The number of relevant records retrieved, FN = The number of relevant records not retrieved, and FP = The number of irrelevant records retrieved. In this example TP = 45, FN = 35 (80-45) and FP = 15 (60-45). Precision = (45 / (45 + 15)) * 100% => 45/60 * 100% = 75% Recall = (45 / (45 + 35)) * 100% => 45/80 * 100% = 56%
Average Precision The average of the precision values from the rank positions where a relevant document was retrieved, i.e., when recall increases. Ranking #1: (1.0 + 0.67 + 0.75 + 0.8 + 0.83 + 0.6) / 6 =0.78 Ranking #2: (0.5 + 0.4 + 0.5 + 0.57 + 0.56 + 0.6) / 6 =0.52
