ROC Curve & AUC Calculator

Advanced Statistical Tests

Input your model's prediction scores and true labels below to generate an ROC curve and calculate the Area Under the Curve (AUC).

Practical Examples

Click on an example to load its data into the calculator.

Cancer Detection Model

Medical Diagnosis

Evaluating a model that predicts the probability of a tumor being malignant (1) or benign (0).

Positive Label: 1

Negative Label: 0

0.95,1
0.85,1
0.80,0
0.70,1
0.55,1
0.45,0
0.40,1
0.30,0
0.25,0
0.10,0

Credit Default Prediction

Financial Risk

Assessing a model that calculates the likelihood of a customer defaulting on a loan ('default') vs. not defaulting ('paid').

Positive Label: default

Negative Label: paid

0.88,default
0.76,paid
0.71,default
0.65,paid
0.61,paid
0.52,default
0.41,paid
0.39,default
0.22,paid
0.15,paid

Spam Email Filter

Marketing

Testing a filter that scores emails on their probability of being spam ('spam') vs. not spam ('ham').

Positive Label: spam

Negative Label: ham

0.99,spam
0.91,spam
0.82,ham
0.75,spam
0.63,ham
0.51,spam
0.49,ham
0.33,ham
0.21,spam
0.11,ham

Ideal Separation

Perfect Classifier

An example of a perfect classifier where all positive samples have higher scores than all negative samples.

Positive Label: 1

Negative Label: 0

0.9,1
0.8,1
0.7,1
0.6,1
0.5,1
0.4,0
0.3,0
0.2,0
0.1,0
0.05,0
Other Titles
Understanding the ROC Curve: A Comprehensive Guide
An in-depth look at the ROC curve, AUC, and their significance in evaluating classification models.

What is a ROC Curve?

  • The Basics of Classification
  • True Positives vs. False Positives
  • Visualizing Performance
A Receiver Operating Characteristic (ROC) curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The curve is created by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) at various threshold settings.
Key Components of a ROC Curve
The two fundamental metrics that form the ROC curve are the True Positive Rate (Sensitivity) and the False Positive Rate. TPR measures the proportion of actual positives that are correctly identified as such. FPR measures the proportion of actual negatives that are incorrectly identified as positives. An ideal classifier would have a TPR of 1 and an FPR of 0, corresponding to the top-left corner of the ROC space.

Step-by-Step Guide to Using the ROC Curve Calculator

  • Data Formatting
  • Defining Classes
  • Interpreting the Results
Using this calculator is straightforward. First, prepare your data. It should consist of two columns or parts per line: the prediction score from your model and the actual true label.
1. Inputting Data
Copy and paste your data into the main text area. Each entry must be on a new line, with the score and label separated by a comma (e.g., 0.85,1).
2. Specifying Labels
In the 'Positive Class Label' and 'Negative Class Label' fields, enter the exact text or number that represents your positive and negative classes (e.g., '1' and '0', or 'spam' and 'ham'). This is case-sensitive.
3. Calculation and Analysis
Click 'Calculate'. The tool will output the Area Under the Curve (AUC), identify the optimal threshold for classification based on Youden's J statistic, and provide the sensitivity and specificity at that threshold. It also generates the (FPR, TPR) points needed to plot the ROC curve yourself.

The Significance of the Area Under the Curve (AUC)

  • AUC as a Performance Metric
  • Interpreting AUC Values
  • Limitations of AUC
The Area Under the Curve (AUC) is the single most important metric derived from the ROC curve. It provides an aggregate measure of performance across all possible classification thresholds. AUC represents the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one.
How to Interpret AUC Values
AUC values range from 0 to 1, where a higher value indicates better performance. An AUC of 1.0 represents a perfect classifier. An AUC of 0.5 suggests no discrimination ability, equivalent to random guessing. An AUC less than 0.5 indicates the model is performing worse than random guessing.

General AUC Interpretation Guide

  • AUC = 1.0: Perfect classifier.
  • AUC > 0.9: Outstanding.
  • AUC > 0.8: Excellent.
  • AUC > 0.7: Acceptable.
  • AUC = 0.5: No predictive value (random chance).
  • AUC < 0.5: Worse than random.

Real-World Applications of ROC Analysis

  • Medical Diagnostics
  • Finance and Credit Scoring
  • Machine Learning Model Selection
Medical Diagnostics
In medicine, ROC curves are used to evaluate the performance of diagnostic tests. For example, a test might be developed to detect a certain disease based on a biomarker level. The ROC curve helps determine the optimal cutoff for that biomarker level to maximize true positives while minimizing false positives.
Finance and Credit Scoring
Banks use scoring models to predict whether a loan applicant will default. ROC analysis helps them choose a credit score threshold that balances the risk of lending to a bad applicant (false positive) against the missed opportunity of denying a good applicant (false negative).

Mathematical Derivation and Finding the Optimal Threshold

  • Calculating TPR and FPR
  • Constructing the Curve
  • Youden's J Statistic
To construct an ROC curve, the data is first sorted by the model's score in descending order. Then, every unique score is treated as a potential threshold. For each threshold, we classify all instances with scores above it as 'positive' and below it as 'negative'.
Formulas for TPR and FPR
TPR = TP / (TP + FN) and FPR = FP / (FP + TN), where TP is True Positives, FN is False Negatives, FP is False Positives, and TN is True Negatives.
Youden's J Statistic
To find the 'optimal' threshold, this calculator uses Youden's J statistic. It is defined for each point on the ROC curve as J = Sensitivity + Specificity - 1 (or TPR - FPR). The threshold that maximizes this value is considered optimal as it represents the point furthest from the line of no-discrimination (the diagonal line).