Receiver operating characteristic

In signal detection theory, a receiver operating characteristic (ROC) is a graphicalical plot of the number of true positives vs. the number of false positives for a binary classifier system as its discrimination threshold is varied. The usage receiver operator characteristic is also common.

ROC curves are used to evaluate the results of a prediction and was first employed in the study of discriminator systems for the detection of radio signals in the presence of noise in the 1940s. They also proved to be useful for the evaluation of machine learning results, such as the evaluation of Internet search engines. They are also used extensively in epidemiology and medical research.

The best possible prediction method would yield a graph that was a point in the upper left corner of the ROC space, i.e. 100% sensitivity (all true positives are found) and 100% specificity (no false positives are found). A completely random predictor would give a straight line at an angle of 45 degrees from the horizontal, from bottom left to top right: this is because, as the threshold is raised, equal numbers of true and false positives would be let in. Results below this no-discrimination line would suggest a detector that gave wrong results consistently, and could therefore be simply used to make a detector that gave useful results by inverting its decisions.

Sometimes, the ROC is used to generate a summary statistic. Two common versions are:

• the intercept of the ROC curve with the line at 90 degrees to the no-discrimination line
• the area between the ROC curve and the no-discrimination line

However, any attempt to summarize the ROC curve into a single number loses information about the pattern of tradeoffs of the particular discriminator algorithm.

The area between the ROC curve and the line is most frequently used statistic, because of its useful mathematical properties as a non-parametric statistic. This area is often simply known as the discrimination.

Illustration of the use of ROC graphs for discrimination. TP - true positives, FP - false p., TN - true negatives, FN - false n.. Starting from two distributions of positives (red) and negatives (blue) one can apply a threshold criterion (vertical line) to arbitrarily separate the two. For overlapping distributions, there is always a tradeoff between sensitivity (TP) and specificity (1-FN). TP and FN as well as TN and FP both add up to 1. Sliding the threshold line towards the distribution of positives will result in a decreased probability for true positive detection P(TP) and FPs, which is equivalent to moving the ROC curve (dashed) downwards. If the two distributions overlap completely, the ROC curve will be the diagonal shown as the dot-dashed curve.

See also:

• Mann-Whitney U test
• Fisher's linear discriminator

External links

• Receiver Operating Characteristic bibliography
• A simple example of a ROC curve
• applets
• A more thorough treatment of ROC curves and signal detection theory