Some Useful Statistics Definitions

• The Four Fundamental Numbers: True Positive, True Negative, False Positive, False Negative.
• The Four (or Eight) Basic Ratios: Sensitivity (and Type II Error), Specificity (and Type I Error), Positive Predictive Value (and False Discovery Rate), Negative Predictive Value (and False Omission Rate).
• The Four Ratios of Ratios: Likelihood Ratios for Positive Tests, Negative Tests, Positive Subjects, Negative Subjects.
• The Test As a Whole: Significance, Power.

The Four Fundamental Numbers

In Tabular Form

For individuals tested for some condition, disease, or other attribute:

	Doesn't Have The Condition (Satisfies Null Hypothesis)	Has The Condition (Does Not Satisfy Null Hypothesis)
Tests Negative (Null Accepted)	True Negative TN or n00	False Negative FN or n10
Tests Positive (Null Rejected)	False Positive FP or n01	True Positive TP or n11

In Words

• True positive
  • The individual has the condition and tests positive for the condition
  • The individual does not satisfy the null hypothesis and the test rejects the null hypothesis
  • TP = n₁₁ = number of such individuals
• True negative
  • The individual does not have the condition and tests negative for the condition
  • The individual satisfies the null hypothesis and the test accepts the null hypothesis
  • TN = n₀₀ = number of such individuals
• False positive
  • The individual does not have the condition but tests positive for the condition
  • The individual satisfies the null hypothesis but the test rejects the null hypothesis
  • FP = n₀₁ = number of such individuals
• False negative
  • The individual has the condition but tests negative for the condition
  • The individual does not satisfy the null hypothesis but the test accepts the null hypothesis
  • FN = n₁₀ = number of such individuals

The Four (or Eight) Basic Ratios

Each of these four fundamental numbers can be divided by its row sum or its column sum.  This gives eight basic ratios, though they come in pairs that sum to one.

• Sensitivity
  • The chance of testing positive among those with the condition
  • The chance of rejecting the null hypothesis among those that do not satisfy the null hypothesis
  • 1 - Type II Error
  • TP / (TP + FN) = n₁₁ / (n₁₀ + n₁₁)
• Specificity or Selectivity
  • The chance of testing negative among those without the condition
  • The chance of accepting the null hypothesis among those that satisfy the null hypothesis
  • 1 - Type I Error
  • TN / (TN + FP) = n₀₀ / (n₀₀ + n₀₁)
• Positive Predictive Value
  • The chance of having the condition among those that test positive
  • The chance of not satisfying the null hypothesis among those that reject the null hypothesis
  • 1 - False Discovery Rate
  • TP / (TP + FP) = n₁₁ / (n₀₁ + n₁₁)
• Negative Predictive Value
  • The chance of not having the condition among those that test negative
  • The chance of satisfying the null hypothesis among those that accept the null hypothesis
  • 1 - False Omission Rate
  • TN / (TN + FN) = n₀₀ / (n₀₀ + n₁₀)
• Type I Error or α (alpha) or p-Value or false positive rate
  • The chance of testing positive among those without the condition
  • The chance of rejecting the null hypothesis among those that satisfy the null hypothesis
  • 1 - Specificity
  • FP / (FP + TN) = n₀₁ / (n₀₀ + n₀₁)
• Type II Error or β (beta) or false negative rate
  • The chance of testing negative among those with the condition
  • The chance of accepting the null hypothesis among those that do not satisfy the null hypothesis
  • 1 - Sensitivity
  • FN / (FN + TP) = n₁₀ / (n₁₀ + n₁₁)
• False Discovery Rate or q-Value
  • The chance of not having the condition among those that test positive
  • The chance of satisfying the null hypothesis among those that reject the null hypothesis
  • 1 - Positive Predictive Value
  • FP / (FP + TP) = n₀₁ / (n₀₁ + n₁₁)
• False Omission Rate
  • The chance of having the condition among those that test negative
  • The chance of not satisfying the null hypothesis among those that accept the null hypothesis
  • 1 - Negative Predictive Value
  • FN / (FN + TN) = n₁₀ / (n₀₀ + n₁₀)

The Four Ratios of Ratios

Each specific test result (positive or negative) and each specific state (having the condition or not) can be used to define a ratio of ratios:

• Likelihood Ratio for Positive Tests
  • The chance of testing positive among those with the condition; divided by the chance of testing positive among those without the condition
  • Sensitivity / Type I Error
  • (1 - Type II Error) / (1 - Specificity)
  • [ TP / (TP + FN) ] / [ FP / (FP + TN) ]
• Likelihood Ratio for Negative Tests
  • The chance of testing negative among those with the condition; divided by the chance of testing negative among those without the condition
  • Type II Error / Specificity
  • (1 - Sensitivity) / (1 - Type I Error)
  • [ FN / (FN + TP) ] / [ TN / (TN + FP) ]
• Likelihood Ratio for Positive Subjects
  • The chance of having the condition among those testing positive; divided by the chance of having the condition among those testing negative
  • Positive Predictive Value / False Omission Rate
  • (1 - False Discovery Rate) / (1 - Negative Predictive Value)
  • [ TP / (TP + FP) ] / [ FN / (FN + TN) ]
• Likelihood Ratio for Negative Subjects
  • The chance of not having the condition among those testing positive; divided by the chance of not having the condition among those testing negative
  • False Discovery Rate / Negative Predictive Value
  • (1 - Positive Predictive Value) / (1 - False Omission Rate)
  • [ FP / (FP + TP) ] / [ TN / (TN + FN) ]

The Test As a Whole

Rather than focusing on what the data implies about any given tested individual, we can measure the quality of the test as a whole.  That is, for a specific distinctiveness criterion, and specific assumptions about the two groups we are testing (individuals who satisfy null hypothesis vs. those who do not), we can ask what is the chance that the distinctiveness criterion will be satisfied.

• Significance
  • The chance that the distinctiveness criterion would indicate a difference between the two groups even when the two groups do not actually differ
  • It is often calculated via χ² which, in this case, simplifies to:

    χ² = [ (TP*TN - FP*FN)² (TP+TN+FP+FN) ] / [ (TP+FP)(TP+FN)(TN+FP)(TN+FN) ]
    χ² = [ (n₀₀ n₁₁ - n₀₁ n₁₀)² (n₀₀ + n₀₁ + n₁₀ + n₁₁) ] / [ (n₀₀ + n₀₁)(n₁₀ + n₁₁)(n₀₀ + n₁₀)(n₀₁ + n₁₁) ]

  • Reported as the α or p-Value (or sometimes 1 - α) of the χ² value
• Power
  • The chance that the distinctiveness criterion would indicate a difference between the two groups when the two groups do actually differ
  • Depends on the choice of the distinctiveness criterion
  • Reported as the 1 - β (or sometimes β) of the distinctiveness criterion

Copyright 2005, 2006, Lee Newberg.  All rights reserved.