Evaluating Multiple Diagnostic Tests with Partial Verification

To evaluate diagnostic tests, one would ideally like to verify, for example, with a biopsy, the disease state of all subjects in a study. Often, however, no all subjects are verified. Previous methods for evaluation assume that the decision to verify depends only on recorded variables. Sometimes, particularly if the disease process is not well understood, the decision to verify may also depend on unrecorded variables related to disease. We propose a method to estimate the true- and false-positive rates of multiple tests while adjusting for the effect, on the decision to verify, of unrecorded variables related to disease. To put the estimates into a more usable form, we develop a simple algorithm for creating a receiver-operating curve which maximizes the true-positive rate, given the false-positive rate. We apply the methodology to data on the early detection of prostate cancer using ultrasonography, digital rectal exam, and prostate specific antigen.