Kane, G. The Methodological Quality of Three Foundational Law Enforcement Drug Influence Evaluation Validation Studies. J Negat Results Biomed. 2013 Nov 4;12(1):16. doi:10.1186/1477-5751-12-16.

Abstract
BACKGROUND: ....This paper first identifies the scientific studies commonly cited in American criminal trials as evidence of DIE accuracy, and second, uses the QUADAS tool to investigate whether the methodologies used by these studies allow them to correctly quantify the diagnostic accuracy of the DIEs currently administered by US law enforcement.

Conclusion
Bigelow, Compton and Adler, the three validation studies commonly cited in Bigelow = Johns Hopkins
Compton = LAPD 173
Adler = AZ DRE Validation American criminal prosecutions to quantify the accuracy of current US law enforcement DIE practice, did no reference testing of driving performance or physical or mental impairment, investigated tests different from those currently employed by US law enforcement, used methodologies that biased accuracies, and reported DIE accuracy statistics that are not externally valid. The LEDA [DRE/DIE] accuracies reported by these studies do not quantify the accuracy of the DIE process now used by US law enforcement. These validation studies do not validate current DIE practice.

Kane 2013 is peer-reviewed science concluding that current DIEs are not validated by their official validation studies

Current US law enforcement DIE testing has no identifiable scientific accuracy. Follow me on this. At your next DUID trial, if asked, the DRE officer will testify that the scientific DIE test he administered to D is scientifically accurate because scientific validation studies —Johns Hopkins, LAPD 173— did a bunch of DIEs, and discovered that, sure enough, DIEs are accurate. 98% accurate. 87% accurate. Something like that. Your D's DRE knows his testing of D was accurate, because he did the same testing the LAPD 173 officers did. They were accurate. He was accurate. It just makes sense.

Kane 2013, a peer-reviewed scientific journal article, demonstrates that the officer is wrong (§ Accuracy). The prosecution has no peer-reviewed science to contradict Kane 2013's conclusions. Thus, the DRE has no scientific basis for testifying to any accuracy for the DIE he administered to D.

Kane 2013 is peer-reviewed science that IDENTIFIES BIASING METHODOLGIES that invalidate any DIE validation project that uses them, including the DRE officer's training and personal experience.

1

The DIE accuracies reported in NHTSA's validation studies (and testified to by DRE officers) are internally valid, but not externally valid. They do not describe the accuracy of the DIE administered to the defendant in any DUID prosecution. I said earlier that science is harder than you'd think. This is one of those times. NHTSA's practice of using the "accuracy" of a test, as measured in a validation study, to describe the predictive power of the same test in a criminal prosecution is scientifically wrong. Although this seems at first to offend our common sense notion of cause and effect, it has been understood by science since 1762. Kane 2013 explains (with references).

"Some diagnostic test accuracy statistics are externally valid (generalizable to populations other than the one in the study), some are not. PPV —“arrest accuracy”, or “accuracy of the arresting officers’ toxicology predictions”— is not externally valid. It depends on prevalence [27-30]. Because the prevalence of drug impairment changes as time and location change, even when officers make exactly the same measurements and interpret them with exactly the same criteria, the accuracy of the officers’ predictions changes depending on where and when the test is administered." Kane 2013, § Accuracy of diagnostic tests, pg 2

Any attempt by the prosecution to use the accuracy of the DIE is, scientifically, wrong.

2

DRE officers' amazing predictions not so amazing after all
NHTSA's validation studies never say DIEs are accurate at predicting driving performance.
NHTSA's validation studies never say DIEs are accurate at predicting driving impairment.
NHTSA's validation studies never say DIEs are accurate at predicting drug impairment.
NHTSA's validation studies never say DIEs are accurate at predicting impairment of any kind.

When NHTSA's validation studies report something is accurate, the thing they say is accurate is DRE officers' predictions of the presence of drugs in a person's blood —after the officer has asked "Are you taking any drugs?"  (Step 2 of the 12-step protocol). What NHTSA's science proves is, if you tell a trained, certified traffic-police Drug Recognition Expert that you have taken a drug, the officer will be highly accurate at predicting that you have taken a drug.

"[T]he circular question DIE validation studies rely on to confirm driver impairment, 'How else could officers have made specific drug predictions?' has a straightforward answer, 'By selecting for inclusion in the study people who confessed or had drugs found in a search.'" Kane 2013, § QUADAS Item 3, misclassification bias

3

Statements about the scientific accuracy of a Defendant's DRE/DIE test are valid and reliable only if they are based on scientific data for D's particular drug. None of the studies the DRE officer has knowledge of give relevant data for D's drug.

"'Drug' is not a homogenous medical entity and the accuracy of DRE Opinions varies from drug to drug. Using Compton's sample group as an example, sensitivity for PCP (phencyclidine) was 91%, but for cocaine 19%. Counting a correct PCP prediction as indicating the accuracy of cocaine predictions misclassifies that PCP result and misstates the accuracy of the LEDA [DRE/DIE] when an officer predicts cocaine." Kane 2013, § QUADAS Item 3

4

Key elements of the traffic-police DIE are not standardized

"None of the three studies described what their LEDA SA [DIE] findings were, or how those findings were combined to formulate DRE Opinions. The validation studies' LEDA DRE Opinions cannot be reproduced.

Current US law enforcement DIE methods do not reproduce the LEDA methods investigated in Bigelow, Compton and Adler. The accuracies reported in these studies do not reflect the accuracy of current DIEs." Kane 2013 § QUADAS Item 8, index test reproducibility

5

The Arizona DRE study reports a high DIE accuracy, calculated by hiding DIE wrong answers. Kane 2013, § QUADAS Item 10 and 11—review bias, pg 8

LAPD 173 reports a 94% DIE accuracy, calculated by counting DRE officers' wrong answers as correct. Kane 2013 § QUADAS Item 10 and 11—review bias, pg 8.

In fact LAPD 173 failed to prove DREs were any more accurate than random guessing.

"Compton's methodology failed to demonstrate that the LEDA-like [DRE/DIE] tests the study investigated were any more accurate than random guessing." Kane 2013, § Accuracies, pg 8, see also Endnote a, pg 10

6

The list of "7 drug categories" in the DRE Student Manual differs from the lists validated by Bigelow, Compton and Adler, whose lists differ from each other. [Table 2, pg 7]

7

Etc. Download a full-text pdf of Kane 2013 from the JNRBM web site, free.