JLE

Epileptic Disorders

MENU

Measuring expertise in identifying interictal epileptiform discharges Volume 24, issue 3, June 2022

Figure 1.

(A) Performance metrics for sensitivity and specificity of clinical experts (blue), the original eight experts used for the reference standard (black), experienced non-clinical experts (green) and novices (red). (B-D) Calibration curves for experts (B), experienced non-experts (C) and novices (D).

Figure 2.

95% confidence interval (CI) for each of the performance metrics: sensitivity (A), false positive rate (B) and calibration error (C) as a function of the number of questions answered. The black vertical dashed lines show the minimum number of questions required to drive the 95% CI below 0.1, corresponding to 549 (A), 514 (B) and 250 (C).

Figure 3.

The “latent trait” framework for analyzing level of expertise in spike detection: (A) schematic of our framework for measuring a scorer’s level of expertise in recognizing epileptiform discharges; and (B) simulation of the decision process for the ideal observer, expert (including the original eight), experienced non-expert and novice (from top to bottom).

Figure 4.

(A) Estimation of scorer’s internal parameters for internal noise levels and σ and θ threshold for experts (blue), the original eight experts used as the reference standard (black), experienced non-experts (green) and novices (red). (B) Updated ROC curves based on estimated internal parameter (blue: experts; green: experienced non-experts; red: novices).