• Low-dose CT screening shown to decrease mortality → early diagnosis
• 85% of cancers found on screening clinical stage I, 10 yr survival 88%
• 75% of diagnosed cancers present on earlier scan

Introduction

• CAD approved for use as second reader
• Little prior investigation into sens/spec of CAD with regard to previously missed cancers
• Interval improvement in technology and increased screening

Introduction

• International Early Lung Cancer Action Program (I-ELCAP)
• 1994-2003
• solid nodules only (more aggressive, CAD for solid nodules more mature)

Methods

• 50 cancers
  • Path proven
  • Missed on prior scan
  • Slice thickness <= 1.25mm
  • Solid nodules only
• Time 1: First detected by radiologist
• Time 0: Prior screening round, retrospectively detected

Methods

• Independently reviewed by 2 radiologists (6 & 10 years experience)
• Differences resolved by consensus or by 3rd radiologist tiebreaker

Methods - Image review

• 4 CAD systems
  • CAD 1: Lung VCAR (GE)
    • Size limit: 3mm
  • CAD 2: ImageChecker CT (R2 Technologies)
    • Limit: 4mm
  • CAD 3: Syngovia Via Va 20 (Siemens)
    • Limit: 3mm
  • CAD 4: Cornell Via (Cornell U)
    • Limit: 3mm (manually set)

Methods - CAD systems

• All CAD annotated images reviewed by 2 radiologists for true and false positives
  • True nodules:
    • Nodules found by initial interpreting radiologist
    • CAD found nodules confirmed by reviewers
    • Additional nodules found by reviewers

Methods - CAD performance

• Interobserver agreement: κ statistic
• Nodule diameter: paired t-test
• CAD agreement on detection and FP rates: Multirater κ statistics
  • ​Fair: κ = 0.2 - 0.4
  • Moderate: κ = 0.41 - 0.6
  • Substantial: κ = 0.61 - 0.8
  • Very good: κ > 0.8

Statistics

Results - Cancer characteristics

Reviewers: moderate - substantial agreement

Avg 4.8 → 11.4 mm

Results - CAD detection

Results - CAD detection of actionable nodules at time 0

Of 37 actionable cancers:

CAD 1: 62%

CAD 2: 84%

CAD 3: 70%

CAD 4: 70%

Results - False positives at time 0

• Percent of identified nodules accepted
  • 34.5% - 85.4%
• Avg # rejected marks per scan
  • 0.6 - 7.4 (CAD 2, 1)

Results - CAD Detection of Cancers at time 1

• Fair agreement between systems
• Range from 74-82% (CAD 1, (2,3))
• None identified all that interpreting radiologist identified

Results - CAD Matchup

Discussion

• CAD detected 56 - 70% of missed cancers
• All were smaller at time 0, suggesting size as factor for human reader performance
• CAD also performs much better when nodules > 3 mm (69-78% vs 0-17%)
• 3 of 4 CAD systems better with 3-6mm nodules than >6 mm
  • >6 mm category triggers add'nl workup on baseline scan
• At time 1, CAD missed up to 26% of cancers (not ready as primary/concurrent reader)

Limitations

• Relatively small n (50)
• Only analyzed performance with missed cancers rather than all cancers

Conclusion

• Most appropriate role of CAD is as a second reader
• Capability of the CAD system to detect at least some missed cancers is compelling

My thoughts

• What strategies do these algorithms use?
• Could they be combined into a super-CAD?
• Illustrates how far CAD still has to go before our jobs are at risk
  • Few more ideal problems for CAD/CV in radiology, esp cross sectional imaging
• What is the threshold for acceptance of CAD?