Retfound Fine-tune

Three disease heads:

  • Glaucoma
  • Diabetic retinopathy
  • AMD

Two ideas:

  • Partial labels, not fake negatives
  • Sentence alignment, image ↔ disease text

Goal

Make RETFound features useful for classification.

Training signal mix:

  • Disease prediction
  • Same-disease feature grouping
  • Cross-dataset control alignment
  • EyeCLIP sentence guidance

Missing labels ignored. Positive labels aligned to disease sentences.

Partial labels: why needed

Datasets uneven.

Dataset Total images Glaucoma cases DR cases AMD cases Why mask?
AIROGS 80,250 2,633 Unknown Unknown Glaucoma labels only
EyePACS 28,100 Unknown 7,496 Unknown DR labels only
ADAM400 400 Unknown Unknown 89 AMD labels only
ODIR7000 7,000 326 1,803 280 All three observed

Unknown ≠ negative.

Unknown = no supervision for that disease.

Partial-label mask

Each disease gets two pieces:

  • Label: positive / negative / unknown
  • Mask: observed / missing

Example:

Disease Label Mask Meaning
Glaucoma 1 1 use as positive
DR -1 0 ignore
AMD 0 1 use as negative

Loss only from mask = 1.

Disease prediction

Model always predicts all three diseases.

only observed entries valid

prediction: [glaucoma, DR, AMD]
label:      [positive, unknown, negative]
mask:       [use,      skip,    use]

Effect:

  • Glaucoma head learns from this image
  • AMD head learns from this image
  • DR head receives no push

No penalty for unknown DR.

No direct false negative.

Partial Labels

Mask controls representation losses.

Same-disease grouping

Control alignment:

  • Use only observed negatives
  • “Glaucoma-negative” = glaucoma ruled out
  • Not same as healthy eye

Why important:

  • Less label noise
  • Better disease-specific features

Sentence alignment

EyeCLIP text encoder gives disease meaning.

Be close to text meaning of observed disease.

Image side:

  • RETFound fundus feature
  • Projected into EyeCLIP text space

Text side:

  • Disease sentences
  • Frozen EyeCLIP embeddings
  • One prototype per disease

Only observed positives aligned.

Disease sentence prototypes

Multiple short clinical sentences per disease.

Glaucoma:

  • fundus photograph showing glaucoma
  • increased cup-to-disc ratio

DR:

  • diabetic retinopathy with microaneurysms
  • retinal haemorrhages

AMD:

  • age-related macular degeneration with drusen
  • pigment changes

Prototype = average sentence meaning, disease-specific direction.

What TextAlign does

For image with observed glaucoma:

image feature ──move closer──▶ glaucoma sentence prototype

For image with DR unknown:

image feature ──no constraint──▶ DR sentence prototype

For image with AMD negative:

no sentence alignment from negative label

TextAlign uses positives only.

Positive, negative, unknown

Negative labels still train the classifier.

Unknown labels are skipped everywhere for that disease.

Label status Prediction loss Text alignment
Positive predict disease present Pull image toward disease text
Negative predict disease absent No alignment
Unknown Ignore No alignment
Made with Slides.com