Retfound Fine-tune

Three disease heads:

• Glaucoma
• Diabetic retinopathy
• AMD

Two ideas:

• Partial labels, not fake negatives
• Sentence alignment, image ↔ disease text

Goal

Make RETFound features useful for classification.

Training signal mix:

• Disease prediction
• Same-disease feature grouping
• Cross-dataset control alignment
• EyeCLIP sentence guidance

Missing labels ignored. Positive labels aligned to disease sentences.

Partial labels: why needed

Datasets uneven.

Unknown ≠ negative.

Unknown = no supervision for that disease.

Partial-label mask

Each disease gets two pieces:

• Label: positive / negative / unknown
• Mask: observed / missing

Example:

Loss only from mask = 1.

Disease prediction

Model always predicts all three diseases.

only observed entries valid

prediction: [glaucoma, DR, AMD]
label:      [positive, unknown, negative]
mask:       [use,      skip,    use]

Effect:

• Glaucoma head learns from this image
• AMD head learns from this image
• DR head receives no push

No penalty for unknown DR.

No direct false negative.

Partial Labels

Mask controls representation losses.

Same-disease grouping

Control alignment:

• Use only observed negatives
• “Glaucoma-negative” = glaucoma ruled out
• Not same as healthy eye

Why important:

• Less label noise
• Better disease-specific features

Sentence alignment

EyeCLIP text encoder gives disease meaning.

Be close to text meaning of observed disease.

Image side:

• RETFound fundus feature
• Projected into EyeCLIP text space

Text side:

• Disease sentences
• Frozen EyeCLIP embeddings
• One prototype per disease

Only observed positives aligned.

Disease sentence prototypes

Multiple short clinical sentences per disease.

Glaucoma:

• fundus photograph showing glaucoma
• increased cup-to-disc ratio

DR:

• diabetic retinopathy with microaneurysms
• retinal haemorrhages

AMD:

• age-related macular degeneration with drusen
• pigment changes

Prototype = average sentence meaning, disease-specific direction.

What TextAlign does

For image with observed glaucoma:

image feature ──move closer──▶ glaucoma sentence prototype

For image with DR unknown:

image feature ──no constraint──▶ DR sentence prototype

For image with AMD negative:

no sentence alignment from negative label

TextAlign uses positives only.

Positive, negative, unknown

Negative labels still train the classifier.

Unknown labels are skipped everywhere for that disease.