Head-to-head comparative evaluation is different because it makes four commercial diabetic retinopathy AI systems sit the same exam, on the same Tanzanian retinal images, with their brand names visible instead of politely hidden behind "Algorithm A" like everyone is in witness protection.
That matters because buying medical AI is not like choosing a toaster. If the toaster burns your bagel, annoying. If an AI screening system misses sight-threatening diabetic retinopathy, that is a very different morning.
The Retina Has Entered the Chat
Diabetic retinopathy happens when diabetes damages tiny blood vessels in the retina, the light-sensitive tissue at the back of the eye. Early on, people may feel totally fine, which is medically inconvenient and very on-brand for diseases that enjoy dramatic timing. By the time vision changes show up, the problem may already be harder to treat.
Screening helps catch disease early. Traditionally, that means taking retinal photos and having trained humans grade them. In places with enough ophthalmologists, cameras, internet, transport, appointment slots, and administrative patience, this can work well. In lower-resource settings, every one of those requirements can become a tiny bureaucratic boss fight.
So AI seems tempting: take retinal images, run them through software, and get a referral decision quickly. The AI is not "thinking" in the human sense. It is pattern-matching retinal features from fundus photos, looking for signs like hemorrhages, exudates, and abnormal blood vessels. Basically, it is a very specialized image nerd with no hobbies and, unlike Reviewer 2, no demand for three extra ablation studies.
The Study: Same Images, Same Test, No Hiding
Cleland and colleagues evaluated four commercially available AI systems for detecting referable diabetic retinopathy in Tanzania: Medios AI/Remidio, MONA, Ophtai, and SELENA+ DOI: 10.2337/dc26-0572, PMID: 42267948. The team first used a scoping review and expert consultation to identify systems that might work in a low-resource setting. Then they tested the systems whose developers confirmed suitability and agreed to participate.
The dataset included retinal images from 689 people in a Tanzanian diabetic retinopathy screening program. Of those, 379 people, or 55.0%, had referable diabetic retinopathy. Another 93 people, or 13.5%, had proliferative diabetic retinopathy, the scarier stage where abnormal new blood vessels show up like uninvited conference attendees at the buffet.
The main results were encouraging but not magical. Sensitivity for detecting referable disease ranged from 83.9% to 93.7%. Specificity was lower, from 70.3% to 79.0%. Translation: the systems were fairly good at catching people who needed referral, but they also flagged a decent number who might not need specialist care. For proliferative disease, sensitivity exceeded 98% across all four systems, which is the kind of result that makes clinicians lean forward instead of just nodding politely at a poster session.
High Sensitivity, Meet Real Life
In screening, sensitivity often gets top billing because missing disease is dangerous. If the AI says "all clear" when someone actually needs treatment, that person may lose precious time. But specificity matters too. Too many false positives can flood referral clinics, stretch staff, and create anxiety for patients. A screening program that refers everyone is technically sensitive, in the same way yelling "fire" in every building is technically cautious.
This is why the study's setting matters. Many AI tools perform nicely in curated datasets, where images are clean, labels are tidy, and reality has been politely asked to wait outside. Real screening programs are messier. Cameras vary. Lighting varies. Eyes vary. Internet connectivity sometimes behaves like it has tenure and cannot be fired.
The study also looked beyond accuracy. All four evaluated systems were CE-marked medical devices, and one, Medios AI/Remidio, worked offline as standard. Offline functionality is not a cute bonus feature in low-resource settings. It can be the difference between a useful tool and a very expensive icon on a laptop.
Why This Fits the Bigger AI-in-Eye-Care Story
Recent work has shown that autonomous AI diabetic eye exams can improve screening and follow-up in youth, as in the ACCESS randomized trial by Wolf and colleagues DOI: 10.1038/s41467-023-44676-z. Other real-world studies suggest these systems can improve access and equity in underserved populations DOI: 10.1038/s41746-024-01197-3, and even improve specialist clinic productivity DOI: 10.1038/s41746-023-00931-7.
But the Tanzania study adds something procurement teams desperately need: named, side-by-side evidence. Not vibes. Not vendor brochures. Not "our internal validation was excellent, trust us, the p-value was wearing a bow tie." Actual comparative data in the population where the system might be used.
A 2025 public health implementation study in India made a similar point: commercial diabetic retinopathy AI systems can vary widely in real-world performance, and local validation matters before deployment DOI: 10.2196/67529. Meanwhile, a 2026 npj Digital Medicine study found that general multimodal AI models still lagged behind clinical experts for diabetic eye screening, despite their impressive medical-exam swagger DOI: 10.1038/s41746-025-02216-7.
The Catch, Because Science Is Contractually Obligated to Have One
This study does not mean every clinic should instantly install one of these systems and call it a day. The dataset had high disease prevalence, and performance can shift across cameras, workflows, patient groups, and image quality. Also, a referral decision is only useful if referral care exists. AI can point at the problem. It cannot build the road, staff the clinic, pay for treatment, or convince the printer to work.
Still, this is exactly the kind of evidence medical AI needs more of: local, comparative, named, practical, and honest about tradeoffs. Assuming these results hold up in broader deployments, AI retinal screening could help more patients get triaged earlier, especially where specialists are scarce.
And that is the grown-up version of AI hype: not a robot doctor with a glowing forehead, but a screening assistant that helps the right patient reach the right clinician sooner. Less sci-fi. More useful. Honestly, much better grant material.
References
-
Cleland CR, Bascaran C, Makupa WU, et al. Head-to-Head Comparative Evaluation of Four Commercially Available Artificial Intelligence Systems for Detecting Referable Diabetic Retinopathy in a Tanzanian Population. Diabetes Care. 2026. https://doi.org/10.2337/dc26-0572, PMID: 42267948
-
Wolf RM, Channa R, Liu TYA, et al. Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial. Nature Communications. 2024;15:421. https://doi.org/10.1038/s41467-023-44676-z
-
Huang JJ, Channa R, Wolf RM, et al. Autonomous artificial intelligence for diabetic eye disease increases access and health equity in underserved populations. npj Digital Medicine. 2024;7:196. https://doi.org/10.1038/s41746-024-01197-3
-
Abramoff MD, Whitestone N, Patnaik JL, et al. Autonomous artificial intelligence increases real-world specialist clinic productivity in a cluster-randomized trial. npj Digital Medicine. 2023;6:184. https://doi.org/10.1038/s41746-023-00931-7
-
Kumar S, et al. Real-World Evaluation of AI-Driven Diabetic Retinopathy Screening in Public Health Settings: Validation and Implementation Study. JMIR Medical Informatics. 2025;13:e67529. https://doi.org/10.2196/67529
-
Hunt MS, Dai T, Abràmoff MD. Evaluating commercial multimodal AI for diabetic eye screening and implications for an alternative regulatory pathway. npj Digital Medicine. 2026;9:42. https://doi.org/10.1038/s41746-025-02216-7
Disclaimer: This blog post is a simplified summary of published research for educational purposes. The accompanying illustration is artistic and does not depict actual model architectures, data, or experimental results. Always refer to the original paper for technical details.