The persistent global STEM knowledge gap disproportionately affects the visually impaired, primarily due to inadequate or absent alt text. This study investigates whether Visual Language Models (VLMs) can bridge this accessibility divide by generating accurate alt text for complex STEM imagery. Using VLMs, we conducted a systematic evaluation against a curated dataset of 533 expert-annotated images from education ebooks. Our analysis reveals a clear dichotomy: VLMs perform well on straightforward visual content such as labeled chemistry diagrams but falter significantly with images requiring quantitative reasoning, including unlabeled charts and intricate 3D figures. These limitations highlight risks of misinformation when relying solely on automated descriptions for critical scientific visuals. These findings underscore the necessity of integrating human expertise with VLM efficiency to ensure accuracy and equity. This research proposes a hybrid intelligence approach combining rapid model output with expert validation as the only ethically sound solution to enhance STEM accessibility and foster inclusive knowledge dissemination.

Risi, M., Donati, N., Farneti, F., Occorso, M., Pio Volgarino, A., Romito, F. (In stampa/Attività in corso). AI and Accessibility: Describing Graphs with Alternative Texts.

AI and Accessibility: Describing Graphs with Alternative Texts

Nicolò Donati
Co-primo
;
In corso di stampa

Abstract

The persistent global STEM knowledge gap disproportionately affects the visually impaired, primarily due to inadequate or absent alt text. This study investigates whether Visual Language Models (VLMs) can bridge this accessibility divide by generating accurate alt text for complex STEM imagery. Using VLMs, we conducted a systematic evaluation against a curated dataset of 533 expert-annotated images from education ebooks. Our analysis reveals a clear dichotomy: VLMs perform well on straightforward visual content such as labeled chemistry diagrams but falter significantly with images requiring quantitative reasoning, including unlabeled charts and intricate 3D figures. These limitations highlight risks of misinformation when relying solely on automated descriptions for critical scientific visuals. These findings underscore the necessity of integrating human expertise with VLM efficiency to ensure accuracy and equity. This research proposes a hybrid intelligence approach combining rapid model output with expert validation as the only ethically sound solution to enhance STEM accessibility and foster inclusive knowledge dissemination.
In corso di stampa
The Journal on Technology and Persons With Disabilities, Volume 14
1
48
Risi, M., Donati, N., Farneti, F., Occorso, M., Pio Volgarino, A., Romito, F. (In stampa/Attività in corso). AI and Accessibility: Describing Graphs with Alternative Texts.
Risi, Milena; Donati, Nicolò; Farneti, Francesco; Occorso, Manuel; Pio Volgarino, Antonio; Romito, Francesco
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/1042103
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact