MULTI-MODEL APPROACH TO SYNTHETIC DATA GENERATION FOR SEMANTIC SIMILARITY ASSESSMENT OF WEB INTERFACE TEXT LABELS

Abstract

Automating web accessibility verification against WCAG standards remains a significant challenge, particularly for criteria requiring semantic understanding of content. WCAG Success Criterion 2.5.3 requires consistency between the visible text of interface elements and their accessible names; however, traditional string matching methods fail to account for semantic nuances of textual modifications. Objective. To develop a methodology for creating and validating a robust dataset for assessing semantic correspondence of text labels in the absence of an objective ground truth. This includes developing a taxonomy of semantic transformations, applying a multi-model approach to generation and annotation of synthetic data in Ukrainian and English languages. Methods. The study employs a multi-model approach involving four leading LLMs to generate over 14 thousand unique text pairs and 17 models for data annotation. A taxonomy of semantic transformations was developed that classifies modification types from permissible contextual extensions to critical contradictions. For validation, a statistical framework for consensus formation based on the median scores of a reference core of models was applied, utilizing ICC2k metrics, coefficients of determination, and Spearman correlation. Results. A validated dataset was created with synthetic data annotated on a semantic similarity scale from -1 to 1. The multi-model approach ensured dataset diversity and minimized biases of individual models. Conclusions. The developed methodology effectively addresses the problem of creating training data without an objective ground truth. The formed dataset enables objective comparison of commercial LLMs and cost-effective knowledge distillation into small models for practical application in automated accessibility testing tools.