Text Readability and Simplification
Open access
Abstract
Abstract
This paper describes the creation of a set of crowd-sourced human evaluations of automated simplifications. We created a synthetic simplification dataset which was assessed by the crowd on dimensions of fluency, clarity, accuracy and complexity. We briefly describe the crowdsourcing environment and some techniques to keep the crowd engaged. This resulted in a dataset of nearly 25,000 responses from 384 respondents on synthetically simplified Dutch data. In order to mitigate the low inter-annotator agreement we investigate the effect of outlier removal techniques on Krippendorf α. The main conclusions stay the same: the synthetic texts are not simpler according to the linguistic proxies investigated, but the crowd judged that in about 80% of the cases the synthetic text was more clear than the original and less complex in about 70% of the cases. The fluency was the same for original sentences versus synthetic sentences, and in the case of accuracy the numbers changed most after outlier removal. A medium accuracy was achieved in about 80%, a high accuracy in 50% before outlier removal and 60% after outlier removal. Both the synthetic parallel data as well as the crowd assessments are made available to the CLARIN infrastructure.
Direct answer
What can I do from this paper page?
Use this page to scan "Results of Crowdsourcing Human Evaluation of SyntheticSimplification" quickly: start with the summary and abstract, then check the authors, source, topics, and related papers. From here, open Scollr to follow Text Readability and Simplification research, save the paper, or map adjacent work.
Research areas
Follow related topics
Citation
BibTeX
@article{Vandeghinste2026Results,
title = {Results of Crowdsourcing Human Evaluation of SyntheticSimplification},
author = {Vincent Vandeghinste and Bram Vanroy and Job Van Doeselaar},
journal = {Linköping electronic conference proceedings},
year = {2026},
doi = {10.3384/ecp222.1595},
url = {https://doi.org/10.3384/ecp222.1595}
}FAQ
Using this paper in a discovery workflow
How do I find related work for this paper?
Use the related papers and topic links on this page as starting points. In Scollr, you can also open the paper and build a literature map around its references, citing papers, and related work.
How can I keep up with new Text Readability and Simplification research papers?
Follow Text Readability and Simplification research in Scollr. New papers from the topic flow into a personalized feed, and you can save useful studies to revisit later.
Can I cite this paper from this page?
This page includes a static BibTeX block for Results of Crowdsourcing Human Evaluation of SyntheticSimplification. Always verify the DOI, source, and publication details against the publisher record before submitting a manuscript.
Follow this research in Scollr
Follow the topics and authors behind this paper, save useful studies, and build a literature map when you are ready to go deeper.
Get the app