An experiment to evaluate constraint diagrams with novice users

Constraint diagrams [1, 2] constitute a graphical notation for program specification. This paper presents an experiment that compares the efficacy of constraint diagrams (CD) with natural language (NL) for understanding program specification statements. In a web-based competition 33 participants were given training either on CD notation or equivalent NL expression, and answered questions about specification statements. It was predicted that the CD participants would find learning and answering questions harder than those in the NL group, because they had no prior experience of CD notation. Although the CD group spent more time on the training examples and were less confident about their answers, they spent less time answering the questions and achieved approximately the same proportion of correct answers as the NL group.