Effects of item headings in aptitude tests: evidence that math-related labels impair students’ performance in deductive reasoning tasks

Two studies examine the effects of domain-related labels used as item headings in aptitude tests. It appears that items and exercises aimed at assessing domain-general cognitive abilities (e.g., hypothesis testing in deductive reasoning) are alternatively labelled under the headings of “math”, “sciences”, “verbal logic”, “reasoning” items, and so on. Past research points out that task labelling may heavily affect solvers’ performances, depending on shared beliefs about intrinsic difficulty of the domain evoked by task label. We hypothesized therefore that when deductive reasoning tasks – administered with the purpose of testing aptitudes - are labelled as diagnostic of math-related abilities, students’ performances may be thwarted, compared to conditions in which the same tasks are labelled as diagnostic of verbal reasoning skills. In study 1, a modified version of the Wason Selection Task was labelled as diagnostic of “formal math demonstration” vs. “verbal reasoning” skills, by means of an heading on the top of the page. Results confirm that the rate of incorrect (confirmatory) answers is significantly higher under math rather than verbal/logic heading, independently from students’ perceived past success in math. In study 2, the Selection Task was inserted in a battery including other six items, three equations labelled as “math” items, and three text comprehension items (labelled as “verbal reasoning”). We hypothesized and found that when math items are indicated as the most diagnostic of students’ aptitudes, the rate of incorrect (confirmatory) answers at the Selection Task is significantly higher when this task is placed under the “math” rather than to the “verbal reasoning” heading, whereas no difference due to the Selection Task heading appears neither when diagnosticity is attributed to verbal reasoning items, nor in the control condition.