No, what they meant is: The control group had 12 questions to get into the flow of solving math problems and then solved three more math problems for good measure.
The AI group on the other hand got into the flow of formulating math problems to ChatGPT and then had to actually solve three math problems themselves
Their critique is, that solving math problems yourself and prompting ChatGPT to solve math problems are not necessarily comparable tasks and require different skill sets so disabling AI after 12 tasks meant the first group had to switch context and therefore had worse performance.
If you want to analyze the first groups general ability of problem solving you should give them again twelve tasks after disabling AI so they get used to this new type of task (solving math problems yourself vs. prompting math problems to the AI) before measuring their performance.
No, what they meant is: The control group had 12 questions to get into the flow of solving math problems and then solved three more math problems for good measure.
The AI group on the other hand got into the flow of formulating math problems to ChatGPT and then had to actually solve three math problems themselves
Their critique is, that solving math problems yourself and prompting ChatGPT to solve math problems are not necessarily comparable tasks and require different skill sets so disabling AI after 12 tasks meant the first group had to switch context and therefore had worse performance.
If you want to analyze the first groups general ability of problem solving you should give them again twelve tasks after disabling AI so they get used to this new type of task (solving math problems yourself vs. prompting math problems to the AI) before measuring their performance.
That’s what the friggin test is about! So of course they did.