Unfortunately your proposed study design does not control for biases. For example, say I'm an audiophile comparing two audio formats, x and y, and I hear y is remarkably better. If I compare them side by side, my own brain might fool me that y is better. The way to control that is to do a blind test, where I won't know which one is x or y.
Also, asking people their impressions can be helpful in the human aspect of the research, but quantitatively you need some sort of metric you can evaluate their experience on. For example, you'd assign a task and see how well people did comparing the two modes, while also making sure the difference is statistically significant (meaning it wasn't just as likely to be chance).
This is just the beginning of where good study design starts. You'd also do things like assigning the modes themselves randomly, so to go back to the audiophile example, I might notice if it's always x and then y, but not if it's scrambled. You'd could go further and try to stratify the groups, so for example making sure one group isn't all elderly people and the other young. It goes on and on...
So while the scientific method is nice, especially for introducing science in educational contexts, the methodology and rationale behind research is much more deliberate and involved. By all means they can try out things themselves, but no, they will not "get better information than any study".