A question we often get asked is ‘how do I know that the difference I see in young people’s well-being is due to my programme, or due to other causes? This question is not new for researchers and one that you will find comes up in any survey or research that you do.
The simple answer is that you can never be 100% certain. In their laboratories, scientists put enormous effort (and money) into isolating different factors or conditions in their experiments. This allows them to focus their experiment on what they want to test.
Unfortunately, social scientists don’t have this luxury. Because they work with people, it is impossible to recreate laboratory conditions.
Instead, the nearest thing to recreating lab conditions is to use a ‘control group’. This is a comparison group, drawn from a population with similar characteristics and selected at random, which allows you to say ‘what would have happened’. By comparing the results of your experiment with what happens to the control group over the same period of time you can isolate the impact of what you are testing.
It is always worth thinking about whether you could run a control group within your survey. However, it may not be possible or desirable. For one, it is expensive and it is not worth doing unless you do it properly. And second, if you work with a particularly vulnerable group it may not be something you want to do as it might mean not providing a service to a comparison group that you know really needs it.
Without a control group your results will still give you a good sense of the difference you make to young people’s lives. When presenting your results it can be useful to talk about ‘contribution’ rather than attribution – recognising that there are many influences on young people’s lives and your work is one of them.
If you would like to use a control group and need help, you can contact us about our consulting services by emailing firstname.lastname@example.org.