There is an art to surveying people. One particular problem faced by generations of social researchers is how to get people to reflect truthfully on their experiences of life, and not to colour their responses by the way we ask the question. One element of this is the important our attention about how direct you are at addressing the subject in hand.
As the influential academic economist Professor Paul Dolan puts it, ‘nothing is ever as important as when you are thinking about it’. To take a trivial example, ask a young person how important chocolate is to them and simply by introducing the idea of ‘chocolate’ you’ve already given them a reason to think it’s important (and potentially influenced the way they will answer).
This can pose great difficulties but is one of the reasons why well-being research can be so useful in evaluation. NPC’s Well-being Measure avoids direct questions about projects or programmes and asks young people instead about things that matter in their lives. It sounds peverse, but this is arguably the the most ‘direct’ way to measure impact.
This point is made in a 2011 lecture by Professor Paul Dolan, at the London School of Economics, which is well worth a watch. You can view the full lecture on YouTube here.