The impact of measurement on your work – choosing the right approach to evaluation

We all know that measuring outcomes can be a tricky business. In particular, measuring ‘soft outcomes’ such as self-esteem or life satisfaction can be difficult because it can affect how we interact with our clients or beneficiaries. In turn this has implications for the objectivity of your results – so you need to tread carefully and decide exactly what you want from your measurement tools before using them.

With this in mind, it’s helpful to think of three approaches, each of which involve a different disagree of interaction.

First, there is measurement as evaluation. This is where measurement is for the purpose of testing the effectiveness or a programme or intervention. Whilst on a programme, a young person may be asked to complete a survey but the impact of measurement on the intervention itself is kept to a minimum. For evaluation the key thing is that methods are as objective as possible so tend to rely on a relatively low degree of interaction with young people – to minimise bias and so that they can answer honestly.  NPC’s Well-being Measure fits into this category – your survey might be gathering detailed information from young people but you are not substantially affecting their experience.

Second, there is measurement as diagnosis. This is where approaches focus on the individual, often using ‘clinical scales’ to focus on mental health. An example of this approach is the Goodman’s Strengths and Difficulties questionnaire (the ‘SDQ’), which measures mental health in young people and focuses on identifying problematic thoughts or behaviours. The SDQ itself involves a relatively low degree of interaction but it may also be combined with individual face-to-face assessment by a professional. Results from the SDQ can also be aggregated used to find out impact at programme level.

Third is measurement as part of the therapeutic process. This is where approaches form a direct and deliberate part of interaction with beneficiaries – the very act of measurement becomes part of the intervention. In this approach, measurement is done in discussions between project worker and young person, often where the two parties agree on a ‘score’ or outcome. Examples of this approach are the Rickter Scale and Outcomes Stars. As a case-working tool this approach can provide a great way of opening up dialogue and working through problems. However, as an evaluation tool it is problematic as it can be open to influence by project workers and risks producing ‘false positive’ results – leading to accusations that it is not an objective way to measure.

Overall, measuring outcomes and how it afffects your intervention is an area where you have to tread carefully. Ultimately the approach you choose depends on why you want to measure, what you want to achieve, and how it will impact upon the experience of your beneficaries.

Interested in discussing your approach to measurement and evaluation? Contact the NPC team to hear more about our consulting services by emailing wellbeing@philanthropycapital.org.

Advertisements

About John Copps

John is part of NPC's research and consulting team and is the founder of NPC's Well-being Measure, a social business that provides an online tool to measure young people’s well-being. He has eight years experience of research and consulting, and is passionate about how data can be used to improve the performance of organisations. John is a regular contributor to NPC's blog and has also contributed to pieces for BBC Radio, the Guardian, and the Financial Times. John is a governor of a secondary school.
This entry was posted in General Well-being. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s