Why outcomes are important for settlement services – HOST International and CSnet

David Keegan is the CEO of HOST International. He spoke to Mandy Doon at CSnet about how organisations can embed outcomes in the work that they do, in the practice, and use the data to contribute to ongoing research for better outcomes.

1. What is your experience of how data is collected and used in community services?

– Why its important to focus on measuring outcomes?

– How do you balance the data collected, the purpose, and time it takes?

My background is as a social worker. I graduated as a social worker just over 20 years ago and I have worked in various different contexts. I started HOST International about six years ago.

We work with refugees and migrant communities now and I set up an organization that works across Asia Pacific. We quite deliberately from early on set up to provide support across multiple countries. A lot of my motivation for this came from wanting to have some independent assurance that what we were doing was worthwhile, and what was needed. It made me very interested in how we collect data, and how we use data.

2. How does that experience shape the approach of HOST International in the measurement of outcomes and ‘change’ for clients and communities?

I’ve worked in many different NGOs from very small community-based organizations through to large charitable organizations, as well as government organizations. I guess through that experience, I’ve seen lots of different ways of collecting data. But unfortunately, the common story I’ve been exposed to is that data is collected according to the contract, or for demographic metrics, but not necessarily collected towards knowing whether outcomes are being achieved or whether the quality of work being done is meeting the objectives of the work.

On the other side of it, over my time I have supervised a lot of direct service workers, for example, social workers, residential workers and now in settlement worker situations. I know a lot of workers struggle with data and increasingly the narrative is that we’re spending more and more time collecting and putting data into systems. From an operational perspective you want your people to be spending as much time with the people we’re trying to help. So getting the right balance – between the data we collect, the purpose of collecting it, and the time it takes – has become important. Not only to know what we’re doing, but also to know the efficiency of how we do our work.

We wanted to try and do data differently, so rather than just starting data collection based on the activities that we’re doing or the contracts that were funded for, we wanted to actually turn that upside down and start by deciding how to collect data based on the kind of difference, or kind of change we wanted to make. So we spent a lot of time early on, trying to think about what were the kind of things we wanted to change, or make a difference to. And that very much involved thinking about our purposes as an organization and the kind of things we wanted to be involved in working with, then working out how we would measure that. What we did was we looked at the kind of contract work we wanted to do and looked at how that would fit into that impact framework.

Consequently we built our CSnet data system around a set of outcomes. Not just a set of tasks … we needed a system that was able to measure change over time and not just measure volume of work – so that was an important factor. We also needed a system that could reconcile both those components. Because you need to record the number of people in the activities you do because funders want that. But you also need to be able to work out ‘Has anything changed?’ and so we were looking for a system that could help us to do both, but also not be too difficult to use.

3. How does a focus on settlement outcomes drive the types of services HOST International delivers, the data collected to understand change, and inform practice from what you learn?

… we ended up determining that what we wanted to be was an organization that would create inclusive communities, and help people on the move, mainly refugees, to get better life outcomes.

We believe [through our theory of change] that is going to happen by people developing their individual coping and capability to self manage and being independent. It also involves working with communities to be more inclusive and building the connections, and it also requires us to work at a systemic level to address disadvantage.

So consequently we ended up aiming to deliver quite different services across the different networks. Obviously, there’s big differences between refugees in South East Asia versus refugees in Australia.

But the common thread was that we wanted to kind of foster individual coping and community inclusion and try to address the systems that got in the way of that and do research and so on … So we needed a system that could measure individual work, but also measure group or community work. That was the other complexity to what we were doing – we ended up developing measures that could be applied individually, but also measures that could be applied at a more macro level and then at the systemic level. We also needed to be able to track project based work that would allow us to document work that we’ve done over time and then we wanted to connect it all together somehow because ultimately a good result for us was that we achieved change across all those levels.

We were really keen to engage with our workers, you know, actually thinking about the effectiveness of their practice and adjusting their practice as they went, according to whether they were getting the results that we wanted.

One of the key drivers for using CSnet was that you had that ability to show workers in real time what was happening – as long as we embedded regular assessments to check progress that data is available at that individual level and that macro level.

But we’ve also discovered this is really complex and we’ve been going for nearly six years. We’re still only now feeling like we’ve got it bedded down and we know what we want to achieve, what kind of outcomes we want to achieve.

But the data is already there, even though it may not be doing everything we want it to, it is already informing that question of whether we’re doing good work or not, and it’s helping us to actually improve and adjust our work based on feedback that we’re getting from that process.