Most organizations define their vision and focus through a strategic plan, but not all our organizations are good about evaluating the impact being made. Maybe we measure the number of people served or services delivered, but how do we track the overall difference we are making in our communities?

Today’s guest is passionate about building fundraising and evaluation capacity with social organizations. Amanda Wallander Roberts has helped over 60 social service organizations fundraise and evaluate programs, including raising over $23 million and developing more than 50 logic models, evaluation plans, and process maps.

During this episode, we discuss:

• How we can track the difference we are making
• Why evaluation plans are important
• What a logic model is and how it’s the first step to measuring our impact

Enjoyed the episode and want to learn more? Visit Amanda’s resource page for her logic model template.

Episode Highlights

• Welcome (1:30)
• The Nonprofit Community (3:35)
• How Do We Track Impact? (4:20)
• What is a Logic Model? (7:30)
• The Five Parts of a Logic Model (10:23)
• Logic Models in Action (14:00)
• Logic Model vs Strategic Plan (16:26)
• How to Get Started (17:18 )
• The Why (24:37)
• Wrap Up (27:40)

The interview transcript is below for reference.

Guest Biography

Amanda Wallander Roberts, MSSW
Founder and Principal Consultant at Capacity Building Consulting

Amanda Wallander Roberts, MSSW, is a fundraising and evaluation coach and consultant passionate about building organizations’ capacity to fulfill their mission.

Using her 10+ years of experience in programs serving children and youth, Amanda helps third sector organizations identify what they need to get out of “emergency mode” and build the skills and create the tools needed to successfully achieve their goals.

She has raised more than $23 million and coached nonprofit leaders and staff in effective planning, implementation, and follow-up for their fundraising and evaluation initiatives. As part of her mission, Amanda provides free fundraising and evaluation resources at capacitybuildingconsulting.com/resources, as well as trainings and online courses.

 

Contact

LinkedIn | Amanda Wallander Roberts
Twitter | @Build_Capacity
Facebook | @CapacityBuildingConsulting

 

Interview Transcript

Emily
Hi Amanda, welcome to the show.

Amanda
Thank you so much for having me. I’m so happy to be here today.

Emily
I am so excited to have you. This is a topic I’m really interested in. I’ve seen contextual models before, but the logic model and how you present it will be so valuable to our audience. So thanks again for being here. I’d love for you to start though with who you are and what you do.

Amanda
I am Amanda Welner Roberts and I am a social worker and so my background is working as a case manager and programs and through that experience I found I was working in organizations that were really underfunded, understaffed, and we didn’t necessarily evaluate our program is making a difference.

So as a case manager I was like, Oh, I hope that this is working for the people. I’m trying to help, but I don’t know for sure. And so I went and got my master’s degree in social work to really focus on building up nonprofits so that they have enough funding and so that they evaluate their programs to really know that they’re making a difference in the world. I’m a nonprofit consultant providing fundraising and evaluation coaching and consulting and I really, really hope to help organizations build up enough capacity so that they can internally fundraise enough and make sure their programs are working or make tweaks or whatever needs to be done.

Emily
I love that mission. And I liked that you touched on hoping that things are helpful for these organizations. When I was in nonprofit and we talked about evaluation, our strategy was that hope is not a strategy because sometimes when we’re in these personal or social organizations and working with them, we just hope things go well, but we really need to assess. We need to evaluate and know how we can make improvements because that’s the only way we will maximize the impact.

Amanda
Absolutely. Well, and I hear so often, I feel like we’re making such a great impact in and my response is always like, okay, well let’s prove it. That’s different than being able to prove it.

Emily
Yes! So from social work to nonprofit, what has been being part of the nonprofit community meant to you?

Amanda
Well, it’s really always been about helping people and so kind of my life mission is really to help the people that help people so that they can help people better. And so that’s really what I’ve seen in so many nonprofits and even nonprofits that don’t necessarily have a human services focused. I’ve worked with other environmental nonprofits where people are always still in mind. So to me working in the nonprofit space really just means like bringing us all up, help, helping us all really live better and create a better world for the people around this.

Emily
Yeah. That positive impact is so important. Absolutely. So in terms of the difference that we’re making, how do we track that? Where should people start?

Amanda
Well, there are really two types of evaluation that I strongly recommend every organization engaged in and the first is process. Really where you start is with process evaluation. This answers the question, are we implementing the programs as planned? So you can understand, okay, across our different sites is the program implemented consistently? Did we end up reaching the number of people we were seeking to serve? If not, why not? It really answers those questions, but you really have to start first with are we doing what we said we were doing or what we were trying to do? Sure. And then moving into, okay, is our program making a difference? So after your program is being implemented properly, you can say, is that properly implemented program making a difference?

Outcomes evaluation is the second type of evaluation and outcomes really says what’s the change in knowledge, behavior skills, belief? What’s the change we’re making with those we’re trying to spur?

And then there is a third type of evaluation that people like to like to jump to. It’s actually very challenging and it’s impact evaluation which really says what’s the community level or longterm difference we’re making. Often most organizations try to jump to that, that long term impact. It’s so popular, especially right now in the nonprofit world, but you have to really say first process evaluation, are we implementing programs correctly? Second outcomes evaluation, what’s the difference our programs are making in the short term for the people we’re serving? And then you can move to that impact. What’s the big picture, community level difference or what’s the longterm change for making?

Emily
I love those three processes and I think too, we get so focused on the qualitative aspect of evaluation. So yes, everyone had fun at our event or yes, every the people we want them to serve were served, but sometimes we don’t include that quantitative, that data in our impact as well. So I love that you have these two but three processes that you’ve laid out. The process evaluation is also something that I’m very passionate about because I think we get into this. We have this idea, we talk about how we’re going to do it, there’s a couple of iterations that happen with the idea and also we’re doing similar to what we had talked about, but it is a little bit different and then evaluating that so

Amanda
Well and with a, with a process evaluation, that’s what a lot of organizations do naturally. And then they kind of stopped there. So answering questions like what’s the number of people we’re serving, how many hours of service are we providing those kinds of easier to track and measure. But if you, if you just stopped there just because you’re providing services, does that mean they’re making a difference necessarily? So you really do have to move to that next outcome step of what’s the change or services or making.

Emily
So you’ve talked a little bit about logic models. I am very interested and maybe we provide the audience first with a definition of what a logic model is and how we use that in these three steps.

Amanda
Absolutely. So a logic model really just connects to your program activities to the difference you’re to make and so it’s this, this kind of five column thing where you connect, what resources do we need to do your program, what activities are going to be a part of our program, and then what differences are program gonna make and a logic model will actually outline, three of those columns are outputs, which is what you measure for process evaluation outcomes, which is what you measured your for an outcome evaluation and then impacts which if you’re able to move to an impact evaluation, that’s what you would be measuring. So it lays out very clearly this is exactly what we’re doing and this is the why. This is the change we’re trying to make with that. And it lays that foundation to then evaluate it. And it’s really interesting because when developing a logic model, sometimes organizations will find like, oh, we’re doing these other activities, but we don’t know why or we don’t know what difference they’re trying to make or this main difference. We thought we were trying to make. Some of our activities don’t align and so it’s a way to plan for your programming but also plan for that evaluation

Emily
It seems too, when we have nonprofits that are looking to implement a new process or program, we get together a work plan or a project plan, but those plans don’t always include the why’s. They don’t always include the measurements, they include these actionable steps, but this is taking it a step further to make sure we’re encompassing all the reasons and like you said, aligning it with what we’re already doing.

Amanda
Absolutely. Well and, and that’s what nonprofits are here for. They have a mission, I have a vision. They have this idea of how they want to change the world and it’s just about lining up your activities, your program and saying, okay, how are we going to change the world? And there’s also this column inputs or it’s sometimes called resources and that’s really helpful to see, okay, what do we need in order to really do this? Well, and sometimes when actually stopping to think about that, we’re like, oh, we need a bunch of more staff, or oh, we need more funding, or oh, we need this system to help us track information that can. That can be a real eyeopening experience from just stopping to ask if we were going to do an amazing job at this. What would we leave? Yeah.

Emily
I think sometimes we’re afraid to dream that big. Right? We have an idea, okay. When we brainstorm, we’re limited by our resources, we’re limited by our own preconceived notions or bias and how far the organization could really go. So this also facilitates a lot of innovation. Dreaming, making sure resources are aligned, but I love that you keep bringing it back to that why.

So what are our steps in this model?

Amanda
So the five steps just to reiterate them are the inputs, activities, outputs, outcomes, and impacts.

Go ahead and picture like columns and you number them – activity number one creates output number one and outcome number one. And it kind of lines up, but sometimes people think of it as arrows, so if we get these inputs, if we get these resources will be able to do these activities. If we do these activities, will you be able to see these evidence evidences that the act activities were completed, those outfits, and then if, if the activities are completed and they’re completed correctly, then we believe we’ll see this difference in outcomes and if we continue to create these outcomes in the community, eventually these impacts will be realized on a community level or in 10 years for an individual. And so it’s really that each category builds off each other to start it with the impacts and work backward and ask, how do we get to our dream? What do we need to get to our June, what do we need to do? Then looking forward, starting with the resources, you say, if we get all this stuff, this is the amazing thing will be.

And it helps beyond just evaluation. It really helps to create that fundraising language about if an organization is providing this amount of funding or this grant, then this is the dream. We’re able to paint a picture for them of how their money or how their partnership or sponsorship is going to help us make this amazing community difference. And so I like to reuse things as much as possible. So I use logic models for creating that branding and fundraising language planning the program and evaluating.

Emily
So all these steps can be used in all aspects.

In terms of the process then, you hit on something that I’m curious about. Do you have folks that you work with start with this model when they have maybe some money that’s been set aside and they’re dreaming up a project or do you do it the other way where maybe they don’t have the funds yet, but we want them to dream and then see what they need to get there? Or do you do both?

Amanda
Honestly, both. I say wherever you are, if you don’t have a logic model yet, now is the perfect time to create a logic model and there are a couple of different ways that I encourage people to create a logic model. If they’re planning their program for the first time, I really doing work backward, start with the impacts and the outcomes. What are you trying to do? And they moved to the activities and say, okay, here’s how we’re going to get there. But if you already have a program that’s been established for a long time, then start by just listing out all of your activities and seeing, okay, what outcomes and impacts do these activities work towards? And that’s where you can kind of tweak and say, Oh, here’s this extra activity that maybe we don’t need anymore. Or Oh, here’s this activity that we need to put more on or resources towards. So wherever you are in the process, if you’re new, if you’ve been around for 30 years, a logic model can be helpful and can be really useful for that. Just alignment, realigning your program or aligning your program for the first time.

Emily
It seems too like a clarity type thing. So if you are an organization that’s maybe making a change in branding or you think you’re making an impact in a certain way, just having a clarity of your message and where you’re headed I think is really valuable. And using this model can do that.

Excellent. So Amanda, what type of case study or where have you seen this in action? It would be great to have an example for our listeners.

Amanda
A few examples come to mind and one for a small organization and one for a large organization. So, I’ve worked with this small organization for several years and when we originally created their logic model, they had not planned a lot of their programming yet. And so they were an amazing, just like inspiring organization with so much energy. And so we really kind of took that energy and said, okay, what does this look like as far as programming goes? And so we made sections of their logic model for different programs and it helped to see, oh, okay, if this is really our goal and maybe this activity doesn’t correct, connect directly enough and we need to rethink it or if this activity is really important to us, then let’s make sure we know exactly what outcome it is.

And so we had these conversations. Then since that initial logic model development, we’ve been able to revisit it together each year and see, okay, what did we meet, what do we end up not doing, what do we need to tweak? Because these are living documents, this is going to change and evolve as your program changes and involved. And so for this organization, it, it also helps them identify the staffing needs that they had. And so now they’ve been able to hire different staff positions to really support their programming. Whereas before it was like, well, we need more staff but we don’t know exactly what to do by outlining it and it makes it really, really clear. And then there was another organization that I, that I worked with on a, on a partner project it, it had so many different departments and their main thing was we, we don’t communicate to each other, they’re doing their program, we’re doing our program. And so by developing logic models we were able to see, Oh, you’re program certainly meet this outcome. So is this other program, how can y’all share information, share data, how can you all work together towards those outcomes in a way that connects different departments within the organization. But it also encourages that culture of evaluation, that culture of sharing and saying, Hey, we would love your data and you can have our data because we’re both working towards similar goals.

Emily
Absolutely. And reducing redundancy, those community partnerships really can make a big impact too. So looking at their logic models, how does the logic model differentiate from a strategic plan or three year action plan that an organization might have?

Amanda
So the strategic plan really says, okay, here’s where we’re trying to go. I view the logic model more of how are we going to get there, but it’s very program and organization specific. So your strategic plan, whatever goals you have there, should be reflected in your logic model, but your logic model is going to be there when that three years is up. Your logic model is going to be there for a lot longer. And so it’s more of an implementation plan almost is how you describe it to say, okay, this is exactly what we’re going to do, but it’s kind of similar from year to year. It doesn’t change dramatically unless your whole organization changes based on that strategic plan.

Emily
Sure. So for an organization or someone who’s listening who wants to implement this type of thing, where would you tell them to start? How should they move forward?

Amanda
So they’re really amazing resources online that I encourage people to have read. The first is the Kellogg Logic Model Development Guide. You can just Google that. It’s a very extensive, very thorough document on logic models and so I always say like, try not to get overwhelmed. It really walks through different types of logic models to help you in different stages. If you are looking for something a little bit easier to digest, then I have a logic model blog posts template you can download on my website at capacitybuilding.com/resources. It really walks you through if you’re a new program, here’s exactly the steps you take to make it and if you’re an existing program, here’s the step side recommend. But in, in my mind, this kind of development of a logic model is a team effort. Even if you’re a one person organization and getting together with the volunteers you have or board members, you have to dream this up together.

And I love using sticky notes or whiteboards or whatever is going to get it on the wall visually for everyone to see. You could do it as a facilitated session or facilitate the session yourself to really say, okay, what do we think about each category? How do these all connect? How do these align? What kind of language are we using? Are we saying youth? Are we saying students? It’s all of those questions that you can answer in a group and then type it up into the template and pass it around to see, okay, does everyone agree? Is this what we’re going for?

Emily
The visual for me is what is most exciting because when we receive reports at the board level, sometimes it’s easy to glaze over them, but this space seems really interactive. It seems like folks can really dream big. It inspires innovation and those are the types of things we need at that board level. Occasionally we’ll find that folks dig into the weeds and this allows them to do that in a way though that’s very productive, right? Because they’re having to think through the staffing and having to think through things that we don’t always want at that level. We want the board to be a little bit more elevated, but it does lend itself to fulfilling that for some folks, but also allowing that innovation and again, the dreaming big.

Amanda
Yeah, well, and I feel like if you do use sticky notes or whiteboard or something, that always helps with that creativity piece because creating a logic model really is outside of your day to day work. So, being able to step away for a second and just get it into a different mind space can really help you say, this is actually what we need. Or this is why we’re doing this or make those kinds of breakthroughs.

Emily
The holistic approach is so important because then everyone is on the same page. Everyone knows where we’re going, how we’re going to get there and can support that.

Amanda
Well and if your staff is pretty big, then you could do this with your staff and then present it to the board too. It just depends on the size of your organization, where you’re at, and how much support you have.

Emily
I’m curious, how did you get into logic models? What drove you to use this in your business and in life?

Amanda
So in my social work program as an undergraduate student, we had a program evaluation class and we had to take a take a program in the community, a real program and create a logic model for them and it just clicked for me. That’s very much how my brain thinks and works. And I was just like, wow, if, if I had this for the nonprofit I was working for at the time, then I would know what we’re trying to do, what difference we’re trying to make. But as it was, I had no idea what our goals were. I had no foundation to start evaluating that. So I just found it was a really useful tool. I’m also just passionate about making sure everything we do is in line with our goals or what we’re aiming for. So I’ve even created a logic model for my business and for different parts of my personal life just because I found it’s such a useful tool to think, how am I going to get there? What am I trying to do? What do I need in order to do that well?

Emily
For the systematic approach, the process person in me, I’ve really enjoyed learning about this. I think it’s so valuable not only for the organizations, but I do think I’ll use it in my personal life too. I’ve seen contextual models and those are always interesting to me. I had a Neen James on the show before and so she talked a little bit about how the visual helps people communicate. I think that that’s so true as well, but this is just in a different way and at a different level.

Excellent. So in terms of now that we have this evaluation process, what tips would you give folks to evaluate if this logic model isn’t working for them? How to make sure that they’re successful?

Amanda
Yeah. So I take the logic model and copy and paste the foundation of my evaluation plans and I typically developed two different evaluation plans. This is similar to what we were talking about before – the process evaluation plan and the outcomes evaluation plan. So I will literally take the logic model outputs column, copy and paste it into another document and ask, okay, how are we going to measure each of these things and what does success look like? What numbers are we reaching for? What’s our goal target there? What method are we going to use to evaluate? Is this something that we’re tracking through surveys or a check-in sheet or a spreadsheet? What are those methods we’re going to use and then who is going to evaluate them and how often? And that’s sometimes the piece that unfortunately we forget.

And so we say we need to make sure we check in every February to see if we got this or we need to do this monthly and this is the person that’s going to be responsible. Then same thing with the outcomes column, I will literally copy and paste the outcome column into another spreadsheet or another document and say, okay, how are we going to evaluate each of these? And it can be a bit overwhelming at first if this is something an organization hasn’t done, but I highly recommend and encourage and use myself, Asterix and put it in a little footnote at the bottom line that says this is an area to grow. We will think about this next year. You can’t do everything at once. And I really encouraged people to not do everything at once. So start where you’re at and create that logic model. Start thinking about how you’re going to evaluate those outputs and outcomes and then build on that from year to year. Because you don’t have to dive in and get overwhelmed with it. And if you, if you do accidentally do that, if you are so jazzed up and excited about it, that you just try and do everything, then it might create some difficulties for your staff or your board where they are much less likely to want to evaluate in the future. You got to focus on those wins.

Emily
Build on your wins, yes. So start with what you have now and as you’re moving forward, feel free to dream but keep things realistic to start, so we’re hitting some of those goals.

Amanda
Absolutely

Emily
You’ve hit on a lot of the reasons why logic models are so important throughout our conversation, but will you please share your input on why they’re so important.

Amanda
So a logic model really clearly outlined and defined for your staff and your board. What are we trying to do and in our organization? It goes beyond your mission and your vision which are so critical and says what are we actually going to do about it? And a creates this unity, this cohesion around, this is what we’re going for, this is what we’re focusing on. It helps create this filter of, oh, there’s this other potential project that we could do, or there’s this partnership that sounds really cool. And it helps you say, okay, let’s look back at our logic model. Would that be in line with our current goals? Would that be in line with what we’re trying to do? If so, let’s add it. If not, let’s let’s skip it.

It creates that direction and it’s really like a roadmap of here’s where we’re going. And logic models are so important too because more and more funders, especially grantors, are requiring that evaluation at least perspective if not the actual results. And it helps you talk in that language and put those pieces together to see, okay, what is this for? Why are we doing this particular activity? What are the outcomes there? And then what can we end up sharing back with that funder or that grantor? So, I mean, I kind of want to say everything!

It’s really things like getting the staff on the same page, creating that unified language that you can use in marketing and fundraising, and then also serving as that, like what are we trying to evaluate so that we know whether or not we’re making the difference we’re trying to make.

Emily
Associations and nonprofits can be messy. Things can feel disorganized, it can feel like people are going in all sorts of different directions. And so I think the importance of this type of model is that people get clarity. Everyone’s on the same page, they’re moving in the same direction and you’re finally measuring that impact beyond the numbers that we like to track. It’s really seeing how we’re impacting people, reminding ourselves of what the why is, and we’re maximizing all of our efforts.

Amanda
And that I think in and of itself, is going to help reduce burnout from what I’ve seen from colleagues and people working in the nonprofit space who just feel inundated with tasks lists and what are we supposed to be doing and I’ve got too much on my plate. It really does help say, Hey, I know that you’ve been doing a lot and here’s the difference that it’s actually making you are making a difference in the world. Or if you’re not, then you can tweak it to decide whether or not it’s working rather than feeling like a hamster on a wheel.

Emily
Always turning back to the how and the why. Well, before we all go back to our desks and get our pen and paper and start on our own logic models, I do have just a couple last questions.

The first being, if you were to look back at when you first started in this industry, what advice would you give yourself?

Amanda
I would say be patient. I came into the industry with a lot of energy and excitement and thinking things were going to move at a million miles an hour and just thinking we can get it all done and get it all done now. And I realized that approaches not beneficial and not necessarily sustainable. So now I’m all about the slow and steady growth and change because that’s really where you will see the results without burning the candle at both ends or without putting too much pressure on yourself or your organization to do everything all at once. So slow and steady change. That’s what I would say.

Emily
I feel that one. You’re a girl after my own heart. As doers, we want to get things done. We see the potential for these organizations, but you’re right, it does take stepping back, slowing down, to really get where we want to go with these organizations.

And then last question, what is the smallest action someone could take to make the biggest impact?

Amanda
This is going to sound biased, but definitely just download or look at a logic model template. It might look overwhelming, but just look at it and your brain will start automatically thinking in that way a little bit. You’ll see this is how everything connects on the logic model and it can help your brain say, how does our program connected in those ways? So if all you do is download a template and look at the format, that’s what I would say to start with because it can be overwhelming.  It will help your brains brain start thinking in that way.

Emily
Yeah. And if your brain is systematic anyways, seeing the visual will really help you understand. And if it’s not, you’ll feel it organized because you’ve seen what it can be and how your programs can look. That’s great advice. Excellent. Well, Amanda, I’m so happy to know you. I’m so thrilled you were able to share some of your knowledge with our audience today. Logic models, I’m sold. I see the value. I see how it fits in supplementally with the things that we’re doing otherwise, like our strategic plans. We really do need to know how we’re going to get there, what we’re going to do, remember our why and see what that impact is going to be. So thanks for sharing your knowledge with us.

Amanda
Yes. Thank you so much for having me. I think this is a great opportunity that you’re providing for organizations and we need more accessible information like this, so I really appreciate what you’re doing here.

Emily
My pleasure. Talk to you soon.