Evaluation (noun)
assessment, appraisal, judgment, gauging, rating, estimation, consideration; analysis, examination, checkup, workup, test, review.
When MEDA worked with San Francisco city agencies and a cadre of community nonprofit service providers to obtain a $30 million federal grant to start the Mission Promise Neighborhood, based on the successful Harlem’s Children Zone in New York, it was clear that an expert evaluation team was needed. This was partly due to federal mandates for collecting data about the impact of the initiative on the participants in the four schools of MPN, plus the surrounding Mission District community. The other need came about because the 18 of the 26 MPN partners that serve clients do not have the data and evaluation capacity to compile data. The funding these agencies received was earmarked for providing services, not for collecting data on the impact of their programs.
So, the MEDA Evaluation Team was tasked with offering capacity building to all partners, thereby ensuring optimized services for clients.
The Need
To ascertain the partners’ ability to meet data requirements, MEDA conducted a data-capacity assessment.
Of the MPN partners that serve children, youths and families (18),
- Four out of 18 track outgoing or incoming referral information.
- All collect outputs, but fewer collect data that depicts quality of effort (16 of 18) or quality of effect (15 of 18).
All 26 partners collect important information about their programs, from activities to participant demographics; however, the main barriers to fully utilizing their data to demonstrate impact were low staff capacity, dearth of in-house evaluation expertise and lack of useful data-management systems.
Explains MEDA Evaluator Elisa Baeza, “Our data capacity assessment was motivated by the need to ensure the sustainability of our partner agencies that do amazing work serving the Mission District community.”
The goal? To lift up partner agencies by building up their capacity to collect and utilize data to analyze, learn from and enhance their programs and services, with the hope that through such technical support, partners will adopt a common practice of strategically using data to make informed decisions about how their services are being implemented.
Continues Baeza, “It doesn’t mean doing more work to get there; it simply means doing the work differently so that organizations may flourish and participants benefit.”
Partner Case
Parents for Public Schools-San Francisco (PPS-SF) needed a streamlined process for collecting demographic information about their participants, which in turn limited their capacity to tailor services to their client base. MEDA’s evaluation team helped PPS-SF create an intake form that would capture essential demographic information on their clients so that they can better understand who is accessing their services, offer more targeted services and enhance their ability to offer funders the pertinent client data they require.
Masharika Maddison, Executive Director, explains her organization’s need: “At PPS-SF, we’re committed to supporting student success through our parent-facing programs and services. Informed parents are better equipped to support their student learners through their academic journey. Meaningful intake forms are a critical first step in ensuring we have the most valuable data possible to align our resources to our parent program participants.”
Dr. Monica Lopez, Associate Director of Evaluation, further explains: “An agency’s ability to communicate the work that they do and the impact that this work is having on the populations they serve in numbers and figures – or data – is extremely important. Being able to utilize data to effectively communicate the value of a program to diverse stakeholders and funders can influence an agency’s ability to remain sustainable. Data utilized effectively can also help improve the quality of programs and services. It is a win-win for everybody. For agencies to be effective advocates for their programs and services, they need data . . . and collecting the right set of data is a task that requires both resources and know-how – this is what we refer to as Data Capacity and this is what we here at the Mission Promise Neighborhood are trying to build for all of our partners.”
Recognition in the field
The team of Lopez and Baeza are working on identifying avenues for sharing their data capacity work with other Promise Neighborhoods. The pair were recently brainstorming at the Promise Neighborhoods National Network Conference, held a couple of weeks ago in Arlington, VA.
There are now discussions about a Webinar, in collaboration with PolicyLink. The date is to be determined.
Another honor bestowed on Lopez and Baeza is their being asked to present this November 15-19 at the American Public Health Association (APHA) National Conference in New Orleans. Getting an abstract accepted is a difficult process, making this honor all the greater.
This year’s conference focuses on healthography, the study of how where you live affects your health. Lopez will be presenting on the impact of the Affordable Care Act on patient enrollment at the Mission Neighborhood Health Center, a project that she and MEDA’s Amelia M. Martinez Cantos are trying to get off the ground; this project endeavors to take into account contextual factors within the changing Mission community.
Baeza will be the lead presenter, with Lopez as backup, on the data capacity work MEDA has been conducting with partners as it contributes to building systems that can track health outcomes for MPN clients.
“It is an honor for Elisa and I to be able to share our work, which is always evolving and being refined. Our evaluation team is dedicated to putting forth the best data available to help MEDA and its partners best serve clients in the Mission,” concludes Lopez.
Presentation links: