Dr. Jillian Martin, PhD, often states that every number has a story, and every story has a number. A component of her role as Assistant Director for Strategy and Evaluation is collecting and interpreting the stories and numbers produced at the Gephardt Institute. Jillian leads the evaluation, assessment, strategic development, and organizational learning activities for the institute. Together, she and the Master’s Research Fellows whom she supervises comprise the GEAR team, which stands for Gephardt Evaluation, Assessment, and Research. Jillian describes the team as the “brain and connective tissue of the institute,” the gatherers and organizers of information, knowledge, and data related to the institute’s programs.
This semester’s Master’s Research Fellows—Shelby Banach, Aaron Pevitz, Emma Nordmeyer, and Taylor Brown—are all Master of Social Work students in the Brown School. Shelby, currently in her third semester as a fellow, explains her perspective on the team’s contributions to the institute: “The GEAR team plays an important role in broadening civic engagement. We take the data from all the awesome work done at the Gephardt Institute and analyze those responses to see what is working well and also possible areas of improvement to create the most positive impact within the WashU community.” Each fellow has an area of focus where they concentrate the bulk of their research efforts.
The work of the GEAR team has two phases. The first is assessment, during which the team determines if a program was effective in meeting intended goals and to what degree. The second phase is evaluation, when the team studies the data to determine how and why the program did or did not meet the outcomes and what changes are needed in the future. In other words, assessment asks if the goals were met, while evaluation explores the how and the why.
The most common way that the WashU community encounters the GEAR team is through feedback forms, surveys, and questionnaires before and/or after events and programs. Jillian emphasizes the importance of this sort of feedback, encouraging people to view these tools as “one huge part of the work of assessment and evaluation.”
Jillian and the fellows do the bulk of their work behind-the-scenes, producing logic models and outcomes in partnership with institute staff for each program or initiative. These models and outcomes define what the institute hopes participants will learn or gain. Jillian explains that the information gathered through surveys, questionnaires, interviews, and focus groups helps to close the gap between the outcomes the GEAR team has articulated and the actual results of a program: “Feedback forms help us to determine what people are actually getting out of the experience.”
The forms, surveys, and interviews used for this sort of data-gathering are carefully designed by the GEAR team to capture participant experiences. For many programs, a variety of tools measure growth and change over time. Jillian pointed out the example of the measurements utilized in the Civic Scholars Program, a two-year, intensive, cohort-based experience. For Civic Scholars, Jillian has designed a comprehensive model for assessment and evaluation, and she applies an additional piece of the model to the program each year. For many initiatives, the GEAR team uses pre- and post-experience tests as a comparative measure of the effectiveness of the intervention. The evaluation tools also collect both qualitative data (stories) and quantitative data (numbers). Jillian prioritizes the intersection of the two, attempting to gather both and explain their relationship to one another.
Jillian sees each of the institute’s programs as “interventions to help something work better.” The GEAR team determines how each intervention works and ensures they are producing the desired outcomes. This is a process of trial and error, shaped by any number of factors that are difficult to control. Jillian terms this work “squishiness.” “Imagine a stress ball,” she says. “When you squeeze it, its shape changes. That’s how our work is. It changes in nature as the dynamics surrounding it change.”
The unchanging guideposts for this work, however, are the institute’s values. Ultimately, the outcomes for each program specifically focus on values rather than on skills gained. “We want to determine the correlations between what students are learning and their likelihood of becoming lifelong engaged citizens,” Jillian explains.
In addition to evaluation and assessment, the third responsibility of the GEAR team is research. Jillian and the fellows constantly survey the fields of higher education, student affairs, and nonprofit work. The GEAR team can often be found working on literature reviews, reading and compiling current research and benchmarking with other institutions on topics related to the institute’s work. Jillian noted the importance of also learning from non-academic sources about what successful implementation looks like in the real world. She says, “We want to see how civic health is being talked about academically, as well as how it’s being put into practice.”
One of Jillian’s favorite projects was last year’s Community Engagement Taskforce, through which the institute conducted a self-study of community engagement at WashU using the Carnegie Classification for Community Engagement. One aim was to enter into a conversation with the broader campus community about community engagement. Projects like this one help the GEAR team and the broader institute to better contextualize civic and community engagement at WashU.
Evaluation, assessment, and research are crucial to the success of all Gephardt Institute programs, acting as guides for how the work of civic engagement can be done most effectively. The two questions that Jillian sees as the heart of this work, informed by Illumilab, a nonprofit evaluation think tank in St. Louis, are, “How do we think change happens, and what change can come from our programs’ outcomes?” The GEAR team’s job is to build a structure within which change can occur.