Case Study: Taking Assessment from Paper to Pioneering
Like many small institutions, Lesley University – located in Cambridge, Massachusetts – had limited resources available for assessment activities, both in terms of budget and personnel. Two years ago, when administrators at Lesley University needed to do an assessment of general education outcomes or collect feedback on students’ internship experiences, they used paper and pencil and manually collected and collated responses from multiple locations on campus. Now, after bringing both those assessment practices into a centralized online solution using the Campus Labs® assessment tool, Lesley University has seen a significant improvement in its data analysis capabilities. Lesley University is living proof that even a small college can benefit from and apply Campus Labs’ cutting-edge assessment solution.
“The Campus Labs tool has helped us track how we’re providing students with opportunities to achieve. Now we have more concrete evidence.”
Assessing General Education Learning Outcomes
When it was accepted into the Engaging Evidence Project of the Council of Independent Colleges (CIC), Lesley University decided to take the opportunity to obtain some tangible results related to student learning, with the help of Campus Labs. Lesley University decided to choose a general education learning outcome and develop a strategy to assess it. “It was another way to look at integrated assessment outcomes,” said Linda Pursley, Director of Assessment and Institutional Research. The Integrated Assessment Team (IAT), composed of faculty representing all divisions in the College of Liberal Arts and Sciences (CLAS), one of the two undergraduate colleges at Lesley University, selected the Critical Reasoning Outcome and slightly modified one of the American Association of Colleges and Universities (AAC&U) VALUE rubrics.
The IAT asked CLAS faculty to submit work they thought provided evidence of critical reasoning, and promptly ran into a problem. “Immediately, we saw that we couldn’t assess using this rubric, because it is weighted toward research papers,” Pursley said. “One of the lessons we learned was to get faculty consensus on what defines critical reasoning.” In addition, it became apparent that they needed to be more specific about the types of artifacts they were pulling. With the help of the expertise provided by their Campus Labs consultant, they were able to fine-tune the process–modifying the rubric to suit their needs, communicating and learning along the way.
Once the rubric had been appropriately adjusted and all 160 pieces of 1000- and 2000-level work had been submitted and screened, the designated three faculty members were ready to begin scoring. Rating was set to take place over the summer, but because Lesley University’s faculty is off campus for those months, the process could have proved problematic. However, the Campus Labs system allows users to access the rubric remotely via an online link, so faculty raters didn’t have to be on campus to participate. “They were able to work wherever they were, whenever they had time,” Pursley said. One Lesley University faculty member even worked from Italy. That access was key to the success of the initiative, as was the fact that it was on a centralized electronic platform. “If we had had separate people rating and entering data into spreadsheets, we would have had to compile it all and it would have been much more manual,” Pursley said. “This made it really easy to look at data in a lot of different ways.”
Evaluating the Impact of Experiential Education
At Lesley University, all students fulfill general education outcomes not only through coursework, but also through experiential learning. Both CLAS and Lesley’s other liberal arts college, the College of Art and Design (CAD), require internships for all students, though only the CLAS requires an accompanying seminar. “Internships are fundamental to the undergraduate student experience because we train a large number of teachers and counselors,” said Pursley. “It’s very important–it’s sort of part of what we do.”
Though the value of internships in Lesley University’s degree programs was assumed, proving the connection was a more difficult matter. “People often said internships are important for general education outcomes, but we didn’t have a way of assessing that,” Pursley said.
Lesley administrators needed to know two things about their students’ internships–feedback on the internship itself, including site and supervision; and how effective internships in general were in achieving three outcomes identified by Lesley’s CIC team–social responsibility, multiple perspectives and lifelong learning. Using the Campus Labs survey tool, Lesley University developed an internship survey for both CLAS and CAD students that fulfilled both those goals by including questions related to general education as well as specifics about the internships themselves.
In the past, Lesley University had made efforts to aggregate and review the results of its paper surveys, but using Campus Labs enabled the internship office staff to slice and dice the data in myriad ways to increase their understanding of and insight into the data. For example, they were able to filter students by year, sort them by various seminars, and cross-reference responses to different questions. Armed with these enhanced data-analysis capabilities, internship office staff were able not only to distribute reports to faculty seminar leaders, but to identify concrete action items to improve both the internship program and the assessment process itself.
Among their findings was evidence that internship seminars provided a valuable way for students to share and process their experiences, and that those seminars were even more effective when they were smaller, with fewer than 10 students. As a result, Lesley may rework its seminar scheduling to allow for more small classes, and may also consider offering seminars at CAD. The data also showed that the quality of site supervision, the nature of the internship and the tasks it required were directly related to the degree to which students felt they were able to effectively meet general education outcomes. As a result, the internship office is planning to implement direct measures of assessment, and possibly ask site leaders to use rubrics for direct observation of general education outcomes.
According to Pursley, the Campus Labs tool has given Lesley University the insight it needs to further strengthen its internship program, and has transformed an assumed value into one that is extensively supported with data. “It has helped us track how we’re providing students with opportunities to achieve,” Pursley said. “Now we have more concrete evidence.”