Using Sequential Surveys To Create Consequential Improvements for Adult Learners

CAEL has always advocated for postsecondary institutions to consider the perception and perspective of their adult students in determining how best to meet their needs and expectations and recognize their unique lived experiences. It is through this understanding that institutions can apply data-driven decision-making to their student-serving initiatives and the implementation of new or improved processes and procedures. However, identifying this information about their adult learners can be challenging. To assist institutions in obtaining the adult learner perspective, CAEL developed the Adult Learner 360, previously known as the ALFI (Adult Learner-Friendly Institution) evaluation, dating back to 1998. The Adult Learner 360 is a comprehensive dual survey, focusing on multiple areas of the student experience through the adult learner lens. In addition to the questions posed to students, the Adult Learner 360 provides a parallel survey to faculty and staff who support this student population, allowing insight into both the student and administrative experience and perspectives. Upon completion of the surveys, CAEL provides raw and comparison data, a gap analysis of the two surveys, assessment relative to national median scores, and a comprehensive written analysis of the results. CAELs deliverables include specific recommendations aligned with CAELs 10 Principles for Effectively Serving Adults and the Framework for Creating Adult Learner Leaders for Institutional Effectiveness (ALLIES). CAEL released the ALLIES Framework in fall 2022 to build on the concepts of the 10 Principles while identifying ways in which institutions can develop an effective change management approach that centers the adult learner. One critical component of the ALLIES Framework is the use of data-driven decision making to inform strategic planning and decision making. The Adult Learner 360, with its targeted questions, gap analysis, and comparison data, is a proven tool for collecting the necessary data to help guide these decisions, both at an institutional level, as well as nationally. Throughout its history, more than 200 institutions have completed the Adult Learner 360, providing meaningful, applicable data of national trends in student needs, objectives, and expectations.

In recent years, as institutions have balanced their COVID-19 responses and amid changing higher education demographics and federal and state legislative action, some have struggled to identify whether the student support work is meeting student needs and making meaningful change. The three following institutions have all completed the Adult Learner 360 at least twice in the last five years, providing longitudinal data of their student and administrative experience and tracking impact of their innovations, process improvements, and policy development. They each agreed to share their experiences of the Adult Learner 360, including changes implemented upon completion of the earlier surveys, lessons learned from their multiple iterations, and the benefits of the comparison to their own results.

Survey Participants
Through the summer of 2023, CAEL team members Kari Shafenberg and Allymyr Atreo met with administrative and academic leadership from Lorain County Community College (Lorain), the University of Central Missouri (UCM), and Virginia Western Community College (VWCC). The Lorain participants included Kelly Zelesnik (dean of engineering, business and technology), Cindy Kushner (director of school and community partnerships), Jonathan Dryden (provost/vice president for academic affairs and university partnership), Marisa Vernon White (vice president for enrollment management and student  services), and Erika Fenik (director of institutional planning and engagement). UCMs Brenda Fuhr (manager, student transition and engagement) and Laurel Hogue (vice provost for online and learning engagement) shared their experiences and insights, and Crystal Hall (Get REAL activity coordinator) and Shonny Cooke (manager of the Hall Associates Career Center) represented VWCC.

One of the first discussion points in the interviews with each institution was identifying the motivating factor that resulted in the decision to complete the Adult Learner 360 more than once. While each institution had its own primary determining event (access to additional funding, tracking of the impact of specific initiatives, etc.), one theme was the use of data in overall strategic planning and decision making. According to Fenik from Lorain, the institution has implemented a long-term plan with Adult Learner 360 surveys across student populations, entering a three-year cycle to identify the impact of campus efforts and determine next steps. Hogue, speaking for UCM, referred to her institutions interest in recruiting and retaining adult learners and noted that the Adult Learner 360 provided a baseline about what their current adult students perceive as strengths and weaknesses, enabling a more specialized approach to meeting those needs. From VWCC, Hall reinforced this interest in obtaining a clear and specific concept of what the needs of students are, rather than relying too heavily on the assumptions or perceptions of faculty and staff. She clarified that while it is possible to rely on existing adult learner research and theory that adults have different needs, there is immense value in the tangible results provided by the survey.

Data Collection and Results
Upon collection of the data, each institution reflected on initiatives and implementation that came from their results. Among the action steps resulting from their data, UCM refocused attention on communication and awareness regarding their robust credit for prior learning (CPL) policies. It also emphasized restructuring and increasing advisor training and resources, including implementing auto-triggers for students who may be eligible to receive credit through CPL assessments. Other adjustments stemming from the AL 360 results include overall greater communication, such as intensified cross-departmental collaboration; increased course offerings for career and life planning; scaffolding information with the admissions team; and optimizing key search terms. According to Hall, VWCC leveraged their results into additional grant funding, which made possible a prior learning specialist, an adult career advisor, and a career affordability navigator. Lorain County Community College had a similar experience in applying data to obtain a grant supporting employer partner connections, allowing Lorain to embed career development in its programs of study through the initiative called  “Train Ohio.”

Surprises and Discoveries

Each institution also shared surprises they encountered in their Adult Learner 360 process. For some, this resulted in refocusing attention to areas highlighted by students that had not been administrative priorities. As Hall from VWCC shared, “Go straight to the source … you’re asking, ‘What’s the perception of what you’re doing?’ and perception is reality for a lot of people. We may be thinking we’re doing something really well, but if the students are receptive to it, or they’re not viewing it that way, then we’re just spinning our wheels.” She went on to add that taking action and implementing change is only meaningful if the work connects with the student needs, even if the work is in alignment with trending implementations. Cooke also shared that when analyzing the student data, it is important to look at their responses objectively, “Sometimes when you take them personally, you get a little defensive and you start to justify why things are the way they are or why students may not be recognizing what you’re already doing, but really [we should be] looking at it as a greater reflection of opportunities we have to make a larger impact as a collective institution.”

Participants also identified some pleasant surprises in their results, namely the increase in student scores over time. Dryden from Lorain and Cooke from VWCC both shared that seeing definitive data that student perceptions were improving over time was positive and encouraging. In fact, each of the institutions saw 6-8% student score improvement in multiple areas between their two most recent survey cycles. Across all three institutions, participants acknowledged the significance of seeing the increased scores, knowing the work that went into moving the needle between surveys. For Vernon White, “It was really affirming that all of these small things, they do add up, and that we sometimes think, ‘Wow, we really have a way to go’ when it comes to supporting adult learners, but it was really affirming to hear from our students that, actually, we’re doing pretty well.” CAEL staff member Atrero reiterated in her comments that “The AL360 surveys do point out what your gaps are, but it also shows your success. I think it’s important that we’re also able to highlight and celebrate your institution’s success.” In addition to using this information to direct additional work, institutions shared they conveyed the results with the student-facing staff and faculty to demonstrate that their work and efforts have measurable impact on the student experience.

Looking Forward

When asked about advice for future Adult Learner 360 participants, each institution provided reflections on their own experiences. Lorain focused in part on the significance of understanding shared language, definitions, and expectations prior to releasing the surveys, including creating a culture of student feedback. By encouraging regular communication to students on the importance of data collection and examples of the impact their responses have to policy and process, Lorain has an excellent student response rate, providing them with robust data. Fuhr at UCM focused on tying the survey results and analysis to future strategic planning decisions. Relying on the recommendations and results, UCM was able to inform their decision-making regarding their adult learner population. VWCC’s Hall reinforced the value of an open and flexible approach to the survey, and acknowledged that the students are the best, most direct source to learn what is working and where there are opportunities for growth and development

Each institution affirmed that the results, both those that indicated gaps and those that demonstrated progress and successes, provided guidance and direction for institutional decisions and development. When asked if they would consider their Adult Learner 360 survey experiences successful, each agreed. Dryden summed up his perception by sharing, “The survey is very helpful as a topic of conversation on campus to help drive change and in this case, also be able to show, ‘Look at the work you’ve done and how the students are responding as a result,’ so you know you’re making a difference.” Hogue from UCM affirmed that the survey was “well worth the time spent,” and Cooke remarked that the survey results have accelerated progress as “adult learners were built into the strategic plan and governance structure.” By collecting the student-focused data, building on the results, and tracking longitudinal impact of the work, institutions are better positioned to identify and meet student needs, while continuing to move forward in the shared goal of supporting and uplifting adult learners.