Taking Assessment Seriously

The challenges that higher education is facing are overwhelming--budget problems, society's need for more graduates, especially in the STEM disciplines; preparation of students  for the knowledge industry in a global environment, external pressures for quality assurance and accountability, inadequate financial aid for students leading to less access,  inadequate K12 preparation of many students and the need for more higher education research.

In the Quality disciplines, we talk about the need to be data-driven, to practice Deming/Skewhart's PDSA (Plan, Do, Study, Act), to provide feedback to the system and continuously improve.    The only sure-fire way to address these complex issues of higher education is to develop strategies based on proven assessment techniques.  In October, the National Institute for Learning Outcomes Assessment released a report on a significant survey involving over 1500 higher education institutions, titled "More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education"  authored by George Kuh and Stanley Ikenberry. The information in this blog is based on the abridged version of the report. (www.learningoutcomesassessment.org)

The survey results tell a story.  Accreditation requirements strongly  influence the amount of assessment.  In a list of uses for assessment, higher education institutions gave the most priority to "Institutional self-study for accreditation".  Measures of  a continuous improvement culture such as  "Improving instructional performance" and "modifying academic support services" were in the middle of this list. In identifying the importance of assessment, the survey looked at the number of staff members that were charged with assessment responsibilities.  It was reported that  "almost half (47%) of doctoral institutions reported having one or more staff, while only one-fifth (19%) of community colleges and other associate-degree-granting schools had at least one person focused on outcomes assessment."  Less than 10% of the institutions reported 5 or more staff in this role.  Obviously, part of the story is the lack of priority being given to staff resources to conduct assessments.  Of course, faculty are responsible for learning outcome assessment in their classrooms; at the same time, it seems that  staff resources need to be dedicated to assessment and continuous improvement.

If we are to use assessment and other research to improve the learning processes of students, a culture of using assessments, both institutional and in the classroom must be adopted.  This report recommends that

1. College and University Presidents and other academic leaders "must make quality assurance an instititional priority"


2. Governing board members "must ensure their institution has a system of academic quality control"


3. Faculty need to be involved with the collection of data about student learning.


4. Institutional research personnel can contribute to this effort by focusing on areas of assessment that are important to the educational institution and provide reports that "determine whether changes in teaching and learning approaches have had the desired effects"


5. Student affairs staff can use assessment to help "inform student affairs practice" and be an active participant in the discussions on campus assessment and subsequent institutional actions.


6. Finally, prospective students and parents "should ask to see learning outcomes information" 

All these recommendations are appropriate.  If we are to adequately address the current challenges of higher education, we must consider a more focused effort for a continuous improvement and quality-oriented culture. This includes using more assessment and education research to inform the practices of teaching,  learning and student support.  In addition, a completed feed back loop to determine if the changes in the teaching/learning process provide the expected improvement are needed.  Higher Education does not need to reinvent the processes of continuous improvement. Many are already out there, in use and very successful. What is needed is a more integrated institutional  dialogue and commitment to a continuous improvement culture and assessment as in being suggested in the recommendations of this report.  Such an effort will help lead to more effective institutional strategies for addressing the current higher education challenges.   

 

Cindy

Can College Student Retention be Improved

The Sept. 8th issue of the Chronicle of Higher Education leads with an article on   Crossing the Finish Line: Completing College at America's Public Universities by William G. Bowen, Matthew M. Chingo and Michael S. McPherson.  It is an exciting and worthwhile book to read.  It shows that for 21 public flagship universities, the 4-year graduation rate was only 49% with a much higher six-year graduation rate of 77%.  The authors present the discussion that family income matters, that for low-income families a little increase in tuition can lead to a  higher drop-out rate.  As we have read in the news, even in this past year in the current economic recession, faculty have received raises and the tuition has increased, placing more financial pressure on families to keep their son or daughter in college.   Colleges need to assume a greater level of accountability on containing tuition and the cost of college.

In their analysis,  the authors found that the high school GPA was much more predictive of the six year graduation rate than the SAT or ACT scores for the universities they studied.    

Based on my research,  I would like to suggest that the results may be different for engineering colleges.   Several studies of engineering studies have shown a significant relationship between the ACT or SAT Math score and upper class retention or graduation rates of engineering students.  Each college should conduct their own study and make an evidence-based decision on whether the high school GPA or SAT/ACT scores  (or both) are significant predictors for student retention. 

As the book suggests, more focus on timely completion of degrees is needed.  More focus on higher four-year graduation rates and systemic interventions that help student success are needed. We can have higher 4-year graduation rates! 

Cindy

Sources: 

"For Certain Types of Students , an Ever-Receding Finish" by D. Glenn, Chronicle of Higher Education, September 8, 2009; Crossing the Finish Line:Completing college at America's Public Universities by W.G. Bowen, M.M. Chingos and M.S. McPerson, Princeton University Press,2009.

 

 

Evidence-based Policies for Student Retention

I was reviewing some past research and came across this quote from  Equity and Excellence in Higher Education by Bowen, Kurzweil and Tobin:

" Poorly conceived and poorly executed programs driven principally by ideology or by partisan political agendas can do no little harm...Finding  the right policies is difficult, and there is generally no way of being sure what is the best approach. Careful analysis and a willingness to change directions in the face of evidence are essential. It is not enough to want to do the right thing. " 

This is so true. Sometimes we get caught up in the latest student retention policy innovation/ trend and assume it applies to our college or university.  The best approach is an evidence-based approach (also good engineering analysis)--that of checking out the hypothesis with data and a good statistical analysis/ assessment. 

Cindy