Results from a National Needs Assessment Survey: A View of Assessment Efforts within Chemistry Departments

Results from a National Needs Assessment Survey: A View of Assessment Efforts within Chemistry Departments

Mary Elizabeth Emenike, Jacob Schroeder, Kristen Murphy, and Thomas Holme

Journal of Chemical Education 2013 90 (5), 561-567

DOI: 10.1021/ed200632c

ABSTRACT: http://pubs.acs.org/doi/abs/10.1021/ed200632c

This article starts out with a summary review of literature on assessments and related topics.  One of the articles cited was Marcy Towns’ article on assessment case studies also featured in my article reading list. As the article title indicates, the authors will be reporting on the results of a survey motivated by Town’s article on a small sample of case studies and the indication that there is quite a variation of assessment plans that exist in colleges and universities.  To “better understand” the current state of assessment efforts, the American Chemical Society Examinations Institute sent out a survey to a national sample of chemistry faculty.  EI plans to use data from this survey to help design and develop interactive professional development resources related to assessment.

The study involved a wide range of studies but the results will be discussed for the following:

1.       Degree to which departments are expected to conduct assessment efforts

2.       Assessments used in these efforts

3.       Motivation for, and faculty roles within, these efforts

4.       Challenges faced by instructors resulting from these efforts.

Information from focus groups interviews held at national meetings helped inform the creation of a pilot survey.

Surveys were sent out by e-mail to 14,000 tenured, tenure-track, and non-tenure track instructors and professors at two-year and four-year colleges and universities.

SOME KEY RESULTS:

Awareness of expectation to enhance assessment efforts: Overall 72% of all faculty across the three institutions report in the affirmative, slightly lower for the two-year faculty (67%) with 4-year faculty reporting a “yes” with the highest fraction.

Assessment/s used: The American Chemical society Standardized Exam was identified as the most commonly used assessment instrument.  This is what we use for our Chemistry 31, 1A, 1B, and 12B at LPC.  Interestingly, only 49% of two-year faculty use this for assessment.  Two-year faculty seem to favor assessing students using specific questions in exams, the method we use for our other 3 courses at LPC, 30A, 30B, and 12A.  We are, however, exploring using the ACS standardized exam for 30A and 30B.  One of the concerns is that the coverage of material may not match adequately.  Another interesting result is that about a quarter of faculty who responded report using lab notebooks for assessment.  This is also something we are looking into in terms of assessing lab performance and skill achievement in our courses.

When asked the question, “What was the primary motivation for these departmental efforts?”, 57% of all participants pointed to “the college as part of an external accreditation”; 70% of two-year faculty identified this as the primary motivation which is what brought the overall higher from 55% of 4-year faculty and 52% of doctoral faculty. The other popular response from two-year faculty was “the college as an internal decision” at 19% and “the faculty thought it was important” at 8%.  It is worth noting that “the faculty thought it was important” did not receive a higher fraction even with the 4-year faculty (7%) nor the doctoral faculty (6%). The researchers looked more deeply and found that of the 76 institutions that had more than three responders, only 10 had responders from the same institution agreeing on the motivation.  As the authors note, “Perhaps more worrisome, however, is the possibility that these differences reflect confusion about the motivation for the assessment.”

On the question of, “What is your primary role in your departmental efforts? (Choose one.)”, the most responders chose “I collect and contribute data from my classes” (59% overall, 55% for two-year faculty).  For two-year faculty, 28% report that they are leading the effort.

The last question was on “challenges individual instructors face with respect to their departmental assessment efforts”:

It is not surprising to me that the challenge of needing extra time needed to report data received the most affirmative responses.  Something notable the authors noted is “ While none of the top four responses differed significantly among participants from different institution types, the challenge that some instructors “teach to the tests” was more likely to be selected by participants from two-year institutions (23%) than from participants at four-year institutions (13%; β = 0.764, OR =

2.1, p < 0.001). “  In the “other” category, the author organized the responses as follows (verbatim):

1. General issues of time requirements (designing and managing assessments, maintaining records, reporting findings)

2. The concern that the results from these assessments will impact faculty members’ evaluation, tenure, and promotion

3. Lack of awareness or apathetic attitudes toward assessment efforts

4. Claims that departmental assessment efforts infringe on instructors’ “academic freedom”

In the Conclusion section, the authors suggest ways of how data for this assessment needs survey can be used by instructors or others for marshalling efforts to carry out assessment.  See article for these suggestions.