The Fifth-Year Interim Report

Part III: The Fifth-Year Compliance Certification


3.3.1.1 Institutional Effectiveness: Educational programs, to include student learning

The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results.

Judgment of Compliance

Compliance

Narrative

Through a continual process of institutional effectiveness administration, Alcorn State University (ASU) identifies expected outcomes for educational programs, assesses whether outcomes are achieved, and provides evidence of improvements based on those results. The institution achieves this by requiring all educational programs to submit institutional effectiveness plans and reports into the university’s software assessment system (TracDat). ASU’s vision, mission, and statement of purpose serves as the foundation for ongoing, continuous institutional effectiveness and assessment of educational programs. These statements were updated and adopted in 2009 through a very-detailed strategic planning process as articulated in the Alcorn State University: 2009-2019 Strategic Plan: Road Map to Greatness.

The institutional mission serves as the underpinning of all continuous improvement efforts because “the university emphasizes intellectual development and lifelong learning” that are strengthened by the core values and university goals of a student-centered campus and academic excellence that are identified in the strategic plan. 

The university goal of a student-centered campus states that “Alcorn State University will continue to offer students an engaging, transformative learning and living environment, empowering them to become globally competitive, socially, and environmentally sensitive, and technologically competent leaders.” 

The university goal of academic excellence states that “Alcorn State University will consistently enhance its academic excellence and become nationally recognized as a premier comprehensive land grant university offering engaging intellectual experiences and collaborative research opportunities.”

Alcorn State University offers one associate’s degree, 29 bachelor’s degree, 11 master’s degrees, and one specialist degree. In accordance with the university’s broad-based, comprehensive institutional effectiveness and assessment process, each academic program is required to identify outcomes, define assessment criteria and procedures, results, and use of results and improvements from this self-assessment on an annual basis. The assessment process at ASU endeavors toward identifying outcomes that demonstrate academic excellence and a student-centered campus.

This narrative will describe the assessment of educational programs at Alcorn State University, and is organized as follows:

  • Overview and History of Institutional Effectiveness
  • Continuous Improvement of Institutional Effectiveness and Assessment Process
  • Institutional Effectiveness Training and Development
  • Student Learning Outcomes Assessment

 

Overview and History of Institutional Effectiveness

Office of Institutional Effectiveness and Assessment

Since Alcorn State University’s successful 2011 SACSCOC reaffirmation visit, the Office of Institutional Effectiveness and Assessment hired a new director to continue and further develop assessment efforts that were already taking place. Dr. La Toya Hart was appointed March, 2012, and has the responsibility of planning, directing, and supporting all campus-wide institutional effectiveness and assessment activities. To ensure that research-based institutional effectiveness and assessment activities are conducted in an integrated, systematic, and cyclical manner, the director and administrative assistant provide workshops for the university community.

Continuous Improvement of Institutional Effectiveness and Assessment Process

Since ASUs last reaffirmation visit, ASU has transitioned from utilizing a paper Institutional Effectiveness Report form to entering all assessment information into TracDat, an electronic software assessment system. In addition to entering assessment information, this software assessment system allows users to run reports and have a repository for all assessment documents and documentation. In conjunction with the Institutional Effectiveness and Assessment Committee, Faculty Senate, Chairs and Deans Council, an assessment calendar was developed that runs from October to September of each academic year. This cycle, explained more fully in the paragraph below, allows academic units the opportunity to review assessment plans, review the feedback from the University Assessment Committee, attend workshops to improve reports, and develop continuous improvement plans. The roles and responsibilities of the University Assessment Committee were expanded so members would serve as advocates for the continued growth of institutional effectiveness and assessment.

University Assessment Committee

The Alcorn State University Faculty Handbook states, “the purpose of this committee is to assist the University in developing academic assessment methods, standards and measures aimed at continuously improving student learning outcomes of the university” p.51. Since our last reaffirmation visit, committee members serve as advocates to expand the institutional effectiveness and assessment process university-wide with an overarching focus on continuously improving student learning and program outcomes. University assessment committee members also serve as a liaison between their respective school, department, or unit and the Office of Institutional Effectiveness and Assessment to provide assessment information. The committee members also work within their school, department, or unit to update and revise assessment plans and reports. Additionally, University assessment committee members serve as peer reviewers to provide feedback on assessment reports for the entire University. Faculty are recommended to serve on the University Assessment Committee by the dean and department chair for three years, and the Institutional Effectiveness and Assessment Director serves as committee chair.

The programs are reviewed annually by two reviewers who serve on the University Assessment Committee using an assessment rubric. A third review by the director is triggered if the two reviewers have wide varying overall scores and/or comments for any program. All scored assessment reports are then shared with the program by the Institutional Effectiveness and Assessment Director. Faculty are encouraged to review the score and the comments from the review, and the University Assessment Committee member for the school, department, or unit continues to work with the program to update their reports based on the recommended suggestions for continuous improvements. It is expected that all assessment reports are updated during subsequent assessment reporting years.

In 2014, this committee developed an assessment cycle that has been adopted by the institution. According to the assessment calendar, all programs will have completed assessment reports in TracDat no later than September 15, and an overall score and substantive feedback are provided to programs in December for the previous academic year.

Rubrics

The committee utilizes a rubric to score and provide feedback on the quality of each report. This rubric provides specific criteria that must be included in each report.  In 2012, the Institutional Effectiveness and Assessment Committee adopted a rubric to evaluate assessment reports that was developed by the Institutional Effectiveness and Assessment Director. After using the 2012 rubric for two years, the committee decided that this rubric left too much room for subjectivity during the review process. Also, the committee felt that double work was being done because two evaluation reports were being submitted for the same assessment report. The reports were scored in Excel and then comments were submitted in a Word document. Committee members began to review other rubrics that could be adopted and revised to fit our institutional effectiveness and assessment needs.

In 2014, the University Assessment Committee revised and adopted an assessment rubric that was being utilized by West Texas A & M University. This assessment rubric guides the University Assessment Committee to provide consistent scoring and facilitate feedback, but also serves as a self-assessment tool to guide programs when developing assessment reports.

 

Institutional Effectiveness Training and Development

The staff of the Office of Institutional Effectiveness and Assessment provides ongoing training for all assessment related activities. Workshops are conducted during the fall and spring semesters on assessment writing and entering assessment information into TracDat. 

Assessment Development

Assessment writing workshops are conducted for the university community by the Institutional Effectiveness and Assessment Director on developing quality assessment documents. These workshops are scheduled and participants are asked to register on the Institutional Effectiveness and Assessment website. The director also conducts specific workshops as requested by schools, departments, and units.

 

TracDat

TracDat can be accessed from the University’s website where all users are required to enter a username and password. Each academic dean and chair has access to all institutional effectiveness data under his/her purview. The chair also has the option of allowing different levels of access (administrative, email only, reports only) to each faculty in the unit. The user follows prompts to enter all of the assessment data in the appropriate fields. An assessment plan or assessment report can be produced after the information has been entered.

Some faculty members expressed challenges to organizing data in a meaningful way prior to entering into TracDat. Two templates were developed to assist faculty members with this challenge. One template allows the user to enter the data in a worksheet format to enter three outcomes; and the other template allows the user to enter the data in the format of how the report will be printed in TracDat.

TracDat workshops are conducted for the university community by the Institutional Effectiveness and Assessment Staff to ensure reports are entered correctly.

 

Introduction to Learning Outcomes

In addition to receiving training from the Institutional Effectiveness and Assessment Director, University Assessment Committee members are provided with the opportunity to take an Introduction to Learning Outcomes course offered by the Data and Decisions Academy sponsored by the Association for Institutional Research. The core lessons for this asynchronous course are: 1) developing intended learning outcomes, 2) assessing student learning outcomes, 3) using assessment data for decisions. Although the course is self-paced, participants are given eight weeks to complete the course, a book is provided to each participant that supplements the online modules. This course allows committee members an additional opportunity to explore techniques and tips for writing intended learning outcome statements to enable more substantive feedback during peer assessment reviews. (http://www.airweb.org/EducationAndEvents/OnlineLearning/Academy/Courses/Pages/LearningOutcomes.aspx)

Student Learning Outcomes Assessment

In accordance with the University’s broad-based comprehensive institutional effectiveness and assessment process, academic programs are expected to complete assessment reports on an annual basis. Each reporting unit follows the University assessment calendar that runs from October to September. This calendar is closely aligned with the academic calendar to allow for data collection and analysis for the entire year. All of the departmental missions flow from the school and University mission and institutional strategic goals that are identified in the strategic plan. Faculty in each department identify program and student learning outcomes, appropriate assessment tools that will yield data that can be used to measure the extent to which the academic unit has achieved the desired outcome. Faculty then use the assessment results to make data-driven decisions to make improvements. All assessment data are entered into TracDat, the University’s assessment software system.

Academic programs that are offered through distance education follow the same institutional effectiveness and assessment process as traditional programs.

The following reporting requirements are expected in each institutional effectiveness and assessment report and are aligned with each rubric component:

  1. A departmental mission that aligns with the university mission and concisely outlines the purpose of the program, who it serves, in what ways, and with what results
  2. Specific, observable, measurable program and student learning outcomes.
  3. Multiple assessment methods and procedures are used to assess the outcome.
  4. Criteria that aligns with measures and outcomes, and is based on benchmarks, previous results, or existing standards.
  5. Assessment results that report the findings in a clear, concise, organized manner.
  6. The action that will be taken based on the results are clearly stated and provide follow up on the implemented actions.

 

Sampling Method
In an effort to demonstrate continuous improvement in the area of reporting, selected samples from the 2014-2015 academic year assessment reports are highlighted and described in the following narrative. The narrative features one randomly selected undergraduate and graduate (if applicable) sample from each academic department.  In situations where there is only one graduate program for the academic discipline, that program was automatically included in the sample below. For instance, all three programs from nursing were included in the narrative because there is only one program for each degree level.

All of the assessment reports for each academic program for 2013-2014, 2014-2015, and 2015-2016 are presented in full detail. Also, the 2016-2017 assessment plan for each academic program is available in full detail.

School of Agriculture, Research, Extension, and Applied Sciences (AREAS)

Agricultural Science (General Agriculture) B.S.
The Agricultural Science program assessed the students’ ability to demonstrate knowledge of weed control practices. The method of assessment was a faculty designed area exam that tested the students’ knowledge of weed control practices. The criterion for success was that 80% of students would score 70% or higher on the exam. Ninety-six (96%) percent of the students scored 70% or higher, thus the criterion was met. Since the criterion was met no further action was taken. Upon following up, it was decided that this outcome would be assessed again in 2016-2017.

Agriculture M.S.
The Agriculture M.S. program assessed the students’ ability to demonstrate knowledge of advanced research methods. This outcome was assessed with the Department of Agriculture Exit Examination designed and validated by members of the curriculum committee. This particular test was designed to only test students’ knowledge of advanced research methods. The criterion for success was that 90% of the students would pass the exam with a score of 80% or higher. Ninety- five (95%) percent  of the students passed with a score of 80% or higher, thus the criterion was met. While the criterion was met, the curriculum advisory committee met and decided to revise the curriculum by adding more tutoring time for low performing students and increased use of technology to enhance learning. When following up on this assessment, the faculty found that the students from 2015-2016 did worse than those from 2014-2015 suggesting that the changes made did not work. The faculty decided to continue with the same actions and to assess the outcome again.

Applied Science B. S.
The Applied Science B. S. program assessed the students’ ability to demonstrate proficiency with the components in their experimental set up and the ability to connect the circuit/assemble in the instrument to write the software code. Seventy-two percent of the students were able to demonstrate this ability on a practical exam that was administered to students in the thesis/project class. Since the criterion of 70% was met, the benchmark will be raised to 75% for the next assessment cycle.

Computer Networking and Information Technology B. S.
The Computer Networking and Information Technology B. S. program assessed students’ ability to demonstrate good research, technical , and communication skills when making thesis presentations. The desired pass rate was 90%, but the actual pass rate was 80%. Some students appeared to have postponed their thesis work, and it was decided by faculty that better advisement would be required to students on their course schedules for graduation.

Child Development B.S.
The Child Development B.S. program assessed the students’ ability to demonstrate a basic understanding of how to conduct a thesis research project. This outcome was assessed by evaluating the students’ ability to complete a thesis presentation. The criterion for success was that 80% of the students enrolled in HS 494 will receive a final score of 80%. 90% of the students achieved a score of 80% or higher with an acceptable rating for the thesis project, thus the criterion was met. Since students demonstrated the ability to complete an outline for a thesis project, faculty will use the outline to work with students to write a complete thesis project. As follow-up, the faculty continue to work with students on developing a thesis project. Faculty will assess projects in a future assessment cycle.

 

School of Arts and Sciences

Biology B.S.
The Biology B.S. program assessed the students’ ability to use a microscope. The method of assessment called for the students to able to identify the parts of a microscope. The criterion for success was that 100% of the students should identify all of the parts of a microscope. 100% of the students were able to identify all of the parts of the microscope, thus the criterion was met. With these results it was found that instructional procedures were adequate. Upon completing follow-up, it was decided that no further action was required for this assessment.

Biology M.S.
The Biology M.S. program assessed the students’ ability to gain admission into a doctoral program based on academic curricula and research training experiences. The assessment method was to gather data showing that at least 70% of the graduates were seeking admission to a graduate or professional school with a 90% success rate. The criterion for success was that 70% of the students would be accepted in a graduate or professional program. 40% of the graduates gained entry into a graduate or professional program, thus the criterion was not met. With these results, the faculty are focused on seeking more graduate opportunities for the students. When following up, it was decided that the faculty needed to encourage the students to begin the applications process earlier.

Biology M.S. (Online)
The Biology M.S. is the only program offered at Alcorn State University in both a traditional and online format. The online program began in fall of 2015 and currently has one student enrolled.  Both programs assess the same student outcomes which include assessing the students’ ability to gain admission into a doctoral program based on academic curricula and research training experiences. There has been no results data collected because the program has only just started.

Chemistry B.S.
The Chemistry B.S. program assessed the students’ understanding of basic concepts in Analytical Chemistry. The outcome was assessed with an exam given to the students taking analytical chemistry. The criterion for success was that 85% of the students would score an 80% or higher on the exam. 80% of the students scored 80% or higher, thus the criterion was not met. With these results, the faculty could see the areas of strengths and weaknesses of the student in Analytical Chemistry allowing for more effective instruction in the course. When following up on this assessment, the faculty decided that it would provide students with a “refresher” session; students’ scores have improved.

Computer and Information Science M.S.
The Computer and Information Science M.S. program assessed the students’ understanding of ethical, legal, security, and social issues and responsibilities. The outcome was measured through student essays gathered in CS 580-Research Project, which were assessed with a faculty developed rubric. The criterion for success was that 80% of the students would score 4 or above per the rubric. 20% scored 4 or above, thus the criterion was not met. Due to these results, it was decided that special lectures and/or materials would be provided for students completing this assignment. Upon follow-up, it was decided that this learning outcome would be assessed again in the 2015-2016 academic year with these procedures in place. The criterion was met the following year and the faculty are continuing to assess this outcome.

Criminal Justice B.S.
The Criminal Justice B.S. program assessed the students’ ability to identify the requirements of each component of the criminal justice system. This outcome was assessed with a written examination consisting of questions pertaining to the components of the criminal justice system. The criterion for success was that 60% of the students would pass the written examination. 70% of the students passed the examination, thus the criterion was met. With these results, the faculty decided that they would continue to emphasize the exam requirements and preparation.

English B.A.
The Literature B.A. program assessed the students’ ability to write supporting claims than would inform their thesis statement. The method of assessment used a faculty developed rubric to assess student arguments written in EN 403 to determine students’ proficiency in using supporting claims to inform their thesis. The faculty criterion for success was that at least 90% of the students would earn a “Proficient” on this element of the rubric. After assessing 10 papers, only 8 (80%) of the writing samples earned a “Proficient,” thus the criterion was not met. The following strategies were implemented because of these findings. The faculty agreed that in all upper level courses students would be required to use evidence to support their spoken claims and that the students would also have to explain this evidence. It was also decided that this model would be used in the written work needed for this assessment in the future as well. Finally, it was decided that all the English majors would be required to attend an Alcorn Writing Center workshop on crafting effective paragraphs. This outcome has remained a part of the assessment plans for the years following 2014-2015. The criterion for success was not met in the following academic year, but to follow-up on the assessment the faculty met to monitor and reinforce the use of results.

Social Work BSW
The Social Work BSW program assessed the students’ ability to identify personal values and biases to ensure behavior that is consistent with professional values. The method of assessment called for the students to write a “Values Paper” in SW 351 which was designed to assist students in comparing and contrasting personal and professional values. The criterion for success was that 70% of students would demonstrate competency by scoring a 70% or higher on the assignment. 100% of students passed the assignment with a score of 70% or higher, thus the criterion was met. The results were used to review and update future assignments to reflect current knowledge and skill levels in the profession.

 

School of Business

Accounting B.S.
The Accounting B.S. program assessed the students’ ability to demonstrate effective writing appropriate to the discipline. The outcome was assessed using an essay assignment. The criterion for success was that 75% of the students were expected to pass the assignment with a C or better. Students assessed under this measure achieved the benchmark, thus the criterion was met. With these results, the faculty decided to continue monitoring for further growth in the students’ written communication. Upon doing follow-up, the faculty decided to add more writing assignments to the students’ management and finance courses.

Business Administration M.B.A.
The M.B.A. program assessed the students’ ability to develop professional papers. This outcome was assessed by how many students were accepted to present at academic conferences or submitted to local/regional/national journals or periodicals. The criterion of success was that one student would present at a conference. One student did present at a conference so the criterion was met. While the criterion was met, the faculty decided that they needed to search for more funding to help send students to conferences in the upcoming year. When following up on this assessment, the faculty decided that it would find ways to encourage student participation at academic sessions which may not require a lot of funding.

 

School of Education and Psychology

Elementary Education Ed.S.
The Elementary Education Ed.S. program assessed the students’ ability to meet all of the requirements for exiting the program in a two-year period. This outcome was assessed by monitoring the students from entrance to end of program. The criterion for success was that 40% of the students entering the Specialist Education Program would exit the program in two years. 33% of the students exited the program, thus the criterion was not met. With these results, the faculty recognized that students were bottle necking at the writing stage and that there was a need to aid with the writing skills of the students. Students identified as weak writers were given referrals to the Writing Center.

Psychology B.S.
The Psychology B.S. program assessed the students’ ability to adopt values that build community at the local and state levels. The method for assessment was student participation in non-classroom learning opportunities. The criterion for success was that 70% of the students enrolled in PH 483 would be selected to participate in off campus community based organizations. 97% of students enrolled in PH 483 were selected to participate in off-campus community based organizations, thus the criterion was met. With these results, the faculty decided to continue assessing this objective with either the same or higher rate of participation required.

Recreation B.S.
The Recreation B.S. program assessed the students’ ability to apply leadership and community service skills at diverse recreational facilities. The method of assessment used a faculty developed rubric to assess individual student’s performance in demonstrating community service and leadership skills at these facilities and with clients. The criterion for success was that at least 70% of the students would score 3 or above on at least five areas on the assessment rubric.80% of students scored 3 or above on the assessment rubric, thus the criterion was met. With these results, the faculty decided to institute leadership assignments in Recreation course offerings. Upon follow-up, the faculty added three leadership activities within the department for the 2016-2017 academic year, two within the HPER Clun and one to assist with Alcorn’s Wellness Day activities.

Teaching MAT
The teaching MAT program assessed the students’ ability to develop effective strategies for discipline and management of the grades 4-12 setting. The outcome was assessed using the MATM-STAR (Domain IV) Learning Environment in ED 521. The criterion for success was that 90% of the students would score a mean score of 2 or better on questions 12-16 of the instrument. 40% of the students scored less the 2, thus the criterion was not met. With these results, faculty decided to use video of classroom settings to analyze strategies to address management in the classroom. Additionally, during the intern course at least one class meeting will be devoted to classroom management. There was a discussion concerning offering the class as a face to face course rather than an online course but with follow-up it was decided to continue offering the course online.

 

School of Nursing

Nursing ASN
The Nursing ASN program assessed the students’ ability to create safe care for clients in a health care setting. The faculty evaluated the students’ ability to provide safe care using the Clinical Evaluation tool. The faculty criterion for success was that at least 80% of the ASN students a final grade of “pass” on the Clinical Evaluation Tool. 100% of the students enrolled in clinical nursing courses achieved a grade of “pass” on the Clinical Evaluation Tool, thus the criterion was met. This data was used to evaluate the Clinical Evaluation Tool. It also revealed that even though the students were not passing the licensure examinations at expected levels of achievement, they were passing didactic and clinical components of the ASN program

Nursing BSN
The Nursing BSN program assessed the students’ ability to demonstrate the scope of knowledge need to practice as a registered nurse delivering client-centered care across the life span. This outcome was assessed using the HESI exit examination to students enrolled in NU 462. The criterion for success was that between 75%-80% of the students would achieve a passing average as a final course grade and that the HESI exit exam would be part of the final course average. 100% of the students enrolled in NU 462 achieved a minimum of 75% in the course, thus the criterion was met. After reviewing the data, the faculty decided to review other vendors that provided standardized testing for nursing. Upon following up on this assessment, the faculty voted to implement the Assessment Technology Institute Essentials Bundle into the curriculum. The Comprehensive Predictor will be administered as the exit examination beginning spring 2016.

Nursing MSN
The Nursing MSN program assessed the students’ ability to utilize effective communication strategies in a variety of healthcare settings. This outcome was assessed through the students’ oral defense of their thesis/research project. A faculty developed rubric was used to measure the students’ performances. The criterion for success was that 100% of the students would achieve a grade of pass as measured by the rubric. 100% of the students received a grade of pass, thus the criterion was met. Due to these results, the faculty decided to continue measuring communication as a learning outcome.

 

 


TextTitle 
institutional mission Alcorn Fact Book Page 2
student-centered campus Alcorn Fact Book p3
academic excellence Alcorn Fact Book p3
Alcorn State University offers Alcorn Fact Book p73
assessment calendar 2014 Assessment Cycle
Faculty Handbook Faculty Handbook p51
scored assessment reports are then shared Assessment Report Feedback
Excel IEP Academic Units Rubrics
Word document Institutional Effectiveness Council Feedback Form
assessment rubric Assessment Rubric
workshops Assessment Training
register Workshop Registrations
TracDat can be accessed TracDat Log In Page
One template Assessment Worksheet
other template Institutional Effectiveness Template
workshops TracDat Workshops
Introduction to Learning Outcomes course Introduction to Learning Outcomes
University assessment calendar Alcorn State University Assessment Cycle 2014
2013-2014 2013-2014 Assessment Reports
2014-2015 2014-2015 Assessment Reports
2015-2016 2015-2016 Assessment Reports
2016-2017 assessment plan 2016-2017 Assessment Plans