Assessment
Teaching and Learning: Evaluation and Improvement
At Otero College our mission is “To be a leader in Colorado’s Community College System when it comes to education.” To achieve that mission we will “provide innovative and responsive educational and economic opportunities in an accessible, inclusive environment that promotes success for students, employees and the community.”
Through our assessment process Otero demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement.
Overview of the Assessment Program
The Assessment philosophy at Otero College is based on the principles of assessing skills, reviewing data and taking action.
Additionally, assessment is a system where faculty use the data collected to improve teaching and student learning. It is not used to give grades, nor is it a faculty evaluation system. Assessment is used by the entire campus for strategic planning and budgetary decisions and assessment aligns with the Higher Learning Commission Accreditation.
Criterion Four. Teaching and Learning: Evaluation and Improvement
The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement.
Assessment is the systematic collection, review, and use of information about educational programs to improve teaching and student learning.
Otero College
2015-2017 Assessment Plan
Universal Assessment Activities
| Task | Timeline | Responsible Parties | Status |
|---|---|---|---|
Create Assessment cycle in Taskstream for ISLOs, PSLOs and Co-curricular | Annually in Fall | Assessment Coordinator |
|
Develop “How To” guide for Taskstream | 2015 | Assessment Coordinator |
|
Create Assessment Handbook | 2016 | Assessment Coordinator and S3 |
|
Increase Taskstream usage | Continuous | Department Chairs | |
Update assessment strategic plan | 2017 | S3 | |
Develop assessment web presence | 2016 | VPI and AVPI |
|
Create assessment libguide | 2017 | AVPI |
Institutional Student Learning Outcomes
| Task | Timeline | Responsible Parties | Status |
|---|---|---|---|
Remap courses being assessed as necessary | 2015 – 2016 | S3, Faculty and Chairs |
|
Re-evaluate ISLO Outcomes, criteria and assessment process | Continuous | S3 |
|
Data collection and analysis process | 2015 – 2016 | Assessment Coordinator |
|
Follow through applying longitudinal data | 2017 | All Staff |
|
Analyze comparison of the Mathematics and Written Communication courses assessed by Otero Math and English, other Otero programs, and concurrent | 2016 | Assessment Coordinator IR |
Program Level Student Outcomes
| Task | Timeline | Responsible Parties | Status |
|---|---|---|---|
Data analysis | Continuously | Assessment Coordinator | |
Data Collection Process | Continuously | Assessment Coordinator | |
Create “to do” for assessment to keep faculty thinking and focused on assessment | 2016 | Assessment Coordinator |
|
Program Reviews
| Task | Timeline | Responsible Parties | Status |
|---|---|---|---|
Process | Annually | Department Chairs | |
Consistency of the data | Annually | IR |
Co-Curricular
| Task | Timeline | Responsible Parties | Status |
|---|---|---|---|
Data Collection Process | Semester basis | Staff |
Assessment at Otero is guided by the Student Success Strategies (S3) committee comprised of department chairs, faculty representatives, assessment coordinator, Vice President of Instruction, Associate Vice President of Instruction, Director of Library & Learning Resources, Director of Instructional Initiatives, Associate Vice President of Enrollment Management and Vice President of Student Services. This committee adopted the ISLOs and developed rubrics and score sheets. Annually, they complete the ISLO assessment cycle, review and analyze ISLO data, oversee the assessment strategic plan, and make recommendations for assessment training.
Assessment at Otero is completed in the following areas:
Institutional Student Learning Outcomes (ISLO) Assessment
This is general education assessment of: Written Communication, Mathematics, Critical Thinking, Information Literacy and Technology. Courses were mapped to determine where ISLOs are assessed.
Program-Level Student Learning Outcomes (PSLO) Assessment
Otero has identified Career and Technical Education (CTE) programs and four academic programs: Arts and Humanities, Mathematics, Social and Behavioral Science, and Natural and Physical Science. The CTE programs are: Agri-Business Management, Agricultural Science, Automotive, Cosmetology, Early Childhood Education, Business, Nursing, Medical Laboratory Technician and Community Health Worker\Health Navigator. Each program developed outcomes and criteria and mapped outcomes and criteria to courses within the program.
Co-curricular Assessment
Otero has 16 co-curricular program categories: Business Office, Computer Center, Physical Plant, SCORE Center, Security, Enrollment Services, Instructional Services, Learning Commons, President’s Office, Advising, Athletics, Financial Aid, Fitness Center, Residential Life, International Department, and Student Life. Each program has developed outcomes and criteria.
The first S3 committee meeting was in November 2007. This committee was formed in response to the Higher Learning Commission accreditation visit recommendation earlier in the fall to begin assessment work across campus. In February 2008, a group of seven attended the Assessment Academy in Chicago, which laid the foundation for implementing a general education assessment. In fall 2008, a Critical Thinking rubric was piloted with the Written Communication assessment to follow in 2010. By fall 2012, a 1/4 time assessment coordinator was hired and Career and Technical Education (CTE) program staff began developing program outcomes. In spring 2013, work began on creating outcomes and criteria for Mathematics, Technology and Information Literacy. By fall 2013, rubrics for the five Institutional Outcomes were developed, along with mapping of those outcomes across all programs. Assessment on these outcomes began shortly after development.
Also in spring 2013, program outcomes were developed for Arts/Humanities, Social and Behavioral Science, Math, and Natural and Physical Science. CTE programs continued to work on and pilot program assessment plans. By January 2014, all programs were doing assessment each semester.
In the summer of 2014, an assessment management tool called Taskstream was purchased; and in the fall of that year, faculty were trained in the basics of its use. Assessment data and analysis became more structured and easier to maintain. By spring 2015, co-curricular assessments were added to Taskstream. With this addition, assessment was happening across campus and beginning to be tied to budgetary planning.
The S3 Committee grew from 10 participants to 15 with participation mainly from faculty and the Vice President and Associate Vice President of Instruction with co-curricular staff invited to attend on an as-needed basis. The duties of this committee were initially to steer an assessment path and have since grown to encompass aggregating ISLO data, writing the yearly plan for assessment, and making recommendations for assessment across campus.
November 2007
Leadership:
- Dr. Tom Armstrong, Vice President of Instruction, from November 1, 1999 to June 30, 2008
- Mr. Jim Herrell, Associate Vice President of Instruction, from January 4, 2000 to January 31, 2011
- Organized first assessment meeting with some faculty and staff.
December 2007
Discussed:
- Syllabus revision
- SLO replace course objectives
- Assessment replace method of grading
- CDHE – GT Pathway
February 2008
- Group of 7 attend the first assessment academy in Chicago
Fall 2008
Leadership:
- Dr. Bob Hayden, Vice President of Instruction, from July 1, 2008 to August 13, 2010
- Mr. Jim Herrell, Associate Vice President of Instruction, from January 4, 2000 to January 31, 2011
Pilot Critical Thinking Rubric
- S3 group and Dept. Chairs were pilot group
Spring 2009
- Full time faculty assess Critical Thinking
- “Data collection is not the deal but looking at teaching is the deal.” Jim Herrell
Fall 2009
- Full time faculty assess Critical Thinking again
Spring 2010
- S3 looked at the English rubric they use to score their written papers
- S3 began working on a technology rubric
- S3 Continuing syllabus discussion
- Linking Outcomes on syllabus to ISLOs
Fall 2010
Leadership:
- Mr. Jim Herrell, Interim Vice President of Instruction and Associate Vice President of Instruction, from September 1, 2010 to January 31, 2011
- S3 adopted English and working on a technical writing rubric to assess writing as an ISLO.
- Departments will decide which rubric it will use.
Spring 2011
Leadership:
- Mr. Jim Herrell, Vice President of Instruction, from February 11, 2011 to November 30, 2012
- Dr. David Cockrell Associate Vice President of Instruction, from June 15, 2011 to June 30, 2014
- S3 discussed that the “n” was too small. Need to get more full time faculty involved. It has not been mandatory up to this point.
- Discussing concurrent classes doing ISLOs
Fall 2011
- Continue to assess Critical Thinking and Written Communication across campus
- Discussing math assessment with focus on rubrics.
Spring 2012
- ECE completed accreditation – S3 considering using the process as a guide
- S3 beginning to discuss CORE content mapping
- Initial Critical Thinking data aggregated and given to S3
- Exit Assessment Academy
Fall 2012
- Assessment coordinator appointed
- Group of faculty attended state assessment workshop in Pueblo
- October – Assessment Workshop for all full time faculty. Beginning to write Outcomes for Gen ED – Written Communications, Mathematics, Technology, Reading, Critical Thinking
- Three people attended assessment workshop in Chicago
- Developed a 2 year assessment plan
- Discussion about what a program is and how to assess program outcomes
- CTE programs began to develop program student learning outcomes (PSLOs)
- Mapping continues – gaps have been identified
- Discussion about common rubrics for ISLOs
Spring 2013
Leadership:
- Dr. David Cockrell, Interim Vice President of Instruction and Associate Vice President of Instruction , from January 1, 2013 to June 30, 2013
- Five ISLOs are finalized – Written Communication, Mathematics, Critical Thinking, Technology and Information Literacy
- ISLO rubric development begins
March 2013
- First draft of academic PSLOs
- English 121 rubric adopted for all Written Communication
- Add Critical Thinking to all ISLO rubrics/score sheets
April 2013
- Mapping for ISLOs continues across campus
May 2013
- All faculty assessment workday
- Math PSLOs are being worked on
- Discussion about whether or not to include Reading as an ISLO
Fall 2013
Leadership:
- Mr. Brad Franz, Vice President of Instruction, from July 1, 2013 to April 30, 2015
- Dr. David Cockrell, Associate Vice President of Instruction, from June 15, 2011 to June 30, 2014
- Ms. Kim Grimsley, Associate Vice President of Instruction, from July 1, 2014 to June 30, 2015
August 2013
- Continue to work on mapping of ISLOs
- Working to map PLSOs in each program
- Discussion was to drop Reading as an ISLO and do Information Literacy instead
September 2013
- Decision to drop Reading as an ISLO but will assess Information Literacy and Technology – both will have the Critical Thinking component
October 2013
- Information Literacy rubric finalized
- Technology rubric finalized
- Mathematics ISLO rubric still in development
November 2013
- Continue with ISLO/PSLO mapping
- Mathematics ISLO rubric finalized
- Mathematics PSLO outcomes and rubric are finalized.
- Cabinet approved revised ISLOs
Spring 2014
January 2014
- Begin to collect PSLO data for programs that are ready
- Still collecting ISLO data on Written Communication and Critical Thinking
March 2014
- AA/AS/AAS will assess at least one ISLO during program.
- Discussion of keeping artifacts – do we need to? How may? Which ones?
April 2014
- Begin researching assessment software
- Collecting artifacts for the first time.
Summer 2014
- Purchase Taskstream and begin to build
Fall 2014
- Taskstream being used for first time
- Decided that only ISLOs will be represented in the syllabi for now
- First data on Spring 2014 ISLO complete
- All rubric and score sheets for all ISLOs complete and ready to go
Spring 2015
Leadership:
- Ms. Kim Grimsley, Interim Vice President of Instruction and Associate Vice President of Instruction, from March 1, 2015 to present
- Some programs will complete Findings in Taskstream – not all ready to complete a whole cycle
- Working to improve Taskstream
Fall 2015
Leadership:
- Ms. Kim Grimsley, Vice President of Instruction, from July 1, 2015 to present
- Mr. Ryan Trosper, Associate Vice President of Instruction, from July 1, 2015 to present
- Implemented a full assessment cycle in Taskstream.
- S3 adopted Information Literacy VALUE Rubric for Spring 2016 implementation
- S3 adopted Critical Thinking VALUE Rubric for Fall 2016 implementation
- Created PSLO reporting system for comparison of data across semesters
- Developed and distributed “How To” guide for Taskstream
Spring 2016
- Remapped all ISLOs for Fall 2016 implementation
- Modified all ISLO score sheets
- Revised and distributed postcard cheat sheets to faculty
Fall 2016
- Changed PSLO reporting system to match the ISLO data reporting system across semesters
- Separated assessment for 2016 cycle in to Spring and Fall and added syllabus repository
- Completed first draft of assessment handbook
How is assessment completed?
The assessment cycle at Otero College runs from January to December, or from Spring Semester to Fall Semester. When the college decided to purchase Taskstream as the tool to guide assessment, it quickly became evident that it was difficult to match the budget process when data was collected during an academic year; thus the change to the calendar year. Once data was collected over several consistent semesters, it has become easier to connect assessment to budget.
What is the timeline?
| Date | Deadline |
|---|---|
| September 2, 2016 | Validate PSLO 2016 spring findings and submit for review |
| December 9, 2016 | Fall 2016 ISLO and PSLO score sheets/artifacts in Taskstream |
| February 10, 2017 | Complete 2016 findings and action plan and submit for review |
| February 10, 2017 | 2017 PSLO assessment plan in Taskstream and submit for review |
| February 10, 2017 | Program reviews submitted to O:Drive and notify Tami |
| May 5, 2017 | Spring 2017 ISLO and PSLO score sheets/artifacts in Taskstream |
| Date | Deadline |
|---|---|
| December 16, 2016 | Fall 2016 ISLO and PSLO score sheets/artifacts in Taskstream |
| June 2, 2017 | Spring 2017 ISLO and PSLO score sheets/artifacts in Taskstream |
What is the Purpose of Program Reviews?
Program reviews are currently completed on a yearly basis. Information collected for program reviews consists of semester-by-semester enrollment, completion rates and faculty information, such as professional development. Strengths and weaknesses are developed and tied to an action plan. Each program develops its plan, which is then added to Taskstream. As this process is streamlined and data is collected, a rotating system for completing program reviews will be developed.