Assessment Reports

Strategy: Multiple committees, led by the Vice Provost and Assistant Provost for Teaching and Learning, the Associate Vice Provost, the Campus Assessment Coordinator, and the Director of UGER, in consultation with the campus-wide University General Education Committee, have endorsed this integrated approach and have worked to build UGER into the system.

Outcome: Current UGER Learning Outcomes are published in the Guide and regularly shared with instructors scheduled to teach Communication, Ethnic Studies, and Quantitative Reasoning Courses. In anticipation of the GER LO’s being accessible to a variety of tools used in the Student Digital Ecosystem, the UGEC has updated and improved language describing GER and GER LO’s. These will be incorporated into tools used to propose new courses and programs. UGEC members participated in pilot projects using the new course evaluation system, provided feedback on how courses meet needs at different levels of curricular analysis (e.g., outcomes that appear on syllabi for GER should reflect GER goals as they are translated through the specific content of the course). This campus-wide course evaluation system should afford systematic and sustainable assessment processes, particularly in areas where the curriculum is most dispersed and most challenging to assess (i.e., “general breadth”), as well as in areas where a broader assessment can complement more narrowly focused direct assessment projects.

Strategy: A working group comprised of faculty and staff who teach QRA and QRB courses revisited and re-articulated learning outcomes for QR that were originally expressed when UGER was founded. See: (Link TBD).

Outcome: The QR liaison, in consultation with a working group of faculty who teach QR courses, found the original course criteria and learning outcomes for QR-A and QR-B were insufficiently distinct for use in the new digital systems that rely on clearly articulated learning outcomes. Departments that teach QR courses have reviewed the revised learning outcomes, which will be proposed for UGEC approval in Fall 2018. Following approval, the committee will develop a plan for sustainable assessment of QR outcomes.

Strategy: Direct Assessment (quizzes, evaluation of student work), instructor survey. See: Libraries Communication-A 2016-2017 Assessment Report.

Outcome: The pilot project determined that extending the online “Sift & Winnow” tutorial program to Communication B courses has been successful; the program will be expanded to reach more Communication B courses. Information from this research will inform further development of the libraries’ information literacy instructional efforts.

Strategy: Benchmarking and consultation to develop rubric (informed by literature survey, AACU VALUE rubrics); field tested by faculty evaluator in two large-enrollment Comm-B courses with recorded video presentations. See: Pilot Assessment Project of Student Oral Communication

Outcomes: This pilot project defined assessable oral and written communication skills more clearly, providing a foundation for a rubric that can be used to evaluate “textual,” “embodied,” and “overall” aspects of oral communication.  The rubric was tested in two Communication B courses on artifacts of student learning and was found to be a useful tool for focused evaluation of the existing oral communication learning outcomes. The UGEC endorsed continuing this work in 2018-19.

Strategy: Consultation with ESR faculty who adopted recommendation through consensus. Assessment Plan endorsed by UGEC April 13, 2018.

Outcome: Per recommendations of Ethnic Studies Subcommittee, ESR courses were revised or removed from course array. New ESR criteria implemented, effective Fall 2019. Best practices for instruction for online ESR courses drafted for further discussion in Fall 2018. Although 2016-17 ESR instructor survey affirmed the alignment between course materials and ESR LO’s and the majority of ESR instructors supported the perception that ESR courses help students achieve outcomes, instructors called for direct assessment. The Ethnic Studies Subcommittee and instructors developed an assessment plan for ESR courses. This plan will be piloted in 2018-19.

Strategy: Statistical analysis of NSSE responses to evaluate changes in students’ responses from first-year to Senior year; compare to responses from peer institutions. See: APIR, “NSSE 2017: General Education”

Outcome: Analysis of responses to questions about critical thinking, skills and associated behaviors in two GER domains (written communication, quantitative reasoning) suggest that UW-Madison students could be exposed to more opportunities to write. Responses to attitudes and behaviors linked to larger UGER goals (participate in a diverse society, expand knowledge of human cultures and natural world, equip them with strategies for understanding complex topics, to exercise personal and social responsibility) are significantly higher and more positive than peer institutions, and well aligned with the goals of GER and of the UW-Madison Experience.

Strategies: (1) Subcommittee review of course syllabi, including interview and discussion with faculty and staff teaching critical courses. (2) Survey of faculty to verify support of ESR outcomes, determine their views of student learning relative to them, and obtain information about faculty experience, challenges and opportunities teaching ESR. See: Report of the Ethnic Studies Subcommittee, August 2017 rev.

Strategy: Consultation with campus partners (Provost’s Office, Registrar’s Office, Advising) about Catalog revisions (to conform to new Guide); consultation with faculty and staff on University General Education Committee and other bodies. See: Guide, “Requirements for Undergraduate Degrees, General Education Requirements”

Outcome: UGER learning outcomes have been clarified and are now published in the Guide. Students, faculty, staff, and other stakeholders have more, and more clear, information about what the requirements are intended to achieve. GER LOs are available to be included in other processes: course proposal system, course evaluation system, etc.

Strategy: Direct Assessment (quizzes, evaluation of student work), instructor survey.

Outcome: Results led to curricular redesign: in-class sessions have added more “authentic” exercises; additional questions are asked in online modules addressing evaluation of scholarly articles; and emphasis in online module has been increased to support student reflection about the learning outcomes for this module. Since the online module has been programmed to allow it to be imported into other courses, students and faculty can revisit this foundational tool in future courses.

Strategy: Review of course syllabi, survey of faculty and staff instructor about courses, pedagogy, and instructors’ evaluation of student attainment of learning outcomes. See: Communication Part B 2016 Curriculum Calibration Report (Appendix D, 2016-17 UGEC Annual Report).

Outcome: Review of course syllabi affirmed that most Comm B courses meet the criteria and have the capacity to support learning outcomes for the requirement. Instructors affirmed course design consilience with Communication learning outcomes, and evaluated student achievement of gains in Comm B courses. However, syllabus review revealed weakness in oral communication instruction, and instructors requested more support for instruction in this area. An online program to offer guidance to instructors in oral communication instruction has been developed to improve teaching in this area.

Strategy: Review of course syllabi, interviews with faculty and staff about courses. Review of course assignments to evaluate capacity to support student attainment of learning outcomes. See: Annual Report of the University General Education Committee to the University Academic Planning Council (Attachment F).

Outcome: Courses found to have insufficient requisites have been modified to ensure that students entering QRA have minimum college qualifications and can proceed through QRA to QRB.  In parallel project, Math Dept. revised pre-college/remedial math courses to better distinguish from and connect to QRA, define appropriate pathways for students seeking Math-intensive majors

• Alignment with HLC expectations to include learning outcomes for all GER
• Development of a sustainable pattern of inquiry for long term assessment that will allow for shorter cycles of study over more requirements
• Consideration of tools that might support this sustainable strategy

See: Annual Report of the University General Education Committee to the University Academic Planning Council(Attachment D).

Outcome: UGEC adopted broad learning outcomes describing four domains of general education and worked on developing more sustainable approaches to GER assessment. These approaches leverage institutional surveys already being conducted, with the goal of complementing those large-scale instruments with focused examination of artifacts and direct assessment.

Strategy: (1) Artifacts of student learning were gathered from a variety of ESR courses and randomly sampled to reflect the range of students enrolled in the participating courses. These artifacts were examined by a group of raters who used a standardized rubric to evaluate the extent to which they show evidence that three of the four ESR learning outcomes have been met.
(2) A survey was administered in Spring 2012 to 2,900 students who met specific criteria: those participating in the artifact study discussed above; a randomly selected sample of students who had not yet completed the ESR and who had not and were not currently taking a Communication B or Quantitative Reasoning B course; and a randomly selected sample of students who had not completed the ESR but who were enrolled in a Communication B or Quantitative Reasoning B course. The survey is intended to elicit information regarding student attitudes about topics related to the ESR, an element of the fourth learning outcome for the requirement.

Status: Complete. For results and more information see the reports Assessing Essential Learning In Ethnic Studies Courses and Assessment of Essential Learning Outcomes of General Education Requirements at UW-Madison.

Goal: To evaluate the relevance of the information literacy learning outcomes identified when the General Education Requirements were established in 1994.

Outcome: Feedback from multiple sessions with Comm A teaching assistants and instructors suggested that the language used to describe student learning with respect to information literacy and technical skills must be updated. See more information in the report Modernizing Information Literacy Language in the General Education Requirements for Communication A Courses.

Next Steps: Updated language has been approved; the changes have been incorporated into training materials. The updated outcomes will be the focus of an assessment project focused on the efficacy of the Computerized Library User Education system.

Outcome:  The small sample size for this study limits the extent to which policy decisions should be made based on this study.  Nevertheless, the results suggest that (1) students who are required to take, and who complete, a Comm A course write better than students who are required to take the course but delay it; (2) students who complete their Comm A course write at levels comparable to those who exempt from the requirement; (3) that placement strategies are appropriately determining which students need to take Comm A to improve their writing skills, and which do not; and (4) that the English Placement test serves as the best measure (as compared to AP, IB, ACT, and SAT scores) for determining the quality of student writing when students enter the university. Final Report.

Next Steps:  The Communication A Subcommittee, the University General Education Committee, and the University Academic Planning Council concur that while this study was limited by the small size of the sample, this study does not suggest that students are failing to meet the learning outcomes associated with the written communication component of Comm A. Results will be shared with the campus community for advising and informational purposes.

Results: Academic planning and Analysis determined that only 2% of students take these courses out of sequence; of those most are taking the courses concurrently or await news concerning exemptions.

Next Steps: Lacking policy directing enforcement and little evidence that students are harmed by the practice, the UGEC determined that the implementation of enforced requisites need not be pursued at this time.

Goal:  Identify best-practices for assessment and administration of first-tier communication requirement, and, if possible, identify instruments used by peers to assess learning outcomes in this area.

Outcome:  The UW-Madison Communication A and B requirements are distinctive, and no clear parallels exist within the comparison group of very large-enrollment research intensive public universities.  Assessment strategies in use are either highly localized (course-based) or very general (large-scale standardized tests).

Next Steps: Although we hoped to identify an effect approach or tool for assessing student learning in this area which we might use, none were well suited to UW-Madison’s distinctive requirements, which define student learning in communication and critical thinking in several dimensions. As a result, a study focused on our local requirements was developed.

Strategy:  Structured discussion with ESR course instructors

Outcomes: In a series of ‘flash focus groups’ with ESR faculty, four learning outcomes were defined. These discussions created a foundation for future evaluation of student learning outcomes for these courses. In addition, three issues were identified by the faculty in the focus groups: (1) a campus-wide policy that General Education courses must be taken on a graded basis was implemented after it was discovered that students could take ESR pass/fail; (2) an online tool for sharing instructional resources has been developed to serve teachers of ESR courses; and (3) Academic Planning and Analysis conducted a study of course-taking patterns for students meeting ESR to help guide development of future assessment projects.

Next Steps:  Having developed a list of measurable learning outcomes for ESR courses, the faculty will in planning and undertaking an assessment study to evaluate student learning in the areas identified in the report.

Related Documents: 
Discussion of Essential Learning in Ethnic Studies Requirement Courses (March 11, 2010)
Analysis of How and When UW-Madison Undergraduates are Satisfying the Ethnic Studies Requirement (March 2011, Academic Planning and Analysis, Office of the Provost)

Results: Two courses were removed from QR-B course array; one course was substantially revised to meet criteria; all courses for which QR-A was not specifically set as s prerequisite were contacted and asked to set and enforce “satisfaction of QR-A” as a requisite. We anticipate that students will perform better in QR-B courses as a result of this change.

Strategy: Revise catalog text and other communication tools (website). The new statement locates General Education within the overall context of a Wisconsin Experience, and describes broad learning goals (or, “ways of knowing”) to be achieved in each of the three breadth areas.

Status: The Undergraduate Catalog entry for General Education has been published online.

Status: This project was completed in May 2008, when the long-range assessment plan was adopted by the UGEC.

Results: Although results revealed high level of preparation for students admitted to UW-Madison, sample size problems suggested the need to reevaluate use of this instrument and the study design.

Results: When compared to students who have not completed Comm-A courses, those who do complete Comm-A report improvements in learning. Furthermore, non-native speakers of English who must also complete Comm-A report equivalent gains, suggesting that they are not at a disadvantage. Although this method of self-report needs to be validated using a direct measure of student learning, prior experience with self-report in QR-A suggest that students can accurately gauge their learning gains.

Strategy: Web-based Survey

Results: Instructors teaching in areas of the curriculum that are regularly assessed report greater understanding of and value of breadth requirements. The study revealed a disconnect between divisional areas and highlighted the need to engage instructors in dialogue about liberal education and breadth. This study has informed communication about breadth requirements, including revision of catalog and advising information about breadth.
The report, “2006 UW-Madison General Education Requirements Survey” has been completed.

Next steps: The 2008 General Education Assessment Plan includes projects related to defining “breadth” areas and assessing student learning in those areas.

Strategy: Survey

Goal: Understand student perceptions of preparation for and learning in range of non-math, non-statistical, non-computational courses with respect to learning goals identified for Quantitative Reasoning.

Outcomes: Students report learning gains in mathematical proficiency; however, they are less confident of “reasoning” or “critical thinking” skills. This requires further student, preferably by looking at what students do in QR-B courses that are not focused on math/quantitative operations.

• Final Report (November 2006)

Results: Descriptive guidelines and student learning outcomes for courses meeting the ethnic studies requirement were established. Course syllabi were evaluated to calibrate course array to learning outcomes. The oversight and administration of the requirement were improved.

Goal: Follow up on issues revealed in focus groups

Study II: How the QR-A Requirement Affects Student Self-Assessments of Quantitative Reasoning Skills and Preparation for Future Courses

Principal Investigator: Charles N. Halaby, Research Director, General Education Assessment Council

Strategy: Pre-test/Post-test, Survey

Goal: Measure student learning in course in light of learning goals identified for Quantitative Reasoning A, using survey of student perception of skills and pre/post-test.

Outcome: Students who complete QR-A courses improve in mathematical proficiency more than students who don’t take this course. In addition, students accurately report their own impressions of that improvement, which suggests that surveys on this topic may serve as future tools for assessment in this area.

• Final Report (October 2005)

Goal: Gauge student awareness and understanding of General Education Requirements during their first semester.

Outcome: Plans are underway to improve communication about “breadth” and general education. Faculty and instructors have been surveyed to understand better their awareness of and communication about “breadth”. (See study in progress discussed above.)

Results: Reduced curricular redundancy in Comm-B course array; identified transfer credit issues. This was an administrative study which allowed the General Education Program to better manage limited resources for a costly requirement by reducing repetition and redundancy.

• Final Report (October 2002)

Strategy: Rubric-based analysis of student workbooks prepared during online research project.

Outcome: This study led to the adjustment of course criteria to improve student learning in the dimension of oral communication skills, and to provide better support for instructors who teach these skills to students. These results also help guide advisors, who can counsel students about which CommA courses might complement their prior courses. These generated substantial discussion within the University General Education Committee and the University Academic Planning Council (as discussed in the General Education Committee report to the UAPC, 6/26/03), and led to the two projects listed immediately above.

Goal: To obtain a better understanding of students (e.g., high school preparation, performance on standardized tests, choice of major) who must take QR-A courses and the impact the requirement has upon their careers at UW-Madison.