Design Lab: Combining Learning, Innovation and Practice in Curriculum Design for Distance Learning Programs
In the ongoing management and iterative development of the OU Design Lab, the empowerment of typically un-empowered participants. Through much of the reorganization of staff and changes in curriculum practice and process, intentional efforts have been made to increase the sense of Agency of the individual participants, as adherence to formal, generic processes has faded to include more collaborative working sessions and goal oriented projects. Ahearn, (2001) provides a deep analysis on agency that results in a usable but incomplete but provisional definition of Agency as “the socioculturally mediated capacity to act”
Much of the administrative curriculum staff have been underutilized in terms of talents and skills as a result of the rigid and outdated structure of the many teams that made up the larger department. As a result frustrations and lowered morale have developed in conjunction with other aforementioned organizational changes in combination with outside pressures. The social constructivist approach of the Design Lab, allows for some release from the constant drudgery and environmental pressures of the previous methods of practice.
Assessment
The Design Lab project will be assessed through using both direct and indirect measures of assessment, analyzed both quantitatively and qualitatively. Curriculum design interventions created in the OU Design Lab, and implemented in the classroom will be evaluated based partly upon data collected from student activity and outcome-achievement data collected within the class. Understanding that the effectiveness of curriculum interventions can be assessed in a variety of ways and through a variety of lenses and theoretical constructs faculty will have the ability to construct and measure against data they deem particularly important to their defined issue and within their particular discipline.
Regardless of the specific methods used by faculty to assess the success of course-specific interventions, two essential components of the process, will be an operational focus for Design Lab Participation. First, all measures and assessments will be co-created by faculty, staff and leadership. Doing so will help encourage ownership of the valuation and examination of quality of design from a measure that represents all stakeholders involved and invested in course design and development. The second aspect of focus for the evaluation of course design quality is that the measures chosen, be consistently applied and continually evaluated resulting in iterative improvement. Ensuring that the co-designed rubrics will not remain completely static, provides an opportunity to both address shortcomings of the measure and also to increase the effectiveness and utility of the rubrics over time. More importantly it provides a Boundary Object for working between communities of faculty and administration.
Assessment across the Curriculum
In order to understand the way student performance is measured and analyzed in the context of the Design Lab, a short overview of the model of assessment at Online University will be provided. The learning outcomes model at Online University aligns closely with Kuh and Ewell (2010) who insist that “…colleges and universities must become smarter and better at assessing student learning outcomes, at using the resulting data to inform resource allocation and other decisions and at communicating to their constituents how well they are performing” (pg. 3). With this goal as a foundational premise, Online University has invested a significant amount of time and energy into assessment tools and practices.
Figure 3.1 below shows a Curriculum Map of Learning Outcomes for the Bachelors of Arts in Instructional Design at Online University. Program Learning Outcomes are listed across the top axis while the course number, title and sequence of progression are listed on the the vertical axis. Within each cell of the matrix are a one of three color coded boxes representing whether a learning outcome is being initially Introduced (Green), continually Reinforced (Blue) or finally Mastered (Red). With this simple visual tool, faculty are able to demonstrate to students, accreditors and program reviewers, exactly how programs are designed to build knowledge intentionally over time.
Table 3.1- Bachelors of Arts in Instructional Design
Throughout the student’s progression in the program, a specific Program Learning Outcome such as the following from EDU120: “Apply universal design principles to instruction”, will be introduced within the first few courses in the program through some sort of an assignment or activity. The PLO is then continuously reinforced in following classes through any number of assignments including formal facilitated discussions, written assignments, quizzes and through the creation of authentic artifacts of learning. Finally, the summative assessment for the program, the Capstone course measures the mastery of each of the program learning outcomes.
Assessment of Design Interventions
Design interventions created in the Design Lab, will be measured in two main ways, primarily through rubrics developed by faculty. One of the central and most powerful tools that will be used to measure the impact of design interventions is the Learning Outcomes Assessment (LOA) system at Online University. The proprietary LOA tool, is not only the tool faculty interface with for tasks related to assessment and grading within Online University, but it aggregates and stores valuable student achievement data which has been essential in establishing a foundation and culture of assessment that is deeply engrained in the design and development process and university culture as a whole.
Effective Learning Outcome Assessment combined with intentional mapping and alignment of learning outcomes within courses and programs are the backbone of the curriculum design model at Online University. Utilizing the Backward Design framework described by Wiggins & McTighe (2005), all courses and programs are designed to scaffold learning and measure outcome achievement of faculty designed Learning Outcomes for each academic program and its compositional courses
Development and revisions of courses and assignments are accompanied by development of rubrics and grading criteria that are then loaded into the LOA tool through a back end interface. On the front end, students submit work and faculty use the system to evaluate and comment upon the student work. Figure 3.3 shows the interface that faculty use in the assessment of student work. The rubric criteria on the left hand side of the image contain specific criteria, designed under the direction of the fulltime faculty and used by all faculty for grading and LOA assessment.
Figure 3.3 Front End of Rubric-based Assessment of Student Work at Online University.
Faculty Developed Quality Rubrics
In order for the participants of the Design Lab to have a common conception of the criteria for quality within courses and programs, an initiative has been undertaken to construct specific design and quality rubrics. Several rubrics have already been created through a process of faculty driven committees charged with helping to improve the quality of curriculum through targeted application of the measure and subsequent redesign of course assignments, assessments and discussion questions. Over the course of several months, faculty led groups worked diligently to discuss and evaluate multiple perspectives on course curriculum quality, narrowing down to individual rubrics which target different course elements. By empowering the faculty to more specifically define what is meant by quality courses and programs, an increased sense of curriculum ownership and a closer association of the faculty role with the identity of curriculum development practitioner, has been less difficult than anticipated.
