Rapid Development Studio: An Intensive, Iterative Approach to Designing Online Learning

Given the increasing demand for online learning at the tertiary level, there currently exists a need to modify or develop instructional design (ID) models/approaches that can effectively facilitate the collaboration between learning designers and teachers, as well as to research the effectiveness of these models/approaches. Against this backdrop, adopting a design-based research approach, we tested a practical ID approach that is developed on two prior models: rapid prototyping and collaborative course development. Accordingly, a 2-week rapid development studio—an agile, intensive, iterative ID process—was arranged. Data from multiple sources were gleaned during the study to generate a comprehensive and in-depth understanding of the proposed approach. Overall, results suggest that the approach is effective for developing online courses in case of a limited time frame and was positively perceived by both course instructors and learning designers. Moreover, practical tips for replicating the process in other contexts are also shared. It is our hope that the study will stimulate further exploration of alternative ID models/approaches to improve online course design efficacy in other higher education institutions.


Introduction
With the expansion of digital cultures and access to information and communication technologies (ICT), online learning has been increasingly integrated into the educational sector as a catalyst to transform teaching and learning, especially at the tertiary level (Broadbent & Poon, 2015;O'Shea et al., 2015;Zhang & Worthington, 2017). Meanwhile, research evidence (e.g., Clifton, 2017;George-Walker & Keeffe, 2010;Lai et al., 2017;Reiser & Dempsey, 2012) suggests that as a multifaceted process, development of effective online learning material benefits from design thinking in order to link potential affordances to pedagogy. Thus, explorations of effective instructional design (ID) for online learning have started to gain momentum (Bonk et al., 2018;Ifenthaler et al., 2018;Maybee et al., 2019;Richardson et al., 2016;Yang et al., 2016). However, current efforts are mostly scaffolded by traditional ID models, such as the Analysis, Design, Development, Implementation, and Evaluation model (ADDIE; Branson et al., 1975), the Attention, Relevance, Confidence, and Satisfaction model (ARCS; Keller, 1987), and the Dick and Carey (1978) model. Further, when employing these models, course instructors and learning designers often struggle to deliver well-designed online learning due to the inflexibility of these linear ID models (Adnan & Ritzhaupt, 2018). Such a dilemma is further exacerbated by the growing offering of online programs by higher educational institutions from around the world (e.g., Bates, 2018;McConnell, 2018). Thus, there is a growing interest in research on exploring alternative ID models/ approaches that allow for rapid production of online courses and efficient collaboration between learning designers and teachers, with some recent examples including the ABC learning design (Young & Perović, 2016), the Carpe Diem learning design (Salmon & Wright, 2014), and Course Design Intensives (Benfield, 2008).
Against this backdrop, adopting a design-based research approach, we, as course developers, tested an ID approach that combines the advantages of rapid prototyping (Tripp & Bichelmeyer, 1990) and collaborative course development (Hixon, 2008). According to Richey and Klein (2014), design-based research refers to "the systematic study of design, development and evaluation processes with the aim of establishing an empirical basis for the creation of instructional and non-instructional products and tools and new or enhanced models that govern their development." (p. 142) Though being a relatively new research scheme, a growing body of research (e.g., Anderson & Shattuck, 2012;Barbera et al., 2017;Gorard et al., 2004;Tracey, 2009) suggest that the participatory and iterative process can bridge the gap between theory and local practice by designing, testing, modifying, retesting, and redesigning contextualized educational interventions. This paper reports how we, as course developers, started the exploration from an initial concept, linked theory to practice, proposed the hybrid ID approach, and conducted a pilot operation within the context of a tertiary education institution. It is our hope that this development process might stimulate active exploration of alternative ID models/approaches to further improve online course design efficacy at the tertiary level and inform future research and practice.

Rapid Prototyping
The term rapid prototyping was initially used in the field of computer software design and engineering (Tripp & Bichelmeyer, 1990). It refers to a "process [which] involves quickly developing a prototype product in the very early stages of an instructional design project and then going through a series of rapid tryout and revision cycles until an acceptable version of the product is produced" (Reiser, 2001, p. 63). Prior research (e.g., Brown & Green, 2015;Desrosier, 2011;Nixon & Lee, 2001) suggests that when compared with traditional ID models, rapid prototyping exhibits the following four advantages: (1) It is especially suitable for designing computer-aided instruction; (2) Its expeditious process can deliver significant cost savings in time, resources, and labor; (3) It allows formative evaluation of the prototypes during the process; and, (4) It facilitates a co-inquiry model integrating direct criticism and feedback among multiple stakeholders.
For instance, adopting rapid prototyping for their projects, Jones and Richey (2000) found that designers were able to spend less time on revision, deliver the product within a shorter time frame, and communicate better with customers. Further, Shih et al. (2008) conducted an experiment to test the use of wiki-based rapid prototyping in designing teaching materials for e-learning grids. The results indicate that the process can reduce redundant human efforts significantly and produce high-quality teaching materials. Moreover, Salmon and Wright (2014) reported findings from their use of the Carpe Diem workshop model, which is essentially a rapid prototyping process. Their findings suggest that this approach can enable teachers to produce online courses quickly and that it should be employed for preparing staff for e-learning design.
However, rapid prototyping for ID is not without its drawbacks. Davidson-Shivers et al. (2018) pointed out that in a typical rapid prototyping scenario, course instructors are generally involved only at the early stage, while the fullscale development around the subject matter usually relies on collaborative efforts among instructional designers. Such disconnection often leads to repeated cycles of revision at the later stage of a project.

Collaborative Course Development
Before the advent of the internet, teachers were mostly responsible for developing their own teaching resources (Clark & Angert, 1981). However, with the rise of online courses, the need for collaboration among teachers and learning designers has become evident (Lowyck & Pöysä, 2001;Mor & Winters, 2007). This is especially true given the increasing awareness of the significance of ID, the affordances of ICT, and the variety of available digital pedagogies (Garrison, 2003). As a result, collaborative development has increasingly been adopted as an approach for developing online courses (Hixon, 2008;Xu & Morris, 2007). Recent studies suggest that collaborative course development could be a practical solution to combining learning designers' technical-pedagogical knowledge together with course instructors' expertise in practice, and Chao et al. (2010) found this particularly evident in the development of new courses "from scratch." Campbell et al. (2009) also determined that collaboration could bring about positive changes among learning designers and make them more active, while Martinez-Maldonado et al. (2017) found that face-to-face collaboration enhanced participants' mutual understanding and facilitated fluid group interactions. Nonetheless, despite the growing research interest in collaborative course development approaches, less is known about effective ways to substantiate such collaboration in practice. For example, collaborative course development is, in practice, mostly organized in the form of scheduled meetings with course instructors and learning designers working in different locations. This physical detachment often impedes the ID process, and can exacerbate common practical constraints such as limited time availability and tight budgets (Hixon, 2008).
Our review demonstrates that both rapid prototyping and collaborative course development have proven effective for online course development. Yet, neither approach offers a balanced solution to the joint constraints of time and space.
In view of this research deficiency, this study tested a practical way to combine these approaches for online course development. The following three research questions guide this study: (1) How can rapid prototyping and collaborative course development be combined for online course development as an ID approach? (2) How do participants of the study, as learning designers and teachers, perceive the ID process? (3) What shared reflections can be gleaned from this collaborative inquiry to inform the future team action?

Method
Considering its focus on course developers, the case study adopted a design-based research approach (Wang & Hannafin, 2005). Specifically, drawing upon prior research (e.g., Coetzee & Smart, 2012), we reported the process of developing a course with the help of a rapid development studio (RDS), which is conceptualized to be an agile, intensive, iterative ID process with a compressed timeframe and thus enables course instructors and learning designers to collaborate in close physical proximity.

Study Site and Participants
This study was conducted within a digital learning team (DLT) from an educational faculty at a New Zealand university, in collaboration with two partner lecturers. The DLT consisted, at the time of this study, of a group of six learning designers and two project assistants, responsible for consulting and collaborating on the design of blended undergraduate and postgraduate courses and special projects within the institution.
Since the implementation of Canvas (a learning management system) in 2017, the team has faced a challenging task to migrate existing courses hosted on prior learning management systems as well as to adapt traditional face-to-face courses for online or blended delivery. In recent years, amid competition from other tertiary providers, demand from the institution's leadership and academic staff for online course delivery as a mean to innovate learning and teaching has rapidly increased, aiming to diversify and increase student enrolments.
Against this background, the DLT, in partnership with two lecturers of a course, carried out the study. The two instructors were responsible for an existing face-to-face, postgraduate course on the subject of digital pedagogies for in-service teachers. Previously, the course bad been delivered in a 1-week, face-to-face block. In view of surging demand for professional development and critical inquiry into the use of digital pedagogies by in-service teachers in New Zealand, the course was identified as one to be made available for general enrolment for online study.

The Proposed Rapid Development Studio
Guided by prior research, the RDS was proposed. Its compressed timeframe requires quick, but incomplete, articulation of the desired end-goal, followed by rapid prototyping of the different components of the course. Equally important is the presence of the teaching staff member(s) throughout the RDS process. This can result in the creation of rapid prototypes with reduced need for revisions and rebuilds.
In terms of structure, the RDS is intended to consist of three phases. Phase 1 starts before the RDS takes place. In this phase, the course content is collated for redevelopment. Next, in Phase 2, multiple stakeholders collaborate to articulate course design needs and desired student experiences, and to develop solutions and content. Phase 3 is the evaluation or quality assurance stage, wherein the final product is reviewed and proofread before being delivered to students.

Data Collection
Following the recommendation of Wang and Hannafin (2005), data from multiple sources were collected. During the RDS, observations and field notes were taken by the project team and collated. After the RDS, an anonymous questionnaire survey was conducted within the course redevelopment team. The questionnaire consisted of eight open-ended questions designed to elicit participants' reflections on the RDS. The questions covered previous understanding of rapid development approaches, experiences of the pilot project, personal opinions on the approach, and suggestions for future modifications (see Appendix for details).

Research Ethics
Research ethics were maintained during this study. The data were gathered as a review of departmental practice and then handed over as deidentified secondary data which are exempt from ethics approval by the University of Auckland. Further, all participants were invited to contribute to the manuscript. They were not forced to participate in the writing and were assured that their identity would be strictly confidential under any circumstances. Hence, all participants were aware that the results of this study would not lead to any punitive consequences. All participants also gave verbal consent for their data within this dataset to be considered when they were invited to contribute to writing the article.

Data Analysis
When the RDS ended, the de-identified secondary data were analyzed within a thematic analysis framework. The data were coded into themes that captured an understanding of participants' perceptions of and experiences in the RDS. As Braun and Clarke (2006) point out, thematic analysis in qualitative research is widely used and offers rich opportunities to identify patterns and understandings that may emerge from subjective experiences of individuals. After analysis of the data, important patterns that were corroborated by respondents were identified. To ensure trustworthiness, the results were uploaded to Google Drive and shared among the participants for peer scrutiny and member checking (Nowell et al., 2017).

Implementation of the RDS
Considering the practical constraints (e.g., both the instructors and learning designers had considerable teaching, research, administrative, and other project commitments), the RDS was planned to be completed within 2 weeks. The detailed structure is presented in Figure 1. Before the RDS started, the course-related content including calendar, description, learning outcomes, weekly topics, assessments, reading list, and key activities were collected and disseminated among the project team and stored in a shared Google Drive to facilitate ongoing access.
Phase 1 (Day 1) of the RDS began with multiple stakeholders (e.g., members of faculty leadership responsible for learning and teaching strategy, and for expanding postgraduate enrolments) meeting with the project team and course instructors to develop a shared understanding of the context of the course, identify strategic and practical needs impacting the course design, and to brainstorm possible solutions. Guiding principles of the course were also discussed and determined.
Next, learning designers and course instructors met to discuss the current implementation of the course, reflect on student feedback, and instructor experience of prior delivery. This was expressed in terms of what had worked well and what could be improved, what students liked and disliked, and the challenges that the new context for the course's delivery might bring. Furthermore, two whiteboards were set up in the shared workspace. The "Responsibilities" board kept track of which particular team member had been assigned responsibility for which overall task (e.g., "content," "assessments," and "design"). The second board, "Progress," was established as a space for team members to update the current status of the project using post-it notes and simple status categories (e.g., "to do," "in progress," and "done").
In Phase 2 (Days 2-9), the project team and teaching staff conducted agile prototyping of course components (assessments, activities, and resources) in semi-structured and fluid small teams, using a tight iterative cycle of ideation, production, peer evaluation, and revision. Regular all-team check-ins, milestone monitoring, and peer evaluations maintained overall course design cohesion. Each day of this phase started with a focused meeting to share updates on progress, and to discuss obstacles. Meanwhile, the course content was outlined in a physical timeline on the floor of the shared workspace to visualize the course in its entirety. Differently colored sheets of paper represented different course components, and were laid out from left-to-right in columns, with each column representing 1 week of study. The scale of the map allowed for a snapshot view of the entire course but left enough room to describe specific topics and activities in more detail, and for multiple groups to interact with the map simultaneously. The choice of color to delineate course components enabled quick identification of balance and variety in those elements, and the timeline sequence structure enabled identification of rhythm, repetition, and progression of those elements. The ability to physically rearrange the elements enabled speedy communication and accelerated the iterative design process. Figure 2 presents how mediating artifacts were used in the workspace.
Finally, in Phase 3 (Day 10), quality control and inclusive design checks were conducted, and participants reflected on the overall process. Despite ongoing informal checks and meetings to ensure course continuity and cohesion, the speed at which activities or assignments were altered resulted in unanticipated impacts throughout the course. Thus, a cognitive walkthrough approach recommended by Mahatody et al. (2010) was adopted to check that the student workload was feasible and that the aims were met and scaffolded into timely and relevant tasks. Furthermore, a member of the design team performed a walkthrough from the student perspective, highlighting any inconsistencies and opportunities for further revision. As a result of these reviews, the decision was made to create a "Getting Started" document to orient learners, and set out expectations and protocols prior to the start of the course. Finally, a checklist for inclusive design was also applied and adjustments made where feasible.

Post-RDS Survey Results
Overall, the open-ended survey results suggest that the RDS is generally well perceived by both course instructors and learning designers. Specifically, when asked about the value of the RDS for online course development, all respondents agreed that the RDS demonstrated an effective approach for this type of online course development. For example, It is a collaborative engagement in an intensive project helps a team to commit to outcomes in a set timeline. (From Respondent E) Such value of the RDS is further reinforced by participants' willingness to engage in future iterations of this practical approach. When asked about their willingness to participate in another RDS, all participants responded positively and made relevant suggestions for managing a second deployment of the RDS. As some respondents clearly indicated below,

It can be a good technique to be added to a mix of approaches. (From Respondent E)
Repeating the process as it was, repeating the experiment with different stakeholders, repeating the experiment with different durations and with different outcomes. (From Respondent F) Furthermore, all participants viewed the RDS as a significant contributor to promote motivation, mutual trust, engagement and connection within the team.
I enjoyed the parts where we came together to share ideas, check on progress, debrief and collaboratively design the next steps. (From Respondent A) When responding to question about effective parts of the RDS, most participants identified with the physical visualization approach, in particular the timeline of the course This focus on the potentially transformative impact of making what are otherwise primarily digital developments in online platforms (which require project participants to actively access and monitor platforms on their computing devices) constantly visible and accessible in physical form is repeated numerous times in the data. One respondent further explains the significance of this element of the RDS's design as relating directly to questions of disembodiment in a professional role largely dedicated to virtual artifacts and experiences:

Another highlight for me [is] to be able to crowd around a physical representation of a virtual component of the course and negotiate, test, critique and develop ideas with the group, in the moment. (From Respondent B)
With regard to personal gains from the RDS, some participants pointed out that participation in the RDS helped increase their understanding of their colleagues' strengths and skills from participating in the RDS.
One of the best things was working as a whole team on a project, [which]  Evidence from the survey data therefore suggests that while the RDS might be well-suited to intensive, short-term projects, its impact on the internal culture and working style of the team might be ongoing and longer-term in nature.
Nevertheless, participants also highlighted points that need to be considered in the future.

Discussion
In order to combine rapid prototyping and collaborative course development, we adopted a design-based research approach and piloted the RDS in order to adapt a course for online delivery in 2 weeks. Overall, the outcome suggests that this agile and collaborative RDS can be an effective alternative model for ID, and worthy of further pursuit for course development projects with limited timeframes. Despite the overall positive feedback, there were participant concerns regarding personal space and ability to focus, suggesting that while the RDS might be productive and efficient as an ID approach, varied individual working styles, and personal needs should be considered as part of its implementation.
First, according to the results, the co-working arrangement of the RDS had a positive impact upon the team productivity. Similar to prior research on space and productivity (e.g., Bednář et al., 2021;Clarke, 2016;Mariotti et al., 2021), feedback from both learning designers and teachers indicated their preference for the arrangement, as it facilitated their interprofessional communication. This positivity may also be attributed to the fact that physical proximity allowed quick distribution of roles responsibilities, and direct communication between team members, and access to rapid peer feedback. The finding has implications for learning design team managers. As team leaders, they may consider devising innovative ways to put both learning designers and teachers together either physically or virtually in order to accelerate the course development process. This is especially the case when large amounts of existing courses might need to be adapted for online delivery during the COVID-19 pandemic (Moorhouse, 2020), and flexible or remote working arrangements continue for university staff in many countries. Just as online teaching requires reconceptualizing the learning environment as a virtual space of community and collaboration, learning design practices will benefit in the post-COVID 19 era by adapting to the possibilities offered by video-conferencing tools and other digital collaborative platforms.
Next, our exploration extends the previous findings on the benefits of utilizing mediating artifacts in similar projects (e.g., Agostinho, 2011;Buckley, 2015;Quintana & Tan, 2021). The materialized visualization offered by the course map proves to be a cogent tool to facilitate collaboration and promote efficiency. In our case, participants of the RDS showed positive perception of the physical course map, which guided the structural design of the course and the project whiteboards, which captured the development's progress. As previously suggested by Kaufman and Flanagan (2016), the findings show that using tangible artifacts as a physical representation of a digital design can help people make sense of "intricate, multi-variable systems with a central problem to solve, rules to learn, and constituent components to negotiate" (p. 2). Thus, it could be helpful for a learning design team to develop their agreed protocol on the use of mediating artifacts, as it could boost both productivity and creativity. Mediating artifacts could also have an important role to play outside of detailed redevelopment projects, and also be used effectively in more targeted learning design activities (e.g., the redesign of a particular module or assessment). The insights these artifacts offer educators about their curricula by making interdependencies, opportunities, and challenges more immediately legible suggests a wide range of contexts for their use by teachers and learning designers.
In addition to the positive feedback, the challenges we met echo Van Rooij's (2010) argument that effective project management strategies are required within ID projects. For example, before a similar ID project is started, its structure, timing, and aim should be well-defined, and shared in accessible formats with all participants in a development project. Also, as it is a labor-intensive sprinting process, activities (e.g., collaborative board games and paid lunches) may also be arranged to keep the team's morale and motivation at an ideal level. Further, a focus on project management dictates that teachers involved in such projects also need to prepare effectively for the ID process by investing time and effort prior to the project into collecting, arranging, and annotating the course content that needs to be adapted.

Limitations and Future Directions
Although empirical support is evident for the RDS, a number of limitations exist in the design of this study. First, this study was small-scale, largely qualitative, and the development was based upon an existing course, thus reducing the generalizability of the findings to other Australasian tertiary environments. Next, due to its primary focus on course developers, the study does not evaluate students' feedback on the redeveloped course. Future studies that address these two limitations would provide more insights on the effectiveness of the RDS approach in practice. Finally, considering the positive role of physical mediating artifacts in supporting collaboration and efficacy in the iterative design process, research efforts can also be made to ascertain the guidelines on how to use such tools.

Conclusion
This design-based study explored the instantiation of RDS, an ID approach based on rapid prototyping and collaborative course development. Our results revealed its effectiveness in rapidly adapting existing offline courses for online delivery. Moreover, involving teachers in both designing and development process can facilitate effective collaboration between learning designers and teachers. Additionally, given the compressed and accelerated nature of learning design practice within RDS, innovative ways may also be employed to encourage collaborative innovation, particularly in the era of the COVID 19 pandemic and a context of increased remote working.