Exploring the Relationship Between Online Lesson Design and Delivery Strategies, and Student-perceived Learning and Engagement in Polytechnics: A Mixed Methods Case Study

Article

Recommended Citation:
Tan Y. K., Pheang, G., Viswanathan, G., Chia C. Y., Ng, P., & Nurin, S. (2023). Exploring the relationship between online lesson design and delivery strategies, and student perceived learning and engagement in polytechnics: A mixed methods case study (Special Issue). Asian Journal of the Scholarship of Teaching and Learning, 13(1), 118-134.

View as PDF      Current Issue

Abstract

The COVID-19 pandemic had escalated the worldwide move to online learning. While many past studies had investigated and supported the impact of online lessons on student learning, some researchers have now emphasised the need to shift research focus to examine the relationships between online lesson design, and delivery strategies and student learning. This convergent mixed methods research aims to identify the design and delivery strategies that can facilitate student learning and engagement in online lessons. The study had a purposive sample of 3,712 Year 1 to 3 students recruited from two polytechnics in Singapore taking diverse modules with 25% or more of their lessons delivered online. Survey responses indicated favourable ratings on Perceived Learning, Engagement, and the Design and Delivery Strategies (i.e., Communication, Online Tools, as well as Interaction and Feedback) in online lessons. Pearson correlation analyses revealed significant moderate to strong positive correlations between the Design and Delivery Strategies, and Perceived Learning and Engagement. Student and staff focus group discussions validated and extended the key design and delivery considerations in effective online lessons. The findings inform the design and delivery of effective online lessons, and the validated instruments can be used to review the efficacy of online lessons.

Keywords: Online learning, design and delivery, engagement, perceived learning, mixed methods

Introduction

The COVID-19 pandemic had forced institutions worldwide to move to online learning. While the speed and scale of adopting online learning had been unprecedented, this mode of learning is not new. The advent of the World Wide Web and rapid advances in information and communication technology (ICT) have propelled the global growth in online learning (Means et al., 2013; Park & Shea, 2020). In Singapore, the Ministry of Education (MOE) embarked on its journey to leverage on technology for teaching and learning decades ago, with the first ICT Masterplan for Education being introduced in 1997 (Kong et al., 2014). In tandem with the increasing emphasis on online learning in Singapore, MOE identified designing quality online learning resources, provided professional development for educators and developed good practices for ICT in education as pertinent focuses in the 4th Masterplan for ICT in Education (MOE, 2015).

The proliferation of online learning has brought with it a need for more research to investigate the impact of online learning. Past studies had mostly focused on the efficacy of online learning and effective online lesson design in both K-12 and higher education (Lee et al., 2013; Lockman & Schirmer, 2020; Means et al., 2010; Means et al., 2013). The following sections review the empirical evidence on the impact of online learning, the effective design and delivery strategies in online lessons and the role of student engagement in online learning.

Impact of online learning

Online learning involves lessons delivered fully or partially online through an asynchronous or synchronous format (Lister, 2014). A blended lesson combines both online and face-to-face learning experiences (Means et al., 2010). Many studies have found positive learning outcomes from online learning or blended learning as indicated by measures such as test scores, passing rate, dropout rate, engagement with learning content, and perceived learning (Nguyen, 2015). A review of the research on online learning supports the view that online learning can be just as good as, or better than, traditional face-to-face learning (Means et al., 2010; Woldeab et al., 2020).

In a meta-analysis of 50 effects from 45 experimental and controlled quasi-experimental studies, Means et al. (2010) found that fully online lessons appeared as effective as conventional lessons. In addition, blended learning was more effective than both face-to-face and fully online lessons. However, Means et al. (2013) cautioned that the positive effects of the blended condition might be attributed to a combination of content, instructor and learner variables.

Lockman and Schirmer (2020) acknowledged the importance of comparing outcomes of online, blended and face-to-face lessons in early studies. However, they emphasised that the research today should focus on the strategies and conditions that facilitate student engagement, learning and satisfaction in online lessons. Hence, the current study aims to examine the impact and design of the online lesson components. 

Effective design and delivery strategies in online lessons

Effective online learning requires more than merely uploading existing course content to online platforms, responding to email messages, and introducing discussion topics on the internet (Means et al., 2013; Park & Shea, 2020). Jaggars and Xu (2016) suggested that the effectiveness of online learning may vary according to the course design, as well as how it is taught. Clearly, online course design and delivery require pedagogical strategies that create learning and engagement opportunities, facilitate students’ interaction with the learning content, other learners and instructors and provide timely feedback for learning (Gray & DiLoreto, 2016; Lockman & Schirmer, 2020; Park & Shea, 2020).

Unlike traditional face-to-face lessons, online lessons (especially when delivered asynchronously) have limited opportunities for students to seek immediate clarification. This highlights the importance of clear communication of course information, instructions for learning activities and content in online lessons. Lister’s (2014) qualitative meta-analysis of 17 studies involving undergraduate and graduate students and lecturers from peer-reviewed educational technology journals suggested that the provision of a clear course overview and introduction, including information about the course purpose, syllabus, schedule, assignments and grading, as well as samples of past assignments can provide clear guidance and direction for the learners.

The quality of instructional materials and resources is also important to engage and facilitate self-paced online learning. Hew’s (2016) qualitative analysis of data from participant observation and participants’ reflections from three top-rated Massive Open Online Courses (MOOCs) in programming, arts and design and literature found that these instructors used a wide variety of course materials including multimedia resources, discussion forums, webcasts, and quizzes. Hence, the appropriate and effective use of technology for teaching and learning can engage students and facilitate their learning. Research suggested that multimodal media learning objects can strengthen course interaction and student engagement (Chen & Williams, 2009).

Online learning can be a lonely process, especially when it focuses on independent learning with limited opportunities to discuss and collaborate with other learners. Therefore, active learning involving learner-learner interaction through discussion forums and online communication platforms, as well as opportunities for learner-instructor interaction can facilitate learning and build a cohesive online learning community (Gray & DiLoreto, 2016; Hew, 2016; Lister, 2014). These place demands on educators’ facilitation skills to keep learners engaged and focused on relevant issues in their online discussions, and ability to provide timely and useful feedback (Lister, 2014; Lockman & Schirmer, 2020).

Beyond quality learning materials and interactive activities, Lister (2014) highlighted the importance of having clearly specified course objectives and assessments; information and guidelines for assignments; use of self-assessments that offer immediate feedback; and educators’ monitoring of students’ learning and provision of timely feedback. Self-assessments allow the learners to review the content and check their level of understanding (Hew, 2016). Beyond assessing surface-level learning, Woldeab et al. (2020) emphasised the need to include critical thinking questions and assess transfer of learning to real-world case scenarios. The use of critical thinking questions would potentially vary across disciplines, intended learning outcomes and learning contexts.

These findings suggest that effective online lessons require clear communication of course information and instructions to guide learners. The learners are engaged through varied learning content and activities that are aligned to the learning outcomes; interactions with peers and instructors; and self-checks with feedback. These design and delivery strategies are aligned with comparable frameworks, such as the Quality Learning and Teaching (QLT) framework by the California State University (2022), and the Higher Education Rubric by Quality Matters (2020). While both frameworks are comprehensive, QLT was selected in the current study for its greater emphasis on Facilitation and Instruction, which may be critical for facilitating online learning amongst polytechnic learners.

The current study aims to establish whether these highlighted strategies identified from overseas contexts are applicable to local polytechnic education. Furthermore, this study aims to extend these findings by identifying the design and delivery strategies associated with student engagement and perceived learning. This is aligned with the shift in research from evaluating the effectiveness of online learning to identifying the evidence-based strategies that can inform and enhance the design and delivery of online lessons (Joosten & Cusatis, 2019; Lockman & Schirmer, 2020).

Role of student engagement in online learning

Researchers define student engagement in different ways. According to Dixson (2015), student engagement refers to students putting time, energy, thought, effort, and to a certain extent, feelings into their learning. Briggs (2015) described student engagement as the level of interest demonstrated by students, how they interact with peers during the course, and their motivation to learn the subject matter. The current study considers the three dimensions of student engagement: affective engagement refers to “students’ feelings about learning”, which reflects their intrinsic motivation; behavioural engagement refers to students’ “effort and persistence in schoolwork”; and cognitive engagement refers to “the cognitive strategies that students adopt and employ during the learning process” (Lam et al., 2014, p. 216).

Student engagement can impact students’ completion of a course, grades, retention and learning, and is considered by many educators to be an important condition in teaching and learning (Appleton et al., 2008; de Freitas et al., 2015; Martin & Bolliger, 2018). Research in higher education found student engagement to be critical in successful learning regardless of programme or instructional format (Trowler, 2010). Student engagement is necessary for learning both online and face-to-face, but may be especially important online where students often feel isolated and disconnected (Dixson, 2015).

Meyer (2014) shared that engagement activities can influence the level of student learning, including higher order thinking. Barkley (2010) emphasised that the activities should be based on active and collaborative learning using simple to more complex projects. He contended that it takes practice to move students from passive to active learners, and it is the instructor’s job to engage their students with the learning outcomes in mind. With the increased number of online lessons offered in institutes of higher learning (IHLs), it is critical to explore online learning strategies that enhance student engagement (Meyer, 2014). The current study examines the relationships between design and delivery strategies, and student engagement and learning.

Research questions

Polytechnics in Singapore offer modules with varying degrees of online learning. In the period of this study 2018-2019, both Nanyang Polytechnic (NYP) and Ngee Ann Polytechnic (NP) had implemented school-wide online learning initiatives in line with national and international directions in online learning. Specifically, NYP had targeted 30% of its course curricula to be delivered online, while NP had required, in each of its diploma courses, 25% of at least six core modules to be delivered online, and at least one fully online core module.

These online lessons were predominantly designed for asynchronous, self-paced learning. The online activities included reading and viewing narrated PowerPoint slides, videos and online resources; sharing perspectives, participating in discussions and completing learning tasks on various online platforms (e.g., Blackboard discussion board, Padlet, Google Slides, Google Classroom); learning through use of simulations; and checking understanding through e-quizzes.

While there are numerous studies on online learning, there seems to be limited published research on online learning in the polytechnic context in Singapore¹. Therefore, this study aims to shed light on the effectiveness of online learning in polytechnic education. And more importantly, to identify the effective design and delivery strategies in relation to student engagement and learning. The specific research questions are:

• Did the students perceive learning and engagement through the online lessons?
• What design and delivery strategies in online learning were associated with students’ perceived learning and engagement in online lessons?

Method

Research ethics approval

Approvals were obtained from the respective Institutional Review Boards at NYP and NP before the start of data collection.

Research design

This study used the convergent mixed methods design (Creswell, 2020). The online survey and focus group discussions (FGDs) were conducted around the same period towards the end of each semester. This allowed triangulation of the quantitative and qualitative findings. More importantly, the FGD student and lecturer responses could provide contextual details on the online lesson design and delivery strategies, and their potential connection with student engagement and learning.

Participants

Participants for online survey 

Participants consisted of 3,712 Year 1 to 3 students (M_age= 19.2 years; SD_age= 3.3 years) from two polytechnics in Singapore. Most of the participants were Year 1 students (n = 2487; 67.0%), followed by Year 2 (n = 985; 26.5%) and Year 3 (n = 240; 6.5%). There were 2,249 (60.6%) female and 1,463 (39.4%) male participants. This purposive sample was recruited from 56 diverse modules at both polytechnics with at least 25% of their lessons delivered online. These modules spanned across disciplines including health sciences, business, engineering, information technology, design, chemical and life sciences, humanities and social sciences, film and media studies and interactive and digital media. Most of the participants (n = 3323; 89.5%) took modules in which around 25 to 30% of their lessons were delivered online. The remaining 8.4% (n = 312) took 100% and 2.1% (n = 77) took 50 to 75% online modules respectively. These modules were either nominated by the Directors of Academic Schools or recruited through contacts of the researchers. Informed consent was obtained from the survey participants.

Participants for focus group discussions (FGDs) 

For triangulation of the quantitative findings, 103 students and 32 lecturers from different disciplines and levels in both polytechnics were interviewed in FGDs. Lecturers helped to invite their students to participate in the FGDs. Participation was voluntary and informed consent was obtained.

Materials

Online survey 

A 47-item online survey measured the participants’ Perceived Learning, Engagement and their ratings of the Design and Delivery Strategies used in the online lessons. Table 1 summarises what these scales measure, the sources of the survey items and reports the Cronbach’s Alpha ranging from 0.84 to 0.96, indicating strong internal consistency. The details on the development and validation of the Perceived Learning, Engagement and Design & Delivery Strategies scales are attached in Appendix A. These survey items used a 5-point scale, ranging from (1) Strongly Disagree to (5) Strongly Agree.

The FGDs with students extended the quantitative findings and highlighted that learning included module-specific knowledge and skills, as well as transferable skills, such as self-directed learning. Specifically, some students shared that online learning facilitated their acquisition of knowledge and skills: 

“…Initially, I don’t know how to use those kind[s] of software [i.e., Photoshop and Illustrator], but through the teacher's guides and everything, his guides were really thorough. [...] The teacher is very clear in his instruction. That usually help[s] me to use online software like Photoshop and Illustrator.” 

Beyond learning academic content, some students also recognised that online learning could promote self-directed learning and equip them with work skills for the future. Some comments illustrating these views include: 

“I feel that through online learning, we get to be more independent, so we don't have to rely on anyone.” 

“Remote learning helped to prepare you for the future. Let's say next time you want to learn something new, you can go online to pick up these skill[s] yourself.” 

Such feedback from students highlighted the potential to facilitate development of transferable skills through online learning.

Design and delivery strategies

Students also rated the Design and Delivery Strategies of the online lessons favourably with means ranging from 3.78 to 4.04 (Table 2). These mean ratings are significantly above 3 (the mid-point on a 5-point Likert scale) based on single-sample t-tests, and align with past findings on the importance of these instructional strategies in online lessons (e.g., Lister, 2014; Lockman & Schirmer, 2020). The next section examines the relationships between these design and delivery strategies, and student engagement and perceived learning. The students’ FGD responses could shed light on the relatively lower mean rating for Assessment and Feedback.

Correlations between design and delivery strategies, and student engagement and perceived learning

Pearson correlation analyses reveal significant moderate to strong positive correlations ranging from 0.56 to 0.86 between the Design and Delivery Strategies (i.e., Communication, Assessment and Feedback, and Online Tools and Interactions) and the student outcome measures, Perceived Learning and Engagement (i.e., Affective, Behavioural and Cognitive Engagement) (Table 3). Consistent with previous studies (Chen & Williams, 2009; Gray & DiLoreto, 2016; Hew, 2016; Lister, 2014; Martin & Bolliger, 2018), the current findings suggest the importance of Communications, Online Tools and Interaction, and Feedback in online lessons. Findings from the FGDs with the students and lecturers provided insights into the strategies that may contribute to student engagement and learning.

Communication 

In general, FGDs with the students and lecturers suggest that clear communications on the module learning outcomes, access and instructions for activities and use of relevant content can keep students engaged and facilitate their learning. Lecturers noted the need to provide clear purpose and instructions for the learning activities to ensure that students know what to do and remain on task. 

“I think it's essential to tell them clearly [...]: what's the purpose of doing this e-learning, and more importantly, how it benefit[s] you.” 

“The last thing students want to do is go in and fumble about, like 'what you expect me to do?'. That has to be spelled out as clear[ly] as possible.” 

Lecturers were mindful that some students "may just give up if they hit some obstacles" and may also be reluctant to reach out during the online activity and choose to wait until the "next lesson and tell you...".

Students also noted that clear deadlines kept them focused: 

“Our online assignments have deadlines. …It gives us a [greater] sense of urgency [...] Pushes us to rush things, get things done by the deadline and basically don't procrastinate.”

Lecturers emphasised that “...relevant content is very important. “What you have learnt, you can actually apply to the real-life scenario.” Beyond helping students to see the relevance of the content, students expected lecturers to have clear lesson structure and organisation. Some comments indicating these views were: 

“What they do is [well] scripted, meaning whatever they wanted to say was very well-planned, very well organised. [...], so it was very easy to absorb.” 

Some also needed varied examples for their learning: 

“...with programming ourselves and other [topics] like mechanics [...] something I had a little trouble visualising, unless you give me some examples on how to look at it and maybe I’ll have a better understanding. With more examples, it does help.”

Clear communication is critical, especially in asynchronous online lessons, in the absence of opportunities for immediate clarification with the lecturers. As Lister (2014) suggested, to provide clear guidance and direction for learners, online lessons need to ensure clarity of communications in key areas, including intended learning outcomes, instructions about access to online materials, expectations regarding online participation, and instructions for online learning activities and assessments.

Online tools 

Students’ responses in the FGDs also highlight how online tools may introduce interactivity and simulate real-world context to enhance their engagement and learning. Some students shared how the use of games engaged them and facilitated learning: 

“E-learning for her module, I think it’s quite fun because she let us play the game, so you have to answer the question and…those at the leaderboard, she will reward [us] with candies [...].”

The use of online 3D models may also facilitate student learning: 

“I learn how the oil is produced, how it’s formed and accumulate[d]. I also learn the drill ship parts. That one definitely learnt because we went to search for it. We have to go like, walk around and look at a model to see, and sometimes, the life-sized model may not be as useful, so having a 3D model would help.”

Lecturers also indicated gamifying learning experiences makes for more fun and engaging learning. The tool Kahoot was mentioned in several FGDs. These concurred with Kumar et al. (2019), who found that award-winning faculty emphasised the importance of authentic and relevant learning materials, and use of varied multimedia resources. These could enhance student engagement and support their learning. A few lecturers shared that they would like to include games and fun elements, but felt they lacked the technical skills to develop games for learning.

Online interaction 

Lecturers preferred to use collaborative tools like Google Docs and Slides that allow students to contribute to the content creation process rather than just consume content passively. As one lecturer put forth, 

"You can be physically present but mentally be switched off (F2F lectures). Here (online activities)...everyone can see if they don't make the effort." 

Lecturers also indicated the value of collaborative tools like Google Slides that allow even the vocally shy students to contribute and learn from each other. Online collaboration can also facilitate peer learning. A student shared, 

“I think it would be helpful because collaborating online would not only give us different perspectives on how to finish the e-learning but it will also give us an opportunity to expand what we already know and learn from what we don’t know.” 

These comments illustrated how intentionally building in online interaction in one’s learning activities can enhance learner engagement, create a safe space for those who are shy to participate, and provide the opportunity for collaborations and peer learning in an online learning environment. The effective design and facilitation of online collaboration needs to consider multiple factors, including the alignment of the collaborative task with the course’s intended learning outcomes; learners’ profile, readiness and perceptions of online communication and collaboration; relationships among the learners (Haythornthwaite, 2008). While these factors are not unique to online collaborations, there may be specific contextual considerations. For example, the use of technology can make communications both visible and permanent (e.g. responses in learning platforms, recorded lessons), which may discourage online participation. With the prevalence of online learning, Haythornthwaite (2008) emphasised the focus on how we can strengthen computer-mediated communication and collaboration, and not dwell on comparing its relative merits against in-person communication and collaboration.

Assessment and feedback 

Lecturers emphasised the need for formative assessment to ascertain students’ learning. From the students and staff FGDs, common formative assessment methods included online quizzes and reflection activities. A student noted, 

“It's really beneficial because most of the time, my lecturer will send out quizzes [...] and it's very good because the way I learn is not through reading and memorizing [...], but through practicing. I can re-attempt as many times as I want. It really gets drilled into my head.” “...We worked in [...] groups of, I think five or six. We submit the [5-minute YouTube] video and then she will grade it… was quite tedious actually, but it’s quite interesting also. It was a good experience.”

Despite the usefulness of formative assessment, it is critical to reconcile these with students’ expectations for timely (even immediate) feedback that has detailed and clear explanation.

“...it’s better if they provide the solution right after we submit [...] so that we can better understand where we went wrong [...]” 

While most online quizzes provide immediate automated answers, students needed more detailed explanations to help them understand the content: 

“You only know whether it's correct or wrong. Most of [the] time, [...] [t]hey don't really explain in-depth where you went wrong. It's for you to understand. If let's say you don't even know the question, you won't even understand the answer.”

Some students also appreciated individualised feedback: 

“For feedback that she gave, it was also very explicit. Like she will really take time to [...] give individual feedback on what could have been done better, how she liked your answer or not. That was very helpful.” 

A few lecturers also concurred that beyond assessing and addressing learning gaps, online feedback help students "feel that you care about the e-learning" and that "doing remote learning, it's not a removed learning".

FGD responses suggested that timely and useful feedback can contribute to student learning and engagement. However, lecturers shared their challenges in providing timely feedback in asynchronous lessons. 

“I guess when they do it on self-paced, unless I’m following the session, I can’t give them immediate feedback. It's only after that. Usually in class, when we do an exercise using the software, whenever they get into some problem, they did something wrong and it doesn’t look right, they get the immediate feedback.[...]” 

Many lecturers also indicated that they "validate learning" at the next F2F lesson using quizzes or activities to "re-cap learning" post online session.

The correlational findings highlight that Communication, Online Tools and Interactions, and Assessment and Feedback are critical factors that may predict student learning and engagement. These are partially consistent with Joosten and Cusatis’ (2019) findings which suggest that design and organisation, student interaction with the instructor, and content design and delivery predict learning. However, they found that learner support, interaction with peers, and assessment were not significant predictors. Clearly, more studies across different learner and lecturer characteristics and contexts will extend and strengthen these findings.

Limitations

While this study contributes to the relatively few published research on the effectiveness of online learning for polytechnic courses in Singapore, it is not without limitations. Although objective measurement of student learning is deemed to be important by some researchers (Meyer, 2014), this study did not use objective measures of student academic performance. Students’ grades across diverse subjects assessed by different lecturers in the two polytechnics were likely to be influenced by multiple variables (e.g., assessment design, scope and level of difficulty), not just the online lessons. Hence, the current study measured students’ perceptions of learning from the online lessons to facilitate the consolidation of data across modules in the two polytechnics. This is supported by Robinson and Hullinger (2008) who proposed that evaluation of the impact of online learning should go beyond objective student outcomes (e.g., test results, grades) to examine subjective student learning experiences.

As the study was based on a convenient sample from two polytechnics and participation in the online survey and FGDs were voluntary, it is not clear the extent to which the findings could be generalised to local polytechnic students. Nevertheless, the key considerations deemed critical in designing and delivering effective online lessons are consistent with existing research findings. This suggests that these online lesson design and delivery strategies and considerations may be generalisable to polytechnic contexts, and perhaps higher education in general. The exploration of how online learning can facilitate professional skills development may be pertinent for polytechnic education, with its emphasis on applied learning. Future studies that examine the complex interplay across online instructional features, student and instructor characteristics and educational contextual factors may be needed to advance the current findings (Lockman & Schirmer, 2020). 

The current study garnered staff’s perspectives through FGDs, which provided opportunities for clarification and elaboration. The online survey was not administered for staff in the current study, which could otherwise allow for triangulation with the findings from the student online survey.

Conclusion

This study found moderate to strong relationships between the Design and Delivery Strategies and students’ Perceived Learning and Engagement in the online lessons. The findings indicated that instructors of online lessons need to learn various strategies in online communications, use of online tools and interactions and provision of feedback to facilitate student learning and engagement online.

The spread of COVID-19 has propelled digital transformation in higher education, with increasing demands on lecturers to design and deliver lessons online. Instructors of online lessons need different knowledge and skillsets (Woldeab et al., 2020). Professional development is critical in building educators’ competencies to be effective designers of learning who purposefully integrate education technology into their lessons. Specifically, referencing staff’s responses in the interviews, the training may focus on strategies and skills to strengthen online engagement and interaction through games, immersive learning experiences and collaborative learning. There is also a need to explore strategies to facilitate assessment for learning and timely feedback in online learning. These findings highlight evidence-based strategies that can inform staff training.

The effectiveness of online learning might vary based on how the online lessons are designed and conducted (Jaggars & Xu, 2016). As more educators move their lessons online, incorporating processes to review the online lessons will help them enhance their online lesson design and delivery. The validated Perceived Learning, Engagement and Design and Delivery Strategies scales that referenced the California State University’s (2019a) domains for the evaluation of online courses may be used by educators to review the efficacy and quality of their online lessons and identify areas for improvement.

Acknowledgements

The authors would like to thank Dr. Cheung Wing Sum, Associate Professor, National Institute of Education, Singapore and Dr. Zachary M. Walker, Associate Professor, University College London for evaluating and providing feedback on the initial survey items on Design & Delivery Strategies, Perceived Learning and Engagement. Also, the authors thank all Nanyang Polytechnic and Ngee Ann Polytechnic faculty members and students who participated in this study.

Funding

This study is funded by the Singapore Ministry of Education Tertiary Education Research Fund (Project MOE2017-TRF-033). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Singapore Ministry of Education.

Endnote

1. Polytechnic education in Singapore focuses on applied and practice-based learning through hands-on learning experiences and internships with industry partners (Ministry of Education, 2022).

Appendix A. Scale Development and Validation of Online Survey

References

Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369-386. http://dx.doi.org/10.1002/pits.20303

Barkley, E. F. (2010). Student engagement techniques: A handbook for college faculty. San Francisco: Jossey-Bass.

Briggs, A. (2015, February 11). Ten ways to overcome barriers to student engagement online. Retrieved from Online Learning Consortium: https://onlinelearningconsortium.org/news_item/ten-ways-overcome-barriers-student-engagement-online/

California State University. (2022). CSU Quality Learning & teaching (QLT) Rubric, 3^rd edition. Retrieved from https://ocs.calstate.edu/sites/default/files/2022-01/QLT-Rubric-3rd-Edition-2022.pdf

California State University. (2019a). Quality Learning and Teaching (QLT) Program Background. Retrieved from http://courseredesign.csuprojects.org/wp/qualityassurance/qlt-informal-review/

California State University. (2015). Quality Online Learning and Teaching (QOLT) Instrument. Retrieved from https://drive.google.com/file/d/0BxN4M6qCVbDPOEl0d1dKWmFXOEk/view

Chen, H-L., & Williams, J. P. (2009). Use of multi-modal media and tools in an online information literacy course: Course students’ attitudes and perceptions. Libraries’ and Librarians’ Publications, 1-34. http://dx.doi.org/doi:10.1016/j.acalib.2008.10.021

Creswell, J. W. (2020). Educational research: Planning, conducting and evaluating quantitative and qualitative research (6th ed.). Pearson Education.

de Freitas, S. I., Morgan, J., & Gibson, D. (2015). Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. British Journal of Educational Technology, 46(3), 455-471. http://dx.doi.org/10.1111/bjet.12268

Dixson, M. D. (2015). Measuring student engagement in the online course: The Online Student Engagement Scale (OSE). Online Learning, 19(4). Retrieved from http://files.eric.ed.gov/fulltext/EJ1079585.pdf.

Gray, J. A., & DiLoreto, M. (2016). The effects of student engagement, student satisfaction, and perceived learning online learning environments. NCPEA International Journal of Educational Leadership Preparation, 11(1). Retrieved from http://files.eric.ed.gov/fulltext/EJ1103654.pdf

Haythornthwaite, C. (2008). Facilitating collaboration in online learning. Online Learning, 10(1). doi: http://dx.doi.org/10.24059/olj.v10i1.1769 

Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS? British Journal of Educational Technology, 47(2), 320-341. http://dx.doi.org/10.1111/bjet.12235

Jaggars, S. S., & Xu, D. (2016). How do online course design features influence student performance? Computers & Education, 95, 270-284. Retrieved from http://dx.doi.org/10.1016/j.compedu.2016.01.014

Joosten, T., & Cusatis, R. (2019). A cross-institutional study of instructional characteristics and student outcomes: Are quality indicators of online courses able to predict student success? Online Learning, 23(4), 354-378. http://dx.doi.org/10.24059/olj.v23i4.1432

Kong, S. C., Chan, T-W, Huang, R., & Cheah, H. M. (2014). A review of e-learning policy in school education in Singapore, Hong Kong, Taiwan, and Beijing: Implications to future policy planning. Journal of Computers in Education, 1(2-3), 187-212. http://dx.doi.org/10.1007/s40692-014-0011-0

Kumar, S., Martin, F., Budhrani, K., & Ritzhaupt, A. (2019). Award-winning faculty online teaching practices: Elements of award-winning courses. Online Learning, 23(4), 160-180. http://dx.doi.org/10.24059/olj.v23i4.2077

Lam, S.-f., Jimerson, S., Wong, B. P. H., Kikas, E., Shin, H., Veiga, F. H., ... Zollneritsch, J. (2014). Understanding and measuring student engagement in school: The results of an international study from 12 countries. School Psychology Quarterly, 29(2), 213-232. http://dx.doi.org/10.1037/spq0000057

Lee, Y-H., Waxman, H., Wu, J-Y., Michko, G., & Lin, G. (2013). Revisit the effect of teaching and learning with technology. Educational Technology & Society, 16(1), 133-146.

Lister, M. (2014). Trends in the design of e-Learning and online learning. MERLOT Journal of Online Learning and Teaching, 10(4), 671-680. Retrieved from http://jolt.merlot.org/vol10no4/Lister_1214.pdf

Lockman, A. S., & Schirmer, B. R. (2020). Online instruction in higher education: Promising, research-based and evidence-based practices. Journal of Education and e-Learning Research, 7(2), 130-152. http://dx.doi.org/10.20448/journal.509.2020.72.130.152

Martin, F., & Bolliger, D.U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205-222. http://dx.doi.org/10.24059/olj.v22i1.1092

Means, B., Toyama, Y., Murphy, R., & Bakia, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115(3), 1-47. Retrieved from https://www.sri.com/sites/default/files/publications/effectiveness_of_online_and_blended_learning.pdf.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, D.C.: U.S. Department of Education. Retrieved from https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

Meyer, K. (2014). Effects from student engagement online. In K. Meyer (Ed.), Student engagement online: What works and why: ASHE Higher Education Report, 40(6), 67-73. http://dx.doi.org/10.1002/aehe.20018

Ministry of Education. (2015). Masterplan 4 for ICT in Education. Retrieved from https://ictconnection.moe.edu.sg/masterplan-4

Ministry of Education. (2022). Overview of Post-Secondary Education Institutions (PSEI). Retrieved from https://www.moe.gov.sg/post-secondary/overview

Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of Online Learning & Teaching, 11(2), 309-319.

Park, H., & Shea, P. (2020). A review of ten-year research through co-citation analysis: Online learning, distance learning and blended learning. Online Learning, 24(2), 225-244. https://doi.org/10.24059/olj.v24i2.2001

Quality Matters. (2020). Specific review standards from the QM Higher Education rubric, sixth edition. Retrieved from https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric

Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101-108.

Trowler, V. (2010). Student engagement literature review. York: Higher Education Academy. Retrieved from https://www.heacademy.ac.uk/system/files/studentengagementliteraturereview_1.pdf

Woldeab, D., Yawson, R. M., & Osafo, E. (2020). A systematic meta-analytic review of thinking beyond the comparison of online versus traditional learning. e-Journal of Business Education & Scholarship of Teaching, 14(1), 1-24. Retrieved from http://ejbest.org/upload/e-JBEST%201a%20Woldeab,%20Yawson%20&%20Osafo%20%2014(1)%202020.pdf

About the Corresponding Authors

Mr TAN Yew Kong is a Senior Education Fellow (Education Research) at the Centre for Learning and Teaching Excellence, Ngee Ann Polytechnic. His current professional and research interests include practicum coaching, data analytics, and pedagogical research. 

He can be reached at TAN_Yew_Kong@np.edu.sg.


Ms Grace PHEANG is a Lead Specialist, Teaching and Learning (T&L) at the Centre for Teaching and Learning Development, Nanyang Polytechnic. Her current professional and research interests include ICT-enabled teaching and learning, flipped learning, and staff professional development. 

She can be reached at grace_pheang@nyp.edu.sg.