Chapter 1 Lessons Learned in Conducting a Systematic Review in Online Learning

In: Handbook of Research in Online Learning
Authors:
Michael Ahlf
Search for other papers by Michael Ahlf in
Current site
Google Scholar
PubMed Close
and
Sara McNeil
Search for other papers by Sara McNeil in
Current site
Google Scholar
PubMed Close
Type:
Chapter
Pages:
1–20
DOI:
https://doi.org/10.1163/9789004702813_001

Purchase instant access (PDF download and unlimited online access):

$40.00
Add to Cart
Redeem Access Token

Abstract

This chapter describes lessons learned while conducting a systematic review of online learning. We share an analysis of our experiences, focused on the tools we used, complications, and items of interest. We describe the processes used for each of the six stages in the systematic review. For each stage, we describe potential challenges and ways to address them. We discuss problems such as difficulties with mis-entered data within search engine results, confusion resulting from vagueness or missing data within sources, and problems locating authors’ biographical information. We recommend that researchers considering a systematic review perform a robust scoping process, make plans to handle unexpected issues with search results and sources, and resist the urge to alter protocols once the systematic review has started. Implementing these suggestions may help researchers avoid time-consuming or frustrating complications when conducting a systematic review.

  • Collapse
  • Expand
Cover Handbook of Research in Online Learning
E-Book ISBN:
9789004702813
Publisher:
Brill
Print Publication Date:
25 Sep 2024
Front Matter
Chapter 2 The Study of Adult Learners in Distance Education
Chapter 3 Theories and Models to Frame, Inform, and Stimulate Online Learning
Chapter 4 What We Know about Online Teacher Education
Chapter 5 A Responsive, Consultative Professional Development Model for Online Engagement
Chapter 6 Active Learning by Design
Chapter 7 An Integrative Review of Adult Learners’ Informal Learning in Online Settings
Chapter 8 Metatextual Dimensions of Multimodal Literacy Engagement
Chapter 9 21 Years of Worldwide Learning Online
Chapter 10 A Critical Review of Two Automated Adaptive Teaching Systems in Mathematics and Chemistry
Chapter 11 Automated Analysis and Visualization of Online Learning Experiences and Engagement
Chapter 12 Evidence-based Practices for Developing a Community of Inquiry
Chapter 13 Pre-service Teachers’ Preparation for Teaching Online
Chapter 14 Micro Investment, Big Reward
Chapter 15 Selecting Media and Methods to Facilitate Blended Learning
Chapter 16 The Impact of Digital Learning on Racially Minoritized and Poverty-Affected College Students
Chapter 17 Remote Academic Advising for Colleges and Universities
Chapter 18 Online Mentoring in the Digital Age
Chapter 19 Mental Fatigue in Online Learning
Chapter 20 Analysis of Patterns in Time for Evaluating Online Instruction
Chapter 21 The Difference between Emergency Remote Teaching and Online Learning
Chapter 22 Exploring the Impact of E-Texts on Student Learning in Online Education
Chapter 23 Supporting AI-Enhanced Active Online Learning Experiences
Chapter 24 Student Perceptions of Learning with Instructional AI Assistance in Online Discussions
Chapter 25 Gamification for Higher Education Applications
Chapter 26 Developing a Quality Assurance Approach for an Online Professional Military Education Institution
Chapter 27 Skills for Student Success in Online Learning
Back Matter

  • Ahlf, M., & McNeil, S. G. (2023). An exploratory review of literature on moderation in asynchronous discussions. Distance Education, 44(1), 137–161. https://doi.org/10.1080/01587919.2022.2150146

    • Search Google Scholar
    • Export Citation

  • Alexander, P. A. (2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/10.3102/0034654319854352

    • Search Google Scholar
    • Export Citation

  • Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology: Theory and Practice, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616

    • Search Google Scholar
    • Export Citation

  • Bearman, M., Smith, C. D., Carbone, A., Slade, S., Baik, C., Hughes-Warrington, M., & Neumann, D. L. (2012). Systematic review methodology in higher education. Higher Education Research and Development, 31(5), 625–640. https://doi.org/10.1080/07294360.2012.702735

    • Search Google Scholar
    • Export Citation

  • Bedenlier, S., Bond, M., Buntins, K., Zawacki-Richter, O., & Kerres, M. (2020). Learning by doing? reflections on conducting a systematic review in the field of educational technology. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic reviews in educational research (pp. 111–127). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_7

    • Search Google Scholar
    • Export Citation

  • Boland, A., Cherry, M., & Dickson, R. (Eds.). (2017a). Doing a systematic review: A student’s guide. Sage.

  • Boland, A., Cherry, M. G., & Dickson, R. (2017b). Carrying out a systematic review as a master’s thesis. In A. Boland, M. Cherry, & R. Dickson (Eds.), Doing a systematic review: A student’s guide (pp. 1–16). Sage.

    • Search Google Scholar
    • Export Citation

  • Bonk, C. J., Wisher, R. A., & Lee, J. (2004). Moderating learner-centered e-learning: Problems and solutions, benefits and implications. In T. S. Roberts (Ed.), Online collaborative learning: Theory and practice (pp. 54–85). IGI Global. https://doi.org/10.4018/978-1-59140-174-2.ch003

    • Search Google Scholar
    • Export Citation

  • Chong, S. W., Lin, T. J., & Chen, Y. (2022). A methodological review of systematic literature reviews in higher education: Heterogeneity and homogeneity. Educational Research Review, 35, 100426. https://doi.org/10.1016/j.edurev.2021.100426

    • Search Google Scholar
    • Export Citation

  • Coverdale, J., Roberts, L. W., Beresin, E. V., Louie, A. K., Brenner, A. M., & Balon, R. (2017). Some potential ‘pitfalls’ in the construction of educational systematic reviews. Academic Psychiatry, 41(2), 246–250. https://doi.org/10.1007/s40596-017-0675-7

    • Search Google Scholar
    • Export Citation

  • Decker, A., & McGill, M. M. (2019). A systematic review exploring the differences in reported data for pre-college educational activities for computer science, engineering, and other STEM disciplines. Education Sciences, 9(2). https://doi.org/10.3390/educsci9020069

    • Search Google Scholar
    • Export Citation

  • Evans, J., & Benefield, P. (2013). Systematic reviews of educational research: Does the medical model fit? British Educational Research Journal, 27(5), 527–541. https://doi.org/10.1080/01411920120095717

    • Search Google Scholar
    • Export Citation

  • Gough, D., Oliver, S., & Thomas, J. (2017). An introduction to systematic reviews (2nd ed.). Sage.

  • Karabulut-Ilgu, A., Cherrez, N. J., & Jahren, C. T. (2018). A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, 49(3), 398–411. https://doi.org/10.1111/bjet.12548

    • Search Google Scholar
    • Export Citation

  • Linnenluecke, M. K., Marrone, M., & Singh, A. K. (2020). Conducting systematic literature reviews and bibliometric analyses. Australian Journal of Management, 45(2), 175–194. https://doi.org/10.1177/0312896219877678

    • Search Google Scholar
    • Export Citation

  • Mallett, R., Hagen-Zanker, J., Slater, R., & Duvendack, M. (2012). The benefits and challenges of using systematic reviews in international development research. Journal of Development Effectiveness, 4(3), 445–455. https://doi.org/10.1080/19439342.2012.711342

    • Search Google Scholar
    • Export Citation

  • Martin, F., Ahlgrim-Delzell, L., & Budhrani, K. (2017). Systematic review of two decades (1995 to 2014) of research on synchronous online learning. American Journal of Distance Education, 31(1), 3–19. https://doi.org/10.1080/08923647.2017.1264807

    • Search Google Scholar
    • Export Citation

  • Martin, F., Dennen, V., & Bonk, C. (2023) Systematic reviews of research on online learning: An introductory look and review. Online Learning, 27(1), 1–14. https://doi.org/10.24059/olj.v27i1.3827

    • Search Google Scholar
    • Export Citation

  • Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers and Education, 159(September), 104009. https://doi.org/10.1016/j.compedu.2020.104009

    • Search Google Scholar
    • Export Citation

  • Mohamed Shaffril, H. A., Samsuddin, S. F., & Abu Samah, A. (2021). The ABC of systematic literature review: The basic methodological guidance for beginners. Quality & Quantity, 55, 1319–1346. https://doi.org/10.1007/s11135-020-01059-6

    • Search Google Scholar
    • Export Citation

  • Munn, Z., Peters, M., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, 143. https://doi.org/10.1186/s12874-018-0611-x

    • Search Google Scholar
    • Export Citation

  • Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology. Contemporary Educational Psychology, 25(1), 3–53. https://doi.org/10.1006/ceps.1999.1019

    • Search Google Scholar
    • Export Citation

  • Nicholson, J., McCrillis, A., & Williams, J. D. (2017). Collaboration challenges in systematic reviews: A survey of health sciences librarians. Journal of the Medical Library Association, 105(4), 385–393. https://doi.org/10.5195/jmla.2017.176

    • Search Google Scholar
    • Export Citation

  • O’Dwyer, L. C., & Wafford, Q. E. (2021). Addressing challenges with systematic review teams through effective communication: A case report. Journal of the Medical Library Association, 109(4), 643–647. https://doi.org/10.5195/jmla.2021.1222

    • Search Google Scholar
    • Export Citation

  • O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002

    • Search Google Scholar
    • Export Citation

  • Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

    • Search Google Scholar
    • Export Citation

  • Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Blackwell.

  • Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & Mcewen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. Research Synthesis Methods, 5(4), 371–385. https://doi.org/10.1002/jrsm.1123

    • Search Google Scholar
    • Export Citation

  • Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic review. Computers and Education, 144(March 2019), 103701. https://doi.org/10.1016/j.compedu.2019.103701

    • Search Google Scholar
    • Export Citation

  • Springer, L. (2023). The confusion around online learning terms. The EvoLLLution. https://evolllution.com/revenue-streams/distance_online_learning/the-confusion-around-online-learning-terms

    • Search Google Scholar
    • Export Citation

  • Tai, J., Ajjawi, R., Bearman, M., & Wiseman, P. (2020). Conceptualizations and measures of student engagement: A worked example of systematic review. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic reviews in educational research (pp. 91–110). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_6

    • Search Google Scholar
    • Export Citation

  • Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850

    • Search Google Scholar
    • Export Citation

  • Zawacki-Richter, O. (2020). Introduction: Systematic reviews in educational research. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic reviews in educational research (pp. v–xiv). Springer Fachmedien Wiesbaden. https://doi.org/https://doi.org/10.1007/978-3-658-27602-7

    • Search Google Scholar
    • Export Citation

  • Zhu, M., Sari, A. R., & Lee, M. M. (2020). A comprehensive systematic review of MOOC research: Research techniques, topics, and trends from 2009 to 2019. Educational Technology Research and Development, 68(4), 1685–1710. https://doi.org/10.1007/s11423-020-09798-x

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past 365 days Past 30 Days
Abstract Views 103 103 19
Full Text Views 0 0 0
PDF Views & Downloads 5 5 0