research literature nursing

Nursing: Literature Review

• Required Texts
• Writing Assistance and Organizing & Citing References
• NCLEX Resources
• Literature Review
• MSN Students
• Physical Examination
• Drug Information
• Professional Organizations
• Mobile Apps
• Evidence-based Medicine
• Certifications
• Recommended Nursing Textbooks
• DNP Students
• Conducting Research
• Scoping Reviews
• Systematic Reviews
• Distance Education Students
• Ordering from your Home Library

Good Place to Start: Citation Databases

Interdisciplinary Citation Databases:

A good place to start your research  is to search a research citation database to view the scope of literature available on your topic.

TIP #1: SEED ARTICLE Begin your research with a "seed article" - an article that strongly supports your research topic.  Then use a citation database to follow the studies published by finding articles which have cited that article, either because they support it or because they disagree with it.

TIP #2: SNOWBALLING Snowballing is the process where researchers will begin with a select number of articles they have identified relevant/strongly supports their topic and then search each articles' references reviewing the studies cited to determine if they are relevant to your research.

BONUS POINTS: This process also helps identify key highly cited authors within a topic to help establish the "experts" in the field.

Begin by constructing a focused research question to help you then convert it into an effective search strategy.

• Identify keywords or synonyms
• Type of study/resources
• Which database(s) to search
• Asking a Good Question (PICO)
• PICO - AHRQ
• PICO - Worksheet
• What Is a PICOT Question?

Seminal Works: Search Key Indexing/Citation Databases

• Google Scholar
• Web of Science

TIP – How to Locate Seminal Works

• DO NOT: Limit by date range or you might overlook the seminal works
• DO: Look at highly cited references (Seminal articles are frequently referred to “cited” in the research)
• DO: Search citation databases like Scopus, Web of Science and Google Scholar

Web Resources

What is a literature review?

A literature review is a comprehensive and up-to-date overview of published information on a subject area. Conducting a literature review demands a careful examination of a body of literature that has been published that helps answer your research question (See PICO). Literature reviewed includes scholarly journals, scholarly books, authoritative databases, primary sources and grey literature.

A literature review attempts to answer the following:

• What is known about the subject?
• What is the chronology of knowledge about my subject?
• Are there any gaps in the literature?
• Is there a consensus/debate on issues?
• Create a clear research question/statement
• Define the scope of the review include limitations (i.e. gender, age, location, nationality...)
• Search existing literature including classic works on your topic and grey literature
• Evaluate results and the evidence (Avoid discounting information that contradicts your research)
• Track and organize references
• How to conduct an effective literature search.
• Social Work Literature Review Guidelines (OWL Purdue Online Writing Lab)

What is PICO?

The PICO model can help you formulate a good clinical question. Sometimes it's referred to as PICO-T, containing an optional 5th factor. 

Search Example

• << Previous: NCLEX Resources
• Next: MSN Students >>

• Last Updated: Apr 9, 2024 1:30 PM
• URL: https://guides.himmelfarb.gwu.edu/Nursing

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 3, Issue 3
• Users' guides to the nursing literature: an introduction
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Nicky Cullum , RN, PhD
• Centre for Evidence Based Nursing, Department of Health Studies University of York, York, UK

https://doi.org/10.1136/ebn.3.3.71

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction to critical appraisal

Evidence-based nursing (EBN) means using the best available evidence from research, along with patient preferences and clinical experience, when making nursing decisions. 1 Nurses are increasingly concerned about ensuring that care is research based, and EBN offers a strategy to help nurses achieve this goal by using 5 steps:

Step 2: translating these areas of uncertainty into focused, searchable questions 2

Step 3: searching the literature for studies that use appropriate designs to help answer the question 3 – 6

Step 4: critically appraising the research

Step 5: changing practice if the research suggests this is necessary.

Previous Notebooks in Evidence-Based Nursing that have described the process of EBN 2 – 6 were written with busy clinical nurses in mind. The next series of Notebooks will explore step 4 in detail—how to assess whether research findings are valid and appropriate for clinical application.

The volume of healthcare literature has been described in graphic terms. Did you know, for example, that Medline has indexed more than 9 million citations in more than 4,000 journals since 1966? 7 Clearly, no individual practitioner can read this volume of research; nor should they try, as only a small proportion of it is of good quality. The trick to keeping abreast of the latest research is to quickly filter the good from the bad. For example, staff of the evidence-based journals office recently calculated that of 136 journals read for 4 evidence-based journals ( Evidence-Based Nursing , Evidence-Based Mental Health , Evidence-Based Medicine , and ACP Journal Club ), an estimated 8% of 21 100 articles passed evidence-based methodologic filters (personal communication, A McKibbon, 20 Mar 2000).

Anyone who has designed or implemented a research study knows that it can be difficult to do well. Researchers, like the rest of us, like to spread “news” and may exaggerate the importance and strength of the findings of their own research. The clinician is faced with the challenge of filtering the thousands of articles published each year into manageable information. This challenge is increased because researchers are notoriously bad at communicating in a language that is accessible to anybody other than researchers. It is not surprising that time and again, nurses tell researchers that they find it extremely difficult, if not impossible to use research findings in their practice.

But don't despair! EBN is possible because (1) there are now a number of clinical evidence sources which present appraised and summarised research for clinicians; (2) good search strategies can identify research that has used an appropriate design to answer your particular clinical question; and (3) critical appraisal is easy to learn, becomes easier with practice, and may be made easier with the Users' Guides series, which will appear in upcoming issues of Evidence-Based Nursing .

Sources of pre-appraised clinical evidence

Clinicians are no longer expected to fend for themselves when trying to deal with the huge volume of research literature. Systematic reviews, for example, are summaries of all the research on a particular topic. An international collaboration, the Cochrane Collaboration, produces and disseminates high quality systematic reviews, and even more importantly, keeps them up to date as new studies become available. The Cochrane Library (available on CD-ROM and via the Internet at http://hiru.mcmaster.ca/cochrane/cochrane/cdsr.htm ) is probably the most important source of reliable information about health care anywhere in the world. The Library contains systematic reviews prepared by Cochrane review groups and others, and a large database of clinical trials. 3

Journals such as Evidence-Based Nursing , Evidence-Based Mental Health , and Evidence-Based Medicine summarise the most valid and clinically applicable research selected from the international research literature. High quality studies and systematic reviews are identified using explicit methodologic criteria and then summarised in the form of structured abstracts. This approach differs from “current awareness” services, which serve to merely announce potentially interesting new research; evidence-based journals use an explicit, quality filtering process to identify only those studies of high methodologic quality. The structured abstracts provide a brief summary of the research question, study design, methods, and main results and the accompanying commentaries help us to interpret the study findings and to suggest how they might be applied in clinical practice.

Clinical Evidence is a new publication that produces and updates rigorous summaries of the research on many common conditions, including, for example, child health and wound care. 8

Sensitive and specific search strategies

Databases such as Medline and CINAHL provide electronic access to the healthcare literature, but they index letters, editorials, and general articles as well as primary studies and systematic reviews. Furthermore, high quality studies and reviews comprise an extremely small proportion of the indexed literature. The trick to searching these databases to find studies to answer your clinical questions lies in using search strategies that have maximum sensitivity (ie, identify a high proportion of the valid and relevant studies) and specificity (ie, do not identify many irrelevant studies). For example, the best single term Medline search to find studies that are likely to have used an appropriate design to evaluate a therapy or intervention is “clinical trial (publication type)”. 4 Of course, any primary studies identified through searches of Medline, CINAHL, or other databases must be appraised for validity and applicability using the critical appraisal techniques that will be addressed in the remainder of this users' guide and the series that follows.

Critical appraisal

Imagine that you have recently been promoted to a new role in your hospital and that you have responsibility for practice development on the medical wards. For the first time in your career you are able to set aside time in your week for reading and keeping abreast of new research—you think you have about 3 hours each week and are keen to use those 3 hours as efficiently has possible. The library is full of medical and nursing journals, but how do you start?

One suggestion is that you start by gathering a list of frequently asked questions ( FAQs ); that is, those questions you often ask yourself during your clinical work, and those often asked by your colleagues. For example:

Is it safe for patients with diabetes to inject insulin directly through their clothing without skin cleansing?

Is it useful to give dietary supplements to critically ill patients on your ward?

Should you be educating and counselling patients who have had a myocardial infarction, while they are still in hospital?

If you apply what you have learned about asking answerable questions to the first question in your list, it becomes 2 :

Is it effective and safe for patients with diabetes to inject insulin through clothing compared with the standard technique of skin cleansing and injection?

The most important lesson to learn about filtering your reading is to read only studies or reviews that have used an appropriate design to answer your particular question. The question about insulin injection is a question of whether a particular treatment (ie, injection of insulin through clothing) is effective and whether it is safe. In an earlier EBN notebook, we learned that randomised controlled trials (RCTs) are the strongest research design for answering this type of question. Systematic reviews of RCTs are better still as they summarise all the RCTs done on a particular topic. 5 Searching Medline using PubMed on the Internet (a free service that can be found at http://www.ncbi.nlm.nih.gov/PubMed/ ) with the terms insulin AND inject* AND cloth* AND clinical trial identifies 2 papers, one of which is a report of a study that appears to directly address our question.

The next step is to look at the study or review and decide whether it is valid and clinically applicable. Some of our medical colleagues have developed an excellent series of Users' Guides to the Medical Literature , and we intend to use these guides as a basis for this series. 9 As you might expect, separate guides exist for different types of clinical questions, as each is best answered using a specific study design. Regardless of the type of study, the clinician should always ask whether the results of the study are valid—that is, is the way the study was done likely to give a true result? The criteria to look for under this broad heading of validity vary, depending on whether the question is about treatment, prognosis, harm, or about understanding complex phenomena, such as feelings or perceptions. The validity checks can be applied quickly, and with a little practice, you will be able to decide efficiently whether to read a paper at all (on the principle that if the methods are not valid, there is no point in reading the paper). For example, the filtering question for a clinical question about whether a treatment works or not is was the assignment of patients to treatments randomised? If a study fails the basic filtering question, you will need to consider carefully whether or not to read it as it is not likely to direct your practice safely.

This approach to filtering your reading helps to keep the amount you do have to read to a manageable amount. If a study passes the validity filtering question and warrants a more detailed review, then there are further questions to help you determine the study validity with greater confidence, and others to help you tease out what the results of the study really mean, and whether they apply to your patients.

Returning to your question of injecting insulin through clothing, a search of Medline on PubMed would have identified a paper by Fleming and colleagues, which compared insulin injection through clothing with conventional techniques. 10 Better still, a search through back copies of Evidence-Based Nursing would have identified that the Fleming study had been summarised in a structured abstract, accompanied by a commentary that addressed the question of clinical applicability. The answer seems to be that injecting insulin through clothing appears to be as safe as conventional techniques and more convenient. 11

To make the most efficient use of the limited time available to keep abreast of clinical evidence, we should filter our reading so as to read only studies or reviews that are likely to provide valid results to our focused clinical questions. We should always look for high quality systematic reviews on a topic for which the author has already identified, appraised, and summarised the relevant studies. Other sources of pre-appraised research, such as the evidence-based journals and Clinical Evidence, can also save us work. Where pre-appraised research is not available or up to date, simple critical appraisal questions can help us to rapidly filter the useful studies from those that contain useless or even harmful information, and help us make decisions about clinical applicability. The figure ⇓ summarises this approach to reading. Critical appraisal checklists for use in appraising different study designs will form the basis of future users' guides.

• Download figure
• Open in new tab
• Download powerpoint
• ↵ DiCenso A, Cullum N, Ciliska D. Implementing evidence-based nursing: some misconceptions [editorial]. Evidence-Based Nursing 1998 Apr; 1 : 38 –40. OpenUrl FREE Full Text
• ↵ Flemming K. Asking answerable questions [editorial]. Evidence-Based Nursing 1998 Apr; 1 : 36 –7. OpenUrl FREE Full Text
• ↵ McKibbon KA, Marks S. Searching for the best evidence. Part 1: where to look [editorial]. Evidence-Based Nursing 1998 Jul; 1 : 68 –70. OpenUrl FREE Full Text
• ↵ McKibbon KA, Marks S. Searching for the best evidence. Part 2: searching CINAHL and Medline [editorial]. Evidence-Based Nursing 1998 Oct; 1 : 105 –7. OpenUrl FREE Full Text
• ↵ Roberts J, DiCenso A. Identifying the best research designs to fit the question. Part 1: quantitative designs. Evidence-Based Nursing 1999 Jan; 2 : 4 –6. OpenUrl FREE Full Text
• ↵ Ploeg J. Identifying the best research designs to fit the question. Part 2: qualitative designs. Evidence-Based Nursing 1999 Apr; 2 : 36 –7. OpenUrl FREE Full Text
• ↵ McKibbon A. PDQ: Evidence-based principles and practice . Hamilton: BC Decker, 1999.
• ↵ How clinical evidence is put together [editorial]. Clinical Evidence 1999 Dec; 2 : xv –xvi. OpenUrl
• ↵ Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. 1. How to get started. JAMA 1993 ; 270 : 2093 –6. OpenUrl CrossRef PubMed Web of Science
• ↵ Fleming DR, Jacober SJ, Vandenberg MA et al . The safety of injecting insulin through clothing. Diabetes Care 1997 ; 20 : 244 –7. OpenUrl Abstract / FREE Full Text
• ↵ Injecting insulin through clothing was safe and convenient [abstract]. Evidence-Based Nursing 1998 Jan; 1 : 12 . Abstract of: Fleming DR, Jacober SJ, Vandenberg MA et al . The safety of injecting insulin through clothing. Diabetes Care 1997; 20 :244–7. OpenUrl FREE Full Text

Read the full text or download the PDF:

Library Research Guides - University of Wisconsin Ebling Library

Uw-madison libraries research guides.

• Course Guides
• Subject Guides
• University of Wisconsin-Madison
• Research Guides
• Nursing Resources
• Conducting a Literature Review

Nursing Resources : Conducting a Literature Review

• Definitions of
• Professional Organizations
• Nursing Informatics
• Nursing Related Apps
• EBP Resources
• PICO-Clinical Question
• Types of PICO Question (D, T, P, E)
• Secondary & Guidelines
• Bedside--Point of Care
• Pre-processed Evidence
• Measurement Tools, Surveys, Scales
• Types of Studies
• Table of Evidence
• Qualitative vs Quantitative
• Types of Research within Qualitative and Quantitative
• Cohort vs Case studies
• Independent Variable VS Dependent Variable
• Sampling Methods and Statistics
• Systematic Reviews
• Review vs Systematic Review vs ETC...
• Standard, Guideline, Protocol, Policy
• Additional Guidelines Sources
• Peer Reviewed Articles
• Systematic Reviews and Meta-Analysis
• Writing a Research Paper or Poster
• Annotated Bibliographies
• Levels of Evidence (I-VII)
• Reliability
• Validity Threats
• Threats to Validity of Research Designs
• Nursing Theory
• Nursing Models
• PRISMA, RevMan, & GRADEPro
• ORCiD & NIH Submission System
• Understanding Predatory Journals
• Nursing Scope & Standards of Practice, 4th Ed
• Distance Ed & Scholarships
• Assess A Quantitative Study?
• Assess A Qualitative Study?
• Find Health Statistics?
• Choose A Citation Manager?
• Find Instruments, Measurements, and Tools
• Write a CV for a DNP or PhD?
• Find information about graduate programs?
• Learn more about Predatory Journals
• Get writing help?
• Choose a Citation Manager?
• Other questions you may have
• Search the Databases?
• Get Grad School information?

What is a Literature Review?

A literature review is an essay that surveys, summarizes, links together, and assesses research in a given field. It surveys the literature by reviewing a large body of work on a subject; it summarizes by noting the main conclusions and findings of the research; it links together works in the literature by showing how the information fits into the overall academic discussion and how the information relates to one another; it assesses the literature by noting areas of weakness, expansion, and contention. This is the essentials of literature review construction by discussing the major sectional elements, their purpose, how they are constructed, and how they all fit together.

All literature reviews have major sections:

• Introduction: that indicates the general state of the literature on a given topic;
• Methodology: an overview of how, where, and what subject terms used to conducted your search so it may be reproducable
• Findings: a summary of the major findings in that field;
• Discussion: a general progression from wider studies to smaller, more specifically-focused studies;
• Conclusion: for each major section that again notes the overall state of the research, albeit with a focus on the major synthesized conclusions, problems in the research, and even possible avenues of further research.

In Literature Reviews, it is Not Appropriate to:

• State your own opinions on the subject (unless you have evidence to support such claims).  
• State what you think nurses should do (unless you have evidence to support such claims).
• Provide long descriptive accounts of your subject with no reference to research studies.
• Provide numerous definitions, signs/symptoms, treatment and complications of a particular illness without focusing on research studies to provide evidence and the primary purpose of the literature review.
• Discuss research studies in isolation from each other.

Remember, a literature review is not a book report. A literature review is focus, succinct, organized, and is free of personal beliefs or unsubstantiated tidbits.

• Types of Literature Reviews A detailed explanation of the different types of reviews and required citation retrieval numbers

Outline of a Literture Review

• << Previous: Peer Reviewed Articles
• Next: Systematic Reviews and Meta-Analysis >>
• Last Updated: Mar 19, 2024 10:39 AM
• URL: https://researchguides.library.wisc.edu/nursing

Literature Reviews for Education and Nursing Graduate Students

(15 reviews)

Linda Frederiksen, Washington State University Vancouver

Sue F. Phelps, Washington State University Vancouver

Copyright Year: 2017

Publisher: Rebus Community

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Rebecca Appleton, Professor of Nursing, Marshall University on 5/7/24

It is very through in covering the steps of a well written literature review read more

Comprehensiveness rating: 5 see less

It is very through in covering the steps of a well written literature review

Content Accuracy rating: 5

I have not read the entire book, but what I did read was very good.

Relevance/Longevity rating: 5

It is up to date, but doing a Literature Review is covered in a step-wise manner, includes writing the LR>

Clarity rating: 5

Very clear step-by-step approach

Consistency rating: 5

It is very consistent!

Modularity rating: 5

Chapters are orderly and succinct

Organization/Structure/Flow rating: 5

Strait forward order.

Interface rating: 5

I did not notice Interface issues.

Grammatical Errors rating: 5

No grammatical errors were noticed.

Cultural Relevance rating: 5

I did not notice any problems with cultural Insensitivity

I plan to use this in a Nursing Research class for Graduate Students, and I am trying a new approach to finding the best Research Evidence on a Nursing Topic. Can't wait to see if this help my graduate students understand research literature better.

Reviewed by Barbara Schneider, Professor, University of Texas at Arlington on 4/29/24

This textbook covers the range of topics important for a literature review, including formulating a research question, finding scholarly articles, evaluating sources, and synthesizing source content. The videos are great supplements to the text. read more

This textbook covers the range of topics important for a literature review, including formulating a research question, finding scholarly articles, evaluating sources, and synthesizing source content. The videos are great supplements to the text.

Overall, the content is accurate. Consider labeling Nursing as a health profession/discipline.

Relevance/Longevity rating: 4

Much of the content remains relevant. Updated examples would be helpful to today's graduate students.

The textbook is clearly written.

Consistency rating: 4

In general, the text is consistent. There could be more consistency in the formatting of the references.

The modularity is an asset.

There is a logical flow to the topics.

The links to outside materials are helpful.

No grammatical errors were evident.

The examples seemed inclusive.

Those who are new to writing a literature review would find this book useful.

Reviewed by Yolanda Griffiths, Professor of Occupational Therapy, Drake University on 12/15/21

The authors were thorough and very organized in stepping readers through the process of conducting and writing a literature review. Each area is appropriately indexed and examples are provided in a variety of ways. The synthesis section is... read more

The authors were thorough and very organized in stepping readers through the process of conducting and writing a literature review. Each area is appropriately indexed and examples are provided in a variety of ways. The synthesis section is especially useful as students often do not understand what this means. Perhaps some content on plagiarism would benefit this section as well. The flow of the material easily guides users logically through each topic.

The content is accurate and unbiased. The content is presented in an easy to understand way with videos, and examples.

The relevance of the content is classic and the text should be pertinent for many years. The links included in the text are very useful and should be easy for authors to check periodically. Using a digital media is more relevant to today's students than print textbooks. Each section addresses a reasonable chunk of information.

The book is user friendly, written in an easy to understand manner, and graphics or links add to the understanding of the content. Definitions are clearly written. Such as clarifying the types of literature reviews will be useful for students. Providing a test yourself section at the end of sections allows the reader to check if any content was confusing or not clear.

The text is consistently laid out in a logical manner which helps to unpack content which may be new or unfamiliar to the reader/student.

The amount of content allocated to each chapter is appropriate and will be easy to assign readings. The chapter headings are clear and the embedded videos, charts and test questions enlighten each subunit. The hyperlinking in the table of contents helps to navigate the chapters well.

The organization of the content is logical and easy to understand the process of completing a literature review. The book is laid out much like a road map where students can see the big picture as well as the supporting parts to the process. The references by chapter are very useful.

The graphics were clear, and the non-serif font aids in eye fatigue. One recommendation is to lower the brightness of the bold blue text in the table of contents to reduce eye fatigue. There was no problem to play the videos and the audio was clear. All links worked well.

There were no grammatical errors. There were a few typos such as 1.3.1.8 needs a space between "A specific", 2.3 in the phrase "Articles by the type of periodical in which an article it is published" perhaps remove the word "it", in the table on page 41. under Nursing , the word clinical is spelled "Cclinical", remove the capital C.

No evidence of cultural bias or insensitivity.

I am very excited to use this textbook in my doctoral level occupational therapy class. The inclusion of concise explanations of PICO and SPICE will be very useful. This will be a wonderful resource for graduate students and being mindful of costs for textbooks is compassionate.

Reviewed by Susan Bassett, Instructor, Nursing Graduate Program, Eastern New Mexico University on 11/9/21

Each chapter presented a different aspect of doing a literature review. This was organized and orderly. The index/table of contents was very detailed which allowed the reader to easily use this book as a reference while conducting a literature... read more

Each chapter presented a different aspect of doing a literature review. This was organized and orderly. The index/table of contents was very detailed which allowed the reader to easily use this book as a reference while conducting a literature review.

The content appeared to be entirely accurate. It did a good job of combining information for both education and nursing students. The authors addressed pertinent points of research study development as well as the specific methodology of approaching a research-focused literature review.

The text was up-to-date in methodology, which should not change frequently. The many links to websites were very helpful and yet were basic enough that they should be relevant for years. If they do need updating, the are clearly presented and should be easily updated. The breakdown to very small "chunks" of information per section will help in easily updating specific parts of information.

The book presented a rather complex topic in an extremely straight-forward, easy to read, clear manner. Each small "chunk" of information was identified per section numbering which correlated with movement through the content. The writing was professional and yet not overwhelmed with discipline-specific terminology. Where potentially new terminology was presented, it was immediately followed with definitions and examples.

The book was well-organized and moved along the structure set out early in the book. Content was gradually unfolded, as divided per chapter. There was a bit of repetition (probably about three examples) where the authors attempted to tie information together. Although this stood out to a reader, it seemed more useful in organizing than detrimental in repetition.

The book was subdivided into chapters and then into many small modules of discrete information. It could easily be assigned in part. It could also readily be used as a reference for students to go back and easily find processes or pieces of information they might need later.

I found the continual clear and succinct organization of information to be a defining highlight of this book. When presenting early steps of the research process and then linking these steps with how to conduct a literature review and subsequenty organize and write a literature review, this book is presenting numerous procews steps that must work in tandem. This book did that in a clear and easily readable fashion.

The one feature that did distract me was within the bullet points of 1.3.1. "Types of Reviews". There was a mix of complete and incomplete sentences that worked to convey information succinctly, but distracted me as a reader.

Grammatical Errors rating: 4

I did find several spelling and grammaticl errors (1.3.1.8, , 1.3.1.9, 2.1.1, 2.3, 2.3.1.1, , 2.3.1.4, 2.3 Table A., p. 41, p. 53, p. 54). Although small errors (a few letters or spacing) they should be corrected.

I did not find any mistakes in cultural appropriateness The content did repeatedly talk about bias reduction in the process of writing a literature review

I thought this book was very well-written and contained great information for my students. The links provided were very appropriate and helpful. The Table "Guide to searching for literature at various stages of the scholarly communication process” was particularly helpful. I will immediately begin using portions of the content in this book to support my research class. Additionally, I will recommend the entire book as a reference for the dedicated student (or one intending to go forward to a doctoral level of education in nursing). Thank you for collating all this information and helpful links into one clear, easily readable and understandable document.

Reviewed by Leah Nillas, Associate Professor, Illinois Wesleyan University on 9/6/21

This book addresses the basic steps in the process of writing a literature review research. Chapter 2 (What is a Literature Review?) needs to be retitled. I think Chapter 1 (Introduction) clearly defines and characterizes literature review as a... read more

Comprehensiveness rating: 4 see less

This book addresses the basic steps in the process of writing a literature review research. Chapter 2 (What is a Literature Review?) needs to be retitled. I think Chapter 1 (Introduction) clearly defines and characterizes literature review as a research category. Chapter 2 focuses more on the creation of information, information cycle, and selecting appropriate sources. Chapter 7 (Synthesizing Sources) and Chapter 8 (Writing the Lit Review) can still be improved to incorporate specific strategies in synthesizing research literature and examples of writing styles through analysis of a variety of published examples. Writing a synthesis is a challenging skill for most novice researchers.

Information shared is accurate. I did not notice any content error.

Main content is up-to-date. A few citations maybe dated but they are necessary in illustrating different examples of literature reviews. It will be easy to include additional relevant examples of research work that are published recently.

I like how this text is written. Tone is reader friendly and narrative is accessible to novice researchers.

Clearly consistent throughout the chapters.

Clear and purposeful "chunking" of information per chapter.

Readers can easily follow the organization of topics and content.

No obvious interface issues. Appropriate use of multimedia tools.

No grammatical errors.

Text is culturally sensitive. Additional readings, references, or examples can easily be added to incorporate research conducted by diverse authors or literature reviews which focus on diversity and inclusion issues in education and nursing.

This is a good introductory literature review text even for undergraduate education students. Clear discussion of the nature of the research and the writing process. The use of videos and images is helpful in providing multimodal approach in explaining topics or processes. Writing style and tone make the text accessible to novice researchers.

Reviewed by Rebecca Scheckler, Assistant Professor, Radford University on 7/6/20

Two missing topics were inter-library loan and how to avoid plagiarism in writing up the literature review. This second is such an important topic that it deserves its own chapter. read more

Two missing topics were inter-library loan and how to avoid plagiarism in writing up the literature review. This second is such an important topic that it deserves its own chapter.

It is accurate. I found no inaccuracies.

This book is very relevant. Every advanced undergraduate or graduate students requires such a book

I found the book clear. The videos interspersed within the book added much to the clarity. There are lots of good diagrams that add to the clarity. They are not all original but their sources are all cited. The section on boolean searches, usage of asterisks and quotes in searches is very helpful and appropriate although often left out of discussion of searches.

The book is consistent in terminology and framework.

The chapters were cohesive.

Organization/Structure/Flow rating: 4

I like the links to within the text to the references and other matter. What is needed are back links to the text from the references. I also would have liked links from the exercises to the answers of the exercises.

Interface rating: 4

See navigation links mentioned above. The grey literature link is broken.

I saw no grammatical problems. There are many bulleted lists rather than text which is appropriate to this topic.

There could be more attention to cultural context in the frequent examples.

I wondered why nursing and education were combined. They are similar in nature but not identical. separation them out into two books might be appropriate.

Reviewed by Lisa Shooman, Associate Professor, Worcester State University on 6/29/20

Overall, this book provides a very comprehensive and thorough roadmap for creating a literature review. The videos assist the reader in crystallizing the information presented in the text. There is an effective index and glossary that provide... read more

Overall, this book provides a very comprehensive and thorough roadmap for creating a literature review. The videos assist the reader in crystallizing the information presented in the text. There is an effective index and glossary that provide helpful navigation to the reader.

The content is detailed, clearly explained, error-free, and unbiased. My students would greatly benefit from the lucid information presented in this text to guide them with developing a literature review. I would be eager to adopt this book for my students.

The content is timely and will not be quickly out-of-date. The quiz questions at the end of each chapter are relevant and will aid students with the consolidation of the material. The online format allows for updating, and the version history at the end of the text clearly indicated that the book was updated recently.

The text is clear and not ridden with any excess jargon /technical terminology. Pictures, graphics, and videos further elucidate the text. There are helpful questions that stimulate thought and lists that help to organize information.

The internal consistency in the text is excellent. However, Chapter 1.1 and Chapter 2 have the same title and it would benefit the reader to have different titles that would highlight the differences between these two sections. Chapter 1.1 is an overview and Chapter 2 dives into more depth.

The text is efficiently divided into smaller reading sections that are demarcated by numbers. The subsections in each chapter can be assigned at different points in the course. The text is organized logically and systematically that assists the reader with comprehension and provides a roadmap for creating an effective literature review.

The entire text is presented coherently and concisely. The organization of the text takes the reader through the process of creating an effective literature review. It can be used by multiple health professions, although the length of the text is relatively short it includes a considerable depth of the material. Other disciplines that would benefit from using this test in their courses may include occupational therapy, physical therapy, and speech and language pathology students.

The interface of the text is simple and easy to follow. The cover of the text would benefit from photos, color, and graphic design to appeal to the modern digital reader.

No grammatical or spelling errors are noted.

No cultural biases existed in the text in any way. There are no individuals highlighted in the book, and due to the technical nature of the subject matter, the text is inclusive to a variety of races, ethnicities, and backgrounds. No offensive statements are included in this book.

The authors should consider including other health professionals in the title and provide examples that can relate to other health professionals throughout the text. Other health professionals that can benefit from reading this text include occupational therapy, physical therapy, and speech and language pathology students. Literature reviews are relevant for many health professionals in their master's and doctorate programs and the text could serve a wider audience.

Reviewed by Ellen Rearick, Assistant Professor, Framingham State University on 6/1/20

This text covers all areas and the process of the integrative review appropriately. It is an engaging text for graduate students new to these assignments. read more

This text covers all areas and the process of the integrative review appropriately. It is an engaging text for graduate students new to these assignments.

This text is well done, very accurate

This text is relevant. The updates needed regarding APA format should be relatively easy to implement.

This text is clear and provides users with definitions and examples of the variety of reviews.

Very well written using consistent terminology throughout.

The text's reading sections are easily accessible and users will find them organized. Each chapter and its sections are presented in the sequence of the process of an integrative review.

Very clear and logical order.

The navigation of this text was problem-free.

No grammatical errors noted.

No issues with cultural insensitivity noted.

This was a well-organized text using videos to reinforce content that would benefit any education or nursing graduate student new to the integrative review process.

Reviewed by Ruth Stoltzfus, Professor of Nursing; Dir., Grad Programs in Nursing, Goshen College on 6/1/19

This text provides everything a graduate student needs to write a literature review in a concise manner. If you look at the digital pdf, there are many strategies to help the reader learn the process - videos, diagrams, and also text. read more

This text provides everything a graduate student needs to write a literature review in a concise manner. If you look at the digital pdf, there are many strategies to help the reader learn the process - videos, diagrams, and also text.

I found no evidence of bias and no errors.

This book has long-term relevance. The content will not quickly out-date.

I really liked the way the textbook is structured. The author is concise which makes the textbook easy to read.

I found no inconsistencies in terminology or other aspects related to the content.

I will adopt this text for a research course I use and will likely assign only specific chapters. I plan to recommend the textbook to another faculty who teaches a comprehensive research course with the idea of assigning only specific sections to read..

The textbook begins with an introduction to the subject matter. Subsequent chapters develop specific aspects related to lit reviews. The textbook provides a nice "how to" for each element of a lit review. Chapters are also organized in a smooth, easy to follow format.

I only looked at the digital pdf and print pdf versions. The print pdf indicates that there are videos to watch, but of course since it is a print pdf, there is no linkage. I think this would be obvious to a savvy reader - that a print pdf will be limited in what the reader can access.

I found no grammatical errors in my quick read.

I found no evidence of cultural bias or insensitivity.

This is the first open textbook that I have encountered. I was expecting it to be flat and boring! However, it was neither of those. There were color diagrams, color photos, and even videos embedded in the textbook.

I have adopted this book for the Research Lit Review course that I am teaching soon. I am impressed!

Reviewed by Melissa Wells, Assistant Professor, University of Mary Washington on 5/1/19

This book helps students in education and nursing complete a literature review, which may be the first time these students are tackling such a task. The chapters break down the process into defining the special genre of a literature review;... read more

This book helps students in education and nursing complete a literature review, which may be the first time these students are tackling such a task. The chapters break down the process into defining the special genre of a literature review; providing tips to get started; suggesting where students can find literature to review; explaining how to evaluate sources; detailing the process of documenting sources; giving advice for synthesizing sources; and finally, putting all of these pieces together into a final literature review. Most significantly, the text provides specific examples of ideas presented in the context of both nursing and education, which makes the content directly relatable to the student's course of study. The conclusion recaps the main points of each chapter in bullet form. The text is lacking both an index and a glossary, which would be additions that could strengthen the text.

Content Accuracy rating: 4

The text explains 11 different types of literature reviews that students may encounter or be asked to create. Also, the text is framed to work with multiple methodologies; for example, steps for writing a research question or a hypothesis to frame the literature review are provided. One inconsistency I noted was in diagram 6.2: the APA citation is incorrectly capitalized for the journal title (which should use sentence, not title, capitalization).

The text also includes external links to sources, such as a videos, which provide students with multiple modalities in which to digest the information. An example of a literature review for both education and nursing is provided at the end of the book; instead of embedding these in the text, the hyperlinks refer the reader to the external site. This will be easy to change to a new example in the future, but checks will need to be done to ensure that all such external sources remain actively accessible.

Each chapter opens with learning objectives to help frame the content with which the reader is about to engage. Throughout the text, the language is approachable and reader-friendly. For example, when the text explains more factual components (i.e., what makes a literature review or what the basics of an effective literature review include), this information is presented in bullet points with hyperlinks to the original sources.

Each chapter follows a similar construction, which makes it accessible to the reader. For example, chapters end with a "Practice" and "Check Yourself" section to apply new learning and self-check responses (an answer key is provided in an appendix). Examples in these exercises are either related to nursing or education, continuing with the stated theme of the text.

When I used this text with my own students, I assigned chapters in isolation, since they had already taken a research methods course and were applying that knowledge to create a research proposal in a specific area of study in my course.

The book is organized in such a way that logically walks the reader through the literature review writing process. Clear headings (which are hyperlinked in the table of contents) also allow the reader to jump to specific parts with which they need additional support.

The interface of this document offered a lot of flexibility. Options allowed users to access the text online, or as a download in multiple file types (EPUB, Digital PDF, MOBI, XHTML, Pressbooks XML, Wordpress XML, and Open Document). These formats provide the reader with an opportunity to pick the interface that works best for them.

I did not see any grammatical errors in the text.

Cultural Relevance rating: 4

No culturally insensitive/offensive content was noted. A variety of examples of research topics were included from both nursing and education. Of the images/video thumbnails embedded in the text that involved people, all depicted White people except for 2 images; therefore, more intentional selection of culturally diverse visuals would be helpful in future versions of this text.

I feel this text was helpful to my students as they wrote their own literature reviews. The only weakness in their papers that I noted was their organization of their literature review based on themes/topic, which was addressed in Chapters 7- 8. I now know to focus more on this part of literature review writing with future students. This text is approachable and field-specific, and I will be using it again!

Reviewed by Bernita (Bernie) Missal, Professor, Bethel University on 12/14/18

This book includes all areas that a graduate student needs to begin a literature review. However metasynthesis could have also been included in types of literature review. read more

This book includes all areas that a graduate student needs to begin a literature review. However metasynthesis could have also been included in types of literature review.

This book is accurate although missing qualitative research.

Although content is up to date, some of the article examples need to be updated. (Example: articles published in 1981 and 1992 need to be updated to more recent articles.)

The book is clear and easy to follow. Bullet points were used throughout the book with short paragraphs which helps the student.

Each chapter follows the same format with narrative followed by practice and test questions.

Clear subheadings are used throughout the book.

This book is presented in a logical way and easy for the student to follow.

Images are clear and appropriate for the content.

No specific grammar issues were seen.

It would be helpful for students to include additional examples of cultural studies throughout the book

This book is an excellent resource for graduate students. It has helpful information for the preparation and process for a literature review. Examples of written literature reviews in chapter 8 or in an appendix would be helpful for students.

Reviewed by Nancyruth Leibold, Associate Professor, Southwest Minnesota State University on 6/19/18

The text is overall comprehensive, yet it breaks the information up into manageable parts. See the table of contents for an overview of the topics. The text is very quantitative driven in that the focus is on reviewing quantitative studies. The... read more

The text is overall comprehensive, yet it breaks the information up into manageable parts. See the table of contents for an overview of the topics. The text is very quantitative driven in that the focus is on reviewing quantitative studies. The book included information about PICO statements, but did not include PICO(T) or the time variable, which is not always used in every case. Population was included in the PICO explanation, but a bit more information on the population or aggregate narrowing could improve the PICO section. These items do not hinder use of the book, but these items would need further inclusion by the faculty member using the text as specific to the discipline.

The content in the book is very accurate.

The content in the book is current and should not be obsolete within a short period of time. Any updates would be easy to add.

The text is clear and easy to understand.

The internal organization and terminology of the book is consistent and logical

The text is set up in small reading sessions. The videos and learning activities are well done and break up some of the content, so there is a variety of presentation. The tutorials, figures, practice and self-test areas are also fantastic in that they are quality and sprinkled throughout the text.

The topics in the book are presented in clear and organized fashion. I particularly like the upbeat and personal writing tone of the book. This tone makes it seem like the authors are speaking to me.

The text is free of any significant interface issues. The book is available in many formats. I used the book online and I did have one navigational problem and that is when clicking on a video, it does not open in a new tab and so the book is lost and have to start over going in the start to the book. One easy solution to this is to right click your mouse and then select open in new tab to watch videos. That way, your place in the book is not lost.

No grammar problems present.

The book is not culturally insensitive or offensive in any way.

Overall, this is a well written textbook and I recommend it!

Reviewed by Marjorie Webb, Professor, Metropolitan State University on 6/19/18

From the Introduction to the Conclusion, the text covers the step-by-step process of conducting a literature review. The text includes topics such as, “Where to find the Literature” and “Synthesizing Sources” that will be useful to graduate... read more

From the Introduction to the Conclusion, the text covers the step-by-step process of conducting a literature review. The text includes topics such as, “Where to find the Literature” and “Synthesizing Sources” that will be useful to graduate nursing students.

The content in the text, including texts, links, and diagrams, is accurate and unbiased. Again, it will aid the graduate nursing student in the long process of conducting a literature review.

The text is current and this type of material does not become dated quickly. The authors did use internet links in the text which will need to be monitored periodically to ensure they are still available. Updates to the text will be relatively easy and straightforward. If media styles change, there may be some challenges to updating.

The text is clear and easy to read. Technical terminology is defined and/or explained.

The text is internally consistent.

The text is organized in sections which facilitates assigning readings based on the subject matter for the class time. It would be pretty easy to divide up this text into easily readable units based on headings and subheadings.

This text is structured well. The topics flow in an organized manner and really help the student see the process of a literature review. The authors discuss the both theory and purpose of the review and the day-to-day logistics of actually performing the review. The day-today organization is not always included in other texts.

The interface is well-done with no distractions.

There was no indication of cultural bias.

I think this text is appropriate for graduate nursing students. Some students struggle with the difference between writing about a topic (generally undergraduate writing) and synthesizing literature on a given topic (generally graduate writing). Chapters seven and eight focus on preparing the graduate student to make the jump to graduate-level writing and should really benefit new graduate students.

Reviewed by Susanna Thornhill, Associate Professor , George Fox University on 3/27/18

This book is fairly comprehensive and offers step-by-step instructions for conceptualizing/researching a literature review. The Table of Contents is well-organized to reflect the book's progression, from establishing the basics of why to write a... read more

This book is fairly comprehensive and offers step-by-step instructions for conceptualizing/researching a literature review. The Table of Contents is well-organized to reflect the book's progression, from establishing the basics of why to write a literature review and the various types of literature reviews, to getting started with formulating a research idea/question, finding and evaluating sources, synthesizing sources, and guidelines on writing the literature review, itself. I found this text to be a straightforward guide for my graduate students in education, and while I worried at first that the merging of education and nursing topics would prove distracting to my education students, I don't believe this was the case.

One thing that was not comprehensive in this book was discussion of qualitative research and methodologies as a valid means of conceptualizing research aims. I hoped for a more balanced discussion between methodological branches as it applied to literature reviews; this book overly favored quantitative methodologies and studies in terms of its direction to readers about how to conceptualize/choose a topic and design a research question in relation to it. Variables that cannot be measured are not inherently un-researchable, which is the conclusion put forth in this textbook. This might serve nursing students better than education students in terms of their discipline's requirements, but it still represents an element that could be improved.

Finally, while the background on what a literature review is, how to conceptualize research, and how to search for and synthesize research was all valuable, the chapter on actually writing the literature review was a bit thin, simply offering tips for introduction, body, and conclusion and some questions for self-evaluation. Some of the most difficult work for students writing a literature review is achieving proper focus, organization, hierarchy of themes, balance in treatment of related topics, etc. None of these issues were discussed in the chapter pertaining to the writing of a literature review.

I did not have any concerns about the book's accuracy. Content was accurate, albeit biased to quantitative and positivist views of research. I would have liked to see it include additional prompts to support students in conceptualizing and valuing qualitative research; this is an area where I had to supplement course readings with additional texts.

The only significant error I could discern in the text was a lack of an Answer Key corresponding to the questions posed at the end of each chapter.

Content is up-to-date and seems like it will hold meaning well over the next few years. The only things I anticipate might go out-of-date is technological information on things like citation managers, search guidelines, and database information. This is easily updatable with future versions of the text. In my view, ERIC is not the best database for educational research and I have confirmed this with educational librarians who support my students, yet it is the only one identified in this text as the best subject-specific source of educational research; this could be revised for additional relevance.

I noticed no issues with the book's clarity. The authors write in a clear and straightforward style, making the text easy to read. Overall, they did well writing for students across two disciplines by avoiding nursing or education-specific terms that would have been problematic to readers in the other discipline.

The book is internally consistent and did not have issues with terminology or framework.

No issues with the book's modularity. Chapter headings and sub-headings were appropriately paced and spaced. I assigned this textbook to my graduate students as a whole text that I wanted them to read at the beginning of a course, but it has been easy to refer them back to particular topics as the course has continued.

In future iterations of the book, I suggest hyperlinking the Answer Key to the exercises at the end of each chapter and/or listing the Answer Key in the Table of Contents for easy referral.

I found the book's organization to be straightforward and sensible. The Table of Contents offers a helpful snapshot of the scope of the book and the authors write in a direct and clear style, which contributes to an appropriate flow for the text.

I did not note any navigation problems with any links. All charts/images loaded well in my iBook app. The authors did a nice job of pulling relevant content and links in to support their ideas; it provided an easy way to seek more information if I wanted it, without feeling like the text was loaded down with unnecessary information.

I only found a few small typos in the text, with no grammar issues. The book is obviously written by two very detail-oriented librarians. I appreciated the clarity of the text and lack of errors.

The text was not culturally insensitive; a variety of topics across nursing and education were discussed as examples, which yielded a fairly balanced text regarding cultural considerations.

Reviewed by Alicia Rossiter, Assistant Professor, University of South Florida on 3/27/18

I believe the book gives a comprehensive overview on how to complete a literature view at the graduate level. It begins with an overview of the purpose of a literature review and moves through the steps to completing the review process. read more

I believe the book gives a comprehensive overview on how to complete a literature view at the graduate level. It begins with an overview of the purpose of a literature review and moves through the steps to completing the review process.

I believe the book was accurate and unbiased. It was easy to read but comprehensive.

Content within the text is relevant and supports the literature view process. It did discuss the various databases for searches which may need updating to include new sites, search engines but otherwise relevant and useful information.

The text is easy to read, provides appropriate examples, includes a section on putting the process into practice as well as a "test yourself" section to ensure the content is understood.

The text is consistent throughout in regards to terminology, framework, and set up.

The text is easy to read and content is leveled for the reader but not over simplified. Content is chunked into sections making it easy for the reader to digest the content. The chapters are well laid out and flow from chapter to chapter. Each chapter contains learning objectives, content sections, practice section, and test yourself section. Well organized and great visuals.

Topics are presented in a logical, clear fashion that flow from chapter to chapter and build as the reader moves through the process.

The text is free of interface issues. I could not get the videos to play but other visuals were appropriate and useful to support content.

The text contains no grammatical errors.

The text is not culturally offensive. There was no evidence of bias or cultural insensitivity.

I think this would be a great resource for graduate student learning to navigate the literature review process. It is easy to read, straightforward, and guides the individual through the process from start to finish. I will recommend this text to my graduate students in evidence-based practice and research courses as a recommended reference.

Table of Contents

• Chapter 1: Introduction
• Chapter 2: What is a Literature Review?
• Chapter 3: How to Get Started
• Chapter 4: Where to Find the Literature
• Chapter 5: Evaluating Sources
• Chapter 6: Documenting Sources
• Chapter 7: Synthesizing Sources
• Chapter 8: Writing the Literature Review

Ancillary Material

About the book.

Literature Reviews for Education and Nursing Graduate Students is an open textbook designed for students in graduate-level nursing and education programs. Its intent is to recognize the significant role the literature review plays in the research process and to prepare students for the work that goes into writing one. Developed for new graduate students and novice researchers just entering into the work of a chosen discipline, each of the eight chapters covers a component of the literature review process. Students will learn how to form a research question, search existing literature, synthesize results and write the review. The book contains examples, checklists, supplementary materials, and additional resources. Literature Reviews for Education and Nursing Graduate Students is written by two librarians with expertise guiding students through research and writing assignments, and is openly licensed.

About the Contributors

Linda Frederiksen is the Head of Access Services at Washington State University Vancouver.  She has a Master of Library Science degree from Emporia State University in Kansas. Linda is active in local, regional and national organizations, projects and initiatives advancing open educational resources and equitable access to information.

Sue F. Phelps is the Health Sciences and Outreach Services Librarian at Washington State University Vancouver. Her research interests include information literacy, accessibility of learning materials for students who use adaptive technology, diversity and equity in higher education, and evidence based practice in the health sciences

Contribute to this Page

In this Guide

• Access Key Resources
• Literature Searching for Evidence-Based Projects
• Find Journal Articles
• Drug Info and Calculators
• Patient Information
• Mobile Apps & Research Tools

Overview & Steps for Searching the Literature

Literature searching and literature reviews are often used interchangeably but are two different steps in the research process guided by EBM.

• Literature search  is searching the literature for some studies. A search strategy is developed for one or more biomedical databases to search the literature, and gather relevant studies.
• Literature review  is reviewing the studies which have been identified through a literature search. As part of the literature review, the retrieved articles are analyzed and critically appraised. 

The following steps will help guide you through the process of literature searching in PubMed. Though we are focusing on PubMed, these steps can be used across bibliographic databases. 

• Formulate a research question
• Identify primary concepts & gather synonyms
• Locate Medical Subject Headings MeSH (database-specific indexing terms)
• Combine search terms using Boolean operators
• Apply search limits or filters

To learn more about the literature searching process, you can explore Lane Library's Literature Searching guide .

Step 1: Formulate a Research Question

The first step in literature searching involves taking a clinical topic or problem and formulating it into a well-defined, answerable question. The development of a clear and focused question will help to streamline the searching process to locate the literature needed to begin answering the question and addressing the clinical problem. A well-defined, answerable question: 

• defines the focus of your literature search
• identifies the appropriate study design and methods
• makes searching for evidence simpler and more effective
• helps you identify relevant results and separate relevant results from irrelevant ones

What type of question are you asking?

Therapy:  effectiveness/risk of a certain treatment

Diagnosis : accuracy/usefulness of a diagnostic test/tool; application to a specific patient  

Prognosis : probable outcome, progression, or survivability of a disease or condition; likelihood of occurrence

Etiology/Harm : cause or risk factors for a disease or condition; questions about the harmful effect of an intervention or exposure on a patient

Tips for formulating a good question:

• The question is directly relevant to the most important health issue for the patient;
• The question is focused and when answered, will help the patient the most;
• The question is phrased to facilitate a targeted literature search for precise answers

Adopted from CEBM:  what makes a good clinical question   and  Center for Evidence Based Medicine: Asking focused questions  

PICO Framework

In EBM, following the PICO framework is a common way to create a focused and answerable question from a general topic. PICO is a mnemonic used to describe the four elements of a sound clinical foreground question.

PICO  stands for:

• P  - Population/Patient/Problem
• I  - Intervention  
• C  - Comparison or Control
• O  - Outcome

Alternative formats of PICO include PICOT and PICOTT:

• T  - Time
• T  - Type of question
• T  - Type of study

What is the effectiveness of Prozac vs Zoloft in treating adolescents with depression?

P : adolescents with depression

Using PICO to formulate your research question makes it easier to follow the next step in the literature searching process -- identifying primary concepts & gathering synonyms.

Step 2: Identify Primary Concepts & Gather Synonyms

Primary concepts for your research question can be identified using the PICO formula from Step 1. Each of the PICO elements can form a primary concept. If your PICO does not have a  C omparison or  O utcome, or if the Outcome is broad or vague, it is okay to leave out these concepts. Sometimes, one of the elements in the PICO framework will include more than one primary concept. For example, the Population for our example includes the concept of adolescents and the concept of depression.

P :  adolescents  with  depression

I :  Prozac

C :  Zoloft

For each primary concept identified, make a list of other terms with the same or related meaning (synonyms). It is important to gather synonyms, because

Terms have different spellings, plural forms, and acronyms

Concepts are described inconsistently across time, geographies, or even among researchers

Terms have the same/close meaning, disciplinary jargon

Umbrella terms vs specific names for issues, interventions, or concepts

These terms will form the keywords of your search strategy. 

Tips for finding synonyms:

• Do a quick search to find a relevant article or two. Look at the words used in the article titles and abstracts.
• Think of specific examples or types
• Use background information to help brainstorm (e.g. UpToDate, DynaMed, textbooks)
• Explore the entry terms and related subject headings in MeSH (see Step 3)

Remember that building a search strategy is iterative. As you learn more about your topic, you can add more keywords to your search to broaden your results, or remove keywords if you are finding too many results.

Step 3: Locate Medical Subject Headings MeSH (database-specific indexing terms)

What is mesh.

Databases like PubMed use subject headings or controlled vocabularies to index (or label) articles. Subject headings are standardized terms for describing what the articles are about. Subject headings are specific to databases, and in PubMed, they are called Medical Subject Headings or MeSH. MeSH terms are structured hierarchically in a tree structure, and when you search a MeSH term, you search automatically includes all the terms that fall beneath it in the tree. Indexers add MeSH terms to journal article records in PubMed to reflect their subject content. 

MeSH terms are useful in a search to aid in locating synonyms and reduce term ambiguities. It facilitates the retrieval of relevant articles even when authors use different words or spelling to describe the same concept. For instance, using the MeSH term "Blood Pressure" will also find articles that use "pulse pressure," "diastolic pressure," and "systolic pressure."

Since MeSH terms are organized in hierarchies or MeSH trees, it also facilitates the searching for broad and narrow concepts. For instance, the MeSH term "Domestic Violence" will retrieve articles containing narrower topics such as "child abuse," "elder abuse," and "spouse abuse." But you can also expand the search, and move to a broader level, such as "Violence."

To look up a MeSH term, click on " MeSH Database " on PubMed's homepage. Type your concept into the search bar. The MeSH database will return appropriate MeSH (terms) if there are any. Not every concept will have a matching MeSH term. Remember to search for one concept at a time.

adolescents => "Adolescent"[Mesh]

Prozac => "Fluoxetine"[Mesh]

Zoloft => "Sertraline"[Mesh]

depression => "Depression"[Mesh]

When you search for a MeSH term in PubMed, use the [Mesh] tag following your search term to specify where to search for the term in the PubMed record.

You can also locate MeSH terms in PubMed by finding a relevant article and scrolling to the heading "MeSH terms" at the bottom of the article. This only works for articles that have been indexed. 

Other PubMed Search Tags

In addition to searching specifically for MeSH terms, you can also use search tags to search for keywords in particular fields of the PubMed record. When you search in PubMed, you are automatically looking for your keywords in all the record fields. Sometimes this might be too broad and bring back too many search results. You can experiment with field tags like [ti] to look for keywords only in the title or [tiab] to look for keywords only in the title or abstract. Explore all of the available search tags and reach out to your liaison librarian if you have questions using search tags.

Step 4: Combine Search Terms Using Boolean Operators

Now that you've identified keywords for your concepts (step 2) and related MeSH terms (step 3), you can combine your search terms with Boolean Operators to build your search strategy.

Boolean Operators are a set of commands that can be used in almost every search engine, database, or online catalog to provide more focus to a search. The most basic Boolean commands are  AND  and  OR . In PubMed, you can use Boolean Operators to combine search terms, and narrow or broaden a set of results.

Narrow Results with AND

Use AND in a search to narrow your results. It tells the search engine to return results that contain ALL the search terms in a record.

adolescents AND depression

Note: Both the words  adolescents  and  depression  will be present in every record in the results.

Broaden Results with OR

Use OR in a search to broaden your results by connecting similar concepts (synonyms). It tells the search engine to return results that contain ANY of the search terms in a record.

adolescents OR youth OR teenagers

Note: Search results need to have at least one of the words  adolescents  or  youth  or teenagers .

Use parentheses ( ) to keep concepts that are alike together, and to tell the database to look for search terms in the parentheses first. It is particularly important when you use the Boolean Operator “OR”.

(adolescents OR youth OR teenagers) AND depression

Tip: You can use" Advanced Search " option in PubMed to help build your search strategy. Search concept by concept, adding ORs between all your keywords and MeSH terms for each concept. After you complete a search for each concept, you can use the "Actions" menu in the Advance Search Search History table to add combine your concept searches with AND. This will look for the overlap between your concept searches and help you avoid nesting errors.

Full Search Strategy Example:

("Adolescent"[Mesh] OR adolescent OR teen OR teens OR teenager OR youth OR youths) AND ("Depression"[Mesh] OR depressive OR depression) AND ("Fluoxetine"[Mesh] OR prozac OR fluoxetin* OR sarafem) AND ("Sertraline"[Mesh] OR zoloft OR sertraline OR altruline OR lustral OR sealdin OR gladem)

Step 5: Apply Search Limits or Filters

You can filter your search results using the PubMed filters in the left sidebar. You can filter by study type to look for the highest level of evidence to answer your question. You can also use date filters or filter to English language materials. If the study type you are looking for is not listed, select "Additional Filters" at the bottom of the left sidebar to see all the available options.

Note: Many PubMed filters depend on indexing, and using filters will exclude articles that do not have indexing.

You can also try PubMed's Clinical Queries to narrow your search results to the type of clinical questions you are asking (Therapy, Diagnosis, etc.).

Getting Too Many Results?

If your search retrieves too many results, you can limit the search results by

• replacing general (e.g. vague or broad) terms with more specific ones
• including additional concepts in your search
• using PubMed's sidebar filters on the left panel of the results page to restrict results by publication date, article type, population, and more

Getting Too Few Results?

If your search returns too few results, you can expand your search by

• browsing the  Similar Articles  on the abstract page for a citation to see closely related articles generated by PubMed's algorithm
• Removing specific or extraneous terms from the search string
• Using alternative terms to describe a similar concept used in the search

Handouts and Checklists

• Search syntax for common databases cheat sheet
• Literature searching in PubMed cheat sheet

Databases to Search Journal Articles

• CINAHL (Cumulative Index to Nursing & Allied Health) Provides access to nursing and allied health literature including 50 nursing specialties, speech and language pathology, nutrition, general health, and medicine.

Provides fulltext access to Lane's resources. Contains coverage of over 5000 journals and more than 35.5 million citations for biomedical articles, including, but not limited to, clinical trials, systematic reviews, case reports, and clinical practice guidelines.

• Embase Biomedical and pharmacological abstracting and indexing database of published literature that contains over 32 million records from over 8,500 currently published journals (1947-present) and is noteworthy for its extensive coverage of the international pharmaceutical and alternative/complementary medicine literature.
• Scopus Largest abstract and citation database of peer-reviewed literature featuring scientific journals, books and conference proceedings.
• Web of Science Multidisciplinary coverage of over 10,000 high-impact journals in the sciences, social sciences, and arts and humanities, as well as international proceedings coverage for over 120,000 conferences. Features systematic reviews that summarize the effects of interventions and makes a determination whether the intervention is efficacious or not.
• Joanna Briggs Institute Evidence Based Practice Database Provides evidence-based health information prepared by expert reviewers at Joanna Briggs Institute (JBI). It includes several databases: Best Practice Information Sheets, Consumer Information Sheets, Evidence Summaries, Recommended Practices, Systematic Review Protocols, Systematic Reviews, and Technical Reports.
• Cochrane Library A collection of databases that contain different types of high-quality, independent evidence to inform healthcare decision-making including systematic reviews conducted by the Conchrane Review Groups and randomized controlled trials.
• PsycINFO Provides systematic coverage of the psychological literature from the 1800s to the present through articles, book chapters and dissertations.
• PsycTESTS Provides downloadable access to psychological tests, measures, scales, and other assessments as well as descriptive and administrative information. It includes both published and unpublished tests developed by researchers but not made commercially available.
• ERIC (Education Resources Information Center) Citations and abstracts to journal and report literature in all aspects of educational research. Access Instructions. . . less... Also available through EBSCO and ProQuest

Lane Classes and Tutorials

• NLM PubMed Online Training PubMed training materials by the National Library of Medicine (NLM)

Finding Systematic Reviews & Evidence Summaries

• ACP Journal Club Summarizes the best new evidence for internal medicine from over 120 clinical journals. In this monthly feature of Annals of Internal Medicine, research staff and clinical editors rigorously assess the scientific merit of the medical literature as it is published. A worldwide panel of over 5000 physicians assesses the clinical relevance and newsworthiness of rigorous studies.
• BMJ Best Practice Provides detailed, evidence-based decision trees and a step-by-step approach, covering prevention, diagnosis, treatment and prognosis. Incorporates guidelines, systematic reviews from BMJ Clinical Evidence, and content from Cochrane Clinical Answers and AHFS Drug Information Essentials.
• JAMA Rational Clinical Examination JAMA's groundbreaking series on evidence-based use of the medical history, physical examination, and testing to diagnosis disease.
• NEJM Journal Watch Reviews over 250 scientific and medical journals. The physician-editors survey high-impact medical journals in specialties and topics to identify key findings. The summaries are presented with a clinical perspective and expert commentary.

Find Practice Guidelines

You can find guidelines in many other databases including PubMed, DynaMed, and ClinicalKey. Select a database and search for a specific condition or topic (e.g. preeclampsia), then follow the directions for each database to limit your search to guidelines:

• ClinicalKey for Nursing Specifically designed to provide the information needed to support nursing care, including administering medication, managing nurse policy, monitoring for complications, and providing patient education. Includes drug monographs, clinical topic summaries, videos, images, and journals.
• DynaMed Provides synthesized evidence and objective analysis to answer clinical questions. Features overviews and recommendations, expert reviews, mobile access, and links to Micromedex drug content. Access Instructions. . . less... Instructions for mobile app access
• UpToDate Point-of-care clinical information resource containing succinct and aggressively updated clinical topic reviews. Offers free DME/CE/CPD. Access Instructions. . . less... Mobile app download instructions

• Canadian Medical Association. CPG Infobase: Clinical Practice Guidelines. This database contains approximately 1,200 evidence-based Canadian clinical practice guidelines (CPGs) developed or endorsed by authoritative medical or health organizations in Canada.
• NICE NICE guidelines are evidence-based recommendations for health and care in England.

Clinical search engine designed to allow users to quickly and easily find and use high-quality research evidence to support their practice and/or care. Users are able to search across other content types including images, videos, patient information leaflets, educational courses and news.

A searchable database of 26 databases of hyperlinks from evidence-based medicine sites around the world.

Books about Practice Guidelines

• Browse Practice Guidelines for Nurses

Useful Websites

• Center for Evidence-Based Medicine (CEBM)
• Cochrane Evidence Essentials
• Joanna Briggs Institute Evidence-Based Practice Resources
• John's Hopkins Nursing Center for Evidence-Based Practice
• Ohio State's Fuld Institute for EBP
• Oncology Nursing Society - Evidence-Based Practice Learning Library
• Sigma Repository It is a profession-based online platform that freely disseminates nursing research, research-related materials, clinical materials related to evidence-based practice and quality improvements, and educational materials.
• << Previous: Access Key Resources
• Next: Find Books >>
• Last Updated: Apr 24, 2024 1:49 PM
• URL: https://laneguides.stanford.edu/nursing

Nursing Education During the SARS-COVID-19 Pandemic: The Implementation of Information and Communication Technologies (ICT)

• Published: 09 May 2024

Cite this article

• Oriette Soto-Luffi   ORCID: orcid.org/0000-0002-7137-6676 1 ,
• Cristian Villegas   ORCID: orcid.org/0000-0001-6224-8974 2 , 3 ,
• Sharon Viscardi   ORCID: orcid.org/0000-0002-5018-4122 4 , 5 , 6 &
• Elizabeth Maria Ulloa-Inostroza   ORCID: orcid.org/0000-0002-2275-2102 7 , 8  

1 Altmetric

In the pandemic, nursing programs found a way to continue the training process through virtual classrooms and technological tools to support the teaching and learning process. The aim of the present research was to analyze the existing literature on the methodologies and technologies used in nursing education during the pandemic. Methods and results, in this context, were integrative review of the literature through the SciELO, PubMed, CUIDEN, Scopus, Web of Science, CINAHL (EBSCO), Redalyc, and Dialnet databases, by running a search on the use of the keywords “nursing students,” “new technology,” and “pandemic,” for literature published between 2020 and 2022. As a search strategy, the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) flowchart was used. Finally, the virtualization of nursing teaching has been positive in giving continuity to the training process, and student learning has been verified. However, there is evidence that some actions and skills are not possible at the moment with the available methodologies and tools and still require attendance for some practical activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Malo Álvarez S, Maldonado-Maldonado A, Gacel Ávila J, Marmolejo F. Impacto Del COVID-19 en la educación superior de México. Rev ESAL. 2020;8:9–14.

Article   Google Scholar  

Shindjabuluka RN, Ashipala DO, Likando GN. COVID-19 as an enabler for enhancing online learning and teaching skills for nurse educators at the University of Namibia: prospects and challenges. Health SA Gesondheid. 2022. https://doi.org/10.4102/hsag.v27i0.1727 .

Ruz-Fuenzalida C. Educación virtual y enseñanza remota de emergencia en El contexto de la educación superior técnico-profesional: posibilidades y barreras. Rev Saberes Educ. 2021;6:128–43. https://doi.org/10.5354/2452-5014.2021.60713 .

Vialart Vidal MN. Estrategias didácticas para la virtualización del proceso enseñanza aprendizaje en tiempos de COVID-19. Rev Educ Med Super. 2020;34(3):e2594 https://ems.sld.cu/index.php/ems/article/view/2594 .

Google Scholar  

Portillo Peñuelas SA, Castellanos Pierra LI, Reynoso González ÓU, Gavotto Nogales OI. Enseñanza remota de emergencia ante la pandemia COVID-19 en educación media superior y educación superior. Propósitos Representaciones. 2020. https://doi.org/10.20511/pyr2020.v8nSPE3.589 .

Carmona C, Mancero PC. Virtualidad como herramienta de apoyo a la presencialidad: Análisis desde la mirada estudiantil. Rev Cienc Soc. 2020;26(1):219–31.

Pardo Iranzo V. La docencia online: ventajas, inconvenientes y forma de organizarla. rBD. 2014;18:622–35.

Paredes-Chacín A, Walles-Peñaloza D. Educación superior e investigación en latinoamérica: transición al uso de tecnologías digitales por covid-19. Rev Cienc Soc. 2020;26(3):98–117.

Vera F. Impact of videoconferencing platforms on higher education during COVID-19. Rev Electrón Transf. 2021;02(01):41–57.

Reinoso-González E. La Videoconferencia como herramienta de educación: ¿qué debemos considerar? Rev Esp Edu Med. 2020;1(1):60–5.

Reuge N, Yameogo JL, Pablo J, Ospino G. How many children and young people have internet access at home? 2020. https://data.unicef.org/resources/children-and-young-people-internet-access-at-home-during-covid19/ .

Umaña-Mata AC. Educación superior en tiempos de COVID-19: oportunidades y retos de la educación a distancia. Rev Innov Educ. 2020;22(Especial):36–49.

Meca JS. Cómo realizar una revisión sistemática y un meta-análisis. Aula Abierta. 2010;38(2):53–64.

Siddaway AP, Wood AM, Hedges LV. How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annu Rev Psychol. 2019;70:747–70. https://doi.org/10.1146/annurev-psych-010418 .

Matthew J, McKenzie JE, Bossuyt PM, Boutron I, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Rev Esp Cardiol. 2021;74:790–9.

Serrano Amarilla N, Cuetos Revuelta MJ, Manzanal Martínez AI. Systematic review: flipped classrooms in the performance of undergraduate science students. J Sci Educ Technol. 2022;31:594–604. https://doi.org/10.1007/s10956-022-09979-8 .

Liberati A, Altman DG, Tetzlaff J, Mulrow C, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009. https://doi.org/10.1136/bmj.b2700 .

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (Pico) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc. 2018;106(4):420–31. https://doi.org/10.5195/jmla.2018.345 .

Grewal A, Kataria H, Dhawan I. Literature search for research planning and identification of research problem. Indian J Anaesth. 2016;60(9):635–9. https://doi.org/10.4103/0019-5049.190618 .

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):245. https://doi.org/10.1186/s13643-017-0644-y .

Hodges C, Moore S, Lockee B, Trust Aaron T. La diferencia entre la enseñanza remota de emergencia y el aprendizaje en línea [Internet]. Educause Rev. 2020;12–22. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remoteteaching-and-online-learning .

Palés Argullós JL, Gomar Sancho C. Simulation in medical education. Teor Educ. 2010;11(2):147–69.

Quirós SM, de Vargas MA. Clinical simulation: a strategy that articulates teaching and research practices in nursing. Texto Contexto Enferm. 2014;23(4):815–6. https://doi.org/10.1590/0104-07072014001200edt .

Haslam MB. What might COVID-19 have taught us about the delivery of nurse education, in a post-COVID-19 world? Nurse Educ Today. 2021;1:97.

Sormunen M, Heikkilä A, Salminen L, Vauhkonen A, Saaranen T. Learning outcomes of digital learning interventions in higher education: a scoping review, vol. 40. Lippincott Williams and Wilkins; 2022. p. 154–64.

Agu CF, Stewart J, Stewart NM, Rae T. Efectos de la pandemia del COVID-19 en la educación en enfermería: mirando bajo el prisma de un país en desarrollo. Int Nurs Rev. 2021;68:153–8.

Lara Jaque R. Educación en Enfermería en Tiempos de Pandemia: Desafíos Presentes y Futuros. Cienc enferm. 2020;26:16 https://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0717-95532020000100102 .

Karl KA, Peluchette JV, Aghakhani N. Virtual work meetings during the COVID-19 pandemic: the good, bad, and ugly. Small Group Res. 2022;3:343–65. https://doi.org/10.1177/10464964211015286 .

McKay MA, Pariseault CA, Whitehouse CR, Smith T, Ross JG. The experience of baccalaureate clinical nursing faculty transitioning to emergency remote clinical teaching during the COVID-19 pandemic: lessons for the future. Nurse Educ Today. 2022;111:105309. https://doi.org/10.1016/j.nedt.2022.105309 .

Guinez-Molinos S, Maragaño Lizama P, Gomar-Sancho C. Simulación clínica colaborativa para el desarrollo de competencias de trabajo en equipo en estudiantes de medicina. Rev Med Chil. 2018;146(5):643–53.

Sánchez Mendiola M, van der Fortoul TI. Zoom y la educación en ciencias de la salud: ¿medio o mensaje? Investig Educ Méd. 2021;38:76–88.

Díaz VL, Escalante EM. Aplicación de la tecnología de la información y comunicación en la enseñanza de anatomía para estudiantes de enfermería 1 palabras clave: anatomía, aprendizaje-significativo, enfermería, tecnología-de-información. Enferm Actual Costa Rica. 2014;26:1–13.

Kanagaraj P, Sakthivel R, Christhumary PC, et al. Nursing student’s satisfaction with virtual learning during COVID-19 pandemic in India. SAGE Open Nurs. 2022. https://doi.org/10.1177/23779608221144933 .

Rojas A, Borja H. ¿Es ética la utilización de técnicas de simulación en la docencia médica de pregrado? Reflexión bioética. Rev Chil Enferm. 2016;32(1):34–7. https://doi.org/10.4067/S0717-73482016000100006 .

Roman JAM. La educación superior en tiempos de pandemia: una visión desde dentro del proceso formativo. Rev Latinoam Estudios Educ. 2020;50(especial):13–40.

Bercheñi V, Itatí Mariño S. Identificación de brechas digitales en pandemia. SciComm Rep. 2021;18:1–12. https://doi.org/10.32457/scr.v1i2.1444 .

Kobayashi M, Koga Y, Kako J, Kakeda T, et al. How has the COVID-19 pandemic influenced nursing students’ academic experience and career choices? A qualitative descriptive analysis. Teach Learn Nurs. 2023;18(1):30–6. https://doi.org/10.1016/j.teln.2022.10.009 .

Prata JA, de Mello AS. Safe return to face-to-face teaching in nursing: challenges to ways of living at the university. Rev Enferm UERJ. 2022;30:e65713. https://doi.org/10.12957/reuerj.2022.65713 .

Cruz JP, Balay-Odao EM, Bajet JB, Mesde J, et al. Face mask use and social distancing attitude of healthcare students: a multi-disciplinary study. Healthc (Basel). 2023;11(6):901. https://doi.org/10.3390/healthcare11060901 .

Moreno-Sánchez E, Merino-Godoy MDLA, Piñero-Claros S, Santiago-Sánchez A, Del-Campo-jiménez Á, Mariscal-Pérez L, et al. Nursing education during the SARS-CoV-2 pandemic: assessment of students’ satisfaction with e-learning environment. Int J Environ Res Public Health. 2022. https://doi.org/10.3390/ijerph19042023 .

Díaz-Guio DA, Arias-Botero JH, Álvarez C, Gaitán-Buitrago MH, Ricardo-Zapata A, Cárdenas L, et al. Telesimulación en la formación en medicina perioperatoria desde la perspectiva colombiana. Sim Clín. 2021;3(3):110–6. https://doi.org/10.35366/103187 .

Fredes CA, Hernández JP, Díaz YDA. Potencial Y Problemas De La Simulación en Ambientes Virtuales Para El Aprendizaje. Form Univ. 2012;5(1):45–56. https://doi.org/10.4067/S0718-50062012000100006 .

Nascimento AAA, Ribeiro SEA, Marinho ACL, Azevedo VD, et al. Repercussions of the COVID-19 pandemic on nursing training: a scoping review. Rev Latino-Am Enfermagem. 2023;31:e3912. https://doi.org/10.1590/1518-8345.6414.3912 .

Barkha D, Nazung L, Shrijana P. Effect of COVID 19 pandemic on clinical learning environment - disruption to recovery in nursing education. IJRSFP. 2022;13:8–13. https://doi.org/10.24327/ijrsr.2022.1301.0003 .

Bassi E, Dal Molin A, Brugnolli A, et al. Moving forward the Italian nursing education into the post-pandemic era: findings from a national qualitative research study. BMC Med Educ. 2023;23:452. https://doi.org/10.1186/s12909-023-04402-1 .

Franco-Coffré JA, Mena-Martin F, Gordillo-Ojeda MV, Vargas-Aguilar GM. La Educación virtual en la formación profesional de enfermeros, durante la pandemia provocada por la COVID 19. Pol Con. 2021;6(12):762–75.

Brown KM, Swoboda SM, Gilbert GE, Horvath C, Sullivan N. Integrating virtual simulation into nursing education: a roadmap. Clin Simul Nurs. 2022;72:21–9. https://doi.org/10.1016/j.ecns.2021.08.002 .

Fernándeza AL, Añasco RV, Hernández LC, Ordoñez YA, Bernal AF. Aprendizajes de estudiantes del área de salud en tiempos de covid-19: una revisión integrativa. Investig Enferm Imagen Desarrollo. 2022. https://doi.org/10.11144/Javeriana.ie24.aeas .

Canova-Barrios CJ, Méndez PG, Sosa LR, Flores MA, et al. Percepción y satisfacción de estudiantes de Enfermería con la educación virtual en época de COVID-19. Enferm Cuid Hum. 2023;12(2):e3304. https://doi.org/10.22235/ech.v12i2.3304 .

Liu Y, Sun X, Zhang P, Han P, et al. Generation Z nursing students’ online learning experiences during COVID-19 epidemic: a qualitative study. Heliyon. 2023;9(4):e14755. https://doi.org/10.1016/j.heliyon.2023.e14755 .

Ramírez-Martinell A, Casillas MA. Háblame de TIC: tecnología digital en educación superior, vol 1. 1st ed. Argentina: Editorial Brujas; 2014. p. 1–179. https://www.uv.mx/personal/albramirez/2014/08/01/hablamedetic/ .

Kwon SJ, Kim Y, Kwak Y. Nursing students’ clinical practice education experience during the COVID-19 pandemic: a qualitative study. BMC Nurs. 2024;23:63. https://doi.org/10.1186/s12912-024-01730-5 .

Bryan V, Corcoran L, Dewart G, Thirsk LM, Bowers E. Clinical learning during the pandemic: experiences of LPN-BN undergraduate nursing students. J Prof Nurs. 2022;42:301–7. https://doi.org/10.1016/j.profnurs.2022.07.020 .

Dabney BW, Linton M, July J, Miles M. RN-to-BSN student reports of work experiences, academic performance, and program progression during the COVID-19 pandemic. Nurs Outlook. 2023;71(4):102019. https://doi.org/10.1016/j.outlook.2023.102019 .

Almonacid-Fierro A, Valdebenito K. Learning and quality of life in undergraduate nursing students: an exploratory study in Chile during the COVID-19 pandemic. J Educ Health Promotion. 2022;9011(1):216. https://doi.org/10.4103/jehp.jehp_295_22 .

Henshall C, Davey Z, Merriman C, et al. The impact of the COVID-19 pandemic on nursing students’ navigation of their nursing programmes and experiences of resilience. A qualitative study. IJNS Adv. 2023;5:100146. https://doi.org/10.1016/j.ijnsa.2023.100146 .

Nodine PM, Arbet J, Jenkins PA, et al. Graduate nursing student stressors during the COVID-19 pandemic. J Prof Nurs. 2021;37(4):721–8. https://doi.org/10.1016/j.profnurs.2021.04.008 .

Austin HM. The effect of COVID-19 on nursing program passing scores. Teach Learn Nurs. 2023;18(3):e84–8. https://doi.org/10.1016/j.teln.2023.01.009 .

Leon Palomino C, Ochoa Marín SC, Restrepo Betancur V, Semenic S. Impact of the COVID-19 pandemic on the nursing students education in a public university in Colombia. Invest Educ Enferm. 2023;41(1):e11. https://doi.org/10.17533/udea.iee.v41n1e11 .

Li Sharpe E, Sykes SR, Marzalik PR. Virtual interprofessional learning due to COVID-19. J Nurs Educ. 2021;60(6):346–51. https://doi.org/10.3928/01484834-20210520-09 .

Bourgault A, Mayerson E, Nai M, Orsini-Garry A, Alexander IM. Implications of the COVID-19 pandemic: virtual nursing education for delirium care. J Prof Nurs. 2022;38:54–64. https://doi.org/10.1016/j.profnurs.2021.11.012 .

Download references

Acknowledgements

We are thankful to URY1999, URY20993 (Centro 3i UAysén), and Semilla Ulloa-2018 UAysén all Universidad de Aysén projects, Coyhaique, Chile.

Moreover, this research was supported by project number VIPUCT2023PRO-SV-03 and 2023FIAS-JC-01, funded by Universidad Católica de Temuco, Chile.

Author information

Authors and affiliations.

Facultad de Ciencias de la Salud, Universidad de las Américas, Santiago, Chile

Oriette Soto-Luffi

Grupo de Investigación Educativa en Formación Inicial Docente (GIEFID), Universidad de Las Américas, Santiago, Chile

Cristian Villegas

Instituto de Educación y Lenguaje, Universidad de Las Américas, Santiago, Chile

Núcleo de Investigación en Producción Alimentaria, Universidad Católica de Temuco, Temuco, Chile

Sharon Viscardi

Laboratorio de Investigación Interdisciplinaria en Microbiología Aplicada, Departamento de Procesos Diagnóstico y Evaluación, Facultad de Ciencias de la Salud, Universidad Católica de Temuco, Temuco, Chile

Biotechnology of Functional Foods Laboratory, Camino Sanquilco, Parcela 18, Padre Las Casas, Chile

Laboratorio de Fisiología Vegetal Aplicada, Universidad de Aysén, Coyhaique, Chile

Elizabeth Maria Ulloa-Inostroza

Departamento de Ciencias Naturales y Tecnología, Universidad de Aysén, Coyahique, Chile

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Elizabeth Maria Ulloa-Inostroza .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Soto-Luffi, O., Villegas, C., Viscardi, S. et al. Nursing Education During the SARS-COVID-19 Pandemic: The Implementation of Information and Communication Technologies (ICT). Med.Sci.Educ. (2024). https://doi.org/10.1007/s40670-024-02056-2

Download citation

Accepted : 25 April 2024

Published : 09 May 2024

DOI : https://doi.org/10.1007/s40670-024-02056-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical students
• New technology in education
• Nursing students
• Find a journal
• Publish with us
• Track your research

Nursing Literature Search

• Evidence-Based Practice (EBP)
• Forming a Question
• How to Research
• Types of Resources
• Database Tutorials
• Evaluate Web Sources
• Read a Journal Article
• Create a Synthesis Matrix
• Write a Literature Review
• Cite My Sources in APA 7th?

Your Librarian!

Tips & Tricks for Searching for Resources

Searching for resources and supportive materials in the library catalog or databases shares similarities with searching for information in Google or another search engine. However, there are key differences and knowing these may help with research frustrations.

Keywords, Not Sentences

Your research question(s) might be written out in sentence form, but you won't have much success if you type a full sentence into the database search bar directly.

Rather, you will want to pull out the keywords (main terms) of the research question(s) and use those first. Below is an example search in the nursing database CINAHL . 

What do you notice about how these keywords are entered? 

Phrase Searching, Boolean Operators, and Truncation

Phrase searching  is useful when your search terms consist of two or more words that must stay together in the results. In the example above, using  quotation marks  around "pain management" and "patient education" will ensure those terms are searched as phrases. Otherwise, you might get results using "pain" and "management" separately, which would clog your results page with irrelevant material to go through. This is not necessary practice in every database, but if you are struggling to find relevant results, you might need to use quotation marks.

Boolean operators  are capital  AND, OR,  or  NOT  used between search terms to denote placement in the results. 

• "Include both these terms"
• The example above uses this Boolean operator to make sure the results list include  all  the search terms. 
• Implied in some search fields, like Google Scholar.
• "Include either of these terms"
• More commonly used for synonym searching
• The example above might use it for synonyms to "pain management" , i.e. "pain management"  OR  "pain relief"
• Some databases, like CINAHL, will auto-fill synonyms with Boolean operator OR 
• "Do not include this term"
• Useful if your search terms consistently return irrelevant results by nature of the term, i.e. "pain management" brings up results on orthopedic pain, but you are focused on other kinds of pain
• The example above might be: "pain management" NOT orthopedic
• Use this Boolean operator sparingly, as it might eliminate useful results over a single use of the term

Truncation  is when you use an asterisk to search for all possible branches of a root word. The example above does this with the term  "nurs * " in order to retrieve results that might use "nurses" or "nursing," without having to type each of those options out. The location of the asterisk matters; if you search for "nurse * " you will not get "nursing," and if you search for "nur * " you might get "nurture" or another non-related term. 

Searching As Exploration

Searching for literature and resources seems as though it would be relatively straightforward. However, going into a search with that mentality can lead to disappointment. Rather, looking at the search itself as an exploratory activity might help ease frustrations. Consider the visual below.

It's rare that any kind of research takes a linear path. Even a "clear" research question might turn up alternative questions or topics that you want or need to explore, first. You might also find that a preconceived notion is coming up as incorrect through your research, and now you need to re-evaluate your topic and question. These events happen to seasoned researchers! Try not to be discouraged, and instead view the research process as exploration for answers. 

Bonus! Organization Tips

Having good organization before, during, and after your search session is important. Whether you're able to search for literature and resources in one long session or are limited to short sessions here and there, see if these organization tips help make the task more manageable. 

Before 

• Create a blank Google Doc or Microsoft 365 document. Keep your work on the Cloud, accessible from different locations.
• Research question(s) 
• Possible keywords and synonyms 
• Databases/websites used 
• As you search, keep track of the information for each of those sections. 
• Begin a new section whenever you adjust something major (i.e. move onto a different database, change keywords substantially) 
• This will help you to recreate a search if you need to at another point.
• The APA citation generated by the database or website will likely need to be refined. They are seldom 100% correct. But this is a starting point! 
• Important:  Do not spend your time reading articles during your search phase. Scan titles and abstracts and set aside. Dedicating a search session to only searching will help you stay on task and focused.
• You will more than likely need to edit the APA citation because generated citations are not 100% accurate. But this method creates a starting point.
• Once you've finished a search session and you're ready to review possible articles, create a bulleted list of information from the article that you think is important. 
• << Previous: Researching Literature
• Next: Types of Resources >>
• Last Updated: May 9, 2024 1:59 PM
• URL: https://library.cf.edu/nursing

This paper is in the following e-collection/theme issue:

Published on 9.5.2024 in Vol 7 (2024)

Privacy Barriers in Health Monitoring: Scoping Review

Authors of this article:

• Luyi Sun 1 , MS   ; 
• Bian Yang 1 , PhD   ; 
• Els Kindt 2 , PhD   ; 
• Jingyi Chu 3 , MS  

1 Department of Information Security and Communication Technology, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Gjøvik, Norway

2 Centre for IT & IP Law, Faculty of Law and Criminology, KU Leuven, Leuven, Belgium

3 Administrative Law, Faculty of Law, China University of Political Science and Law, Beijing, China

Corresponding Author:

Luyi Sun, MS

Department of Information Security and Communication Technology

Faculty of Information Technology and Electrical Engineering

Norwegian University of Science and Technology

Teknologivegen 22

Gjøvik, 2815

Phone: 47 96864781

Email: [email protected]

Background: Health monitoring technologies help patients and older adults live better and stay longer in their own homes. However, there are many factors influencing their adoption of these technologies. Privacy is one of them.

Objective: The aim of this study was to provide an overview of the privacy barriers in health monitoring from current research, analyze the factors that influence patients to adopt assisted living technologies, provide a social psychological explanation, and propose suggestions for mitigating these barriers in future research.

Methods: A scoping review was conducted, and web-based literature databases were searched for published studies to explore the available research on privacy barriers in a health monitoring environment.

Results: In total, 65 articles met the inclusion criteria and were selected and analyzed. Contradictory findings and results were found in some of the included articles. We analyzed the contradictory findings and provided possible explanations for current barriers, such as demographic differences, information asymmetry, researchers’ conceptual confusion, inducible experiment design and its psychological impacts on participants, researchers’ confirmation bias, and a lack of distinction among different user roles. We found that few exploratory studies have been conducted so far to collect privacy-related legal norms in a health monitoring environment. Four research questions related to privacy barriers were raised, and an attempt was made to provide answers.

Conclusions: This review highlights the problems of some research, summarizes patients’ privacy concerns and legal concerns from the studies conducted, and lists the factors that should be considered when gathering and analyzing people’s privacy attitudes.

Introduction

The proportion of older adults around the world is growing sharply. According to the 2021 aging report published by the European Commission [ 1 ], the ratio in the European Union (EU) between people aged ≥65 years and those aged 20 to 64 years (also known as the demographic old-age dependency ratio) will increase greatly in the coming decades, from approximately 34% in 2019 to 59% in 2070. In Norway, the population of older adults (aged ≥65 years) will increase from 17.4% in 2019 to 27.8% in 2070 [ 1 ]. Because of this, the term “aging in place” was put forward in social policy, which refers to providing assisted living facilities to enable older adults to remain in their own homes for as long as possible [ 2 ]. Various ongoing research projects in ambient assisted living technologies are being, or have been, conducted to help older adults, such as the European AALIANCE2 project; the Ambient Assisted Living Joint Programme, financed by the European Commission [ 3 ]; and the Active and Assisted Living Programme, also financed by the European Commission [ 4 ].

In the meantime, patients are also in need of health monitoring systems. The aging population and patients’ growing needs for health care support have facilitated the development of different types of health monitoring and assisted living technologies, such as socially assistive robots, wearable trackers, telemedicine, image sensors, and so on. According to the study by Rostad and Stokke [ 5 ], these technologies can be categorized into localization technologies (eg, GPS), compensation technologies (eg, remote control of light and heating, robot vacuums, and cognitive or physical aids), safety technologies (eg, social alarm systems and fall detection sensors), technologies for social contact (eg, tablet computers, smartphones, and gaming devices), therapeutic robots, and treatment technologies (eg, medical remote monitoring and automated pill dispensers), all used for different purposes in health monitoring.

The application of the aforementioned intelligent devices is supposed to enhance the quality of patients’ lives. Nevertheless, various factors impact patients’ acceptance of health monitoring devices [ 6 ], including intervention factors (eg, privacy concerns, security concerns, frequency, a lack of awareness, and the observability of outcomes), environmental factors (eg, social influence, social implication, change in technology use by society over time, and group participation), technology factors (eg, complexity, reliability, interface design, compatibility, functionalities, features, aesthetics, and cost), individual factors (eg, level of innovativeness, level of technology learnability, and living alone), psychological factors (eg, attitudinal factors and perception factors), support and training factors (eg, training, quality of training, and facilitating support) in general. Among all these factors, our attention was seized by privacy concerns .

There are different types of privacy; therefore, privacy concerns can be analyzed from different perspectives. The way that researchers distinguish privacy has reflected their different concerns arising from their professional backgrounds; for example, Rosenberg [ 7 ] distinguished 3 kinds of privacy: territorial privacy, individual privacy, and informational privacy. Clarke [ 8 ] outlined 4 types of privacy: privacy of a person, privacy of personal data, privacy of personal behavior, and privacy of personal communication. However, with the development of emerging technologies, different categories arose. Some researchers extended the categories formulated by Rosenberg [ 7 ] and added 3 more categories: privacy of thoughts and feelings, privacy of location and space, privacy of association (including group privacy) [ 9 ]. More specifically, in different scenarios, users have different privacy concerns, and these concerns can be categorized case by case; for instance, web-based social network users can have 4 dimensions of privacy concerns: virtual territorial privacy, factual privacy, interactional privacy, and psychological privacy [ 10 ]. Furthermore, Serenko [ 11 ] narrowed the scope in health care and put forward 3 privacy dimensions: informational privacy, physical privacy, and psychological privacy. These 3 privacy dimensions were regarded as determinants of patient behavior in health care.

Although privacy concerns are known to be barriers for patients with regard to adopting health monitoring technologies according to some studies [ 12 ], it is not sufficient to analyze factors impacting the adoption of health monitoring technologies individually because these influential factors may impact each other; for example, despite the fact that privacy concerns are included in intervention factors [ 6 ], psychological factors, as well as support and training factors with regard to privacy awareness, can also impact patients’ privacy concerns. Patients’ awareness of privacy-related laws in the health care environment will also influence their privacy concerns and decision-making out of respect for authority [ 13 ]. Considering privacy issues in society, privacy attitudes and concerns are always analyzed from the legal perspective. For patients, their privacy attitudes may have a straight impact on the informed consent process, and they are expected to know how to obtain legal aid in case they encounter technology abuse, or their privacy is intruded upon. For technology vendors, industry standards or privacy policies need to be carefully checked and complied with before their product is released. They need to carry out data protection impact assessments to minimize privacy risks [ 14 ]. Apart from older patients who are most in need of health monitoring and assisted living technologies, members of the general public are also potential users of these technologies as they age or develop health conditions. Furthermore, with regard to members of the general public, their prevalent uncertainty regarding, and trust issues with, technologies may prompt legislators to adopt a more cautious and conservative approach with regard to such technologies. However, privacy-friendly approaches can be seen as a way to motivate technology vendors to be more transparent and, on the one hand, foster privacy by design , while, on the other hand, promote social awareness and trust by bridging the information gap. In all, various factors and their relationships need to be always taken into consideration at the same time. Meanwhile, the question persists as to whether privacy concerns are truly barriers because of the rapid changes in society, such as the development of privacy-enhancing technologies. Thus, the rationality of privacy concerns should also be discussed.

In the past few years, researchers have conducted a series of studies to gather different privacy attitudes or privacy concerns regarding health monitoring and assisted living technologies from people with different demographic characteristics. However, the results vary from study to study not only because of the different user groups to which the participants belong (eg, older adults’ attitudes compared to those of younger adults and female participants’ attitudes compared to those of male participants) but also because of the different experimental approaches used and the different scenarios provided in these studies.

Research Questions

In summary, the implementation of assisted living technologies in the aging population era faces several challenges. First, multiple factors impede patients’ adoption of these technologies, and the interrelations among these factors have not been thoroughly explored. Second, the extent to which privacy concerns affect technology adoption needs further investigation. Third, the study design and current results require consolidation for researchers to make meaningful improvements. Addressing these challenges, we pose 4 research questions and conduct a scoping review with the aim of providing an overview of the privacy barriers in health monitoring from current research and elucidating answers to these research questions. The four research questions are listed herein, and the answers to these questions are provided in the Results section.

• What are the influential factors that lead to different privacy attitudes and concerns?
• How will the methodologies used in different studies influence participants’ privacy awareness with regard to health monitoring technologies from the perspective of social psychology?
• What are the legal challenges regarding people’s privacy attitudes and concerns today?
• What should be taken into consideration in subsequent studies related to privacy attitudes and concerns in the context of social psychology?

On the basis of the selected articles, we have summarized 5 hypotheses particularly related to the second challenge, which are clarified in the Results section. The contribution of this paper includes observing the inconsistency of these hypotheses, looking into experimental approaches in each article, and seeking answers to the 4 research questions. We have tried to come up with suggestions that should be taken into consideration comprehensively before implementing health monitoring technologies.

A scoping review was conducted to explore the privacy attitudes of different groups of participants in the context of legal norms and social psychology in health monitoring technologies by adopting the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement ( Multimedia Appendix 1 ) [ 15 ]. Scoping reviews include all quantitative, qualitative, and mixed methods studies that are identified as literature on a particular topic or research area [ 16 , 17 ]. They differ from systematic reviews but can be used to inform systematic reviews because more specific questions are usually addressed in a more precise systematic review [ 18 ]. Of note, there are other approaches to evidence synthesis for systematic reviews, such as realist reviews [ 19 ], mixed methods reviews [ 19 ], concept analyses [ 20 ], and so on. In this study, with the aim of identifying and mapping the available studies, examining how research is conducted in a certain field, summarizing findings, and analyzing results, a scoping review is the best choice compared to other approaches.

Eligibility Criteria

We conducted a review for articles published between January 1, 2016, and March 31, 2022. Search parameters were established to identify articles published during this period regarding different participants’ privacy attitudes with regard to health monitoring technologies as well as legal norms regarding privacy in health monitoring in Norway, the EU, and the United States. For an in-depth investigation into the research questions, we acknowledge the regional characteristics evident in previous studies, often shaped by factors such as cultural backgrounds and legal norms. To address this, we have selected these regions. Specifically, this decision is motivated by 2 key considerations. First, EU policies extend to Norway, the authors’ country of residence, thereby potentially impacting health service delivery and the deployment of assisted living technologies. Second, certain EU countries share a common cultural background, suggesting that individuals in these regions may harbor more similar privacy perspectives than individuals in other locations. Furthermore, studies published in the United States were included because it is one of the most developed countries owning quantities of health monitoring technologies. It is worth mentioning that for the articles we identified, even if the authors did not specify the review region or if the authors’ countries of residence were outside the region, we still included these articles because they provided comprehensive views. Other than region specification, studies were included if they (1) reflected the privacy attitudes or privacy concerns of any group of people, (2) reflected any legal concerns or legal frameworks that should be taken into account, (3) were peer-reviewed publications, and (4) were written in English. All study methods (quantitative, qualitative, and multimethod) were eligible for the review.

Search Terms, Strategy, and Sources

Instead of searching for privacy barriers directly, we sought studies relevant to people’s privacy attitudes or legal norms regarding privacy in the health monitoring environment and tried to summarize the barriers described in these studies. The literature search was conducted by listing the following search terms in the search string: (“privacy attitudes” OR “legal norms”) AND ((“healthcare monitoring” AND “nursing homes”) OR “homecare monitoring”) AND (“Norway” OR “EU” OR “the U.S.”). The sources of the articles on privacy attitudes and legal norms were mainly 5 databases: Semantic Scholar, PubMed, IEEE Xplore, ScienceDirect, and Scopus. As no relevant articles met the eligibility criteria in IEEE Xplore and ScienceDirect, only articles in the rest of the 3 databases were included. Additional works identified in other databases, such as ACM Digital Library, were categorized into other sources because we sorted the articles by relevance and scanned the results directly based on the title and abstract provided at the first attempt instead of following the PRISMA steps strictly, which was the approach we followed for the 5 main databases. Therefore, instead of making an exhaustive selection, we merely added the most relevant and important works.

Study Selection

The PRISMA flow diagram is presented in Figure 1 . The search process resulted in the identification of 953 studies. Before the screening, 122 (12.9%) duplicate records were removed from these 953 studies. The first screening was performed on the title, abstract, and language, and 341 (41%) of the 830 articles were identified as not meeting the eligibility criteria. Of the remaining 489 articles, 226 (46.3%) could not be retrieved, leaving 263 (53.7%) reports for assessment. After the second screening, of the 263 articles, we excluded 198 (75.2%) because they (1) were not conducted in the regions specified, (2) were not relevant to privacy attitudes or privacy concerns, and (3) were not relevant to health monitoring or assisted living technologies, leaving 66 (24.8%) articles for the final review, from which we extracted and categorized useful information.

Data Extraction and Categorization

The methods for data extraction and categorization were established through the literature review process. Useful information was extracted and input into a form, including title; authors; year of publication; region; topic; technology; participant inclusion criteria (if any); study design; location of the study; key findings; and laws, rules, regulations, directives, and policies mentioned.

The categories we formulated are presented in Textbox 1 .

• Article information: title, authors, year of publication, and region
• Topic: identified and categorized based on the field covered by the articles; instead of setting the topic as “privacy attitudes” or “legal norms” in general, 5 topics were defined (privacy attitudes, privacy concerns, legal concerns, legal frameworks, and privacy barriers); some of the articles have covered several topics at the same time; reasons for classification are clarified in the Results section
• Technology: includes health monitoring technologies mentioned in the article; some of the articles have covered a specific device (eg, human behavior modeling [ 21 ]), while some provide privacy attitudes or legal norms of a general designation, such as smart home technology
• Participant inclusion criteria: specifically created for studies with participants invited to take part; inclusion criteria include the number of participants, age, nationality, whether they have any diseases, and place of residence
• Key findings: any information related to the 5 topics listed in the Topic category
• Laws, rules, regulations, directives, and policies mentioned: specifically created for studies covering legal frameworks or legal barriers; all legal documents mentioned in the articles were extracted

In this section, we will provide the results in the form of categories. As mentioned in the Eligibility Criteria subsection, the review was focused on Norway, the EU, and the United States, or reviews worldwide. When it comes to review articles, they are included regardless of region. Among the 65 articles, there were 4 (6%) from France, 2 (3%) from Finland, 1 (2%) from Sweden, 3 (5%) from Germany, 1 (2%) from Ireland, 2 (3%) from Italy, 3 (5%) from the Netherlands, 5 (8%) from Norway, 2 (3%) from Poland, 1 (2%) from Portugal, 1 (2%) from Spain, 1 (2%) conducted jointly in Germany and Denmark, and 30 (46%) from the United States. Of the remaining 9 articles, 1 (11%) discussed telemonitoring at the EU level; 1 (11%) reviewed laws, standards, and recommendations applicable at the EU level; and 7 (78%) were literature reviews whose authors’ countries of residence were not part of the specified regions. As the results of the reviews were general in nature, they were not categorized into specific areas.

Privacy Attitudes, Privacy Concerns, and Privacy Barriers

Articles reflecting privacy concerns were classified into 3 subcategories: privacy attitudes, privacy concerns, and privacy barriers. According to Kokolakis [ 22 ], although privacy attitudes and privacy concerns have a close relationship, they differ from each other because privacy attitudes are bound to specific contexts such as the appraisal of specific privacy behaviors, while privacy concerns are not, and they can be generic. It is worth mentioning that the articles that either gathered participants’ general privacy concerns or participants’ privacy attitudes were included in the review. The key findings extracted from the results should be categorized as privacy concerns according to the definition. However, we set the topic as privacy attitudes if any privacy attitudes were gathered in these studies. Thus, articles covering either privacy concerns or privacy attitudes were included when we compared the findings.

In contrast to the studies that gathered and analyzed participants’ privacy attitudes or concerns, 11 (17%) of the 66 studies [ 23 - 33 ] discussed people’s adoption of health monitoring technologies. Although some of these studies, such as the study by Charness et al [ 26 ], also recruited participants, gathered their privacy attitudes, and analyzed results from the attitudes (similar to the methods used by Sánchez et al [ 21 ] and Caldeira et al [ 34 ]), these studies provide a broad view from the perspective of technology adoption and acceptance; for instance, in the study by Biermann et al [ 25 ], researchers came up with several barriers to technology adoption, such as financial reasons, restriction of privacy, and a feeling of surveillance. Among all barriers, concern regarding privacy is merely one of the factors that may influence participants’ technology adoption. Therefore, even if the findings reflecting privacy concerns are similar to those reflecting privacy attitudes and privacy barriers, articles stating that concern regarding privacy is one of the barriers to the adoption and acceptance of assisted living technologies were classified into a different category: privacy barriers.

Contradictions Among Perceptions of Privacy in Key Findings

All included studies reflect privacy attitudes, privacy concerns, and privacy barriers, which show many contradictions. In general, the results can be classified into five hypotheses according to the extent of participants’ privacy concerns: (1) participants do not have privacy concerns ( Table 1 ); (2) participants have privacy concerns, and they are a major barrier ( Table 2 ); (3) participants have privacy concerns, but they do not seem to be a significant barrier ( Table 3 ); (4) participants’ privacy concerns vary from person to person, and there is insufficient statistical evidence across a large population to validate whether the concerns will have a significant influence ( Textbox 2 ); and (5) participants have privacy concerns, and their perceptions of privacy are influenced by their background ( Table 4 ). It is worth mentioning that some of the studies satisfied 2 hypotheses at the same time because the fifth hypothesis does not violate the second, third, or fourth hypothesis.

Study and methodology

• Randall et al [ 50 ]: focus group or qualitative approach
• Gerłowska et al [ 51 ]: literature review
• Soro et al [ 52 ]: review
• Wang et al [ 53 ]: focus group+survey (multimethod approach)
• Pilozzi and Huang [ 54 ]: no methods directly related to the results
• Pekmezaris et al [ 55 ]: focus group+interview (multimethod approach)
• Biermann et al [ 25 ]: web-based questionnaire (empirical quantitative approach)
• Kodate et al [ 56 ]: questionnaire (quantitative approach)
• Berridge et al [ 57 ]: survey (qualitative approach)
• Mittelstadt [ 58 ]: systematic survey
• Koo and Fallon [ 59 ]: interview (qualitative approach)
• Joe et al [ 60 ]: focus group+questionnaire (multimethod approach)
• Chan et al [ 27 ]: literature review
• Chan et al [ 29 ]: literature review
• Sánchez et al [ 61 ]: literature review
• Hjelm et al [ 62 ]: semistructured interview (qualitative approach)
• Cristiano et al [ 63 ]: interview+focus group (qualitative approach)
• Zhang et al [ 64 ]: questionnaire (quantitative approach)
• Mallinson and Shafi [ 32 ]: review
• Guazzini et al [ 65 ]: questionnaire+focus group (multimethod approach)
• Wan et al [ 66 ]: literature review+semistructured interviews (multimethod approach)
• Zheng et al [ 67 ]: semistructured interviews (qualitative approach)
• Yao et al [ 68 ]: focus group+co-design activities (qualitative approach)
• Ahmad et al [ 69 ]: semistructured interviews (qualitative approach)
• Kheirinejad et al [ 70 ]: questionnaire (quantitative approach)

The aforementioned contradictory hypotheses have led to the formulation of our research questions, which we attempt to answer in this section.

Research Question 1: What Are the Influential Factors That Lead to Different Hypotheses?

To identify the influential factors, we looked into the methodologies used in these studies in detail. To sum up, qualitative, quantitative, and multimethod studies as well as reviews were included in these studies. Except for reviews, the other 3 approaches recruited participants during the study. On the basis of the participant inclusion criteria, we found that the number of participants would influence the results. For those studies that concluded that the majority of participants do not have privacy concerns, the number of participants recruited was small [ 21 , 34 ]. Hence, one could argue that there might have been sampling bias in the qualitative approach applied by the studies conducted. As the number of participants increased, the fact that people had privacy concerns seemed to become a common conclusion.

Nevertheless, it still seemed hard for researchers to come to an agreement on the importance of privacy issues. Some stated that privacy is an important barrier without verification and regarded it as a consensus [ 35 ]. However, according to a scoping review of ambient assisted living technology adoption, most studies found that privacy was not a barrier [ 28 ]. Because of the uncertainty mentioned above [ 22 ], we agree with the fourth and fifth hypotheses. Although some of the studies presented in Textbox 2 have not provided sufficient evidence in support of any conclusions, we regard this fact as indirect evidence for the fourth hypothesis as well. Furthermore, people’s privacy concerns, as presented in Table 4 , may be influenced by the following factors: (1) context and type of technology; (2) age; (3) health needs; (4) personal trust and the device’s usability; (5) trade-off among privacy, autonomy, assistance, safety, or independence; (6) health status; (7) region; (8) gender; (9) user roles; (10) sociocultural context; (11) emotion; (12) previous knowledge; (13) personality; and (14) potential use of personal data.

Research Question 2: How Will the Methodologies Influence Participants’ Privacy Awareness With Regard to Health Monitoring Technologies From the Perspective of Social Psychology?

Even if most of the studies satisfied the fourth and fifth hypotheses, the researchers’ confirmation bias could have influenced the results and participants’ answers. More specifically, in reviews, such bias exists when researchers search for evidence that can support their own beliefs [ 80 ]. For the other 3 approaches (qualitative, quantitative, and multimethod), researchers’ confirmation bias could also impact their interaction with participants, such as raising inducible questions or providing insufficient information [ 81 ]. It has already been pointed out that most people lack the cognitive ability to calculate privacy risks and to make rational privacy decisions because of incomplete information, bounded rationality, and information asymmetries [ 49 ]. Therefore, the information provided to the interviewees might compel them to give answers that match researchers’ expectations. For multimethod studies that include several experiments, the design of the experiments will also guide participants to make different privacy decisions; for example, because several studies found that there is a trade-off among privacy, autonomy, assistance, safety, or independence, we believe that a privacy-related question in the first experiment might encourage the participants to be concerned more about privacy rather than autonomy in the experiments that follow.

Furthermore, social influence in groups should also be emphasized because participants’ privacy awareness might be influenced not only by the sociocultural context but also by the other participants; for example, a herd mentality can lead participants to converge on a consensus answer and make irrational privacy decisions [ 82 ].

On the one hand, researchers found that the wisdom of small groups of people tends to outstrip that of both individuals and a large group of people. According to one of the findings, when there are 4 groups, and the number of participants in a focus group is 5, although opinions within a group might converge, there are still diversities among the different groups, and researchers will be able to gather different views from these groups [ 82 ]. On the basis of these findings, we analyzed the experiment design of focus groups in the selected studies and found that most experiments lacked diversity in terms of participants’ backgrounds [ 35 , 50 , 55 , 63 ].

On the other hand, even if the diversity in terms of participants’ backgrounds is enhanced, the results of a group cannot always represent personal privacy attitudes because of group polarization [ 83 ], that is, it remains questionable whether the decisions made by these groups can represent the views of individuals in the group accurately. In all, we cannot ignore the need and significance with regard to clarifying the ultimate goal of gathering and analyzing privacy attitudes.

Legal Concerns and Legal Frameworks

Legal norms, overall, refer to social norms that are enforced by a relatively strong degree of coercion [ 84 ]. However, few of the articles we identified can be categorized into the topic of legal norms straightforwardly. Therefore, instead of categorizing them into legal norms, we classified the articles describing legal issues into 2 subcategories: legal concerns and legal frameworks. More precisely, the articles reflecting legal concerns were not describing participants’ concerns about the existing legal frameworks; rather, they were describing participants’ attitudes toward technology adoption in light of their awareness of legal obligations; for example, Sánchez et al [ 21 ] presented the fact that even if participants were aware of municipalities’ legal obligations to provide health care services for older people, they preferred to buy anything they could afford or adopt welfare technologies. The study by Sánchez et al [ 61 ] also did not present any legal concerns directly gathered from participants, but it highlighted the importance of legal liability for different user roles (physicians, nurses, or relatives of the patients) during a visit to patients and regarded it as legal concern. As a matter of fact, among the studies we selected, there were only a few conducted for gathering people’s legal concerns regarding privacy with respect to health monitoring and assisted living technologies. However, people’s legal concerns regarding privacy and the problems in current legal frameworks might be considered one of the influential factors when it comes to adopting health monitoring technologies, which constitute one of the privacy barriers.

Regarding legal frameworks, we extracted regulations, laws, policies, directives, and rules from the findings ( Textbox 3 ). These documents are not limited to assisted living technologies; rather, they cover legal aspects in health care in general; for instance, the legal challenges in the home care or health care environment include data privacy, data management, stakeholders’ interests, and informed consent [ 85 ].

It is worth mentioning that not all legal documents concerning health monitoring or assisted living technologies are exhaustively listed in Textbox 3 because different countries have different laws or rules regulating aspects of health care. Some of the articles pointed out the shortcomings in the existing frameworks. Among these articles, Ryu [ 91 ] revealed the fact of the absence of legal guidelines in the mobile health domain regarding privacy and confidentiality in more than half of the EU countries and the United States and suggested that mobile health should be included within the framework in different countries; Ambrosino et al [ 92 ] provided the conclusion that a full legal framework for telemedicine was still lacking in European countries; and Sánchez et al [ 61 ] stated that the standardization, research, and assessment of the legal aspects should be addressed in an international perspective. However, in this paper, we only focus on the impact of legal norms on privacy concerns.

Study and the laws, rules, regulations, directives, and policies mentioned

• Sánchez et al [ 21 ]: Norwegian Municipal Health and Care Services Act of 2011 (ACT 24/06/2011 no. 30; act relating to municipal health and care services, and so on)
• Gerłowska et al [ 51 ]: European Parliament resolution of February 16, 2017, with recommendations to the European Commission on Civil Law Rules on Robotics (European Parliament, 2017)
• Garg et al [ 86 ]: Health Information Technology for Economic and Clinical Health Act, United States (2009); Health Insurance Portability and Accountability Act, United States (1996)
• Costa et al [ 87 ]: Article 8 of the Charter of Fundamental Rights of the European Union (2010); Article 16 of the Treaty on the Functioning of the European Union (consolidated version of the Treaty on the Functioning of the European Union, 2012); Portuguese data protection law; General Data Protection Regulation
• Jin et al [ 88 ]: Health Insurance Portability and Accountability Act, United States (1996)
• Garzo and Garay-Vitoria [ 89 ]: Regulation 2016/679 (also known as General Data Protection Regulation); Regulation 2017/745 on medical devices (2017); Regulation 536/2014 relating to clinical tests with medication for human use (2014); harmonized standard ISO 14155 related to good clinical practice (International Organization for Standardization, 2020)
• Ross et al [ 90 ]: General Data Protection Regulation; Health Insurance Portability and Accountability Act, United States (1996)

Research Question 3: What Are the Legal Challenges Regarding People’s Privacy Attitudes and Concerns Today?

The articles included in the review show the absence of research on legal norms regarding privacy or people’s legal concerns with regard to assisted living technologies in the health monitoring environment; for example, informed consent for various scenarios in health care, different user roles (eg, device owners, bystanders [ 68 ], and technology developers), and different types of health monitoring technologies require researchers to pay more attention to the legal frameworks rather than merely point out that they are inadequate.

To this end, a few of the included studies have investigated patients’ informed consent requirements in a health monitoring environment. As stated by Demiris and Hensel [ 93 ], when patients or older adults approach the end of their lives, they have opportunities to become familiar with smart home applications and perhaps change their minds and consent to use them in light of their value. Patients with cognitive impairment [ 68 ] who are gradually losing their cognitive ability to make decisions might prefer to disclose more information in exchange for better medical help when giving informed consent. Thus, informed consent requirements need to be updated.

This also applies to privacy decision-making in health care; for instance, informed consent could be obtained through a shared decision-making framework [ 93 ]. Generally, informed consent includes data processing, such as storage, transmission, collection, erasure, and sharing. However, when it is applied to a specific field, more concrete explanations of the risks and benefits need to be provided. In shared decision-making, which requires the involvement of patients and clinicians, informed consent serves as a legal process used to promote patient autonomy and self-determination as well as legal rights [ 94 ]. While shared decision-making includes treatment decision-making, it can also include, for example, privacy decision-making because it allows people to discuss how confidential information can be used and shared [ 95 , 96 ].

As shared decision-making is a collaborative process and aims to help patients better understand problems and make rational decisions with support from clinicians, both patients’ and clinicians’ opinions need to be taken into consideration. To be compliant with patients’ privacy needs and the cognitive changes they may be experiencing, we believe that informed consent requirements need to be updated continually as well [ 97 ].

The aforementioned cases only serve as examples of applications of legal concerns. More scenarios and elements remain to be clarified, such as identifying direct and indirect stakeholders and their responsibilities and distinguishing the need for informed consent when there are more user roles to be considered (formal caregivers as well as informal caregivers such as friends or relatives) in the health monitoring environment. Some scholars have pointed out that informed consent is not always necessary if the disclosure of information is consistent with respect for underlying human dignity or individual autonomy, which is referred to as “reasonable expectations of privacy” [ 98 , 99 ]. They argue there are circumstances in which confidential information can be better protected, precluding the need to rely on implied consent. By shifting from implied consent to “reasonable expectations of privacy,” the pressure to classify cases as implied consent could be eased [ 100 ].

As researchers are currently focusing more on reasonable expectations of privacy with regard to the sharing of confidential health information, reasonable expectations of privacy for adopting health monitoring and assisted living technologies or privacy decision-making concerning these technologies can possibly be taken into account in data protection legislation as well. Although we agree that reasonable expectations of privacy can help reduce participants’ burden when giving consent, the scope of reasonable expectations of privacy still relies on social psychological factors, such as the quality of the physician-patient relationship [ 99 ]; for example, trust between physicians and patients will increase the level of reasonable expectations of privacy when patients are making decisions, such as whether to allow the health monitoring system to send alerts to the clinician staff under some circumstances.

Research Question 4: What Should Be Taken Into Consideration in Subsequent Studies Related to Privacy Attitudes and Concerns in the Context of Social Psychology?

Vassli and Farshchian [ 23 ] state that one of the most cited reasons that the authors found that might influence participants’ adoption of assisted living technologies was that monitoring devices made them feel observed. This has inspired us to suggest experiments (refer to the following paragraphs) that should be conducted in future studies.

First, as far as we could find, the selected studies had not looked into the problem of the Hawthorne effect [ 101 ], which refers to a phenomenon in which people alter their behavior in response to being watched or monitored, that is, they might make an instantaneous modification in their behavior once they become aware that they are being observed. People behave differently even when looking into a mirror (rather than being watched by someone else) [ 102 ]. In this sense, installing monitoring devices might affect people’s behavior even if they have consented to the use of these technologies.

Holden [ 103 ] suggested in 2001 that the possible presence of a Hawthorne effect could lead to participants drawing conclusions subconsciously. Therefore, we cannot predict the influences wrought by the Hawthorne effect, while this remains of key importance because it will consequently impact user experience and influence their decision-making in real life. Although some participants in the studies by Vassli and Farshchian [ 23 ] and Biermann et al [ 25 ] tended to ignore the feeling of being observed, the Hawthorne effect can cause positive impacts as well; for example, in the study by Cristiano et al [ 63 ], even if participants had negative feelings of privacy intrusion when being monitored, this was not always the case because older adults stated that they felt secure when being monitored. This also reflected the trade-off between privacy concerns and security concerns. The researchers claimed in their paper that negative feelings of privacy intrusion could be overcome by providing older adults with appropriate information. In another study of clinical trials in dementia, researchers who were aware of the Hawthorne effect found that more intensive follow-ups would cause better cognitive functioning outcomes [ 104 ]. Another observation from the Norwegian University of Science and Technology Nord-Trøndelag Health Study [ 103 ] showed that participants surveyed by the project regularly over many years exhibited statistically better health states than those not surveyed, which might be attributed to the Hawthorne effect as well. In this way, the feeling of being observed can turn out to be a good thing, although some technology researchers try to hide the monitoring devices to reduce patients’ feeling of being observed.

In all, the Hawthorne effect is a complex phenomenon that can lead to unknown bias. There should be more experiments to compare patients’ or older adults’ behaviors when they are aware of being observed and their behaviors with hidden observation during such research.

Second, in the follow-up experiment design of observing participants’ privacy behaviors, researchers should keep an eye not only on the privacy paradox phenomenon but also on the stress of cognitive dissonance caused by the phenomenon. The privacy paradox reveals the fact that there are discrepancies between users’ self-declared privacy attitudes and their privacy behavior [ 105 , 106 ]. These discrepancies will cause cognitive dissonance, which appears when people hold conflicting beliefs, or their behaviors contradict their beliefs [ 107 , 108 ].

It is mentally stressful to cope with contradictory experiences or beliefs, and cognitive dissonance will make conditions for patients or older adults in health care settings even more stressful [ 109 ]. But there can also be positive effects if researchers use a patient’s or an older adult’s motivation to mitigate the dissonance to change their behaviors [ 105 ]. A few researchers also found that the contrast between privacy concerns and privacy-protecting behaviors is caused by privacy fatigue [ 106 ], referring to the reduced intention of privacy protection when faced with the increasing complexity of privacy settings or regulations. Because of this, some participants even became confused about the laws or regulations and lacked the ability to make appropriate decisions or give consent [ 107 ].

Third, there is a lack of longitudinal studies on privacy attitudes with regard to assisted living technologies. Even if some studies had adopted multimethod approaches, and participants had been invited to take part in several experiments, it is hard for us to identify their cognitive changes over time. Because of the problem we have outlined in research question 1, current experiments might even induce participants to give the answers we want. Thus, we need long-term studies to test patients’ cognitive changes over the technologies.

From the findings we extracted, we aimed to provide a comprehensive understanding of privacy barriers in health monitoring. We have explained the interaction of different factors, especially people’s privacy concerns and legal concerns, and pointed out the impact of social psychological factors on these factors. We suggest that to ensure people’s autonomy while protecting their privacy, the rules applied to them need to meet their demographic characteristics, health conditions, and health needs. Among the listed hypotheses and research questions, we tend to support the fourth hypothesis: people’s privacy concerns vary from person to person, and there is insufficient evidence to validate the importance of privacy barriers currently. As information asymmetries will also lead people to make different privacy decisions, we suppose that the more accurate and useful the information they provide, the more precise the decisions they will make. To intuitively present the influential factors we found in research question 1, we highlight the elements that should be considered and analyzed to measure a person’s privacy concerns ( Figure 2 ). The categorization of the elements is flexible; for instance, both social trust and technical trust can affect privacy concerns (refer to the inner relations among the elements [solid lines] and subelements [dashed lines] plotted in Figure 2 ).

On the basis of these factors, we have determined that there are few studies investigating the privacy attitudes of other user roles with regard to these technologies. We found that, of the 66 included articles, only 1 (2%) [ 74 ] has conducted interviews with caregivers, while 2 (3%) [ 68 , 69 ] discuss the privacy concerns of bystanders. In addition, there are only a few studies that were not conducted within the specified geographic regions that collected technology researchers’ perceptions of ethical issues (privacy is one of the ethical issues interviewed) with regard to smart home technologies [ 108 ]. Although bystanders will not be the majority of the users of assisted living technologies, we insist that privacy concerns should be gathered from them too. In addition, clinicians’ and technical researchers’ views need to be explored because they can provide more information based on their professional background. Therefore, more studies are required to be conducted from the perspective of different user roles, enabling shared privacy decision-making among them.

This review also reveals the problems in current research, such as an insufficient number of participants recruited, a lack of diversity regarding focus groups, the confirmation bias of researchers during study design, and the fuzzy definitions of different concepts, and provides suggestions for some of the barriers especially from the perspective of social psychology, such as improving cognitive functioning by applying the Hawthorne effect or reducing cognitive inconsistency by using cognitive dissonance.

To help participants make more accurate and stable decisions, we suggest that more empirical studies should be conducted that observe participants’ behaviors and measure the distances between behaviors and attitudes. In combination with the self-perception theory [ 110 ], the participants are expected to observe themselves, notice the inconsistencies, and interpret their attitudes from their behaviors. In subsequent steps, researchers can also guide the participants appropriately based on the social learning theory [ 109 ], notify participants about the inconsistencies, and ask them to adjust their attitudes or behaviors to reduce the distances. On the basis of the newly gathered attitudes, participants’ preferences and behaviors in real life are expected to be predicted more precisely on the machine level, by using appropriate predicting algorithms.

Study Limitations

Despite all the interesting findings, we acknowledge the limitations of the review. First, the scope of findings deviated somewhat from the search terms we set at the beginning. Because of the limited number of articles identified regarding legal concerns and legal frameworks, the legal frameworks and documents that we have listed are not exhaustive; therefore, we have not ventured in depth in this direction. Although we have classified the findings into subcategories of our creation, we cannot deny the fact that few studies are directly related to legal norms regarding privacy in health monitoring. Second, although some search terms were updated continually based on the new ideas we generated, to be compliant with the inclusion criteria (eg, the region specification), some important studies might have been excluded, although their findings may not be applicable and adaptable to the authors’ country of residence. However, we encourage future works to be carried out in other regions to obtain a more comprehensive overview of the problem. Last but not least, although we have tried to interpret the findings from the perspective of social psychology, the evidence we have presented is inconclusive, and they remain to be investigated in long-term studies.

Conclusions

This scoping review has synthesized existing published research on privacy barriers with regard to the adoption of assisted living technologies. On the basis of the findings and main topics, the studies were classified into five categories: (1) privacy attitudes, (2) privacy concerns, (3) legal concerns, (4) legal frameworks, and (5) privacy barriers. Subsequently, we investigated the methodology and participant inclusion criteria. We have listed the factors that influence people’s privacy concerns and analyzed the social psychological influence of the experiments on people’s privacy awareness. Example legal challenges regarding privacy attitudes have been put forward, and the interaction between privacy factors and legal factors has been discussed. Future research might involve longitudinal studies on the privacy attitudes of different user roles and the informed consent obtained, with more psychological impacts such as the Hawthorne effect and confirmation bias carefully considered.

Acknowledgments

This work has been cofunded by the Initiative for ICT and digital innovation program of the Research Council of Norway under the scope of and as part of the outcome from the research project Reinforcing the Health Data Infrastructure in Mobility and Assurance through Data Democratization (Health Democratization, 2019–2024, project number 288856).

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

• The 2021 ageing report: underlying assumptions and projection methodologies. The European Commission. URL: https:/​/economy-finance.​ec.europa.eu/​publications/​2021-ageing-report-underlying-assumptions-and-projection-methodologies_en [accessed 2024-04-04]
• Lewis C, Buffel T. Aging in place and the places of aging: a longitudinal study. J Aging Stud. Sep 2020;54:100870. [ FREE Full text ] [ CrossRef ] [ Medline ]
• The Ambient Assisted Living joint programme. ERA LEARN. URL: https://www.era-learn.eu/network-information/networks/aal-jp [accessed 2023-10-09]
• Active and Assisted Living 2. ERA LEARN. URL: https://www.era-learn.eu/network-information/networks/aal-2 [accessed 2023-10-09]
• Rostad HM, Stokke R. Integrating welfare technology in long-term care services: nationwide cross-sectional survey study. J Med Internet Res. Aug 16, 2021;23(8):e22316. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vichitvanichphong S, Talaei-Khoei A, Kerr D, Ghapanchi AH. Assistive technologies for aged care: comparative literature survey on the effectiveness of theories for supportive and empowering technologies. Inf Technol People. 2018;31(2):405-427. [ CrossRef ]
• Rosenberg RS. The Social Impact of Computers. Amsterdam, The Netherlands. Elsevier; Feb 1992.
• Roger Clarke's privacy introduction and definitions. Xamax Consultancy. URL: http://www.rogerclarke.com/DV/Intro.html [accessed 2023-10-10]
• Gutwirth S, Leenes R, de Hert P, Poullet Y. European Data Protection: Coming of Age. Dordrecht, The Netherlands. Springer; 2013.
• Wang A, Zhang AN, Xu Y. Privacy in online social networks. In: Proceedings of the 11th International Conference on Computer and Information Science. 2012. Presented at: ICIS 2012; May 30-June 1, 2012; Shanghai, China. [ CrossRef ]
• Serenko N. Informational, physical, and psychological privacy as determinants of patient behaviour in health care. In: Patient Safety and Quality Care through Health Informatics. Hershey, PA. IGI Global; 2014.
• Ferguson C, Hickman LD, Turkmani S, Breen P, Gargiulo G, Inglis SC. "Wearables only work on patients that wear them": barriers and facilitators to the adoption of wearable cardiac monitoring technologies. Cardiovasc Digit Health J. Apr 06, 2021;2(2):137-147. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tyler TR, Lind EA. A relational model of authority in groups. Exp Soc Psychol. 1992;25:115-191. [ FREE Full text ] [ CrossRef ]
• Data protection impact assessment (DPIA). General Data Protection Regulation. URL: https://gdpr.eu/data-protection-impact-assessment-template/ [accessed 2023-10-09]
• Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. Jul 21, 2009;6(7):e1000100. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. Dec 2014;5(4):371-385. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Price ML, Surr CA, Gough B, Ashley L. Experiences and support needs of informal caregivers of people with multimorbidity: a scoping literature review. Psychol Health. Jan 2020;35(1):36-69. [ CrossRef ] [ Medline ]
• Armstrong R, Hall BJ, Doyle J, Waters E. Cochrane update. 'Scoping the scope' of a Cochrane review. J Public Health (Oxf). Mar 2011;33(1):147-150. [ CrossRef ] [ Medline ]
• Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. Jul 2005;10 Suppl 1:21-34. [ CrossRef ] [ Medline ]
• Matarese M, Lommi M, De Marinis MG, Riegel B. A systematic review and integration of concept analyses of self-care and related concepts. J Nurs Scholarsh. May 2018;50(3):296-305. [ CrossRef ] [ Medline ]
• Sánchez VG, Anker-Hansen C, Taylor I, Eilertsen G. Older people's attitudes and perspectives of welfare technology in Norway. J Multidiscip Healthc. Oct 16, 2019;12:841-853. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kokolakis S. Privacy attitudes and privacy behaviour: a review of current research on the privacy paradox phenomenon. Comput Secur. Jan 2017;64:122-134. [ FREE Full text ] [ CrossRef ]
• Vassli LT, Farshchian BA. Acceptance of health-related ICT among elderly people living in the community: a systematic review of qualitative evidence. Int J Hum Comput Interact. 2018;34(2):99-116. [ CrossRef ]
• Shin G, Jarrahi MH, Fei Y, Karami A, Gafinowitz N, Byun A, et al. Wearable activity trackers, accuracy, adoption, acceptance and health impact: a systematic literature review. J Biomed Inform. May 2019;93:103153. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Biermann H, Offermann-van Heek J, Ziefle M. Older users’ benefit and barrier perception of using ultrasonic whistles in home care. In: Proceedings of the Human Aspects of IT for the Aged Population. Design for the Elderly and Technology Acceptance. 2019. Presented at: HCII 2019; July 26-31, 2019; Orlando, FL. [ CrossRef ]
• Charness N, Best R, Evans J. Supportive home health care technology for older adults: attitudes and implementation. Gerontechnology. Feb 23, 2016;15(4):233-242. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chan M, Estève D, Fourniols JY, Escriba C, Campo E. Smart wearable systems: current status and future challenges. Artif Intell Med. Nov 2012;56(3):137-156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gettel CJ, Chen K, Goldberg EM. Dementia care, fall detection, and ambient-assisted living technologies help older adults age in place: a scoping review. J Appl Gerontol. Dec 2021;40(12):1893-1902. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chan M, Campo E, Estève D, Fourniols JY. Smart homes - current features and future perspectives. Maturitas. Oct 20, 2009;64(2):90-97. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lanne M, Leikas J. Ethical AI in the re-ablement of older people: opportunities and challenges. Gerontechnology. 2021;20(2):1-13. [ CrossRef ]
• Lederman R, Ben-Assuli O, Vo TH. The role of the internet of things in healthcare in supporting clinicians and patients: a narrative review. Health Policy Technol. Sep 2021;10(3):100552. [ FREE Full text ] [ CrossRef ]
• Mallinson DJ, Shafi S. Smart home technology: challenges and opportunities for collaborative governance and policy research. Rev Policy Res. Feb 20, 2022;39(3):330-352. [ CrossRef ]
• Kolakowski H, Shepley MM, Valenzuela-Mendoza E, Ziebarth NR. How the COVID-19 pandemic will change workplaces, healthcare markets and healthy living: an overview and assessment. Sustainability. Sep 09, 2021;13(18):10096. [ FREE Full text ] [ CrossRef ]
• Caldeira C, Bietz M, Vidauri M, Chen Y. Senior care for aging in place: balancing assistance and independence. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 2017. Presented at: CSCW '17; February 25-March 1, 2017; Portland, OR. [ CrossRef ]
• Schomakers EM, Vervier L, Ziefle M. The impact of user diversity on the acceptance of mHealth for aftercare - identifying the user types ‘assistance seekers’ and ‘privacy supporters’. In: Proceedings of the Human Aspects of IT for the Aged Population. 2020. Presented at: HCII 2020; July 19-24, 2020; Copenhagen, Denmark. [ CrossRef ]
• Harrington EE, Bishop AJ, Do HM, Sheng W. Perceptions of socially assistive robots: a pilot study exploring older adults’ concerns. Curr Psychol. Mar 18, 2021;42:2145-2156. [ CrossRef ]
• Choi YK, Thompson HJ, Demiris G. Use of an internet-of-things smart home system for healthy aging in older adults in residential settings: pilot feasibility study. JMIR Aging. Nov 10, 2020;3(2):e21964. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tural E, Lu D, Austin Cole D. Safely and actively aging in place: older adults' attitudes and intentions toward smart home technologies. Gerontol Geriatr Med. May 22, 2021;7:23337214211017340. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Attié E, Meyer-Waarden L. The acceptance and usage of smart connected objects according to adoption stages: an enhanced technology acceptance model integrating the diffusion of innovation, uses and gratification and privacy calculus theories. Technol Forecast Soc Change. Mar 2022;176:121485. [ FREE Full text ] [ CrossRef ]
• Karlsen IC, Colomo-Palacios R, Holone H. The medical use of smartwatches: a tertiary study comparing northern and southern European nursing homes. In: Proceedings of the Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality. 2021. Presented at: TEEM'21; October 26-29, 2021; Barcelona, Spain. [ CrossRef ]
• Gimpel H, Manner-Romberg T, Schmied F, Winkler TJ. Understanding the evaluation of mHealth app features based on a cross-country Kano analysis. Electron Mark. 2021;31(4):765-794. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mujirishvili T, Maidhof C, Florez-Revuelta F, Ziefle M, Richart-Martinez M, Cabrero-García J. Acceptance and privacy perceptions toward video-based active and assisted living technologies: scoping review. J Med Internet Res. May 01, 2023;25:e45297. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wilczewski H, Soni H, Ivanova J, Ong T, Barrera JF, Bunnell BE, et al. Older adults' experience with virtual conversational agents for health data collection. Front Digit Health. Mar 15, 2023;5:1125926. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jaschinski C, Allouch SB, Peters O, van Dijk J. The influence of privacy on the acceptance of technologies for assisted living. In: Proceedings of the 6th International Conference on Human Aspects of IT for the Aged Population. 2020. Presented at: ITAP 2020; July 19-24, 2020; Copenhagen, Denmark. [ CrossRef ]
• Chung J, Bleich M, Wheeler DC, Winship JM, McDowell B, Baker D, et al. Attitudes and perceptions toward voice-operated smart speakers among low-income senior housing residents: comparison of pre- and post-installation surveys. Gerontol Geriatr Med. Mar 26, 2021;7:23337214211005869. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fruchter N, Liccardi I. Consumer attitudes towards privacy and security in home assistants. In: Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 2018. Presented at: CHI EA '18; April 21-26, 2018; Montreal, QC. [ CrossRef ]
• Piau A, Steinmeyer Z, Mattek N, Lindauer A, Sharma N, Bouranis N, et al. Caregiving in older adults; experiences and attitudes toward smart technologies. J Clin Med. Feb 23, 2023;12(5):1789. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tan NH, Wong RY, Desjardins A, Munson SA, Pierce J. Monitoring pets, deterring intruders, and casually spying on neighbors: everyday uses of smart home cameras. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022. Presented at: CHI '22; April 29-May 5, 2022; New Orleans, LA. [ CrossRef ]
• Schomakers EM, Ziefle M. Privacy vs. security: trade-offs in the acceptance of smart technologies for aging-in-place. Int J Hum Comput Interact. Jun 12, 2022;39(5):1043-1058. [ CrossRef ]
• Randall N, Bennett CC, Šabanović S, Nagata S, Eldridge L, Collins S, et al. More than just friends: in-home use and design recommendations for sensing socially assistive robots (SARs) by older adults with depression. Paladyn J Behav Robot. 2019;10:237-255. [ CrossRef ]
• Gerłowska J, Furtak-Niczyporuk M, Rejdak K. Robotic assistance for people with dementia: a viable option for the future? Expert Rev Med Devices. Jun 2020;17(6):507-518. [ CrossRef ] [ Medline ]
• Soro A, Ambe AH, Brereton M. Minding the gap: reconciling human and technical perspectives on the IoT for healthy ageing. Wirel Commun Mob Comput. 2017;2017:1-15. [ CrossRef ]
• Wang S, Bolling K, Mao W, Reichstadt J, Jeste D, Kim HC, et al. Technology to support aging in place: older adults' perspectives. Healthcare (Basel). Apr 10, 2019;7(2):60. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pilozzi A, Huang X. Overcoming Alzheimer's disease stigma by leveraging artificial intelligence and blockchain technologies. Brain Sci. Mar 23, 2020;10(3):183. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pekmezaris R, Williams MS, Pascarelli B, Finuf KD, Harris YT, Myers AK, et al. Adapting a home telemonitoring intervention for underserved Hispanic/Latino patients with type 2 diabetes: an acceptability and feasibility study. BMC Med Inform Decis Mak. Dec 07, 2020;20(1):324. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kodate N, Donnelly S, Suwa S, Tsujimura M, Kitinoja H, Hallila J, et al. Home-care robots - attitudes and perceptions among older people, carers and care professionals in Ireland: a questionnaire study. Health Soc Care Community. May 2022;30(3):1086-1096. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Berridge C, Halpern J, Levy K. Cameras on beds: the ethics of surveillance in nursing home rooms. AJOB Empir Bioeth. 2019;10(1):55-62. [ CrossRef ] [ Medline ]
• Mittelstadt B. Ethics of the health-related internet of things: a narrative review. Ethics Inf Technol. Jul 4, 2017;19:157-175. [ FREE Full text ] [ CrossRef ]
• Koo SH, Fallon K. Explorations of wearable technology for tracking self and others. Fash Text. Mar 28, 2018;5:8. [ CrossRef ]
• Joe J, Chaudhuri S, Chung J, Thompson H, Demiris G. Older adults' attitudes and preferences regarding a multifunctional wellness tool: a pilot study. Inform Health Soc Care. 2016;41(2):143-158. [ CrossRef ] [ Medline ]
• Sánchez VG, Taylor I, Bing-Jonsson PC. Ethics of smart house welfare technology for older adults: a systematic literature review. Int J Technol Assess Health Care. Nov 20, 2017;33(6):691-699. [ CrossRef ]
• Hjelm K, Hedlund L. Internet-of-things (IoT) in healthcare and social services - experiences of a sensor system for notifications of deviant behaviours in the home from the users' perspective. Health Informatics J. 2022;28(1):14604582221075562. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cristiano A, Musteata S, De Silvestri S, Bellandi V, Ceravolo P, Cesari M, et al. Older adults' and clinicians' perspectives on a smart health platform for the aging population: design and evaluation study. JMIR Aging. Feb 28, 2022;5(1):e29623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang D, Lim J, Zhou L, Dahl AA. Breaking the data value-privacy paradox in mobile mental health systems through user-centered privacy protection: a web-based survey study. JMIR Ment Health. Dec 24, 2021;8(12):e31633. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guazzini A, Fiorenza M, Panerai G, Duradoni M. What went wrong? Predictors of contact tracing adoption in Italy during COVID-19 pandemic. Future Internet. Nov 15, 2021;13(11):286. [ FREE Full text ] [ CrossRef ]
• Wan L, Müller C, Randall D, Wulf V. Design of a GPS monitoring system for dementia care and its challenges in academia-industry project. ACM Trans Comput Hum Interact. Oct 10, 2016;23(5):1-36. [ CrossRef ]
• Zheng S, Apthorpe N, Chetty M, Feamster N. User perceptions of smart home IoT privacy. Proc ACM Hum Comput Interact. Nov 01, 2018;2(CSCW):1-20. [ CrossRef ]
• Yao Y, Basdeo JR, Mcdonough OR, Wang Y. Privacy perceptions and designs of bystanders in smart homes. Proc ACM Hum Comput Interact. Nov 07, 2019;3(CSCW):1-24. [ CrossRef ]
• Ahmad I, Farzan R, Kapadia A, Lee AJ. Tangible privacy: towards user-centric sensor designs for bystander privacy. Proc ACM Hum Comput Interact. Oct 15, 2020;4(CSCW2):1-28. [ CrossRef ]
• Kheirinejad S, Alorwu A, Visuri A, Hosio S. Contrasting the expectations and experiences related to mobile health use for chronic pain: questionnaire study. JMIR Hum Factors. Sep 06, 2022;9(3):e38265. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reeder B, Chung J, Lyden K, Winters J, Jankowski CM. Older women's perceptions of wearable and smart home activity sensors. Inform Health Soc Care. Jan 2020;45(1):96-109. [ CrossRef ] [ Medline ]
• Halvorsrud L, Holthe T, Karterud D, Thorstensen E, Lund A. Perspectives on assistive technology among older Norwegian adults receiving community health services. Disabil Rehabil Assist Technol. Jul 2023;18(5):685-692. [ CrossRef ] [ Medline ]
• Langer A, Meleis A, Knaul FM, Atun R, Aran M, Arreola-Ornelas H, et al. Women and health: the key for sustainable development. Lancet. Sep 19, 2015;386(9999):1165-1210. [ CrossRef ] [ Medline ]
• Jaschinski C, Ben Allouch S. Listening to the ones who care: exploring the perceptions of informal caregivers towards ambient assisted living applications. J Ambient Intell Humaniz Comput. May 25, 2018;10:761-778. [ CrossRef ]
• Łukasik S, Tobis S, Kropińska S, Suwalska A. Role of assistive robots in the care of older people: survey study among medical and nursing students. J Med Internet Res. Aug 12, 2020;22(8):e18003. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Simpson E, Brown R, Sillence E, Coventry L, Lloyd K, Gibbs J, et al. Understanding the barriers and facilitators to sharing patient-generated health data using digital technology for people living with long-term health conditions: a narrative review. Front Public Health. Nov 23, 2021;9:641424. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Seberger JS, Patil S. Post-COVID public health surveillance and privacy expectations in the United States: scenario-based interview study. JMIR Mhealth Uhealth. Oct 05, 2021;9(10):e30871. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chaparro JD, Ruiz JF, Romero MJ, Peño CB, Irurtia LU, Perea MG, et al. The SHAPES smart mirror approach for independent living, healthy and active ageing. Sensors (Basel). Nov 28, 2021;21(23):7938. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Felber NA, Tian YJ, Pageau F, Elger BS, Wangmo T. Mapping ethical issues in the use of smart home health technologies to care for older persons: a systematic review. BMC Med Ethics. Mar 29, 2023;24(1):24. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sun L, Yang B. Your privacy preference matters: a qualitative study envisioned for homecare. In: Proceedings of the IEEE Symposium on Computers and Communications (ISCC). 2021. Presented at: ISCC 2021; September 5-8, 2021; Athens, Greece. [ CrossRef ]
• Elridge S. Cognitive bias. Britannica. URL: https://www.britannica.com/science/cognitive-bias [accessed 2023-10-09]
• Nickerson RS. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol. Jun 01, 1998;2(2):175-220. [ CrossRef ]
• Groupthink vs the wisdom of crowds. The Learning NGO. URL: https://thelearningngo.wordpress.com/2010/05/15/groupthink-vs-the-wisdom-of-crowds/ [accessed 2024-04-04]
• Vicsek L. A scheme for analyzing the results of focus groups. Int J Qual Method. Nov 30, 2016;6(4):20-34. [ CrossRef ]
• Daci J. Legal principles, legal values and legal norms: are they the same or different? Academicus Int Sci J. Jul 2010;(2). [ CrossRef ]
• Garg SK, Lyles CR, Ackerman S, Handley MA, Schillinger D, Gourley G, et al. Qualitative analysis of programmatic initiatives to text patients with mobile devices in resource-limited health systems. BMC Med Inform Decis Mak. Feb 06, 2016;16:16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Costa A, Yelshyna A, Moreira TC, Andrade FC, Julián V, Novais P. A legal framework for an elderly healthcare platform: a privacy and data protection overview. Comput Law Secur Rev. Oct 2017;33(5):647-658. [ FREE Full text ] [ CrossRef ]
• Jin MX, Kim SY, Miller LJ, Behari G, Correa R. Telemedicine: current impact on the future. Cureus. Aug 20, 2020;12(8):e9891. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garzo A, Garay-Vitoria N. Ethical and legal implications for technological devices in clinical research in Europe: flowchart design for ethical and legal decisions in clinical research. In: Proceedings of the XXI International Conference on Human Computer Interaction. 2021. Presented at: Interacción '21; September 22-24, 2021; Málaga, Spain. [ CrossRef ]
• Ross G, Zhao Y, Bosman A, Geballa-Koukoula A, Zhou H, Elliott C, et al. Best practices and current implementation of emerging smartphone-based (bio)sensors – part 1: data handling and ethics. TrAC Trend Anal Chem. Jan 2023;158:116863. [ FREE Full text ] [ CrossRef ]
• Ryu S. Book Review: mHealth: new horizons for health through mobile technologies: based on the findings of the second global survey on eHealth (Global Observatory for eHealth Series, Volume 3). Healthc Inform Res. 2012;18(3):231-233. [ FREE Full text ] [ CrossRef ]
• Ambrosino N, Vitacca M, Dreher M, Isetta V, Montserrat JM, Tonia T, et al. Tele-monitoring of ventilator-dependent patients: a European Respiratory Society statement. Eur Respir J. Sep 2016;48(3):648-663. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Demiris G, Hensel B. “Smart homes” for patients at the end of life. J Housing Elder. Feb 20, 2009;23(1-2):106-115. [ CrossRef ]
• Cognitive impairment: a call for action, now!. Centers for Disease Control and Prevention. URL: http://www.cdc.gov/aging/pdf/cognitive_impairment/cogimp_poilicy_final.pdf [accessed 2024-03-19]
• Sample informed consent language library: describing technologies used in research. Harvard Catalyst. URL: https:/​/catalyst.​harvard.edu/​publications-documents/​sample-informed-consent-language-library-describing-technologies-used-in-research/​ [accessed 2023-10-10]
• Whitney SN, McGuire AL, McCullough LB. A typology of shared decision making, informed consent, and simple consent. Ann Intern Med. Jan 06, 2004;140(1):54-59. [ CrossRef ] [ Medline ]
• Sanchez VG, Pfeiffer CF. Legal aspects on smart house welfare technology for older people in Norway. In: Novais P, Konomi S, editors. Intelligent Environments. Amsterdam, The Netherlands. IOS Press; 2016;125-134.
• Mosca F, Such JM, McBurney P. Value-driven collaborative privacy decision making. King's College London. 2019. URL: https:/​/nms.​kcl.ac.uk/​hasp/​pubs/​Value_driven_Collaborative_Decision_Making__Privacy_Application__PAL2019_.​pdf [accessed 2024-03-19]
• Taylor MJ, Wilson J. Reasonable expectations of privacy and disclosure of health data. Med Law Rev. Aug 01, 2019;27(3):432-460. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shared decision making. National Institute for Health and Care Excellence. Jun 17, 2021. URL: https://www.nice.org.uk/guidance/ng197 [accessed 2023-10-10]
• Wilkins RG. Defining the "reasonable expectation of privacy": an emerging tripartite analysis. Vanderbilt Law Rev. 1987;40(5):1077. [ CrossRef ]
• Kent G. Shared understandings for informed consent: the relevance of psychological research on the provision of information. Soc Sci Med. Nov 1996;43(10):1517-1523. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Holden JD. Hawthorne effects and research into professional practice. J Eval Clin Pract. Feb 07, 2001;7(1):65-70. [ CrossRef ] [ Medline ]
• Martin DU, Perry C, Kaufman J. Effects of a mirror on young children's transgression in a gift-delay task. Br J Dev Psychol. Jun 27, 2020;38(2):205-218. [ CrossRef ] [ Medline ]
• Barth S, de Jong MD. The privacy paradox – Investigating discrepancies between expressed privacy concerns and actual online behavior – A systematic literature review. Telemat Inform. Nov 2017;34(7):1038-1058. [ CrossRef ]
• Norberg PA, Horne DR, Horne DA. The privacy paradox: personal information disclosure intentions versus behaviors. J Consum Affairs. Mar 06, 2007;41(1):100-126. [ CrossRef ]
• Balakrishnan J, Dwivedi YK. Role of cognitive absorption in building user trust and experience. Psychol Mark. Feb 24, 2021;38(4):643-668. [ CrossRef ]
• Ghosh I, Singh V. Using cognitive dissonance theory to understand privacy behavior. Proc Assoc Inf Sci Technol. Oct 24, 2017;54(1):679-681. [ CrossRef ]
• Krokstad S, Langhammer A, Hveem K, Holmen TL, Midthjell K, Stene TR, et al. Cohort Profile: the HUNT Study, Norway. Int J Epidemiol. Aug 2013;42(4):968-977. [ CrossRef ] [ Medline ]
• McCarney R, Warner J, Iliffe S, van Haselen R, Griffin M, Fisher P. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol. Jul 03, 2007;7:30. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by E Borycki; submitted 11.10.23; peer-reviewed by Y Zhuang, I Mircheva; comments to author 03.11.23; revised version received 20.12.23; accepted 13.03.24; published 09.05.24.

©Luyi Sun, Bian Yang, Els Kindt, Jingyi Chu. Originally published in JMIR Nursing (https://nursing.jmir.org), 09.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Nursing, is properly cited. The complete bibliographic information, a link to the original publication on https://nursing.jmir.org/, as well as this copyright and license information must be included.

Why Nursing Research Matters

Affiliation.

• 1 Author Affiliation: Director, Magnet Recognition Program®, American Nurses Credentialing Center, Silver Spring, Maryland.
• PMID: 33882548
• DOI: 10.1097/NNA.0000000000001005

Increasingly, nursing research is considered essential to the achievement of high-quality patient care and outcomes. In this month's Magnet® Perspectives column, we examine the origins of nursing research, its role in creating the Magnet Recognition Program®, and why a culture of clinical inquiry matters for nurses. This column explores how Magnet hospitals have built upon the foundation of seminal research to advance contemporary standards that address some of the challenges faced by healthcare organizations around the world. We offer strategies for nursing leaders to develop robust research-oriented programs in their organizations.

Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.

• Credentialing / standards
• Hospitals / standards*
• Leadership*
• Nursing Research / standards*
• Nursing Service, Hospital / standards*
• Nursing Staff, Hospital / standards*
• Organizational Culture
• Quality of Health Care / standards
• United States

• Open access
• Published: 10 May 2024

Community-based participatory-research through co-design: supporting collaboration from all sides of disability

• Cloe Benz   ORCID: orcid.org/0000-0001-6950-8855 1 ,
• Will Scott-Jeffs 2 ,
• K. A. McKercher   ORCID: orcid.org/0000-0003-4417-585X 3 ,
• Mai Welsh   ORCID: orcid.org/0000-0002-7818-0115 2 , 4 ,
• Richard Norman   ORCID: orcid.org/0000-0002-3112-3893 1 ,
• Delia Hendrie   ORCID: orcid.org/0000-0001-5022-5281 1 ,
• Matthew Locantro 2 &
• Suzanne Robinson   ORCID: orcid.org/0000-0001-5703-6475 1 , 5  

Research Involvement and Engagement volume  10 , Article number:  47 ( 2024 ) Cite this article

Metrics details

As co-design and community-based participatory research gain traction in health and disability, the challenges and benefits of collaboratively conducting research need to be considered. Current literature supports using co-design to improve service quality and create more satisfactory services. However, while the ‘why’ of using co-design is well understood, there is limited literature on ‘ how ’ to co-design. We aimed to describe the application of co-design from start to finish within a specific case study and to reflect on the challenges and benefits created by specific process design choices.

A telepractice re-design project has been a case study example of co-design. The co-design was co-facilitated by an embedded researcher and a peer researcher with lived experience of disability. Embedded in a Western Australian disability organisation, the co-design process included five workshops and a reflection session with a team of 10 lived experience and staff participants (referred to as co-designers) to produce a prototype telepractice model for testing.

The findings are divided into two components. The first describes the process design choices made throughout the co-design implementation case study. This is followed by a reflection on the benefits and challenges resulting from specific process design choices. The reflective process describes the co-designers’ perspective and the researcher’s and organisational experiences. Reflections of the co-designers include balancing idealism and realism, the value of small groups, ensuring accessibility and choice, and learning new skills and gaining new insights. The organisational and research-focused reflections included challenges between time for building relationships and the schedules of academic and organisational decision-making, the messiness of co-design juxtaposed with the processes of ethics applications, and the need for inclusive dissemination of findings.

Conclusions

The authors advocate that co-design is a useful and outcome-generating methodology that proactively enables the inclusion of people with disability and service providers through community-based participatory research and action. Through our experiences, we recommend community-based participatory research, specifically co-design, to generate creative thinking and service design.

Plain language summary

Making better services with communities (called co-design) and doing research with communities (e.g. community-based participatory research) are ways to include people with lived experience in developing and improving the services they use. Academic evidence shows why co-design is valuable, and co-design is increasing in popularity. However, there needs to be more information on how to do co-design. This article describes the process of doing co-design to make telepractice better with a group of lived experience experts and staff at a disability organisation. The co-design process was co-facilitated by two researchers – one with a health background and one with lived experience of disability. Telepractice provides clinical services (such as physiotherapy or nursing) using video calls and other digital technology. The co-design team did five workshops and then reflected on the success of those workshops. Based on the groups’ feedback, the article describes what worked and what was hard according to the co-designers and from the perspective of the researchers and the disability organisation. Topics discussed include the challenge of balancing ideas with realistic expectations, the value of small groups, accessibility and choice opportunities and learning new skills and insights. The research and organisational topics include the need to take time and how that doesn’t fit neatly with academic and business schedules, how the messiness of co-design can clash with approval processes, and different ways of telling people about the project that are more inclusive than traditional research. The authors conclude that co-design and community-based participatory research go well together in including people with lived experience in re-designing services they use.

Peer Review reports

Introduction

Co-design has the potential to positively impact co-designers and their community, researchers, and organisations. Co-design is defined as designing with, not for, people [ 1 ] and can reinvigorate business-as-usual processes, leading to new ideas in industry, community and academia. As co-design and community-based participatory research gain traction, the challenges and benefits of collaborative research between people with lived experience and organisations must be considered [ 2 ].

Disability and healthcare providers previously made decisions for individuals as passive targets of an intervention [ 3 ]. By contrast, the involvement of consumers in their care [ 4 ] has been included as part of accreditation processes [ 4 ] and shown to improve outcomes and satisfaction. For research to sufficiently translate into practice, consumers and providers should be involved actively, not passively [ 4 , 5 ].

Approaches such as community-based participatory research promote “a collaborative approach that equitably involves community members, organisational representatives and researchers in all aspects of the research process” [ 6 ] (page 1). This approach originated in public health research and claims to empower all participants to have a stake in project success, facilitating a more active integration of research into practice and decreasing the knowledge to practice gap 6 . Patient and public involvement (PPI) increases the probability that research focus, community priorities and clinical problems align, which is increasingly demanded by research funders and health systems [ 7 ].

As community-based participatory research is an overarching approach to conducting research, it requires a complementary method, such as co-production, to achieve its aims. Co-production has been attributed to the work of Ostrom et al. [ 8 ], with the term co-design falling under the co-production umbrella. However, co-design can be traced back to the participatory design movement [ 9 ]. The term co-production in the context of this article includes co-planning, co-discovery, co-design, co-delivery, and co-evaluation [ 10 ]. Within this framework, the concept of co-design delineates the collaborative process of discovery, creating, ideating and prototyping to design or redesign an output [ 11 ]. The four principles of co-design, as per McKercher [ 1 ], are sharing power, prioritising relationships, using participatory means and building capacity [ 1 ]. This specific method of co-design [ 1 ] has been used across multiple social and healthcare publications [ 10 , 12 , 13 , 14 ].

A systematic review by Ramos et al. [ 15 ] describes the benefits of co-design in a community-based participatory-research approach, including improved quality and more satisfactory services. However, as identified by Rahman et al. [ 16 ], the ‘ why ’ is well known, but there is limited knowledge of ‘ how ’ to co-design. Multiple articles provide high-level descriptions of workshops or briefly mention the co-design process [ 13 , 17 , 18 , 19 ]. Pearce et al. [ 5 ] include an in-depth table of activities across an entire co-creation process, however within each part i.e., co-design, limited descriptions were included. A recent publication by Marwaa et al. [ 20 ] provides an in-depth description of two workshops focused on product development, and Tariq et al. [ 21 ] provides details of the process of co-designing a research agenda. Davis et al. [ 11 ] discuss co-design workshop delivery strategies summarised across multiple studies without articulating the process from start to finish. Finally, Abimbola et al. [ 22 ] provided the most comprehensive description of a co-design process, including a timeline of events and activities; however, this project only involved clinical staff and did not include community-based participation.

As “We know the why, but we need to know the how-to” [ 16 ] (page 2), of co-design, our primary aim was to describe the application of co-design from start to finish within a specific case study. Our secondary aim was to reflect on the challenges and benefits created by specific process design choices and to provide recommendations for future applications of co-design.

Overview of telepractice project

The case study, a telepractice redesign project, was based at Rocky Bay, a disability support service provider in Perth, Australia [ 23 ]. The project aimed to understand the strengths and pain points of telepractice within Rocky Bay. We expanded this to include telepractice in the wider Australian disability sector. The project also aimed to establish potential improvements to increase the uptake and sustainability of Rocky Bay’s telepractice service into the future. Rocky Bay predominantly serves people under the Australian National Disability Insurance Scheme (NDIS) [ 24 ] by providing a variety of services, including allied health (e.g. physiotherapy, dietetics, speech pathology, etc.), nursing care (including continence and wound care), behaviour support and support coordination [ 23 ]—Rocky Bay services metropolitan Perth and regional Western Australia [ 23 ].

The first author, CB, predominantly conducted this research through an embedded researcher model [ 25 ] between Curtin University and Rocky Bay. An embedded researcher has been defined as “those who work inside host organisations as members of staff while also maintaining an affiliation with an academic institution” [ 25 ] (page 1). They had some prior contextual understanding which stemmed from being a physiotherapist who had previously delivered telehealth in an acute health setting. A peer researcher, WSJ, with lived experience of disability, worked alongside CB. They had no previous experience in research or co-design, this was their first paid employment and they had an interest in digital technology. Peer Researcher is a broad term describing the inclusion of a priority group or social network member as part of the research team to enhance the depth of understanding of the communities to which they belong [ 26 ]. Including a peer researcher in the team promoted equity, collective ownership, and better framing of the research findings to assist with connecting with people with lived experience. These outcomes align with key components of community-based participatory research and co-design [ 27 , 28 , 29 , 30 ].

Person-first language was used as the preference of experts with lived experience who contributed to this research to respect and affirm their identity. However, we respect the right to choose and the potential for others to prefer identity-first language [ 31 ].

A summary of the structure of the phases completed before co-design workshops are represented in Fig.  1 below. Ethical approval for the project was received iteratively before each phase on the timeline (Fig.  1 ) from the Curtin Human Research Ethics Committee (HRE2021-0731). The reporting of this article has been completed in line with the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) checklist [ 7 ].

Summary of telepractice co-design project structure [ 1 ]

Here, we present an outline of the chosen research methods with descriptions of each process design choice and supporting reasons and examples specific to the study. The format is in chronological order, with further details of each step provided in Appendix 1 (Supplementary Material 1).

Methods and results

Process of co-production and preparation for co-design.

Co-production was chosen as the planning method for the study, as the inclusion of community members (Rocky Bay Lived experience experts and Staff) in each step of the research process would increase buy-in and make the research more likely to meet their needs [ 5 ]. An example of co-planning (part of co-production) includes the study steering committee, with a lived experience expert, clinician and project sponsor representatives collaborating on the selection of study aim, methods and recruitment processes. Another example of co-planning, co-design, and co-delivery was recruiting a peer researcher with disability, who worked with the embedded researcher throughout the study design and delivery.

The second process design choice was to attempt to build safe enough conditions for community participation, as people who feel unsafe or unwelcome are less likely to be able to participate fully in the research [ 1 ]. Building conditions for safety was applied by repeatedly acknowledging power imbalances, holding space for community input, and anticipating and offering accessibility adjustments without judgment.

Getting started

Understanding and synthesising what is already known about telepractice experiences and learning from lived experience was prioritised as the first step in the process. We paired a scoping review of the literature with scoping the lived experiences of the community [ 32 ]. Our reasoning was to understand whether the findings aligned and, secondly, to learn what had already been done and to ask what was next, rather than starting from the beginning [ 1 ]. Examples of strategies used in this step included interviewing clinicians and service provider Managers across Australia to establish how they implemented telepractice during the pandemic and understand their views of what worked and what did not. The second learning process occurred onsite at Rocky Bay, with people with lived experience, clinicians and other support staff, whom the embedded researcher and peer researcher interviewed to understand experiences of telepractice at Rocky Bay.

The authors presented the interview findings during focus groups with Rocky Bay participants to share the learnings and confirm we had understood them correctly. The groups were divided into staff and lived experience cohorts, allowing for peer discussions and sharing of common experiences. This helped build relationships and a sense of familiarity moving into the workshop series.

Co-design workshops

This section outlines specific components of the co-design workshop preparation before describing each of the five workshops and the final reflection session.

Staff and community co-designers

Two process design choices were implemented to form the co-design group. The first was to prioritise lived experience input as there are generally fewer opportunities for lived experience leadership in service design [ 16 ], and because the disability community have demanded they be included where the focus impacts them [ 33 ]. To acknowledge the asymmetry of power between community members, people with lived experience of disability and professionals, we ensured the co-design group had at least the same number of lived experience experts as staff.

The second priority for the co-design group was to include people for whom involvement can be difficult to access (e.g. people who are isolated for health reasons and cannot attend in-person sessions, people who live in supported accommodation, part-time staff, and people navigating the dual-role of staff member while disclosing lived experience). It was important to learn from perspectives not commonly heard from and support equity of access for participants [ 4 ].

Workshop series structure

When structuring the workshop series, lived experience co-designers nominated meeting times outside standard work hours to reduce the impact of co-design on work commitments and loss of income while participating. The workshops were designed to be delivered as a hybrid of in-person and online to give co-designers a choice on how they wanted to interact. The workshops were designed as a series of five sequential 90-minute workshops, where co-designers voted for the first workshop to be predominantly in-person and the remainder of the workshops online. Some co-designers chose to attend the initial session in person to build rapport. However, the virtual option remained available. The subsequent online sessions reduced the travel burden on co-designers, which the co-designers prioritised over further face-to-face meetings.

Workshop facilitators

To maintain familiarity and ensure predictability for co-designers, the workshops were co-facilitated by the embedded researcher and peer researcher. The co-facilitators built on relationships formed through previous interactions (interviews and focus groups), and each facilitator represented part of the co-designer group as a clinician or a person with disability. An extra support person was tasked with supporting the co-designers with disability to break down tasks and increase the accessibility of activities. The reason for selecting the support person was that they could contribute their skills as a school teacher to support the communication and completion of activities, and they had no previous experience with disability services to influence the co-designers opinions. This role was adapted from the provocateur role described by McKercher [ 1 ].

Pre-workshop preparations

To prepare for the workshops, each co-designer was asked to complete a brief survey to ensure the co-facilitators understood co-designers collect preferences and needs ahead of the session to enable preparation and make accommodations. The survey included pronouns, accessibility needs and refreshment preferences. Following the survey, the co-facilitators distributed a welcome video; the peer researcher, a familiar person, was videoed explaining what to expect, what not to expect and expected behaviours for the group to support a safe environment [ 1 ]. This process design choice was made to allow co-designers to alleviate any potential anxieties due to not having enough information and to increase predictability.

Workshop resources and supports

As the first workshop was in-person, specific process choices were made to ensure co-designers felt welcome and to uphold the dignity of co-designers with lived experience [ 34 ]. Examples of process design choices include facilitating transport and parking requests, providing easy access to the building and room, making a sensory breakout room available and having the peer researcher waiting at the entrance to welcome and guide people to the workshop room.

After reaching the workshop room, all co-designers received an individualised resource pack to equalise access to workshop materials, aiming again to balance power in a non-discriminatory way [ 11 ]. The resource pack included name tags with pronouns, individualised refreshments, a fidget toy [ 35 ] whiteboard markers and a human bingo activity described in a later section. An easy-to-apply name tag design was selected after consulting a co-designer with an upper limb difference. Further details on the resource packs are included in Appendix 1 (Supplementary Material 1).

Enabling different kinds of participation

We provided non-verbal response cards to each co-designer as communication preferences vary significantly within the disability community. The cards were intended to benefit any co-designer who struggled to use the response buttons on MS teams. The co-facilitators co-created the Yes, No, and In-the-middle response cards (Fig.  2 ) and were guided by recommendations by Schwartz and Kramer [ 29 ]. They found that people with intellectual disability were more likely to respond “yes” if the negative option included a frowning face or red-coloured images, as choosing these types of alternatives was perceived as being negative or would cause offence [ 29 ].

Non-verbal response cards

A summary of the structure and purpose of each of the five workshops is shown in Fig.  3 , followed by a more in-depth discussion of the strategies employed in each workshop.

Outline of workshop and group structures

Workshop 1: the beginning

Human Bingo was the first workshop activity, as it aimed to support relationship building in an inclusive way for both in-person and online attendees. The activity asked each co-designer to place a name in each worksheet box of someone who fit the described characteristic of that square(for example, someone who likes cooking). To include the two online attendees, laptops were set up with individual videocall streams and noise cancelling headphones enabling the online co-designers to interact one-on-one with others during the activities.

The second activity used The Real Deal cards by Peak Learning [ 36 ] to ask the co-designers to sort cards to prioritise the top five experiences and feelings they would want in a future version of telepractice. This activity aimed to set initial priorities for the redesign of telepractice [ 1 ]. Small groups with a mix of lived experience experts and staff were tasked with negotiating and collaborating to produce their top five desired experiences and feelings for future service success.

A follow-up email was sent after the session to thank co-designers, provide closure, invite feedback and let co-designers know what to expect from the next session.

Workshop 2: mapping the journey

In the second workshop, held online, the co-facilitators explained the journey mapping process and showed a draft of how the visual representation would likely look (Fig.  4 ). As the first step, co-designers were tasked with completing a series of activities to analyse lived experience interview data on the current experience of telepractice for lived experience experts. Small mixed groups were created, prioritising the needs of the lived experience experts to have staff who would be the best fit in supporting them to work through the task [ 1 ]. The small groups were allocated interview quotes corresponding to the steps of a customer journey through telepractice and asked to identify strengths, challenges and emotions associated with the current Telepractice service journey at Rocky Bay [ 1 ]. Further details on the journey map analysis are described in Appendix 1 (Supplementary Material 1) and in a published article co-authored by the co-designers (Benz et al. [ 37 ]).

Draft journey map visualisation

After workshop two, the embedded researcher drafted a journey map by compiling the co-designer group responses to the analysis activity, which was then circulated for feedback and confirmation. The completed journey map is published with further details on the process in an article co-authored with the co-designers, Benz et al. [ 37 ].

Workshop 3: ideas for addressing pain points

For the third workshop, the co-facilitators selected activities to be completed separately by lived experience and staff co-designers. The lived experience expert activity involved exploring preferences for improving pain points identified through the journey map. The lived experience expert activity was facilitated by the peer researcher and support person and included questions such as, how would it be best to learn how to use telepractice? Visual prompt cards were shared to support idea creation, where lived experience expert co-designers could choose any option or suggest an alternative (Fig.  5 ).

Option cards for Lived experience expert co-designer workshop activity

Simultaneously, the staff co-designers completed a parallel activity to address pain points from a service delivery point of view. These pain points were identified in the clinical and non-clinical staff interviews and from the journey map summary of lived experience expert interviews (analysed in Workshop 2). Staff co-designers completed a mind map based on service blueprinting guidelines by Flowers and Miller [ 38 ]. The activity used service blueprinting to identify a list of opportunities for improvement, with four prompts for co-designers to commence planning the actions required to implement these improvements. The foci of the four prompts were roles, policies, technology and value proposition [ 38 ] (described further in Appendix 1 (Supplementary Material 1)). Each of the four prompts were completed for the ten proposed opportunities for improvement to draft plans for future telepractice service delivery.

Workshop 4: story telling and generation of future state solutions

In the fourth workshop, we introduced the concept of prototyping [ 39 ] as a designerly way to test co-designers’ ideas for improving telepractice according to desirability, feasibility and viability with a wider audience of lived experience experts and staff. The co-designers helped to plan the prototyping, and accessibility was a key consideration in selecting a prototype, as the group were conscious of the target audience.

Creating the prototype was collaborative, allowing co-designers to produce an output representing their ideas. They selected a video storyboard prototype with a staff and customer version formatted similarly to a children’s book. It included cartoon animations completed on PowerPoint, voiceover narration, closed captioning and an introductory explanation from two co-designers.

After workshop four, the co-designers collaborated on the customer and staff prototypes during the two weeks between workshops four and five, with support and input from the facilitators. The prototype files were co-produced, with different co-designers working on the visual aspects, the script for the main audio narration and the introductory explanation.

Workshop 5: finishing the story

The co-design group reviewed the draft prototypes in the final workshop, with specific attention paid to the story’s cohesiveness.

The feedback questionnaire was then created to be completed by viewers outside of the co-design group after engaging with either the staff or the customer prototype. The survey allowed Rocky Bay customers and staff to contribute ideas. Following thoughtful discussions, consensus was reached by all co-designers on the final survey questions (Appendix 2 (Supplementary Material 1)).

A reflection activity concluded the final workshop, allowing co-designers to provide feedback on the co-design process, elements for improvement and aspects they valued in participating in the project. Their reflections on the benefits and challenges of co-design in this study are included in the section Co-designer’s perspectives of the workshop series , with the reflection questions included in Appendix 3 (Supplementary Material 1).

Post prototype reflection session

The prototype feedback responses were reviewed with co-designers in a final reflection session. The group then discussed adaptations to the implementation plan for proposal to Rocky Bay. Following the survey discussion, co-designers reviewed proposed service principles for the new telepractice implementation recommendations. These principles aim to align any future decisions in the implementation and service provision stages of the telepractice project with the intentions of the co-designers. An additional reflection activity was completed, specific to the telepractice proposal they had produced and the prototyping process. Feedback relevant to subsequent discussions of the challenges and benefits of co-design is included in the following section: Co-designer’s perspectives of the workshop series , with the reflection prompts in Appendix 3 (Supplementary Material 1).

Benefits and challenges

Learnings derived from completing a study of this kind are complex. However, it is necessary to reflect on which strategies used in the project were beneficial and which strategies created challenges - anticipated and unexpected. These reflections are discussed in two sections, the first being the challenges and benefits reflected upon by co-designers. The second set of reflections relates to organisational and research project-level benefits and challenges from the perspective of clinical department managers and researchers involved in the project.

Co-designer’s perspectives of the workshop series

Co-designers were positive overall about the workshop series. Responses to a prompt for one-word descriptors of their experience included “captivating, innovative, fulfilling, exciting, insightful, helpful, eye-opening and informative ” .

Co-designing as a team

A foundational strategy implemented in this project was the intentional collaboration of lived experience experts with staff; this linked to the co-design principle of prioritising relationships and sharing power. Multiple reflections commented on feeling like a team and that having diverse perspectives across the group was beneficial.

It was especially interesting to hear the perspective of clinicians (for us, the other side of Telepractice). [Lived experience expert Co-designer]

Additionally, the combination of facilitators, including an embedded researcher with an allied health clinical background, a peer researcher with lived experience and a support person with strengths in breaking down tasks, provided different facets of support and task modelling to the co-designers throughout the process.

Balancing idealism and realism

There is an inherent challenge in collaboration between lived experience experts and service providers, whereby co-designers formulate ideas for service improvement and then, in good faith, propose required changes to be implemented. Strategies to support imagination and idealism while being honest about the constraints of what can be delivered were implemented in the context of this project. This was essential to reinforce to co-designers that their contributions and ideas are valid while tempering their hopes with the truth that organisational change is challenging and funding for change is limited. Co-designers were encouraged to be cognisant of ideas that would require high investment (cost and time) and which ideas faced fewer barriers to implementation. This strategy did not prevent the ideation of changes and prioritising what mattered most to them, and co-designers felt it was beneficial in adding a level of consideration regarding what investments they deemed necessary versus those that would be nice to have. For example, having a person to call for help was viewed as necessary, while a nice to have was more advanced technological features.

I feel that the prototype is useful; however, I worry that nothing will be carried over to the Rocky Bay Service. I feel like more customers will want to access telepractice, and Rocky Bay now needs to start the implementation process to ensure that telepractice is utilised, including processes, education and training. [Clinician Co-designer]

The value of small groups

Working in small groups was another beneficial strategy, aiming to create a more hospitable environment for co-designers to voice their thoughts. The small groups varied across activities and workshops, with facilitators intentionally pairing groups that would best support the lived experience of expert co-designers completing activities. As described in the workshop sections, some activities suited mixed groups, whereas others suited lived experience expert and staff-specific groups. Two reflective comments demonstrated the benefit of the small groups, one from a clinician who reflected on supporting a fellow co-designer:

I found that in our group, all of us had a say; however, [Lived Experience Co-designer name] was a bit overwhelmed at times, so I tried to support her with that. [Clinician Co-designer]

And a lived experience expert co-designer additionally reflected:

The breakout rooms were a very good idea. It can be quite intimidating speaking in front of the main group. I found it much easier to participate in the smaller groups . [Lived experience expert Co-designer]

The second session included an unplanned whole group activity, which challenged co-designers. Co-designers reflections of this experience demonstrate the benefits of smaller groups:

I did feel that at the end when the whole group did the task, there wasn’t as much collaboration as there were quite a few more assertive participants, so the quieter ones just sat back. [Clinician Co-designer]

Accessibility and choice

A challenge navigated throughout the workshop series with a diverse group of co-designers was meeting their varying individual health and other needs. This required responding in sensitive, non-judgemental, and supportive ways to encourage co-designers to engage fully. Examples of support include the presence of a support person and adaption of resource packs for co-designers who have difficulty swallowing (re: refreshments), as well as the previously mentioned non-verbal response cards and accessible name tags.

Accessibility supports were also provided for the peer researcher during facilitation activities, including pre-written scripts to provide clarity when explaining tasks to the co-design group, written reminders and regular check-ins. A lived experience expert co-designer reflected that it was beneficial that they could tell the peer researcher was nervous but appreciated that he was brave and made them feel like they did not need to be perfect if the peer researcher was willing to give it a go.

When facilitating the sessions, the embedded researcher and peer researcher identified that the workshops were long and, at times, mentally strenuous. One co-designer requested “more breaks during each session” . Breaks were offered frequently; however, upon reflection, we would schedule regular breaks to remove the need for co-designers to accept the need for a break in front of the group. The instructions for each activity were visual, verbal and written and given at the start of a task. However, once the co-designers were allocated to breakout rooms, they could no longer review the instructions. Many co-designers suggested that having the instructions in each breakout room’s chat window would have been a valuable visual reminder.

One thing I think might of helped a little is having the instructions in the chat as I know I that I listened but couldn’t recall some of the instructions for the group task. [Lived experience expert Co-designer]

Learning new skills and gaining new insight

The co-designers considered that the benefits of working together included learning new skills and widening their understanding of research, the services they provide or use, and the differences between the priorities of lived experience experts and staff. Two lived experience experts commented that the opportunity to learn collaboration skills and create cartoons using PowerPoint were valuable skills for them to utilise in the future. One clinician reflected that the process of co-design had improved their clinical practice and increased their use of telepractice:

My practice is 100% better. I am more confident in using telepractice and more confident that, as a process, it doesn’t reduce the impact of the service- in some ways, it has enhanced it when customers are more relaxed in their own environments. I have not seen my stats, but my use of telepractice has increased significantly, too. [Clinician Co-designer]

The management co-designer acknowledged that although ideas across the group may be similar, prioritisation of their importance can vary dramatically:

Whilst all the feedback and potential improvements were very similar, some things that I viewed as not an issue, was very different to a customer’s perspective. [Management Co-designer]

Overall, the workshop series challenged co-designers. However, the provision of a supportive and accessible environment resulted in mutual benefits for the research, organisation, and co-designers themselves. The strategy for facilitating the workshops was to pose challenges, support the co-designers in rising to meet them, and take into account their capabilities if provided with the right opportunity. A lived experience expert co-designer summarised the effectiveness of this strategy:

I found the activities to be challenging without being too difficult. Each activity provided enough guidance and structure to encourage interesting group discussions and make collaboration easy. [Lived experience expert Co-designer]

Research and organisational reflections of benefits and challenges of co-design

A significant challenge in completing this project was that building foundational relationships and trust takes time. While the authors view this trust as the foundation on which community-based participatory research and co-design are built, they note the direct tension of the time needed to develop these foundational relationships with the timeline expectations of academic and organisational decision-making. The flexibility required to deliver a person-centred research experience for the co-designers resulted in regular instances when timeline extensions were required to prioritise co-designer needs over efficiency. The result of prioritising co-designer needs over research timeline efficiency was an extended timeline that was significantly longer than expected, which sometimes created a disconnect between the flexibility of co-design and the rigidity in traditional academic and organisational processes.

The impacts of a longer-than-expected timeline for completion of the co-design process included financial, project scope, and sponsorship challenges. The project’s initial scope included a co-implementation and co-evaluation phase; however, due to the three-year time constraint, this was modified to conclude following the prototyping process. Whilst the three-year period set expectations for project sponsors and other collaborators from Rocky Bay, the wider context for the project varied significantly and rapidly over this period. This included two changes in Rocky Bay supervisor and one change in Rocky Bay project sponsor. Additionally, one of the academic supervisors left Curtin. This challenge indicates that the project would benefit from key role succession planning.

The peer researcher role was beneficial in providing an opportunity for a person with lived experience to join the study in a strength-based role and experience academic and business processes. However, challenges arose with the timeline extensions, which required this part-time, casual role to be extended by seven months. While the contract extension posed budgetary challenges, the role was viewed as vital to the completion of the project.

While an essential component of research, particularly involving vulnerable populations, ethical approvals proved challenging due to the non-traditional research methods involved in co-design. It was evident to the authors that while the ethics committee staff adhered to their processes, they were bound by a system that did not have adequate flexibility to work with newer research methods, such as co-design. Multiple methods in this study were heavily integrated into the community, including embedded research, peer research and co-design.

The present ethics process provided a comprehensive review focusing on planned interactions within research sessions (e.g. interviews and workshops). Unfortunately, this failed to account for a wider view, including the initial co-production prior to ethical application and anecdotal interactions that occurred regularly in the organic co-design process. In addition to the repeated submissions required to approve the sequential study format, these interactions created a significant workload for the research team and ethics office. These challenges were compounded by the need to navigate Rocky Bay’s organisational processes and changing business needs within ethical approval commitments.

In the authors’ opinion, prioritising the inclusion of lived experience experts in co-creating outputs to disseminate findings was beneficial. The co-creation enabled an authentic representation of the study to audiences regarding community-based participatory research and co-design method implementation. For example, the presentation of a panel discussion at a conference in which the peer researcher could prerecord his responses to questions as his preferred method of participation. All posters presented by the project were formatted to be accessible to lay consumers and were collaboratively produced, with the additional benefit of the posters being displayed across Rocky Bay hubs for customers and staff to gain study insights.

Due to the co-design method’s dynamic nature, some budgetary uncertainty was challenging to navigate. However, financial and non-financial remuneration for all non-staff participants in the project was prioritised. As previously discussed, the position of peer researcher was a paid role; additionally, all lived experience expert participants were remunerated at a rate of AUD 30/hour in the form of gift cards. The carer representative on the steering committee recommended using gift cards to avoid income declaration requirements from government benefits people may receive. Non-financial remuneration for the valuable time and contribution of the co-designer group included co-authorship on an article written regarding the Journey Map they produced (Benz et al. [ 37 ]) and acknowledgement in any other appropriate outputs. The implementation proposal provided to Rocky Bay included recommendations for continued inclusion and remuneration of co-designers.

Setting a new bar for inclusion

Another benefit to reflect upon, which may be the most significant legacy of the project, was setting the precedence for the inclusion of people with disability in decision-making roles in future projects and research conducted by the University and Rocky Bay. After this project commenced, other Rocky Bay clinical projects have similarly elevated the voices of lived experience in planning and conducting subsequent quality improvement initiatives.

I’m lucky enough to have been part of a lot of projects. But I guess I probably haven’t been a part of continuous workshops, pulling in all perspectives of the organisation perfectly… So, collaboration and getting insight from others I haven’t usually was a very unique experience, and I definitely found value if this were to continue in other projects. [Manager Co-designer]

In summary, the findings from using a co-design method for the telepractice research study produced a series of benefits and presented the researchers with multiple challenges. The findings also addressed a literature gap, presenting in-depth descriptive methods to demonstrate how co-design can be applied to a specific case.

Drawn from these findings, the authors identified six main points which form the basis of this discussion. These include (1) the fact that the necessary time and resources required to commit to co-design process completion adequately were underestimated at the outset, (2) there is a need to support the health, well-being and dignity of lived experience expert participants, (3) academic ethical processes have yet to adapt to address more participatory and integrated research methods, (4) strategies used to foster strong collaborative relationships across a diverse group were valued by all participants, (5) better delineation between terminologies such as co-design and community-based participatory research or patient and public involvement would improve the clarity of research methods and author intent and, (6) broader non-traditional impacts that participatory research can create should be better quantified and valued in the context of research impact. Each point will now be discussed in further detail.

In underestimating the time and resources required to complete the telepractice study, a scope reduction was required. This scope reduction removed the study’s originally planned co-implementation and co-evaluation phases. While Harrison et al. [ 40 ] and Bodden and Elliott [ 41 ] advocate for more frequent and comprehensive evaluation of co-designed initiatives, the authors acknowledge that this became no longer feasible within the study constraints. A growing body of literature indicates expected timelines for completed co-production projects from co-planning to co-evaluation. An example by Pearce et al. [ 5 ] indicated that a timeline of five years was reasonable. In contrast, a more limited co-design process was completed with a shorter timeline by Tindall et al. [ 13 ]. Although neither of these articles were published when this study commenced, they are complementary in building an evidence base for future research to anticipate an adequate timeline.

While co-design and other co-production processes are resource and time-intensive, the investment is essential to prioritise the health and other needs of potentially vulnerable population groups in the context of an imbalance of power [ 42 ]. In exploring the concept of dignity for people with disability, Chapman et al. [ 34 ] indicated that recognising the right to make decisions and proactively eliminating or minimising barriers to inclusion are key to protecting dignity. Community participation in decision-making processes such as this study can result in messy and unpredictable outcomes. However, the onus must be placed on policymakers, organisations, and academia to acknowledge this sufficiently rather than demand conformity [ 15 ].

The authors posit that the study would have benefited from an alternative ethics pathway, which may provide additional required flexibility while upholding the rigour of the ethical review process. The increasing frequency of participatory research studies indicates that challenges experienced by the authors of this study are unlikely to be isolated. Lloyd [ 43 ] described challenges regarding information gathered in-between, before and after structured research sessions, reflecting that they relied on personal judgement of the intent to consent for research use. Similarly, Rowley [ 44 ] reflected on the ethical complexities of interacting with families and respecting their confidentiality within the context of being integrated within an organisation. While these studies were co-production in child protection and education, the ethical challenges of their reflections parallel those experienced in the telepractice study. The risks posed by inadequate ethical support in these contexts are that increased poor ethical outcomes will occur, especially in the in-between times of co-design. Therefore, an ethics pathway that involves more frequent brief liaisons with a designated ethics representative to update project progress and troubleshoot ethical considerations may better support researchers to safeguard study participants.

We believe the decision to complete a sequential workshop series with a consistent group of diverse co-designers, led by co-facilitators, was a strength of the co-design process implemented in the telepractice re-design project. The group worked together across a series of workshops, which enabled them to build solid working relationships. Pearce et al. [ 5 ], Rahman et al. [ 16 ] and Tindall et al. [ 13 ] also demonstrated a collaborative whole-team approach to co-design. By contrast, studies that involved separate workshops with different cohorts or multiple of the same workshop did not demonstrate strong collaboration between co-designers [ 18 , 19 , 20 ]. Nesbitt et al. [ 19 ] explicitly highlighted that they would improve their method by completing sequential workshops with a continuous cohort. Stephens et al. [ 45 ] found that small mixed groups were not sufficient to support the participation of people with disability, indicating that the choice to intentionally balance groups to meet the lived experience expert co-designer’s needs may have been an impacting factor on our success.

A lack of clarity in the terminology used in co-design and community-based participatory practice was identified during the completion of this study. We found that co-design frequently meant either a collaborative design process or good participatory practices [ 46 ]. When viewing the structure of the telepractice re-design project, the overarching research approach was community-based participatory-research, and the method was co-design [ 9 ]. The delineation between the overarching approach and methods clarifies the misappropriation of the term co-design with the intent of meaning public participation [ 46 ] rather than the joint process of creative thinking and doing to design an output [ 11 ]. The use of the two-level structure appears more prominent in the United Kingdom, whereas Fox et al. [ 47 ] systematic review assessing public or patient participants identified that 60% of studies originated from the United Kingdom, compared to the next highest 16% for Canada or 4% from Australia and the United States. To improve clarity and reduce confusion about the terminology used, the authors advocate for greater awareness and implementation of the delineation between the concepts of a community-based-participatory-research/patient or public involvement approach versus the co-design method.

An example of co-design being used where alternate terms such as community-based participatory processes (or research) may be more relevant was the most recent amendment to the act governing the NDIS under which this project resided [ 48 ]. The term co-design could be interpreted as an intent to collaborate with people with disability for equitable involvement in all aspects of the NDIS [ 48 ]. It is proposed that the differentiation of these terms would assist in clarifying the intent of the study and dissuade inaccurate expectations of community involvement or design processes.

Implementing community-based participatory research has demonstrated the potential to create an impact that expands further than the original aim of the study. The skills learned by co-designers, the learning of the research team in collaboration with people with disability, the engagement and skill-building of a peer researcher with lived experience, the organisations who engaged in the co-design process and the academic and lay people who engaged with research outputs, all carry a piece of the impact of the co-design process. Rahman et al. [ 16 ] contend that co-design processes positively impact communities. In the context of this study, the peer researcher was included in the National Disability Insurance Agency’s quarterly report as an example of strength-based employment opportunities, which significantly positively impacted his career prospects [ 49 ]. This project provided skills for people with disability that they value and improved the clinical practice of clinician co-designers, which echoes the conclusions of Ramos et al. [ 15 ], who described that participants felt valued and experienced improved self-esteem. There is additional intent from the authors to positively impact disability providers and academia, to advocate for greater collaboration, and to provide open-access publications to provide a stronger evidence base for co-design in clinical practice and service delivery.

Strengths and limitations

The study provides reflective evidence to support the challenges and benefits experienced during the implementation of the study. However, a limitation in the project’s design was the exclusion of outcome measures to assess the impact of process design choices directly. Stephens et al. [ 45 ] completed targeted outcome measures correlating to accessibility adaptations in co-design and conceded that the variability of findings and individual needs reduced the usefulness of these measures.

The reduction of project scope enabled the completion of the study within the limitations of budgeting and timeline restrictions. Although the scope of the project had some flexibility, there were limitations to how far this could be extended as resources were not infinite, and staffing changes meant that organisational priorities changed. Including implementation and evaluation would have improved the study’s rigour. However, Rocky Bay now has the opportunity to implement internally without potential research delays and restrictions.

The blended and flexible approach to the co-design process was a strength of the study as it met the co-designers needs and maximised the project’s potential inclusivity. This strength has the potential to positively impact other studies that can modify some of the process design choices to suit their context and increase inclusivity [ 11 ]. It is believed that the messiness of co-design is important in meeting the needs and context of each individual study; therefore, no two co-design processes should look the same.

The authors concede that the inclusion of a cohort of people with disability and clinical staff does not represent the entirety of their communities, and their proposed changes may cause some parts of the disability community to experience increased barriers [ 50 ]. It is important to note that while the co-designers who participated in this project provided initial design developments, future opportunities remain to iterate the proposed telepractice service and continue to advocate for equitable access for all.

Recommendations for future studies

Recommendations from this study fall into two categories: recommendations for those intending to utilise the described methods and recommendations for future avenues of research inquiry. For those intending to implement the methods, the primary recommendations are to build ample time buffers into the project schedule, implement key role succession planning and set remuneration agreements at the outset, and work together as partners with the mindset that all contributors are creative [ 51 ] with important expertise and invaluable insights if supported appropriately.

Regarding avenues for future inquiry, we recommend investigating a more dynamic and flexible ethics process that may utilise more frequent short consultations to respond to ethical considerations during the emergent co-design and participatory research.

In the authors’ opinion, supported by co-designers experiences, co-design is a useful and outcome-generating methodology that can proactively enable the inclusion of people with disability and service providers in a community-based participatory research approach. The process is both time and resource-intensive; however, in our opinion, the investment is justified through the delivery of direct research benefits and indirect wider community benefits. We advocate for using community-based participatory-research/processes paired with co-design to generate creative thinking within service design processes. Through co-design processes, we recommend collaborating with a single diverse group of co-designers who have the time and space to build trusting working relationships that enable outputs representative of the group consensus.

Data availability

The dataset supporting the conclusions of this article is predominantly included within the article (and its additional files). However, due to the small number of co-designers reflecting upon the research, despite deidentification, there is a reasonable assumption of identification; therefore, the reflection activity response supporting data is not available.

Abbreviations

Australian Dollar

Guidance for Reporting Involvement of Patients and the Public 2 Checklist

Human Research Ethics Committee

Doctor of Philosophy

Patient and Public Involvement

Microsoft Teams

National Disability Insurance Scheme

McKercher KA. Beyond Sticky Notes doing co-design for Real: mindsets, methods, and movements. 1 ed. Sydney, NSW: Beyond Sticky Notes; 2020. p. 225.

Google Scholar  

Mullins RM, Kelly BE, Chiappalone PS, Lewis VJ. No-one has listened to anything I’ve got to say before’: co-design with people who are sleeping rough. Health Expect. 2021;24(3):930–9. https://doi.org/10.1111/hex.13235 .

Article   PubMed   PubMed Central   Google Scholar  

Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, et al. Person-centered Care — Ready for Prime Time. Eur J Cardiovasc Nurs. 2011;4248–51. https://doi.org/10.1016/j.ejcnurse.2011.06.008 . [cited 3/9/2022];10.

National Commission on Safety and Quality in Healthcare. Partnering with Consumers Standard. Australia: National Commission on Safety and Quality in Healthcare. 2021. https://www.safetyandquality.gov.au/standards/nsqhs-standards/partnering-consumers-standard .

Pearce T, Maple M, McKay K, Shakeshaft A, Wayland S. Co-creation of new knowledge: good fortune or good management? Res Involv Engagem. 2022;8(1):1–13. https://doi.org/10.1186/s40900-022-00394-2 .

Article   Google Scholar  

Bordeaux BC, Wiley C, Tandon SD, Horowitz CR, Brown PB, Bass EB. Guidelines for writing manuscripts about community-based participatory research for peer-reviewed journals. Prog Community Health Partnersh. 2007;1(3):281–8. https://doi.org/10.1353/cpr.2007.0018 .

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3(1):1–11. https://doi.org/10.1186/s40900-017-0062-2 .

Ostrom E, Baugh W, Guarasci R, Parks R, Whitaker G. Community Organization and the Provision of Police Services. Sage; 1973.

Masterson D, Areskoug Josefsson K, Robert G, Nylander E, Kjellström S. Mapping definitions of co-production and co-design in health and social care: a systematic scoping review providing lessons for the future. Health Expect. 2022;25(3):902–13. https://doi.org/10.1111/hex.13470 .

Bibb J. Embedding lived experience in music therapy practice: Towards a future of co-designed, co-produced and co-delivered music therapy programs in Australia. Australian Journal of Music Therapy [Journal Article]. 2022 [cited 2023/08/21];33(2):25–36. https://doi.org/10.3316/informit.829441047529429 .

Davis A, Gwilt I, Wallace N, Langley J. Low-contact Co-design: considering more flexible spatiotemporal models for the co-design workshop. Strategic Des Res J. 2021;14(1):124–37. https://doi.org/10.4013/sdrj.2021.141.11 .

Claborn KR, Creech S, Whittfield Q, Parra-Cardona R, Daugherty A, Benzer J. Ethical by design: engaging the community to co-design a Digital Health Ecosystem to Improve Overdose Prevention efforts among highly vulnerable people who use drugs. Front Digit Health [Original Research]. 2022;4:1–13. https://doi.org/10.3389/fdgth.2022.880849 .

Tindall RM, Ferris M, Townsend M, Boschert G, Moylan S. A first-hand experience of co‐design in mental health service design: opportunities, challenges, and lessons. Int J Ment Health Nurs. 2021;30(6):1693–702. https://doi.org/10.1111/inm.12925 .

Article   PubMed   Google Scholar  

Wahlin DW, Blomkamp DE. Making global local: global methods, local planning, and the importance of genuine community engagement in Australia. Policy Des Pract. 2022;5(4):483–503. https://doi.org/10.1080/25741292.2022.2141489 .

Ramos M, Forcellini FA, Ferreira MGG. Patient-centered healthcare service development: a literature review. Strategic Des Res J. 2021;14(2):423–37. https://doi.org/10.4013/sdrj.2021.142.04 .

Rahman A, Nawaz S, Khan E, Islam S. Nothing about us, without us: is for us. Res Involv Engagem. 2022;8(1):1–10. https://doi.org/10.1186/s40900-022-00372-8 .

Harrison R, Manias E, Ellis L, Mimmo L, Walpola R, Roxas-Harris B, et al. Evaluating clinician experience in value-based health care: the development and validation of the Clinician experience measure (CEM). BMC Health Serv Res. 2022;22:1–11. https://doi.org/10.1186/s12913-022-08900-8 .

Kerr JAS, Whelan M, Zelenko O, Harper-Hill K, Villalba C. Integrated Co-design: a model for co-designing with multiple stakeholder groups from the ‘Fuzzy’ front-end to Beyond Project Delivery. Int J Des. 2022;16(2):1–17. https://doi.org/10.57698/v16i2.06 .

Nesbitt K, Beleigoli A, Du H, Tirimacco R, Clark RA. User experience (UX) design as a co-design methodology: lessons learned during the development of a web-based portal for cardiac rehabilitation. Eur J Cardiovasc Nurs. 2022;21(2):178–83. https://doi.org/10.1093/eurjcn/zvab127 .

Marwaa MN, Guidetti S, Ytterberg C, Kristensen HK. Using experience-based co-design to develop mobile/tablet applications to support a person-centred and empowering stroke rehabilitation. Res Involv Engagem. 2023;9(1):1–17. https://doi.org/10.1186/s40900-023-00472-z .

Tariq S, Grewal EK, Booth R, Nat B, Ka-Caleni T, Larsen M, et al. Lessons learned from a virtual community-based Participatory Research project: prioritizing needs of people who have diabetes and experiences of homelessness to co-design a participatory action project. Res Involv Engagem. 2023;9(1):1–11. https://doi.org/10.1186/s40900-023-00456-z .

Abimbola S, Li C, Mitchell M, Everett M, Casburn K, Crooks P, et al. On the same page: co-designing the logic model of a telehealth service for children in rural and remote Australia. Digit Health. 2019;5:2055207619826468–2055207619826468. https://doi.org/10.1177/2055207619826468 .

Rocky Bay. Rocky Bay Annual Report FY 2021–2022. Perth. 2022. https://www.rockybay.org.au/wp-content/uploads/2022/12/Rocky-Bay-Annual-Report-21-22.pdf .

National Disability Insurance Agency. What is the NDIS? [Internet]. 2021 [updated 14.08.2021. https://www.ndis.gov.au/understanding/what-ndis .

Reen G, Page B, Oikonomou E. Working as an embedded researcher in a healthcare setting: a practical guide for current or prospective embedded researchers. J Eval Clin Pract. 2022;28(1):93–8. https://doi.org/10.1111/jep.13593 .

Bell S, Aggleton P, Gibson A. Peer Research in Health and Social Development 1st Edition ed. London: Routledge; 2021. p. 286.

Book   Google Scholar  

Curran T, Jones M, Ferguson S, Reed M, Lawrence A, Cull N, et al. Disabled young people’s hopes and dreams in a rapidly changing society: a co-production peer research study. Disabil Soc. 2021;36(4):561–78. https://doi.org/10.1080/09687599.2020.1755234 .

Kelly B, Friel S, McShane T, Pinkerton J, Gilligan E. I haven’t read it, I’ve lived it! The benefits and challenges of peer research with young people leaving care. Qualitative Social work: QSW: Res Pract. 2020;19(1):108–24. https://doi.org/10.1177/1473325018800370 .

Schwartz AE, Kramer JM. Inclusive approaches to developing content valid patient-reported outcome measure response scales for youth with intellectual/developmental disabilities. Br J Learn Disabil. 2021;49(1):100–10. https://doi.org/10.1111/bld.12346 .

Webb P, Falls D, Keenan F, Norris B, Owens A, Davidson G, et al. Peer researchers’ experiences of a co-produced research project on supported decision-making. Res Involv Engagem. 2022;8(1):1–10. https://doi.org/10.1186/s40900-022-00406-1 .

People with Disability Australia. PWDA Language Guide: A guide to language about disability. Sydney, Australia. 2021. https://pwd.org.au/wp-content/uploads/2021/12/PWDA-Language-Guide-v2-2021.pdf .

Peters MDJGC, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E MZ, editor. JBI Manual for Evidence Synthesis, JBI, 2020: JBI; 2020.

Australian Broadcasting Commission. ‘My purpose is changing perceptions’: Australian of the Year Dylan Alcott’s speech in full [Internet]. 2022 [cited 17.08.2023]. https://www.abc.net.au/news/2022-01-26/dylan-alcott-australian-of-the-year-speech-in-full/100783308 .

Chapman K, Dixon A, Ehrlich C, Kendall E. Dignity and the importance of acknowledgement of Personhood for people with disability. Qual Health Res. 2024;34(1–2):141–53. https://doi.org/10.1177/10497323231204562 .

Flattery S. Stim Joy: Using Multi-Sensory Design to Foster Better Understanding of the Autistic Experience: ProQuest Dissertations Publishing; 2023.

Peak Learning. The Real Deal [Internet]. 2023 [cited 6.10.2023]. https://www.peaklearning.com/trd/ .

Benz C, Scott-Jeffs W, Revitt J, Brabon C, Fermanis C, Hawkes M, et al. Co-designing a telepractice journey map with disability customers and clinicians: partnering with users to understand challenges from their perspective. Health Expect. 2023;1–11. https://doi.org/10.1111/hex.13919 .

Flowers E, Miller ME. Your Guide to Blueprinting The Practical Way. 1 ed. USA: Practical By Design 2022. 134 p. pp. 1-134.

Blomkvist J. Benefits of Service Level Prototyping. Des J. 2016;19(4):545–64. https://doi.org/10.1080/14606925.2016.1177292 .

Harrison R, Ní Shé É, Debono D, Chauhan A, Newman B. Creating space for theory when codesigning healthcare interventions. J Eval Clin Pract. 2023;29(4):572–5. https://doi.org/10.1111/jep.13720 .

Bodden S, Elliott J. Finding space for Shared futures. Edinb Archit Res. 2022;37:90–104.

Page K. Ethics and the co-production of knowledge. Public Health Research & Practice. 2022:1–5. https://www.phrp.com.au/issues/june-2022-volume-32-issue-2/ethics-and-co-production/ .

Lloyd J. Life in a lanyard: developing an ethics of embedded research methods in children’s social care. J Children’s Serv. 2021;16(4):318–31. https://doi.org/10.1108/JCS-12-2019-0047 . [cited 2023/12/05];.

Rowley H. Going beyond procedure:engaging with the ethical complexities of being an embedded researcher. Manage Educ. 2014;28(1):19–24. https://doi.org/10.1177/0892020613510119 .

Stephens L, Smith H, Epstein I, Baljko M, McIntosh I, Dadashi N, et al. Accessibility and participatory design: time, power, and facilitation. CoDesign. 2023;1–17. https://doi.org/10.1080/15710882.2023.2214145 .

Gardner G, McKercher KA. But is it co-design? And if it is, so what? 2021. https://healthvoices.org.au/issues/nov-2021/but-is-it-co-design-and-if-it-is-so-what .

Fox G, Lalu MM, Sabloff T, Nicholls SG, Smith M, Stacey D, et al. Recognizing patient partner contributions to health research: a systematic review of reported practices. Res Involv Engagem. 2023;9(1):1–30. https://doi.org/10.1186/s40900-023-00488-5 .

National Disability Insurance Agency. 2022 NDIS legislation amendments Australia; 2022. https://www.ndis.gov.au/news/7975-2022-ndis-legislation-amendments-july-update .

National Disability Insurance Agency. Report to disability ministers for Q4 of Y10 Summary Part A Australia. 2023. https://www.ndis.gov.au/about-us/publications/quarterly-reports .

Lid IM. Universal Design and disability: an interdisciplinary perspective. Disabil Rehabil. 2014;36(16):1344–9. https://doi.org/10.3109/09638288.2014.931472 .

Sanders E, Stappers PJ. Co-creation and the New landscapes of Design. CoDesign. 2008;4:5–18. https://doi.org/10.1080/15710880701875068 .

Download references

Acknowledgements

The authors acknowledge the contribution of Rocky Bay as the industry partner of this project and would like to thank the Co-designers of this project, without whom none of this was possible. The research team would also like to thank Katie Harris for her time and support throughout the workshop series, which were invaluable to the completion of the project and the formation of the published study.

The article forms part of a PhD project funded by the first author, CB’s Australian Government Research Training Program (RTP) scholarship.

Author information

Authors and affiliations.

School of Population Health, Curtin University, Bentley, Australia

Cloe Benz, Richard Norman, Delia Hendrie & Suzanne Robinson

Rocky Bay, Mosman Park, WA, Australia

Will Scott-Jeffs, Mai Welsh & Matthew Locantro

Beyond Sticky Notes, Sydney, Australia

K. A. McKercher

Therapy Focus, Bentley, Australia

Deakin Health Economics, Institute for Health Transformation, Deakin University, Melbourne, Australia

Suzanne Robinson

You can also search for this author in PubMed   Google Scholar

Contributions

CB and MW liaised with the steering committee and conceived the study and structure. SR, DH and RN guided the protocol development and ethics approval. KAM provided methodological support to the project and subject matter expertise. CB and WJS completed participant recruitment, facilitation of workshops and data collection. KAM and CB ideated the format and content of the article. CB completed data analysis and wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved of the final version of the manuscript.

Corresponding author

Correspondence to Cloe Benz .

Ethics declarations

Ethical approval and consent.

The study was approved by the Curtin University Human Research Ethics Committee (ID# HRE2021-0731), and all participants provided written informed consent before engaging in any research activity.

Consent for publication

Not applicable.

Competing interests

Cloe Benz, Richard Norman, Delia Hendrie & Suzanne Robinson do not have any competing interests to declare. Will Scott-Jeffs, Matthew Locantro and Mai Welsh, for all or part of the study period were employed by Rocky Bay a Not-For-Profit Disability Service provider who function as the industry partner for the project. K.A. McKercher is the author of a co-design method book referenced in the article. McKercher also runs a business that helps people co-design.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1:

Appendix 1–3

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Benz, C., Scott-Jeffs, W., McKercher, K.A. et al. Community-based participatory-research through co-design: supporting collaboration from all sides of disability. Res Involv Engagem 10 , 47 (2024). https://doi.org/10.1186/s40900-024-00573-3

Download citation

Received : 13 November 2023

Accepted : 12 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1186/s40900-024-00573-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Community-based participatory-research
• Telepractice
• Lived experience
• Embedded researcher
• Digital health
• Patient and public involvement

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

• Partner with us
• Apply Online

Pirogov Medical University

The university.

• Recognitions
• Eligibility
• Fees Structure

Founded in 1906 in the city of Moscow, Pirogov Medical University—officially known as Russian National Research Medical University named after N. I. Pirogov— is one of the oldest medical universities in Russia.  The first lecture took place on September 26, 1906, with 206 students, and the first graduation ceremony was celebrated in 1912. The University claims to be the first university in Russia that started offering medical education to women in the Russian Federation.It independently carries out various research projects in medicine and has received the status of National Research University in 2010.The Library of Pirogov Medical University has a collection of more than 7,50,000 books. For world-class clinical training and diverse practical exposure, the University collaborates with more 120 hospitals across the city of Moscow. Pirogov Medical University started accepting international students in 1959. Presently, more than 8,000 students are studying medicine at the University, out of which about 700 are international students.  The University has a strong team of about 2,000 faculty members. Pirogov Medical University is approved by the Medical Council of India (MCI) and offers a 6-Year Program for MBBS in Russia. Students in India, who have qualified NEET, can apply for direct admission to the MBBS Program of Pirogov Medical University.

Ministry of Science and Higher Education, Russia

World Directory of Medical Schools (WDOMS)

Medical Council of India (MCI)

Foundation for Advancement of International Medical Education and Research (FAIMER)

To get admission to the MBBS Program of Pirogov Medical University, the student must qualify NEET-UG (National Eligibility cum Entrance Test-Undergraduate). 

Besides NEET-UG, there is no requirement to go through any additional entrance examination.

RUS EDUCATION SUPPORT

INDIAN FOOD

MODERN CLASSROOMS

Medical Laboratories

Clinical Training

Recreational Facilities

Ensured Safety

FMGE (Foreign Medical Graduates Examination) Preparation

USMLE (United States Medical Licensing Examination) Preparation

• Pirogov Medical University was founded in 1906 in the Moscow city of Russia.
• In 1930, Pirogov Medical University organized the World’s first pediatric faculty.
• In 1963, Pirogov Medical University organized the World’s first biomedical faculty.
• In 2010, Pirogov Medical University received the status of National Research University.
• Presently, more than 8,000 students are studying at Pirogov Medical University, out of which about 700 are international students.

University Address

Mbbs program, admission & support, medical licensing examination support, student life.

Founded in 1906 in the city of Moscow, Pirogov Medical University—officially known as Russian National Research Medical University named after N. I. Pirogov— is one of the oldest medical universities in Russia.  The first lecture took place on September 26, 1906, with 206 students, and the first graduation ceremony was celebrated in 1912. 

Focused on constantly improving the quality of education, the University entered the list of Times Higher Education World University Rankings and QS World University Rankings in 2019. 

The University claims to be the first university in Russia that started offering medical education to women in the Russian Federation. Among all milestones achieved by the University, two most celebrated achievements are that Pirogov Medical University created the world’s first pediatric faculty in 1930 and the world’s first biomedical faculty in 1963.

To provide students and visitors a glimpse into the decades of the glorious history of the University, Museum of the History of Pirogov Medical University was established in 1981, which continues to capture astonishing achievements of the University, its students, and members.

The University maintains a leading position in Russia by actively participating in activities of medical research and healthcare and delivering excellence in medical education and care. 

The University independently carries out various research projects in medicine and has received the status of National Research University in 2010. Conducting pre-clinical as well as clinical studies for a better understanding of human diseases, new medicines, and medical devices have been the center of research at Pirogov Medical University.  

To keep the students and healthcare professionals updated on the latest research and innovations in medicine, the University publishes its own scientific journal with articles on biomedical sciences and clinical medicine.

The Library of Pirogov Medical University has a collection of more than 7,50,000 books. Students can also access scientific journals and eBooks through the electronic library system. In collaboration with other universities, academic mobility and exchange programs are also arranged for students to help them get experience in other institutions and build new connections.

For the healthcare of locals, the University operates clinical centers in Moscow. Students are provided hands-on clinical training in these University-operated clinics and also involved in various clinical studies. For world-class training and diverse practical exposure, the University collaborates with more 120 hospitals across the city of Moscow.

Pirogov Medical University started accepting international students in 1959. Until now, the University has trained more than 80,000 doctors. Presently, more than 8,000 students are studying medicine at the University, out of which about 700 are international students.  The University has a strong team of about 2,000 faculty members.

Pirogov Medical University is listed in the World Directory of Medical Schools (WDOMS) and certified by the Educational Commission for Foreign Medical Graduates (ECFMG), United States of America. Pirogov Medical University is also approved by the Medical Council of Canada (MCC) and the Medical Council of India (MCI). The University offers a 6-Year Program for MBBS in Russia for local as well as international medical aspirants. Students in India, who have qualified NEET, can apply for direct admission to the MBBS Program of Pirogov Medical University.

Pirogov Medical University Faculty of Medicine 1 Ostrovityanov Str Moscow, 117997 Russian Federation

Pirogov Medical University offers a 6-Year MBBS Program in the Russian language. For international students, classes for initial years may be organized in English medium.

The Program for MBBS in Russia is focused on building a strong academic base with a pragmatic approach to education and medical research. To provide hands-on clinical experience, the students studying MBBS in Russia are involved in clinical training from the second year of MBBS. While education in classrooms and laboratories helps the students develop academic skills and sound theoretical understanding, clinical training in University-affiliated hospitals help them apply their knowledge into practice.

To get admission to the MBBS Program of Pirogov Medical University, you can apply online at Rus Education website.

Rus Education is duly authorized by the Russian Centre for Science and Culture (Cultural Department of The Embassy of the Russian Federation in India) to promote Russian Education among Indian Citizens. Rus Education is also an authorized associate of Pirogov Medical University. We facilitate one-window admission to the MBBS Program of Pirogov Medical University with no requirement of any donation or capitation and without any entrance examination.

Pirogov Medical University offers a healthy student life and an opportunity to experience life in Moscow, the capital city of Russia, and also the most vibrant and exciting location in the largest country in the world! 

For affordable accommodation of students and make their living experience safe and better, the University maintains a comfortable dormitory. Every room is shared by two or three students, and each floor has a shared kitchen where students can cook their food. Members of the dormitory help the newcomers to settle in their new homes. For the safety of the students, the University’s security team maintains 24-hour surveillance and is capable of providing emergency response, if required.

To help students adjust to life at university, it has a dedicated Student Support System in place. Every group of new students is assigned to two professors who guide the students not only about studying but about living as well, helping students adjust to the new environment and feel comfortable.

To keep students fit and active, Sports Center on the campus is equipped with facilities to play various sports, including badminton, basketball, volleyball, table tennis, swimming, football, hockey, chess, etc. Student can unleash their creativity by indulging in extracurricular adventures offered by Student Organizations and Societies. On the campus, there are ample opportunities for self-improvement and taking part in music, dance, sports competitions, and theater. 

For peer support, the University has a Student Council in place which offers support in academic as well as non-academic matters making student life stress free.

For the social upliftment and help students connect with the society and local people, they are involved in community and welfare organized by the University, including medical outreach, health awareness programs, and blood donation camps.  The University Volunteer Center organizes a number of volunteer activities to help students contribute to social causes.

Living in Moscow, students can explore its cultural heritage, museums, historic buildings, the world-famous Alexander Garden, and much more. For traveling in Moscow, students don’t face any problems, thanks to its convenient and cheap transportation system, especially the Moscow Metro.

With the charm of Moscow and all the student facilities and support services offered by the University, student life at Pirogov Medical University is a delight.

TOP MEDICAL UNIVERSITIES IN RUSSIA

Perm State Medical University

Tver State Medical University

Orenburg State Medical University

Mari State University

Siberian State Medical University

A php error was encountered.

Severity: Notice

Message: Undefined variable: countries

Filename: includes/footer.php

Line Number: 26

File: /home/mbbsinrussia/public_html/application/views/includes/footer.php Line: 26 Function: _error_handler

File: /home/mbbsinrussia/public_html/application/controllers/University.php Line: 46 Function: view

File: /home/mbbsinrussia/public_html/index.php Line: 315 Function: require_once

Severity: Warning

Message: Invalid argument supplied for foreach()

Message: Undefined variable: state

Line Number: 44

File: /home/mbbsinrussia/public_html/application/views/includes/footer.php Line: 44 Function: _error_handler

©2024-25 Rus Education.