research project review sace

Research Project A & B (Stage 2)

Length of course, compulsory or elective, pre-requisites, subject description.

Students choose a research question that is based on an area of interest to them. They explore and develop one or more capabilities in the context of their research.

The term ‘research’ is used broadly and may include practical or technical investigations, formal research, or exploratory inquiries.

The Research Project provides a valuable opportunity for SACE students to develop and demonstrate skills essential for learning and living in a changing world. It enables students to develop vital skills of planning, research, synthesis, evaluation, and project management.

The Research Project enables students to explore an area of interest in depth, while developing skills to prepare them for further education, training, and work. Students develop their ability to question sources of information, make effective decisions, evaluate their own progress, be innovative, and solve problems.

The content of both Research Project A and B consists of:

• developing the capabilities
• applying the research framework 

In Research Project students choose a research question that is based on an area of interest . They identify one or more capabilities that are relevant to their research.

Students use the research framework as a guide to developing their research and applying knowledge, skills, and ideas specific to their research question. They choose one or more capabilities, explore the concept of the capability or capabilities, and how it or they can be developed in the context of their research.

Students synthesise their key findings to produce a Research Outcome, which is substantiated by evidence and examples from the research. They review the knowledge and skills they have developed, and reflect on the quality of their Research Outcome.

Students must achieve a C– grade or better to complete the subject successfully and gain their SACE.

For Research Project A, students can choose to present their external assessment in written, oral, or multimodal form.

For Research Project B, the external assessment must be written.

The following assessment types enable students to demonstrate their learning in Stage 2 Research Project A and B:

School Assessment (70%)

Assessment Type 1: Folio (30%)

Assessment Type 2: Research Outcome (40%)

External Assessment (30%)

Assessment Type 3: Review (30% Research Project A)

Assessment Type 3: Evaluation (30% Research Project B)

Research Project A and B contribute to an ATAR

Pathways for Cross Disciplinary

Click a subject to view its pathway

Subject added

THS Curriculum Handbook

Year 11 – Research Project

Length: Single Semester (10 Stage 2 credits) Contact: SACE Leader ALL students must complete the 10-credit Research Project at Stage 2 of the SACE, with a C− grade or better. Course Description Students will:

• Choose a topic of interest and develop a research question
• Learn and apply research processes and the knowledge and skills specific to their research topic
• Record their research and evaluate what they have learnt.

The term research is used broadly and may include practical or technical investigations, formal research, or exploratory enquiries. Students are expected to:

• Work independently and with others to initiate an idea, and to plan and manage a research project
• Demonstrate the learning capability and 1 other chosen capability
• Analyse information and explore ideas to develop their research
• Develop and apply specific knowledge and skills
• Communicate and evaluate their research outcome
• Evaluate the research processes used and their chosen capability.

Assessment (Both ATAR accredited) Research Project A

• Folio (30%)
• Research Outcome (40%)
• Review (external assessment – 30%).

Maximum of 1500 words if written. Maximum of 10 minutes for an oral presentation. Equivalent in multimodal form. Research Project B

• Evaluation (30%).

A maximum of 2000 words if written or a maximum of 12 minutes for an oral presentation, or the equivalent in multimodal form. Note: We strongly advise that Research Project B be undertaken for those students on a University pathway.

Enterprising Research and the SACE Research Project

Topic outline, 2.2 what makes a good sace research project question.

While we now know a bit more about the research questions that researchers at UniSA are working with, it’s important to contextualise your own research questions within the Research Project itself. University-level research and Research Projects are of course very different, and the sorts of questions you ask reflect that. 

While our researchers at UniSA have whole labs, teams, and sometimes years at their disposal, you only have a short amount of time to complete your projects. This means your questions should look a little different. 

So, what should Research Project questions look like?

You shouldn’t be able to answer it with a Google search, or by asking one expert what they think. It should require research using multiple sources, both primary and secondary. 

You should be looking for a gap, a question that someone else hasn’t asked yet. A good rule of thumb is to think local. How does this question impact someone like you? Or year 12 students? Or people in Adelaide? These are often questions often in need of an answer, and questions that will require a suitable amount of effort on your part.

KIS Academics

We'll mark your

Sace research project, assignment feedback starting at $30.

Our expert tutors can help you ace your

Last minute cram? Get your

back as quickly as you need it

Get your assignment back on time, or we'll give you a full refund and send it to you anyway

It's as easy as 1, 2, 3 🎶

Here's why kis academics is your #1 choice:.

Marked by the top 3% of tutors in Australia

Year 7-12 Science Practicals

Assignment marking starts at $30

Science Poster Presentations and EPIs

Year 5-10 English Essays

Research Projects (SACE)

Year 11-12 ATAR English Essays

Maths Investigations

SACE SHE tasks. Learn how to ace it here .

...and more!

Getting an expert to look over your

Won't just boost your mark... it'll make sure you actually learn and improve with the help of useful feedback, how does the sace research project work 🔬, so what is the research project.

Unlike your other SACE stage 2 subjects being 20 credits, the research project is a 10-credit SACE subject you will either complete in year 11 or 12 depending on which high school you attend. The subject consists of three parts: the folio, outcome, and review for research project A or the evaluation if you are undertaking research project B.

Despite research projects A and B having different performance standards, both encourage you to explore a topic of choice in depth, gathering various sources and writing reflections on your learning. In the first few weeks of the subject, your teacher will guide you when developing your question.

The folio is 10 pages in length and typically consists of your reflections and the main sources you have collected through your research (both primary and secondary sources!). You will then write an outcome that is essentially answering your original research question. Lastly, comes the evaluation or review where you will write an overall reflection and evaluate the findings in the outcome.

How do I develop the best question for my topic of interest?

The most important part of the research is picking the right topic.

You want to pick something you have a strong interest in. This way, it will be much easier for you to feel more motivated to sit down and do your research. However, at the same time, you want to pick a topic that will have lots of research behind it, you don't want to be stuck for sources!

To avoid this, write down a list of topics you have an interest in and do some research on each - see what is available online or at a local library. This way, you will be more prepared when your teacher comes over to your desk to ask you what you have done so far!

Once you have picked your topic, create another list of possible questions you could investigate. These questions should be open-ended, not just with a simple yes or no answer. Keep in mind you will be writing a 1500 to 2000-word answer to this question, so make it a question you can go into complete depth with. Typical questions should be specific and may begin with ‘to what extent’, ‘evaluate’, ‘what’ or ‘how’. For example, if you picked social media as your topic, your question could be ‘to what extent does social media use impact the attention spans of teenagers aged 13-17?’ rather than ‘does social media impact attention spans?’.

You may then have to break down your main question into four more guiding questions to help you structure your folio and outcome. For example, ‘how much time do teenagers aged 13-17 spend on social media every day?’. It is important that you keep documentation of this process as you will be displaying it in your folio.

Want an expert tutor to read over your Research Project before handing it in? You've come to the right place.

Send it through to us now and we'll return your Research Project to you with detailed marking and feedback. We'll even compare it to any criteria or rubric you've been given to make sure it aligns with your marking standards 🔥

Trusted Tutors by families all over Australia 🇦🇺

Real results and verified reviews, why you should get your, marked by kis academics:, review before submitting, maximise your marks.

If you really want to achieve the highest grades for your assignments, submit them to be marked by KIS Academics' expert tutors before handing them in. ‍ Having someone look over your work and provide feedback is one of the best ways to pre-screen assignments before submission and get the best advantage. As soon as you send your assignment to us, our tutors will review and return it to you with thorough and useful recommendations to improve not just this assignment, but your academic skills overall. All assignment reviewing is done in accordance with our Academic Integrity Policy ‍

get expert advice

Don't just hand assignments in... learn from them.

Assignments aren't just a way to test your knowledge. You should be able to learn from them too. Receive valuable feedback on: 👉 Spelling, grammar and punctuation 👉 Sentence and essay structure 👉 Themes, quotes and analysis 👉 Scientific and Mathematical Accuracy 👉 Sticking to curriculum and rubric requirements 👉 And more...

Let's ace your

Our academic integrity policy: ‍ provide help, not answers 🎓.

KIS Academics is committed to maintaining the highest levels of academic integrity by creating, using, and sharing information in a manner that is responsible , honourable , and, at all times, ethical .

KIS Academics considers the upholding of academic integrity to be the responsibility of all employees within the company, as well as all private tutors.

KIS Academics takes an educative - not punitive - approach to ensure students and private tutors are provided with the support needed to effectively address any academic integrity issues that may be identified.

KIS Academics mandates our tutors to maintain the highest levels of academic integrity in all their interactions with their students.

Australia’s Tertiary Education Quality and Standards Agency (TEQSA) defines academic integrity as ‘a commitment ... to six fundamental values: honesty, trust, fairness, respect, responsibility, and courage’ (International Centre for Academic Integrity, 2014 AS CITED IN TEQSA).

These fundamental values are the basis on which principles and behaviours within the scope of academic integrity are developed. For the purpose of this policy, academic integrity is the moral and ethical code of conduct related to the use, generation and communication of information in an honourable, fair, and responsible manner.

Submitting your assignment to be reviewed by a KIS Academics tutor will ensure you not only receive top-notch feedback and guidance on how to improve your academic skills, but you do so in a way that is ethical, legal and maintains academic integrity

Assignment and essay marking

Frequently asked questions 🙋, what is the marking service.

How much does it cost?

How does it work, who will mark my assignment, is it confidential, what if it's not sent back in time.

That's why we set out to hire only the best tutors in Australia .

We have hundreds of amazing, qualified online tutors offering top quality services, starting at just $40 per hour. Whether the student is in high school or primary school, we cater for all. So, why are our tutors so amazing compared to other companies? It's simple, we pay them better. We take a low commission so you can get top scoring private tutors at the best rates. Discover our team of dedicated tutors who specialize in various subjects, from Chemistry and Maths to English and UCAT , as well as offering support for high school, HSC , and VCE , and many more. View all tutoring locations .

Paying for an initial lesson with a tutor you've never met is absolutely crazy! And surprisingly, that's what most tutoring places will ask you to do. We think you deserve to meet your tutor and make sure they're the right fit for you - free of charge. Request a tutor and they will reach out to you to organise this free trial session!

Skeptical about online courses ? Trust us, you won't be for long. With tutors specifically trained to deliver content online and use the latest and greatest teaching tools, we'll make you wish you had done things with KIS from the start. You can watch, rewatch, revise and review content as often as you need to. Starting at only $8 per week.

It's hard to express just how much we care that you have a great experience with us. We invite you to look us up on Facebook, TrustPilot, Instagram and Google. There's plenty of amazing stories from our previous customers. Click below to see all our reviews on posted on our Wall of Love ❤️

• VCE Chemistry tutors Melbourne
• VCE English tutors Melbourne
• VCE  Methods tutors Melbourne
• Year 5-12 Maths tutors Melbourne
• Year 5-12 English tutors Melbourne
• HSC Chemistry tutors Sydney
• HSC English tutors Sydney
• HSC Physics tutors Sydney
• Year 5-12 Maths tutors Sydney
• Year 5-12 English tutors Sydney
• ATAR Chemistry tutors Brisbane
• ATAR English tutors Brisbane
• ATAR Methods tutors Brisbane
• Year 5-12 Maths tutors Brisbane
• Year 5-12 English tutors Brisbane
• WACE Chemistry tutors Perth
• WACE English tutors Perth
• WACE Maths Methods tutors Perth
• Year 5-12 Maths tutors Perth
• Year 5-12 English tutors Perth
• Medical Entry
• Study Skills
• Private Tutoring
• Bring a Friend and Save
• Online Courses
• Assignment and Essay Marking
• How Private Tutoring Works
• How The ATAR Works
• Atar Calculator
• Free Events
• Tutoring Reviews ❤️
• Help Centre
• Become a Tutor
• Terms of Service
• Privacy Policy
• Jobs at KIS Academics 🚀
• Chemistry Tutors in Adelaide
• English Tutors in Adelaide
• Maths Tutors in Adelaide
• Mathematical Methods Tutors in Adelaide
• Chemistry Tutors in Perth
• English Tutors in Perth
• Maths Tutors in Perth
• Mathematical Methods Tutors in Perth
• Chemistry Tutors in Brisbane
• English Tutors in Brisbane
• Maths Tutors in Brisbane
• Mathematical Methods Tutors in Brisbane
• Chemistry Tutors in Melbourne
• English Tutors in Melbourne
• Maths Tutors in Melbourne
• Mathematical Methods Tutors in Melbourne

• How it Works
• Find a Tutor

• Read Today's Paper

Controversial SACE research project to stay in curriculum despite criticism

STUDENTS must continue to study the controversial research project despite many submissions asking for it to be dumped, Education Minister Grace Portolesi says.

STUDENTS must continue to study the controversial research project despite many submissions to a review asking for it to be dumped, made optional or strictly a Year 11 subject, Education Minister Grace Portolesi says.

Responding to the SACE first year evaluation, the State Government says it will now encourage students to study five full subjects in Year 12.

But it is not willing to enforce the move, instead leaving it up to the state's universities to boost entry requirements.

Ms Portolesi yesterday said the Government would also provide more support for teaching and marking, more practical research project options and an optional preparatory research skills subject in Year 11.

There will be no changes for students introduced next year. Revised research project options and the research skills subject will be introduced in 2014.

"We are strengthening the research project and we are backing the SACE," Ms Portolesi said.

She revealed plans to provide students with an incentive to study a fifth Year 12 subject and had written to each of SA's three universities asking them to consider calculating the university entry score using a 90-point system rather than the current 80 points.

Under the current system, the score is based on students' results in four subjects because each subject is worth 20 points. Moving to a 90-point system would allow the fifth subject to also be counted.

"The decision to have a 90-point aggregate is up to universities and I am asking them to consider this in time for students studying for their SACE in 2015," Ms Portolesi said.

"This means students could study five full-year subjects in their final year and have each of them count towards their ATAR (university entrance score).

An Institute of Educational Assessors would also be established to assist teachers and the SACE Board would receive an extra $7.6 million investment over the next five years.

The independent evaluation of the first year of the new SACE found the research project provides an "inherent advantage" for female students and recommended an easier version be available to Year 11 students who wanted to do five subjects in Year 12.

The government did not support this recommendation and instead would continue to allow schools to offer the research project designed for Year 12s in Year 11.

Each of SA's three universities said they were willing to look at Ms Portolesi's request to change the entry requirements.

Australian Education Union SA branch president Correna Haythorpe said teachers wanted to see more flexibility and thought the research project should not be compulsory.

"I believe members would continue to hold the view that students should be able to decide whether to engage in the research project or not," she said.

Association of Independent Schools of SA chief executive Garry Le Duff said there were very diverse views about the SACE among private schools but some leaders were critical of the research project.

"Our submission to the review wanted students who want to study five subjects be given recognition so the government's response has gone some way to meeting that request," he said.

"(It) is a step in the right direction and we will continue to monitor the SACE and its development."

St Peter's Girls principal Fiona Godfrey said she applauded the consideration given to all the major issues but was concerned the changes did not go far enough.

"In terms of the research project I still firmly believe it shouldn't be a compulsory element of Stage 2 (Year 12) and should be a Stage 1 (Year 11) subject or an elective in Stage 2," she said.

"Changes to university entrance doesn't force students to study five subjects and most will just continue to study four subjects and the compulsory research project.

"The real issues not addressed at all are the falling number of students studying languages other than English and humanities."

SA Secondary Principals Association president Jan Paterson said she welcomed the decision because it would enhance schools' capacity to deliver a high school certificate that was valuable to all students.

Opposition Leader Isobel Redmond said the new SACE was a "failed experiment".

"I understand some people were doing planning their wedding or building a dog kennel and all sorts of things (for the Research Project). Those things should be extra curricular, not part of a serious qualification," she said.

"Having only four subjects was not good. I believe that we need to have a bigger focus on English as a language being taught. I don't believe that we do that well enough." 

Add your comment to this story

To join the conversation, please log in. Don't have an account? Register

Join the conversation, you are commenting as Logout

• STUDY PROGRAMS
• OVERSEAS STUDY CONSULTING
• SCHOLARSHIPS

Australian courses

Research project.

I. INTRODUCTION

Stage 2 Research Project is a compulsory 10-credit subject. Students must achieve a C–grade or better to complete the subject successfully and gain their SACE.

Students enrol in either Research Project A or Research Project B.

The external assessment for Research Project B must be written. Students can choose to present their external assessment for Research Project A in written, oral, or multimodal form.

Research Project B may contribute to a student’s Australian Tertiary Admission Rank (ATAR).

Students choose a research question that is based on an area of interest to them. They explore and develop one or more capabilities in the context of their research.

The term ‘research’ is used broadly and may include practical or technical investigations, formal research, or exploratory inquiries.

The Research Project provides a valuable opportunity for SACE students to develop and demonstrate skills essential for learning and living in a changing world. It enables students to develop vital skills of planning, research, synthesis, evaluation, and project management.

The Research Project enables students to explore an area of interest in depth, while developing skills to prepare them for further education, training, and work. Students develop their ability to question sources of information, make effective decisions, evaluate their own progress, be innovative, and solve problems.

II. LEARNING SCOPE AND REQUIREMENTS 

1. Learning Requirements

The learning requirements summarise the knowledge, skills, and understanding that students are expected to develop and demonstrate through their learning in Stage 2 Research Project B.

In this subject, students are expected to:

• generate ideas to plan and develop a research project
• understand and develop one or more capabilities in the context of their research
• analyse information and explore ideas to develop their research
• develop specific knowledge and skills
• produce and substantiate a Research Outcome
• evaluate their research.

Stage 2 Research Project B is a 10-credit subject.

The content of Research Project B consists of:

• developing the capabilities
• applying the research framework.

In Research Project B students choose a research question that is based on an area of interest. They identify one or more capabilities that are relevant to their research.

Students use the research framework as a guide to developing their research and applying knowledge, skills, and ideas specific to their research question. They choose one or more capabilities, explore the concept of the capability or capabilities, and how it or they can be developed in the context of their research.

Students synthesise their key findings to produce a Research Outcome, which is substantiated by evidence and examples from the research. They evaluate the research processes used, and the quality of their Research Outcome.

2.1. Developing the Capabilities

The purpose of the capabilities is to develop in students the knowledge, skills, and understanding to be successful learners, confident and creative individuals, and active and informed citizens.

The capabilities that have been identified are:

• information and communication technology capability
• critical and creative thinking
• personal and social capability
• ethical understanding
• intercultural understanding.

The capabilities enable students to make connections in their learning within and across subjects in a wide range of contexts.

2.1.1. Literacy

In Research Project B, students develop their capability for literacy by, for example:

• communicating with a range of people in a variety of contexts
• asking questions, expressing opinions, and taking different perspectives into account
• using language with increasing awareness, clarity, accuracy, and suitability for a range of audiences, contexts, and purposes
• accessing, analysing, and selecting appropriate primary and secondary sources
• engaging with, and reflecting on, the ways in which texts are created for specific purposes and audiences
• composing a range of texts — written, oral, visual, and multimodal
• reading, viewing, writing, listening, and speaking, using a range of technologies
• developing an understanding that different text types (e.g. website, speech, newspaper article, film, painting, data set, report, set of instructions, or interview) have their own distinctive stylistic features
• acquiring an understanding of the relationships between literacy, language, and culture.

2.1.2. Numeracy

In Research Project B, students develop their capability for numeracy by, for example:

• using appropriate language and representations (e.g. symbols, tables, and graphs) to communicate ideas to a range of audiences
• analysing information displayed in a variety of representations and translating information from one representation to another
• justifying the validity of the findings, using everyday language, when appropriate
• applying skills in estimating and calculating, using thinking, written, and digital strategies to solve and model everyday problems
• interpreting information given in numerical form in diagrams, maps, graphs, and tables
• visualising, identifying, and sorting shapes and objects in the environment
• interpreting patterns and relationships when solving problems
• recognising spatial and geographical features and relationships
• recognising and incorporating statistical information that requires an understanding of the diverse ways in which data are gathered, recorded, and presented.

2.1.3. Information and Communication Technology Capability

In Research Project B, students develop their capability for information and communication technology by, for example:

• understanding how contemporary information and communication technologies affect communication
• critically analysing the limitations and impacts of current technologies
• considering the implications of potential technologies
• communicating and sharing ideas and information, to collaboratively construct knowledge and digital solutions
• defining and planning information searches of a range of primary and secondary sources when investigating research questions
• developing an understanding of hardware and software components, and operations of appropriate systems, including their functions, processes, and devices
• applying knowledge and skills of information and communication technology to a range of methods to collect and process data, and transmit and produce information
• learning to manage and manipulate electronic sources of data, databases, and software applications
• applying technologies to design and manage projects.

2.1.4. Critical and Creative Thinking

In Research Project B, students develop their capability for critical and creative thinking by, for example:

• thinking critically, logically, ethically, and reflectively
• learning and applying new knowledge and skills
• accessing, organising, using, and evaluating information
• posing questions, and identifying and clarifying information and ideas
• developing knowledge and understanding of a range of research processes
• understanding the nature of innovation
• recognising how knowledge changes over time and is influenced by people
• exploring and experiencing creative processes and practices
• designing features that are fit for function (e.g. physical, virtual, or textual)
• investigating the place of creativity in learning, the workplace, and community life
• examining the nature of entrepreneurial enterprise
• reflecting on, adjusting and explaining their thinking, and identifying the reasons for choices, strategies, and actions taken.

2.1.5. Personal and Social Capability

In Research Project B, students develop their personal and social capability by, for example:

• developing a sense of personal identity
• reviewing and planning personal goals
• developing an understanding of, and exercising, individual and shared obligations and rights
• participating actively and responsibly in learning, work, and community life
• establishing and managing relationships in personal and community life, work, and learning
• developing empathy for and understanding of others
• making responsible decisions based on evidence
• working effectively in teams and handling challenging situations constructively
• building links with others, locally, nationally, and/or globally.

2.1.6. Ethical Understanding

In Research Project B, students develop their capability for ethical understanding by, for example:

• identifying and discussing ethical concepts and issues
• considering ethical and safe research processes, including respecting the rights and work of others, acknowledging sources, and observing protocols when approaching people and organisations
• appreciating the ethical and legal dimensions of research and information
• reflecting on ethics and honesty in personal experiences and decision-making
• exploring ideas, rights, obligations, and ethical principles
• considering workplace safety principles, practices, and procedures
• developing ethical sustainable practices in the workplace and the community
• inquiring into ethical issues, selecting and justifying an ethical position, and understanding the experiences, motivations, and viewpoints of others
• debating ethical dilemmas and applying ethical principles in a range of situations.

2.1.7. Intercultural Understanding

In Research Project B, students develop their capability for intercultural understanding by, for example:

• identifying, observing, analysing, and describing characteristics of their own cultural identities and those of others (e.g. group memberships, traditions, values, religious beliefs, and ways of thinking)
• recognising that culture is dynamic and complex and that there is variability within all cultural, linguistic, and religious groups
• learning about and engaging with diverse cultures in ways that recognise commonalities and differences, create connections with others, and cultivate mutual respect
• developing skills to relate to and move between cultures
• acknowledging the social, cultural, linguistic and religious diversity of a nation, including that of Aboriginal and Torres Strait Islander societies in Australia
• recognising the challenges of living in a culturally diverse society and of negotiating, interpreting, and mediating difference.

2.2. Applying the Research Framework

The four parts of the research framework for Research Project B are:

• initiating and planning the research
• developing the research
• producing and substantiating the Research Outcome
• evaluating the research.

The four parts of the research framework are explained below.

2.2.1. Students Initiate and Plan their Research

Students plan their research by making decisions, seeking help, responding to and creating opportunities, and solving problems.

Students Formulate and Refine a Research Question

Formulating and refining the question help students to focus their research.

A research question:

• could be based on an idea or issue, a technical or practical challenge, a hypothesis, creating an artefact, or solving a problem
• may be an area of interest that is not related to a subject or course
• may be linked to content in an existing subject or course. Work that has been previously assessed for another subject or course cannot be used in this subject. However, information gained or ideas expressed in one assessment task can be extended in another assessment task.

Students refine their question, ensuring that the question lends itself to being researched and that the research is likely to be manageable and achievable. Refining a question may involve identifying a precise context, for example, place, type, age group, or time period.

Students and teachers must ensure that the research question and processes proposed do not compromise the principles of honest, safe, and ethical research.

Students Plan their Research

• consider, select, and/or design research processes (e.g. qualitative and quantitative research, practical experimentation, fieldwork) that are appropriate to their research question
• investigate and propose safe and ethical research processes
• identify knowledge, skills, and ideas that are specific to their research question
• identify people with whom to work (e.g. their teacher, a community expert, or a peer group) and negotiate processes for working together
• plan the research in manageable parts
• explore ideas in an area of interest
• explore the concept of a capability or capabilities in the context of their research
• consider the form of and audience for the Research Outcome.

2.2.2. Students Develop their Research

• develop a capability or capabilities in ways that are relevant to their research question
• develop and apply specific knowledge and skills
• develop and explore ideas
• locate, select, organise, analyse, use, and acknowledge information from different sources
• consult teachers and others with expertise in their area of interest
• participate in discussions with the teacher about the progress of their research
• apply safe and ethical research processes
• review and adjust the direction of their research in response to feedback, opportunities, questions, and problems as they arise
• maintain a record of progress made and sources used.

2.2.3. Students Produce and Substantiate their Research Outcome

Students synthesise their key findings (knowledge, skills, and ideas) to produce a Research Outcome.

The Research Outcome is substantiated by evidence and examples from the research, and shows how the student resolved the research question.

Substantiation should be relevant to the Research Outcome, and is usually provided in one or both of the following ways:

• By referencing the key findings from the research to sources, using, for example, in‑text references and thereby demonstrating the origin of ideas and thoughts.
• By explaining the validity of the methodology adopted and thereby demonstrating that it is able to be reproduced.

The Research Outcome must include the key findings and substantiation. The Research Outcome can take the form of:

• the key findings and substantiation, which together form a product

Examples include: an essay, a report, an oral or written history with appropriate in-text referencing and a bibliography and/or references list; a multimedia presentation; a documented science experiment

• the key findings and substantiation, with elements of or reference to a separate product

Examples include: a supporting statement and annotated photographs of a product that has been created; an extract from a student-developed children’s story, with a record of the background research

• the key findings presented as annotations on a product, and substantiated by evidence and examples of the research

Examples include: a recorded dance performance with notes and a director’s statement.

Students negotiate with their teacher suitable forms for producing their Research Outcome.

2.2.4 Students Evaluate their Research

• explain the choice of research processes used (e.g. qualitative and quantitative research, practical experimentation, fieldwork) and evaluate the usefulness of the research processes specific to the research question
• evaluate decisions made in response to challenges and/or opportunities (e.g. major activities, insights, turning points, and problems encountered)
• evaluate the quality of the Research Outcome
• organise their information coherently and communicate ideas accurately and appropriately
• communicate in written form.

III. ASSESSMENT SCOPE AND REQUIREMENTS

All Stage 2 subjects have a school assessment component and an external assessment component.

1. Evidence of Learning

The following assessment types enable students to demonstrate their learning in Stage 2 Research Project B:

School Assessment (70%)

Assessment Type 1: Folio (30%)

Assessment Type 2: Research Outcome (40%)

External Assessment (30%)

• Assessment Type 3: Evaluation (30%).

2. Assessment Design Criteria

The assessment design criteria are based on the learning requirements and are used by:

• teachers to clarify for the student what he or she needs to learn
• teachers and assessors to design opportunities for the student to provide evidence of his or her learning at the highest possible level of achievement.

The assessment design criteria consist of specific features that:

• students should demonstrate in their learning
• teachers and assessors look for as evidence that students have met the learning requirements.

For this subject the assessment design criteria are:

• development

The specific features of these criteria are described below.

The set of assessments, as a whole, must give students opportunities to demonstrate each of the specific features by the completion of study of the subject.

2.1. Planning

The specific features are as follows:

• P1: Consideration and refinement of a research question.
• P2: Planning of research processes appropriate to the research question.

2.2. Development

• D1: Development of the research.
• D2: Analysis of information and exploration of ideas to develop the research.
• D3: Development of knowledge and skills specific to the research question.
• D4: Understanding and development of one or more capabilities.

2.3. Synthesis

• S1: Synthesis of knowledge, skills, and ideas to produce a resolution to the research question.
• S2: Substantiation of key findings relevant to the Research Outcome.
• S3: Expression of ideas.

2.4. Evaluation

• E1: Evaluation of the research processes used, specific to the research question.
• E2: Evaluation of decisions made in response to challenges and/or opportunities specific to the research processes used.
• E3: Evaluation of the quality of the Research Outcome.

3. School Assessment

The Folio is a record of the student’s research. Students develop a research question and then select and present evidence of their learning from the planning and development stages of the research project. The Folio includes a proposal (evidence of planning), and evidence of the research development, which may take a variety of forms, including a discussion.

• consider and define a research question, and outline their initial ideas for the research
• consider and select research processes that are likely to be appropriate to their research question (i.e. valid, ethical, and manageable research processes).

Evidence could include:

• guiding questions
• a written statement
• an oral discussion
• a multimedia presentation,

that may lead to the development of, and its incorporation in, a management plan.

Research Development

• develop the research, including knowledge and skills specific to the research question
• organise and analyse information gathered
• explore ideas
• understand and develop one or more capabilities.
• information collected, selected, annotated, and analysed, and ideas explored in relation to the research question

Examples include: notes, drafts, letters, sketches, plans, models, interview notes, observations, trials, reflections, data from experiments, records of visits or fieldwork, photographs, annotations, feedback, translations, and interpretations

• responses to feedback, interactions, questions, and problem-solving

Examples include: major activities, insights, turning points, and problems encountered

• recordings of discussions with the teacher (either digital or in the form of notes taken by the student) about how the research is developing, the research processes used, ideas that are developing through the research, and the knowledge and skills being developed and applied.

For this assessment type, students provide evidence of their learning in relation to all specific features of the following assessment design criteria:

Refer to the subject operational information on the Research Project minisite on the SACE website ( www.sace.sa.edu.au ) for details about materials to be submitted for moderation.

The Research Outcome is the resolution of the research question, through the presentation of the key findings from the research.

Students identify the intended audience for their Research Outcome, and consider the value of their research to this audience. The form and language of the Research Outcome should be appropriate to the intended audience.

In resolving the research question, students come to a position or conclusion as a response to their research question.

Students synthesise their key findings (knowledge, skills, and ideas) to produce a Research Outcome and substantiate these with evidence and examples from their research to show how they resolved the research question.

The Research Outcome must include the key findings and substantiation. The Research Outcome can take the form of:

Examples include: an essay, a report, an oral or written history, with appropriate in-text referencing and a bibliography and/or references list; a multimedia presentation; a documented science experiment

Students negotiate with their teacher suitable forms for producing their Research Outcome, for example:

• written results, conclusions, recommendations, or solutions to a problem or question (e.g. an essay, a report, a booklet, or an article)
• a product (e.g. an artefact, a manufactured article, or a work of art or literature) and a producer’s statement
• a display or exhibition with annotations
• a multimedia presentation and podcast
• a performance (live or recorded) with a supporting statement
• a combination of any of the above.

Students submit their Research Outcome to the teacher and, if they choose, present it to a broader audience (e.g. other students or community members).

Evidence of the Research Outcome must be:

• a maximum of 2000 words if written
• a maximum of 12 minutes for an oral presentation
• the equivalent in multimodal form.

For this assessment type, students provide evidence of their learning in relation to all specific features of the following assessment design criterion:

4. External Assessment

Assessment Type 3: Evaluation (30%)

The Evaluation is a series of judgments about the research processes used and the Research Outcome produced.

For this assessment type, students:

• evaluate the usefulness of the research processes used specific to the research question.

Students make judgments about the effectiveness of processes they used to collect information as part of their research (e.g. qualitative and quantitative research, practical experimentation, fieldwork). They make reference to specific sources of information to provide examples of the usefulness of the research processes.

• evaluate decisions made in response to challenges and/or opportunities specific to the research processes used.

Students make judgments about their actions when faced with challenges and/or opportunities while using research processes. They draw conclusions about the effect of these actions on the research.

• evaluate the quality of the Research Outcome.

Students make balanced judgments about the quality of their Research Outcome with a focus on the significance of their findings, and the particular features that influence the overall value and worth of their Research Outcome, including the extent to which the question has been resolved.

• organise their information coherently and communicate ideas accurately and appropriately.

Students prepare a written summary of the research question and the Research Outcome, to a maximum of 150 words. This summary is assessed.

Students must present their Evaluation in written form to a maximum of 1500 words (excluding the written summary).

The Evaluation can include visual material (e.g. photographs and diagrams), integrated into the written text.

The following specific features of the assessment design criteria for this subject are assessed in the external assessment component:

• evaluation — E1, E2, and E3
• synthesis — S3.

5. Performance Standards

The performance standards describe five levels of achievement, A to E.

Each level of achievement describes the knowledge, skills, and understanding that teachers and assessors refer to in deciding how well a student has demonstrated his or her learning on the basis of the evidence provided.

During the teaching and learning program the teacher gives students feedback on their learning, with reference to the performance standards.

At the student’s completion of study of each school assessment type, the teacher makes a decision about the quality of the student’s learning by:

• referring to the performance standards
• assigning a grade between A+ and E- for the assessment type.

The student’s school assessment and external assessment are combined for a final result, which is reported as a grade between A+ and E-.

6. Assessment Integrity

The SACE Assuring Assessment Integrity Policy outlines the principles and processes that teachers and assessors follow to assure the integrity of student assessments. This policy is available on the SACE website (www.sace.sa.edu.au) as part of the SACE Policy Framework.

The SACE Board uses a range of quality assurance processes so that the grades awarded for student achievement, in both the school assessment and the external assessment, are applied consistently and fairly against the performance standards for a subject, and are comparable across all schools.

Information and guidelines on quality assurance in assessment at Stage 2 are available on the SACE website (www.sace.sa.edu.au).

IV. SUPPORT MATERIALS 

1. Subject-specific Advice

Online support materials are provided for each subject and updated regularly on the SACE website ( www.sace.sa.edu.au ). Examples of support materials are sample learning and assessment plans, annotated assessment tasks, annotated student responses, and recommended resource materials.

2. Advice on Ethical Study and Research

Advice for students and teachers on ethical study and research practices is available in the guidelines on the ethical conduct of research in the SACE on the SACE website ( www.sace.sa.edu.au ).

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Environment | Climate-change research project aboard USS…

Share this:.

• Click to share on Facebook (Opens in new window)
• Click to share on Twitter (Opens in new window)
• Click to print (Opens in new window)

Today's E-Edition

• Commercial Real Estate
• Marketplace
• The Mercury News

Environment

Environment | climate-change research project aboard uss hornet paused for environmental review, after city raised concerns, scientists agreed to delay but reasserted safety of spraying salt water into air.

The city asked the Hornet’s administrators and the University of Washington to stop the experiment, stating it was in violation of the Hornet’s lease with the city and was taking place without the city’s knowledge, officials announced in a Facebook post May 4. The experiment is not allowed under the ship’s museum operations outlined in its lease, Jennifer Ott, Alameda’s city manager, wrote in a letter to the Hornet which was shared with Bay Area News Group by the city.

The city has contracted biological and hazardous material consultants to independently investigate the environmental safety and health of the experiment, officials said in the post, adding that “there is no indication that the spray from the previous experiments presented a threat to human health or the environment.”

The program stopped its experiments prior to Alameda’s public announcement, according to a statement released by Dr. Rob Wood, principal investigator and Dr. Sarah Doherty, program director. The scientists added that the city was informed of the study’s corresponding educational exhibit in advance but asked for a closer review of the study after news articles released details in April.

“This type of review was not unexpected given that the approach in undertaking the studies and engaging with the public on the USS Hornet … is something new,” Wood and Doherty wrote. “We are happy to support their review and it has been a highly constructive process so far.”

The Marine Cloud Brightening Project aims to test whether ejecting plumes of microscopic droplets of salt water into the clouds will make them more reflective, helping to counteract warming climates by sending heat back up into the sky instead of allowing it down to the ground. Based out of the University of Washington, the program partnered with the U.S.S. Hornet, a World War II-era aircraft carrier-turned museum which is perpetually docked on the coast of Alameda, to conduct experiments on its top deck.

The team of scientists and engineers developed the spray technology and nozzle designs over the course of several years in the lab and launched the next phase of the study — testing whether the theory works in actual atmospheric conditions — in April. Scientists had planned to test the technology over the course of several months and measure its effectiveness with computer models.

Before beginning tests on the Hornet, the program went through an expert assessment of requirements and “found that the study does not exceed established regulatory or permitting thresholds,” Wood and Doherty wrote. The plumes of salt water “operate well below established thresholds for environmental or human health impact for emissions.”

A comment on the city’s Facebook post from the USS Hornet’s account read in part: “We believed that our existing permits and lease covered these activities when we started. As we now know, there was a gap in communication and understanding of the scope of the project and we are committed to working with the City to meet all of their needs regarding this effort.”

The findings will be presented to the Alameda City Council in June and will be shared with the public, according to the post.

“We continue to appreciate our engagement with the community on the nature of this type of research study, which is not designed to impact clouds, the environment or climate,” Wood and Doherty wrote.

More in Environment

Environment | Can’t install your own solar panels? Some areas let you join a community project

Environment | Tire toxicity faces fresh scrutiny after salmon die-offs

Environment | Exxon and Chevron output booms in world’s hottest oil patches

Environment | Share water bottles? Use refill shampoo pouches? New concepts in cutting plastic pollution

• Open access
• Published: 13 May 2024

What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review

• Olivia R. Phillips 1 , 2   na1 ,
• Cerian Harries 2 , 3   na1 ,
• Jo Leonardi-Bee 1 , 2 , 4   na1 ,
• Holly Knight 1 , 2 ,
• Lauren B. Sherar 2 , 3 ,
• Veronica Varela-Mato 2 , 3 &
• Joanne R. Morling 1 , 2 , 5  

Research Involvement and Engagement volume  10 , Article number:  48 ( 2024 ) Cite this article

1 Altmetric

Metrics details

There is increasing interest in using patient and public involvement (PPI) in research to improve the quality of healthcare. Ordinarily, traditional methods have been used such as interviews or focus groups. However, these methods tend to engage a similar demographic of people. Thus, creative methods are being developed to involve patients for whom traditional methods are inaccessible or non-engaging.

To determine the strengths and limitations to using creative PPI methods in health and social care research.

Electronic searches were conducted over five databases on 14th April 2023 (Web of Science, PubMed, ASSIA, CINAHL, Cochrane Library). Studies that involved traditional, non-creative PPI methods were excluded. Creative PPI methods were used to engage with people as research advisors, rather than study participants. Only primary data published in English from 2009 were accepted. Title, abstract and full text screening was undertaken by two independent reviewers before inductive thematic analysis was used to generate themes.

Twelve papers met the inclusion criteria. The creative methods used included songs, poems, drawings, photograph elicitation, drama performance, visualisations, social media, photography, prototype development, cultural animation, card sorting and persona development. Analysis identified four limitations and five strengths to the creative approaches. Limitations included the time and resource intensive nature of creative PPI, the lack of generalisation to wider populations and ethical issues. External factors, such as the lack of infrastructure to support creative PPI, also affected their implementation. Strengths included the disruption of power hierarchies and the creation of a safe space for people to express mundane or “taboo” topics. Creative methods are also engaging, inclusive of people who struggle to participate in traditional PPI and can also be cost and time efficient.

‘Creative PPI’ is an umbrella term encapsulating many different methods of engagement and there are strengths and limitations to each. The choice of which should be determined by the aims and requirements of the research, as well as the characteristics of the PPI group and practical limitations. Creative PPI can be advantageous over more traditional methods, however a hybrid approach could be considered to reap the benefits of both. Creative PPI methods are not widely used; however, this could change over time as PPI becomes embedded even more into research.

Plain English Summary

It is important that patients and public are included in the research process from initial brainstorming, through design to delivery. This is known as public and patient involvement (PPI). Their input means that research closely aligns with their wants and needs. Traditionally to get this input, interviews and group discussions are held, but this can exclude people who find these activities non-engaging or inaccessible, for example those with language challenges, learning disabilities or memory issues. Creative methods of PPI can overcome this. This is a broad term describing different (non-traditional) ways of engaging patients and public in research, such as through the use or art, animation or performance. This review investigated the reasons why creative approaches to PPI could be difficult (limitations) or helpful (strengths) in health and social care research. After searching 5 online databases, 12 studies were included in the review. PPI groups included adults, children and people with language and memory impairments. Creative methods included songs, poems, drawings, the use of photos and drama, visualisations, Facebook, creating prototypes, personas and card sorting. Limitations included the time, cost and effort associated with creative methods, the lack of application to other populations, ethical issues and buy-in from the wider research community. Strengths included the feeling of equality between academics and the public, creation of a safe space for people to express themselves, inclusivity, and that creative PPI can be cost and time efficient. Overall, this review suggests that creative PPI is worthwhile, however each method has its own strengths and limitations and the choice of which will depend on the research project, PPI group characteristics and other practical limitations, such as time and financial constraints.

Peer Review reports

Introduction

Patient and public involvement (PPI) is the term used to describe the partnership between patients (including caregivers, potential patients, healthcare users etc.) or the public (a community member with no known interest in the topic) with researchers. It describes research that is done “‘with’ or ‘by’ the public, rather than ‘to,’ ‘about’ or ‘for’ them” [ 1 ]. In 2009, it became a legislative requirement for certain health and social care organisations to include patients, families, carers and communities in not only the planning of health and social care services, but the commissioning, delivery and evaluation of them too [ 2 ]. For example, funding applications for the National Institute of Health and Care Research (NIHR), a UK funding body, mandates a demonstration of how researchers plan to include patients/service users, the public and carers at each stage of the project [ 3 ]. However, this should not simply be a tokenistic, tick-box exercise. PPI should help formulate initial ideas and should be an instrumental, continuous part of the research process. Input from PPI can provide unique insights not yet considered and can ensure that research and health services are closely aligned to the needs and requirements of service users PPI also generally makes research more relevant with clearer outcomes and impacts [ 4 ]. Although this review refers to both patients and the public using the umbrella term ‘PPI’, it is important to acknowledge that these are two different groups with different motivations, needs and interests when it comes to health research and service delivery [ 5 ].

Despite continuing recognition of the need of PPI to improve quality of healthcare, researchers have also recognised that there is no ‘one size fits all’ method for involving patients [ 4 ]. Traditionally, PPI methods invite people to take part in interviews or focus groups to facilitate discussion, or surveys and questionnaires. However, these can sometimes be inaccessible or non-engaging for certain populations. For example, someone with communication difficulties may find it difficult to engage in focus groups or interviews. If individuals lack the appropriate skills to interact in these types of scenarios, they cannot take advantage of the participation opportunities it can provide [ 6 ]. Creative methods, however, aim to resolve these issues. These are a relatively new concept whereby researchers use creative methods (e.g., artwork, animations, Lego), to make PPI more accessible and engaging for those whose voices would otherwise go unheard. They ensure that all populations can engage in research, regardless of their background or skills. Seminal work has previously been conducted in this area, which brought to light the use of creative methodologies in research. Leavy (2008) [ 7 ] discussed how traditional interviews had limits on what could be expressed due to their sterile, jargon-filled and formulaic structure, read by only a few specialised academics. It was this that called for more creative approaches, which included narrative enquiry, fiction-based research, poetry, music, dance, art, theatre, film and visual art. These practices, which can be used in any stage of the research cycle, supported greater empathy, self-reflection and longer-lasting learning experiences compared to interviews [ 7 ]. They also pushed traditional academic boundaries, which made the research accessible not only to researchers, but the public too. Leavy explains that there are similarities between arts-based approaches and scientific approaches: both attempts to investigate what it means to be human through exploration, and used together, these complimentary approaches can progress our understanding of the human experience [ 7 ]. Further, it is important to acknowledge the parallels and nuances between creative and inclusive methods of PPI. Although creative methods aim to be inclusive (this should underlie any PPI activity, whether creative or not), they do not incorporate all types of accessible, inclusive methodologies e.g., using sign language for people with hearing impairments or audio recordings for people who cannot read. Given that there was not enough scope to include an evaluation of all possible inclusive methodologies, this review will focus on creative methods of PPI only.

We aimed to conduct a qualitative systematic review to highlight the strengths of creative PPI in health and social care research, as well as the limitations, which might act as a barrier to their implementation. A qualitative systematic review “brings together research on a topic, systematically searching for research evidence from primary qualitative studies and drawing the findings together” [ 8 ]. This review can then advise researchers of the best practices when designing PPI.

Public involvement

The PHIRST-LIGHT Public Advisory Group (PAG) consists of a team of experienced public contributors with a diverse range of characteristics from across the UK. The PAG was involved in the initial question setting and study design for this review.

Search strategy

For the purpose of this review, the JBI approach for conducting qualitative systematic reviews was followed [ 9 ]. The search terms were (“creativ*” OR “innovat*” OR “authentic” OR “original” OR “inclu*”) AND (“public and patient involvement” OR “patient and public involvement” OR “public and patient involvement and engagement” OR “patient and public involvement and engagement” OR “PPI” OR “PPIE” OR “co-produc*” OR “co-creat*” OR “co-design*” OR “cooperat*” OR “co-operat*”). This search string was modified according to the requirements of each database. Papers were filtered by title, abstract and keywords (see Additional file 1 for search strings). The databases searched included Web of Science (WoS), PubMed, ASSIA and CINAHL. The Cochrane Library was also searched to identify relevant reviews which could lead to the identification of primary research. The search was conducted on 14/04/23. As our aim was to report on the use of creative PPI in research, rather than more generic public engagement, we used electronic databases of scholarly peer-reviewed literature, which represent a wide range of recognised databases. These identified studies published in general international journals (WoS, PubMed), those in social sciences journals (ASSIA), those in nursing and allied health journals (CINAHL), and trials of interventions (Cochrane Library).

Inclusion criteria

Only full-text, English language, primary research papers from 2009 to 2023 were included. This was the chosen timeframe as in 2009 the Health and Social Reform Act made it mandatory for certain Health and Social Care organisations to involve the public and patients in planning, delivering, and evaluating services [ 2 ]. Only creative methods of PPI were accepted, rather than traditional methods, such as interviews or focus groups. For the purposes of this paper, creative PPI included creative art or arts-based approaches (e.g., e.g. stories, songs, drama, drawing, painting, poetry, photography) to enhance engagement. Titles were related to health and social care and the creative PPI was used to engage with people as research advisors, not as study participants. Meta-analyses, conference abstracts, book chapters, commentaries and reviews were excluded. There were no limits concerning study location or the demographic characteristics of the PPI groups. Only qualitative data were accepted.

Quality appraisal

Quality appraisal using the Critical Appraisal Skills Programme (CASP) checklist [ 10 ] was conducted by the primary authors (ORP and CH). This was done independently, and discrepancies were discussed and resolved. If a consensus could not be reached, a third independent reviewer was consulted (JRM). The full list of quality appraisal questions can be found in Additional file 2 .

Data extraction

ORP extracted the study characteristics and a subset of these were checked by CH. Discrepancies were discussed and amendments made. Extracted data included author, title, location, year of publication, year study was carried out, research question/aim, creative methods used, number of participants, mean age, gender, ethnicity of participants, setting, limitations and strengths of creative PPI and main findings.

Data analysis

The included studies were analysed using inductive thematic analysis [ 11 ], where themes were determined by the data. The familiarisation stage took place during full-text reading of the included articles. Anything identified as a strength or limitation to creative PPI methods was extracted verbatim as an initial code and inputted into the data extraction Excel sheet. Similar codes were sorted into broader themes, either under ‘strengths’ or ‘limitations’ and reviewed. Themes were then assigned a name according to the codes.

The search yielded 9978 titles across the 5 databases: Web of Science (1480 results), PubMed (94 results), ASSIA (2454 results), CINAHL (5948 results) and Cochrane Library (2 results), resulting in 8553 different studies after deduplication. ORP and CH independently screened their titles and abstracts, excluding those that did not meet the criteria. After assessment, 12 studies were included (see Fig.  1 ).

PRISMA flowchart of the study selection process

Study characteristics

The included studies were published between 2018 and 2022. Seven were conducted in the UK [ 12 , 14 , 15 , 17 , 18 , 19 , 23 ], two in Canada [ 21 , 22 ], one in Australia [ 13 ], one in Norway [ 16 ] and one in Ireland [ 20 ]. The PPI activities occurred across various settings, including a school [ 12 ], social club [ 12 ], hospital [ 17 ], university [ 22 ], theatre [ 19 ], hotel [ 20 ], or online [ 15 , 21 ], however this information was omitted in 5 studies [ 13 , 14 , 16 , 18 , 23 ]. The number of people attending the PPI sessions varied, ranging from 6 to 289, however the majority (ten studies) had less than 70 participants [ 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Seven studies did not provide information on the age or gender of the PPI groups. Of those that did, ages ranged from 8 to 76 and were mostly female. The ethnicities of the PPI group members were also rarely recorded (see Additional file 3 for data extraction table).

Types of creative methods

The type of creative methods used to engage the PPI groups were varied. These included songs, poems, drawings, photograph elicitation, drama performance, visualisations, Facebook, photography, prototype development, cultural animation, card sorting and creating personas (see Table  1 ). These were sometimes accompanied by traditional methods of PPI such as interviews and focus group discussions.

The 12 included studies were all deemed to be of good methodological quality, with scores ranging from 6/10 to 10/10 with the CASP critical appraisal tool [ 10 ] (Table  2 ).

Thematic analysis

Analysis identified four limitations and five strengths to creative PPI (see Fig.  2 ). Limitations included the time and resource intensity of creative PPI methods, its lack of generalisation, ethical issues and external factors. Strengths included the disruption of power hierarchies, the engaging and inclusive nature of the methods and their long-term cost and time efficiency. Creative PPI methods also allowed mundane and “taboo” topics to be discussed within a safe space.

Theme map of strengths and limitations

Limitations of creative PPI

Creative ppi methods are time and resource intensive.

The time and resource intensive nature of creative PPI methods is a limitation, most notably for the persona-scenario methodology. Valaitis et al. [ 22 ] used 14 persona-scenario workshops with 70 participants to co-design a healthcare intervention, which aimed to promote optimal aging in Canada. Using the persona method, pairs composed of patients, healthcare providers, community service providers and volunteers developed a fictional character which they believed represented an ‘end-user’ of the healthcare intervention. Due to the depth and richness of the data produced the authors reported that it was time consuming to analyse. Further, they commented that the amount of information was difficult to disseminate to scientific leads and present at team meetings. Additionally, to ensure the production of high-quality data, to probe for details and lead group discussion there was a need for highly skilled facilitators. The resource intensive nature of the creative co-production was also noted in a study using the persona scenario and creative worksheets to develop a prototype decision support tool for individuals with malignant pleural effusion [ 17 ]. With approximately 50 people, this was also likely to yield a high volume of data to consider.

To prepare materials for populations who cannot engage in traditional methods of PPI was also timely. Kearns et al. [ 18 ] developed a feedback questionnaire for people with aphasia to evaluate ICT-delivered rehabilitation. To ensure people could participate effectively, the resources used during the workshops, such as PowerPoints, online images and photographs, had to be aphasia-accessible, which was labour and time intensive. The author warned that this time commitment should not be underestimated.

There are further practical limitations to implementing creative PPI, such as the costs of materials for activities as well as hiring a space for workshops. For example, the included studies in this review utilised pens, paper, worksheets, laptops, arts and craft supplies and magazines and took place in venues such as universities, a social club, and a hotel. Further, although not limited to creative PPI methods exclusively but rather most studies involving the public, a financial incentive was often offered for participation, as well as food, parking, transport and accommodation [ 21 , 22 ].

Creative PPI lacks generalisation

Another barrier to the use of creative PPI methods in health and social care research was the individual nature of its output. Those who participate, usually small in number, produce unique creative outputs specific to their own experiences, opinions and location. Craven et al. [ 13 ], used arts-based visualisations to develop a toolbox for adults with mental health difficulties. They commented, “such an approach might still not be worthwhile”, as the visualisations were individualised and highly personal. This indicates that the output may fail to meet the needs of its end-users. Further, these creative PPI groups were based in certain geographical regions such as Stoke-on-Trent [ 19 ] Sheffield [ 23 ], South Wales [ 12 ] or Ireland [ 20 ], which limits the extent the findings can be applied to wider populations, even within the same area due to individual nuances. Further, the study by Galler et al. [ 16 ], is specific to the Norwegian context and even then, maybe only a sub-group of the Norwegian population as the sample used was of higher socioeconomic status.

However, Grindell et al. [ 17 ], who used persona scenarios, creative worksheets and prototype development, pointed out that the purpose of this type of research is to improve a certain place, rather than apply findings across other populations and locations. Individualised output may, therefore, only be a limitation to research wanting to conduct PPI on a large scale.

If, however, greater generalisation within PPI is deemed necessary, then social media may offer a resolution. Fedorowicz et al. [ 15 ], used Facebook to gain feedback from the public on the use of video-recording methodology for an upcoming project. This had the benefit of including a more diverse range of people (289 people joined the closed group), who were spread geographically around the UK, as well as seven people from overseas.

Creative PPI has ethical issues

As with other research, ethical issues must be taken into consideration. Due to the nature of creative approaches, as well as the personal effort put into them, people often want to be recognised for their work. However, this compromises principles so heavily instilled in research such as anonymity and confidentiality. With the aim of exploring issues related to health and well-being in a town in South Wales, Byrne et al. [ 12 ], asked year 4/5 and year 10 pupils to create poems, songs, drawings and photographs. Community members also created a performance, mainly of monologues, to explore how poverty and inequalities are dealt with. Byrne noted the risks of these arts-based approaches, that being the possibility of over-disclosure and consequent emotional distress, as well as people’s desire to be named for their work. On one hand, the anonymity reduces the sense of ownership of the output as it does not portray a particular individual’s lived experience anymore. On the other hand, however, it could promote a more honest account of lived experience. Supporting this, Webber et al. [ 23 ], who used the persona method to co-design a back pain educational resource prototype, claimed that the anonymity provided by this creative technique allowed individuals to externalise and anonymise their own personal experience, thus creating a more authentic and genuine resource for future users. This implies that anonymity can be both a limitation and strength here.

The use of creative PPI methods is impeded by external factors

Despite the above limitations influencing the implementation of creative PPI techniques, perhaps the most influential is that creative methodologies are simply not mainstream [ 19 ]. This could be linked to the issues above, like time and resource intensity, generalisation and ethical issues but it is also likely to involve more systemic factors within the research community. Micsinszki et al. [ 21 ], who co-designed a hub for the health and well-being of vulnerable populations, commented that there is insufficient infrastructure to conduct meaningful co-design as well as a dominant medical model. Through a more holistic lens, there are “sociopolitical environments that privilege individualism over collectivism, self-sufficiency over collaboration, and scientific expertise over other ways of knowing based on lived experience” [ 21 ]. This, it could be suggested, renders creative co-design methodologies, which are based on the foundations of collectivism, collaboration and imagination an invalid technique in the research field, which is heavily dominated by more scientific methods offering reproducibility, objectivity and reliability.

Although we acknowledge that creative PPI techniques are not always appropriate, it may be that their main limitation is the lack of awareness of these methods or lack of willingness to use them. Further, there is always the risk that PPI, despite being a mandatory part of research, is used in a tokenistic or tick-box fashion [ 20 ], without considering the contribution that meaningful PPI could make to enhancing the research. It may be that PPI, let alone creative PPI, is not at the forefront of researchers’ minds when planning research.

Strengths of creative PPI

Creative ppi disrupts power hierarchies.

One of the main strengths of creative PPI techniques, cited most frequently in the included literature, was that they disrupt traditional power hierarchies [ 12 , 13 , 17 , 19 , 23 ]. For example, the use of theatre performance blurred the lines between professional and lay roles between the community and policy makers [ 12 ]. Individuals created a monologue to portray how poverty and inequality impact daily life and presented this to representatives of the National Assembly of Wales, Welsh Government, the Local Authority, Arts Council and Westminster. Byrne et al. [ 12 ], states how this medium allowed the community to engage with the people who make decisions about their lives in an environment of respect and understanding, where the hierarchies are not as visible as in other settings, e.g., political surgeries. Creative PPI methods have also removed traditional power hierarchies between researchers and adolescents. Cook et al. [ 13 ], used arts-based approaches to explore adolescents’ ideas about the “perfect” condom. They utilised the “Life Happens” resource, where adolescents drew and then decorated a person with their thoughts about sexual relationships, not too dissimilar from the persona-scenario method. This was then combined with hypothetical scenarios about sexuality. A condom-mapping exercise was then implemented, where groups shared the characteristics that make a condom “perfect” on large pieces of paper. Cook et al. [ 13 ], noted that usually power imbalances make it difficult to elicit information from adolescents, however these power imbalances were reduced due to the use of creative co-design techniques.

The same reduction in power hierarchies was noted by Grindell et al. [ 17 ], who used the person-scenario method and creative worksheets with individuals with malignant pleural effusion. This was with the aim of developing a prototype of a decision support tool for patients to help with treatment options. Although this process involved a variety of stakeholders, such as patients, carers and healthcare professionals, creative co-design was cited as a mechanism that worked to reduce power imbalances – a limitation of more traditional methods of research. Creative co-design blurred boundaries between end-users and clinical staff and enabled the sharing of ideas from multiple, valuable perspectives, meaning the prototype was able to suit user needs whilst addressing clinical problems.

Similarly, a specific creative method named cultural animation was also cited to dissolve hierarchies and encourage equal contributions from participants. Within this arts-based approach, Keleman et al. [ 19 ], explored the concept of “good health” with individuals from Stoke-on Trent. Members of the group created art installations using ribbons, buttons, cardboard and straws to depict their idea of a “healthy community”, which was accompanied by a poem. They also created a 3D Facebook page and produced another poem or song addressing the government to communicate their version of a “picture of health”. Public participants said that they found the process empowering, honest, democratic, valuable and practical.

This dissolving of hierarchies and levelling of power is beneficial as it increases the sense of ownership experienced by the creators/producers of the output [ 12 , 17 , 23 ]. This is advantageous as it has been suggested to improve its quality [ 23 ].

Creative PPI allows the unsayable to be said

Creative PPI fosters a safe space for mundane or taboo topics to be shared, which may be difficult to communicate using traditional methods of PPI. For example, the hypothetical nature of condom mapping and persona-scenarios meant that adolescents could discuss a personal topic without fear of discrimination, judgement or personal disclosure [ 13 ]. The safe space allowed a greater volume of ideas to be generated amongst peers where they might not have otherwise. Similarly, Webber et al. [ 23 ], , who used the persona method to co-design the prototype back pain educational resource, also noted how this method creates anonymity whilst allowing people the opportunity to externalise personal experiences, thoughts and feelings. Other creative methods were also used, such as drawing, collaging, role play and creating mood boards. A cardboard cube (labelled a “magic box”) was used to symbolise a physical representation of their final prototype. These creative methods levelled the playing field and made personal experiences accessible in a safe, open environment that fostered trust, as well as understanding from the researchers.

It is not only sensitive subjects that were made easier to articulate through creative PPI. The communication of mundane everyday experiences were also facilitated, which were deemed typically ‘unsayable’. This was specifically given in the context of describing intangible aspects of everyday health and wellbeing [ 11 ]. Graphic designers can also be used to visually represent the outputs of creative PPI. These captured the movement and fluidity of people and well as the relationships between them - things that cannot be spoken but can be depicted [ 21 ].

Creative PPI methods are inclusive

Another strength of creative PPI was that it is inclusive and accessible [ 17 , 19 , 21 ]. The safe space it fosters, as well as the dismantling of hierarchies, welcomed people from a diverse range of backgrounds and provided equal opportunities [ 21 ], especially for those with communication and memory difficulties who might be otherwise excluded from PPI. Kelemen et al. [ 19 ], who used creative methods to explore health and well-being in Stoke-on-Trent, discussed how people from different backgrounds came together and connected, discussed and reached a consensus over a topic which evoked strong emotions, that they all have in common. Individuals said that the techniques used “sets people to open up as they are not overwhelmed by words”. Similarly, creative activities, such as the persona method, have been stated to allow people to express themselves in an inclusive environment using a common language. Kearns et al. [ 18 ], who used aphasia-accessible material to develop a questionnaire with aphasic individuals, described how they felt comfortable in contributing to workshops (although this material was time-consuming to make, see ‘Limitations of creative PPI’ ).

Despite the general inclusivity of creative PPI, it can also be exclusive, particularly if online mediums are used. Fedorowicz et al. [ 15 ], used Facebook to create a PPI group, and although this may rectify previous drawbacks about lack of generalisation of creative methods (as Facebook can reach a greater number of people, globally), it excluded those who are not digitally active or have limited internet access or knowledge of technology. Online methods have other issues too. Maintaining the online group was cited as challenging and the volume of responses required researchers to interact outside of their working hours. Despite this, online methods like Facebook are very accessible for people who are physically disabled.

Creative PPI methods are engaging

The process of creative PPI is typically more engaging and produces more colourful data than traditional methods [ 13 ]. Individuals are permitted and encouraged to explore a creative self [ 19 ], which can lead to the exploration of new ideas and an overall increased enjoyment of the process. This increased engagement is particularly beneficial for younger PPI groups. For example, to involve children in the development of health food products, Galler et al. [ 16 ] asked 9-12-year-olds to take photos of their food and present it to other children in a “show and tell” fashion. They then created a newspaper article describing a new healthy snack. In this creative focus group, children were given lab coats to further their identity as inventors. Galler et al. [ 16 ], notes that the methods were highly engaging and facilitated teamwork and group learning. This collaborative nature of problem-solving was also observed in adults who used personas and creative worksheets to develop the resource for lower back pain [ 23 ]. Dementia patients too have been reported to enjoy the creative and informal approach to idea generation [ 20 ].

The use of cultural animation allowed people to connect with each other in a way that traditional methods do not [ 19 , 21 ]. These connections were held in place by boundary objects, such as ribbons, buttons, fabric and picture frames, which symbolised a shared meaning between people and an exchange of knowledge and emotion. Asking groups to create an art installation using these objects further fostered teamwork and collaboration, both at an individual and collective level. The exploration of a creative self increased energy levels and encouraged productive discussions and problem-solving [ 19 ]. Objects also encouraged a solution-focused approach and permitted people to think beyond their usual everyday scope [ 17 ]. They also allowed facilitators to probe deeper about the greater meanings carried by the object, which acted as a metaphor [ 21 ].

From the researcher’s point of view, co-creative methods gave rise to ideas they might not have initially considered. Valaitis et al. [ 22 ], found that over 40% of the creative outputs were novel ideas brought to light by patients, healthcare providers/community care providers, community service providers and volunteers. One researcher commented, “It [the creative methods] took me on a journey, in a way that when we do other pieces of research it can feel disconnected” [ 23 ]. Another researcher also stated they could not return to the way they used to do research, as they have learnt so much about their own health and community and how they are perceived [ 19 ]. This demonstrates that creative processes not only benefit the project outcomes and the PPI group, but also facilitators and researchers. However, although engaging, creative methods have been criticised for not demonstrating academic rigour [ 17 ]. Moreover, creative PPI may also be exclusive to people who do not like or enjoy creative activities.

Creative PPI methods are cost and time efficient

Creative PPI workshops can often produce output that is visible and tangible. This can save time and money in the long run as the output is either ready to be implemented in a healthcare setting or a first iteration has already been developed. This may also offset the time and costs it takes to implement creative PPI. For example, the prototype of the decision support tool for people with malignant pleural effusion was developed using personas and creative worksheets. The end result was two tangible prototypes to drive the initial idea forward as something to be used in practice [ 17 ]. The use of creative co-design in this case saved clinician time as well as the time it would take to develop this product without the help of its end-users. In the development of this particular prototype, analysis was iterative and informed the next stage of development, which again saved time. The same applies for the feedback questionnaire for the assessment of ICT delivered aphasia rehabilitation. The co-created questionnaire, designed with people with aphasia, was ready to be used in practice [ 18 ]. This suggests that to overcome time and resource barriers to creative PPI, researchers should aim for it to be engaging whilst also producing output.

That useable products are generated during creative workshops signals to participating patients and public members that they have been listened to and their thoughts and opinions acted upon [ 23 ]. For example, the development of the back pain resource based on patient experiences implies that their suggestions were valid and valuable. Further, those who participated in the cultural animation workshop reported that the process visualises change, and that it already feels as though the process of change has started [ 19 ].

The most cost and time efficient method of creative PPI in this review is most likely the use of Facebook to gather feedback on project methodology [ 15 ]. Although there were drawbacks to this, researchers could involve more people from a range of geographical areas at little to no cost. Feedback was instantaneous and no training was required. From the perspective of the PPI group, they could interact however much or little they wish with no time commitment.

This systematic review identified four limitations and five strengths to the use of creative PPI in health and social care research. Creative PPI is time and resource intensive, can raise ethical issues and lacks generalisability. It is also not accepted by the mainstream. These factors may act as barriers to the implementation of creative PPI. However, creative PPI disrupts traditional power hierarchies and creates a safe space for taboo or mundane topics. It is also engaging, inclusive and can be time and cost efficient in the long term.

Something that became apparent during data analysis was that these are not blanket strengths and limitations of creative PPI as a whole. The umbrella term ‘creative PPI’ is broad and encapsulates a wide range of activities, ranging from music and poems to prototype development and persona-scenarios, to more simplistic things like the use of sticky notes and ordering cards. Many different activities can be deemed ‘creative’ and the strengths and limitations of one does not necessarily apply to another. For example, cultural animation takes greater effort to prepare than the use of sticky notes and sorting cards, and the use of Facebook is cheaper and wider reaching than persona development. Researchers should use their discretion and weigh up the benefits and drawbacks of each method to decide on a technique which suits the project. What might be a limitation to creative PPI in one project may not be in another. In some cases, creative PPI may not be suitable at all.

Furthermore, the choice of creative PPI method also depends on the needs and characteristics of the PPI group. Children, adults and people living with dementia or language difficulties all have different engagement needs and capabilities. This indicates that creative PPI is not one size fits all and that the most appropriate method will change depending on the composition of the group. The choice of method will also be determined by the constraints of the research project, namely time, money and the research aim. For example, if there are time constraints, then a method which yields a lot of data and requires a lot of preparation may not be appropriate. If generalisation is important, then an online method is more suitable. Together this indicates that the choice of creative PPI method is highly individualised and dependent on multiple factors.

Although the limitations discussed in this review apply to creative PPI, they are not exclusive to creative PPI. Ethical issues are a consideration within general PPI research, especially when working with more vulnerable populations, such as children or adults living with a disability. It can also be the case that traditional PPI methods lack generalisability, as people who volunteer to be part of such a group are more likely be older, middle class and retired [ 24 ]. Most research is vulnerable to this type of bias, however, it is worth noting that generalisation is not always a goal and research remains valid and meaningful in its absence. Although online methods may somewhat combat issues related to generalisability, these methods still exclude people who do not have access to the internet/technology or who choose not to use it, implying that online PPI methods may not be wholly representative of the general population. Saying this, however, the accessibility of creative PPI techniques differs from person to person, and for some, online mediums may be more accessible (for example for those with a physical disability), and for others, this might be face-to-face. To combat this, a range of methods should be implemented. Planning multiple focus group and interviews for traditional PPI is also time and resource intensive, however the extra resources required to make this creative may be even greater. Although, the rich data provided may be worth the preparation and analysis time, which is also likely to depend on the number of participants and workshop sessions required. PPI, not just creative PPI, often requires the provision of a financial incentive, refreshments, parking and accommodation, which increase costs. These, however, are imperative and non-negotiable, as they increase the accessibility of research, especially to minority and lower-income groups less likely to participate. Adequate funding is also important for co-design studies where repeated engagement is required. One barrier to implementation, which appears to be exclusive to creative methods, however, is that creative methods are not mainstream. This cannot be said for traditional PPI as this is often a mandatory part of research applications.

Regarding the strengths of creative PPI, it could be argued that most appear to be exclusive to creative methodologies. These are inclusive by nature as multiple approaches can be taken to evoke ideas from different populations - approaches that do not necessarily rely on verbal or written communication like interviews and focus groups do. Given the anonymity provided by some creative methods, such as personas, people may be more likely to discuss their personal experiences under the guise of a general end-user, which might be more difficult to maintain when an interviewer is asking an individual questions directly. Additionally, creative methods are by nature more engaging and interactive than traditional methods, although this is a blanket statement and there may be people who find the question-and-answer/group discussion format more engaging. Creative methods have also been cited to eliminate power imbalances which exist in traditional research [ 12 , 13 , 17 , 19 , 23 ]. These imbalances exist between researchers and policy makers and adolescents, adults and the community. Lastly, although this may occur to a greater extent in creative methods like prototype development, it could be suggested that PPI in general – regardless of whether it is creative - is more time and cost efficient in the long-term than not using any PPI to guide or refine the research process. It must be noted that these are observations based on the literature. To be certain these differences exist between creative and traditional methods of PPI, direct empirical evaluation of both should be conducted.

To the best of our knowledge, this is the first review to identify the strengths and limitations to creative PPI, however, similar literature has identified barriers and facilitators to PPI in general. In the context of clinical trials, recruitment difficulties were cited as a barrier, as well as finding public contributors who were free during work/school hours. Trial managers reported finding group dynamics difficult to manage and the academic environment also made some public contributors feel nervous and lacking confidence to speak. Facilitators, however, included the shared ownership of the research – something that has been identified in the current review too. In addition, planning and the provision of knowledge, information and communication were also identified as facilitators [ 25 ]. Other research on the barriers to meaningful PPI in trial oversight committees included trialist confusion or scepticism over the PPI role and the difficulties in finding PPI members who had a basic understanding of research [ 26 ]. However, it could be argued that this is not representative of the average patient or public member. The formality of oversight meetings and the technical language used also acted as a barrier, which may imply that the informal nature of creative methods and its lack of dependency on literacy skills could overcome this. Further, a review of 42 reviews on PPI in health and social care identified financial compensation, resources, training and general support as necessary to conduct PPI, much like in the current review where the resource intensiveness of creative PPI was identified as a limitation. However, others were identified too, such as recruitment and representativeness of public contributors [ 27 ]. Like in the current review, power imbalances were also noted, however this was included as both a barrier and facilitator. Collaboration seemed to diminish hierarchies but not always, as sometimes these imbalances remained between public contributors and healthcare staff, described as a ‘them and us’ culture [ 27 ]. Although these studies compliment the findings of the current review, a direct comparison cannot be made as they do not concern creative methods. However, it does suggest that some strengths and weaknesses are shared between creative and traditional methods of PPI.

Strengths and limitations of this review

Although a general definition of creative PPI exists, it was up to our discretion to decide exactly which activities were deemed as such for this review. For example, we included sorting cards, the use of interactive whiteboards and sticky notes. Other researchers may have a more or less stringent criteria. However, two reviewers were involved in this decision which aids the reliability of the included articles. Further, it may be that some of the strengths and limitations cannot fully be attributed to the creative nature of the PPI process, but rather their co-created nature, however this is hard to disentangle as the included papers involved both these aspects.

During screening, it was difficult to decide whether the article was utilising creative qualitative methodology or creative PPI , as it was often not explicitly labelled as such. Regardless, both approaches involved the public/patients refining a healthcare product/service. This implies that if this review were to be replicated, others may do it differently. This may call for greater standardisation in the reporting of the public’s involvement in research. For example, the NIHR outlines different approaches to PPI, namely “consultation”, “collaboration”, “co-production” and “user-controlled”, which each signify an increased level of public power and influence [ 28 ]. Papers with elements of PPI could use these labels to clarify the extent of public involvement, or even explicitly state that there was no PPI. Further, given our decision to include only scholarly peer-reviewed literature, it is possible that data were missed within the grey literature. Similarly, the literature search will not have identified all papers relating to different types of accessible inclusion. However, the intent of the review was to focus solely on those within the definition of creative.

This review fills a gap in the literature and helps circulate and promote the concept of creative PPI. Each stage of this review, namely screening and quality appraisal, was conducted by two independent reviewers. However, four full texts could not be accessed during the full text reading stage, meaning there are missing data that could have altered or contributed to the findings of this review.

Research recommendations

Given that creative PPI can require effort to prepare, perform and analyse, sufficient time and funding should be allocated in the research protocol to enable meaningful and continuous PPI. This is worthwhile as PPI can significantly change the research output so that it aligns closely with the needs of the group it is to benefit. Researchers should also consider prototype development as a creative PPI activity as this might reduce future time/resource constraints. Shifting from a top-down approach within research to a bottom-up can be advantageous to all stakeholders and can help move creative PPI towards the mainstream. This, however, is the collective responsibility of funding bodies, universities and researchers, as well as committees who approve research bids.

A few of the included studies used creative techniques alongside traditional methods, such as interviews, which could also be used as a hybrid method of PPI, perhaps by researchers who are unfamiliar with creative techniques or to those who wish to reap the benefits of both. Often the characteristics of the PPI group were not included, including age, gender and ethnicity. It would be useful to include such information to assess how representative the PPI group is of the population of interest.

Creative PPI is a relatively novel approach of engaging the public and patients in research and it has both advantages and disadvantages compared to more traditional methods. There are many approaches to implementing creative PPI and the choice of technique will be unique to each piece of research and is reliant on several factors. These include the age and ability of the PPI group as well as the resource limitations of the project. Each method has benefits and drawbacks, which should be considered at the protocol-writing stage. However, given adequate funding, time and planning, creative PPI is a worthwhile and engaging method of generating ideas with end-users of research – ideas which may not be otherwise generated using traditional methods.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Critical Appraisal Skills Programme

The Joanna Briggs Institute

National Institute of Health and Care Research

Public Advisory Group

Public and Patient Involvement

Web of Science

National Institute for Health and Care Research. What Is Patient and Public Involvement and Public Engagement? https://www.spcr.nihr.ac.uk/PPI/what-is-patient-and-public-involvement-and-engagement Accessed 01 Sept 2023.

Department of Health. Personal and Public Involvement (PPI) https://www.health-ni.gov.uk/topics/safety-and-quality-standards/personal-and-public-involvement-ppi#:~:text=The Health and Social Care Reform Act (NI) 2009 placed,delivery and evaluation of services . Accessed 01 Sept 2023.

National Institute for Health and Care Research. Policy Research Programme – Guidance for Stage 1 Applications https://www.nihr.ac.uk/documents/policy-research-programme-guidance-for-stage-1-applications-updated/26398 Accessed 01 Sept 2023.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019. https://doi.org/10.1111/hex.12888

Article   PubMed   PubMed Central   Google Scholar  

Street JM, Stafinski T, Lopes E, Menon D. Defining the role of the public in health technology assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care. 2020. https://doi.org/10.1017/S0266462320000094

Article   PubMed   Google Scholar  

Morrison C, Dearden A. Beyond tokenistic participation: using representational artefacts to enable meaningful public participation in health service design. Health Policy. 2013. https://doi.org/10.1016/j.healthpol.2013.05.008

Leavy P. Method meets art: arts-Based Research Practice. New York: Guilford; 2020.

Google Scholar  

Seers K. Qualitative systematic reviews: their importance for our understanding of research relevant to pain. Br J Pain. 2015. https://doi.org/10.1177/2049463714549777

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis JBI. 2020. https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03

CASP. CASP Checklists https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf (2022).

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006. https://doi.org/10.1191/1478088706qp063oa

Article   Google Scholar  

Byrne E, Elliott E, Saltus R, Angharad J. The creative turn in evidence for public health: community and arts-based methodologies. J Public Health. 2018. https://doi.org/10.1093/pubmed/fdx151

Cook S, Grozdanovski L, Renda G, Santoso D, Gorkin R, Senior K. Can you design the perfect condom? Engaging young people to inform safe sexual health practice and innovation. Sex Educ. 2022. https://doi.org/10.1080/14681811.2021.1891040

Craven MP, Goodwin R, Rawsthorne M, Butler D, Waddingham P, Brown S, Jamieson M. Try to see it my way: exploring the co-design of visual presentations of wellbeing through a workshop process. Perspect Public Health. 2019. https://doi.org/10.1177/1757913919835231

Fedorowicz S, Riley V, Cowap L, Ellis NJ, Chambers R, Grogan S, Crone D, Cottrell E, Clark-Carter D, Roberts L, Gidlow CJ. Using social media for patient and public involvement and engagement in health research: the process and impact of a closed Facebook group. Health Expect. 2022. https://doi.org/10.1111/hex.13515

Galler M, Myhrer K, Ares G, Varela P. Listening to children voices in early stages of new product development through co-creation – creative focus group and online platform. Food Res Int. 2022. https://doi.org/10.1016/j.foodres.2022.111000

Grindell C, Tod A, Bec R, Wolstenholme D, Bhatnagar R, Sivakumar P, Morley A, Holme J, Lyons J, Ahmed M, Jackson S, Wallace D, Noorzad F, Kamalanathan M, Ahmed L, Evison M. Using creative co-design to develop a decision support tool for people with malignant pleural effusion. BMC Med Inf Decis Mak. 2020. https://doi.org/10.1186/s12911-020-01200-3

Kearns Á, Kelly H, Pitt I. Rating experience of ICT-delivered aphasia rehabilitation: co-design of a feedback questionnaire. Aphasiology. 2020. https://doi.org/10.1080/02687038.2019.1649913

Kelemen M, Surman E, Dikomitis L. Cultural animation in health research: an innovative methodology for patient and public involvement and engagement. Health Expect. 2018. https://doi.org/10.1111/hex.12677

Keogh F, Carney P, O’Shea E. Innovative methods for involving people with dementia and carers in the policymaking process. Health Expect. 2021. https://doi.org/10.1111/hex.13213

Micsinszki SK, Buettgen A, Mulvale G, Moll S, Wyndham-West M, Bruce E, Rogerson K, Murray-Leung L, Fleisig R, Park S, Phoenix M. Creative processes in co-designing a co-design hub: towards system change in health and social services in collaboration with structurally vulnerable populations. Evid Policy. 2022. https://doi.org/10.1332/174426421X16366319768599

Valaitis R, Longaphy J, Ploeg J, Agarwal G, Oliver D, Nair K, Kastner M, Avilla E, Dolovich L. Health TAPESTRY: co-designing interprofessional primary care programs for older adults using the persona-scenario method. BMC Fam Pract. 2019. https://doi.org/10.1186/s12875-019-1013-9

Webber R, Partridge R, Grindell C. The creative co-design of low back pain education resources. Evid Policy. 2022. https://doi.org/10.1332/174426421X16437342906266

National Institute for Health and Care Research. A Researcher’s Guide to Patient and Public Involvement. https://oxfordbrc.nihr.ac.uk/wp-content/uploads/2017/03/A-Researchers-Guide-to-PPI.pdf Accessed 01 Nov 2023.

Selman L, Clement C, Douglas M, Douglas K, Taylor J, Metcalfe C, Lane J, Horwood J. Patient and public involvement in randomised clinical trials: a mixed-methods study of a clinical trials unit to identify good practice, barriers and facilitators. Trials. 2021 https://doi.org/10.1186/s13063-021-05701-y

Coulman K, Nicholson A, Shaw A, Daykin A, Selman L, Macefield R, Shorter G, Cramer H, Sydes M, Gamble C, Pick M, Taylor G, Lane J. Understanding and optimising patient and public involvement in trial oversight: an ethnographic study of eight clinical trials. Trials. 2020. https://doi.org/10.1186/s13063-020-04495-9

Ocloo J, Garfield S, Franklin B, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Sys. 2021. https://doi.org/10.1186/s12961-020-00644-3

National Institute for Health and Care Research. Briefing notes for researchers - public involvement in NHS, health and social care research. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 Accessed 01 Nov 2023.

Download references

Acknowledgements

With thanks to the PHIRST-LIGHT public advisory group and consortium for their thoughts and contributions to the design of this work.

The research team is supported by a National Institute for Health and Care Research grant (PHIRST-LIGHT Reference NIHR 135190).

Author information

Olivia R. Phillips and Cerian Harries share joint first authorship.

Authors and Affiliations

Nottingham Centre for Public Health and Epidemiology, Lifespan and Population Health, School of Medicine, University of Nottingham, Clinical Sciences Building, City Hospital Campus, Hucknall Road, Nottingham, NG5 1PB, UK

Olivia R. Phillips, Jo Leonardi-Bee, Holly Knight & Joanne R. Morling

National Institute for Health and Care Research (NIHR) PHIRST-LIGHT, Nottingham, UK

Olivia R. Phillips, Cerian Harries, Jo Leonardi-Bee, Holly Knight, Lauren B. Sherar, Veronica Varela-Mato & Joanne R. Morling

School of Sport, Exercise and Health Sciences, Loughborough University, Epinal Way, Loughborough, Leicestershire, LE11 3TU, UK

Cerian Harries, Lauren B. Sherar & Veronica Varela-Mato

Nottingham Centre for Evidence Based Healthcare, School of Medicine, University of Nottingham, Nottingham, UK

Jo Leonardi-Bee

NIHR Nottingham Biomedical Research Centre (BRC), Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, NG7 2UH, UK

Joanne R. Morling

You can also search for this author in PubMed   Google Scholar

Contributions

Author contributions: study design: ORP, CH, JRM, JLB, HK, LBS, VVM, literature searching and screening: ORP, CH, JRM, data curation: ORP, CH, analysis: ORP, CH, JRM, manuscript draft: ORP, CH, JRM, Plain English Summary: ORP, manuscript critical review and editing: ORP, CH, JRM, JLB, HK, LBS, VVM.

Corresponding author

Correspondence to Olivia R. Phillips .

Ethics declarations

Ethics approval and consent to participate.

The Ethics Committee of the Faculty of Medicine and Health Sciences, University of Nottingham advised that approval from the ethics committee and consent to participate was not required for systematic review studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

40900_2024_580_MOESM1_ESM.docx

Additional file 1: Search strings: Description of data: the search strings and filters used in each of the 5 databases in this review

Additional file 2: Quality appraisal questions: Description of data: CASP quality appraisal questions

40900_2024_580_moesm3_esm.docx.

Additional file 3: Table 1: Description of data: elements of the data extraction table that are not in the main manuscript

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Phillips, O.R., Harries, C., Leonardi-Bee, J. et al. What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review. Res Involv Engagem 10 , 48 (2024). https://doi.org/10.1186/s40900-024-00580-4

Download citation

Received : 28 November 2023

Accepted : 25 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s40900-024-00580-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Public and patient involvement
• Creative PPI
• Qualitative systematic review

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

REVIEW article

Shaping the foundation of the supercyberkids learning framework: a comprehensive analysis of cybersecurity education initiatives provisionally accepted.

• 1 Institute of Didactic Technologies, Department of Human and Social Sciences, Cultural Heritage, National Research Council (CNR), Italy
• 2 Institute of Informatics and Telematics, Department of Engineering, ICT and Technology for Energy and Transport, National Research Council (CNR), Italy
• 3 European Cyber Security Organisation, Belgium
• 4 School of Digital Technologies, Tallinn University, Estonia

The final, formatted version of the article will be published soon.

In the rapidly evolving digital landscape, cybersecurity education for children is paramount. The 'SuperCyberKids' project, funded under the EU Erasmus+ programme, aims to address this need by developing a comprehensive educational ecosystem for children aged 8 to 13 and their teachers. Central to this initiative is the SuperCyberKids Learning Framework (SCKLF), which incorporates a game-based approach to enhance engagement and motivation through a bespoke digital learning platform and broader ecosystem. This paper focuses on the second pillar of SCKLF, a detailed survey of 65 cybersecurity education initiatives, offering a practical perspective on the current state of cybersecurity education. The survey covered initiatives within and beyond the European Union, emphasizing the target age group. The analysis of these initiatives provides invaluable insights into the practical application of cybersecurity education and plays a crucial role in shaping the SCKLF. By highlighting the diversity of approaches and strategies in cybersecurity education, this research contributes to a more holistic and applied perspective, ensuring the framework's relevance and effectiveness in fostering digital resilience among young learners.

Keywords: Cybersecurity education, SuperCyberKids Learning Framework, Digital learning, Children's Online Safety, Cybersecurity Education Initiatives

Received: 24 Jan 2024; Accepted: 13 May 2024.

Copyright: © 2024 Manganello, Earp, Fante, Bassi, Fabbri, Matteucci, Vaccarelli, Olesen, De Vibraye, Callaghan and Gentile. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Flavio Manganello, Institute of Didactic Technologies, Department of Human and Social Sciences, Cultural Heritage, National Research Council (CNR), Genova, Italy

People also looked at

Contribution of the Chinese Meridian Project to space environment research: Highlights and perspectives

• Open access
• Published: 14 June 2023
• Volume 66 , pages 1423–1438, ( 2023 )

Cite this article

You have full access to this open access article

• Chi Wang 1 , 2 ,
• Jiyao Xu 1 , 2 ,
• Libo Liu 2 , 3 ,
• Xianghui Xue 4 ,
• Qinghe Zhang 5 ,
• Yongqiang Hao 6 ,
• Gang Chen 7 ,
• Hui Li 1 , 2 ,
• Guozhu Li 3 ,
• Bingxian Luo 2 , 8 ,
• Yajun Zhu 1 &
• Jiangyan Wang 1  

453 Accesses

2 Citations

Explore all metrics

The Chinese Meridian Project (CMP) is devoted to establishing a comprehensive ground-based monitoring network for China’s space weather research. CMP is a major national science and technology infrastructure project with the participation of more than 10 research institutions and universities led by the National Space Science Center of the Chinese Academy of Sciences. CMP is planned to be constructed in two phases: CMP phases I and II. The first phase (CMP-I) started construction in 2008 and completed in 2012, after which it entered the operation stage. The 10-year observation of CMP-I has made significant scientific discoveries and achievements in the research fields of the middle and upper atmospheric fluctuations, metal layers in the mesosphere and lower thermosphere, ionospheric disturbances and irregularities, geomagnetic disturbances, and influences of solar activity. The review summarizes the main observations and research achievements, space weather forecast modeling and methods based on CMP-I over the past 10 years, and presents a future extension perspective along with the construction of CMP-II.

Article PDF

Download to read the full article text

Similar content being viewed by others

Scientific challenges and instrumentation for the International Meridian Circle Program

The Changing Earth Science Network: Projects 2009–2011

Australian Space Research Program—Platform Technologies for Space, Atmosphere and Climate Project: Selected Innovations

Avoid common mistakes on your manuscript.

Aa E, Liu S, Huang W, Shi L, Gong J, Chen Y, Shen H, Li J. 2016. Regional 3-D ionospheric electron density specification on the basis of data assimilation of ground-based GNSS and radio occultation data. Space Weather, 14: 433–448

Article   Google Scholar  

Chen G, Ding F, Wan W, Hu L, Zhao X, Li J. 2020. Structures of multiple large-scale traveling ionospheric disturbances observed by dense Global Navigation Satellite System networks in China. J Geophys Res-Space Phys, 125: e27032

Chen G, Jin H, Yan J, Zhang S, Li G, Yokoyama T, Yang G, Yan C, Wu C, Wang J, Zhong D, Li Y, Wang Z. 2017. Low-latitude daytime F region irregularities observed in two geomagnetically quiet days by the Hainan coherent scatter phased array radar (HCOPAR). J Geophys Res-Space Phys, 122: 2645–2654

Chen G, Wang J, Zhang S, Deng Z, Zhong D, Wu C, Jin H, Li Y. 2018. Opposite latitudinal dependence of the premidnight and postmidnight oscillations in the electron density of midlatitude F layer. J Geophys Res-Space Phys, 123: 796–807

Chen G, Wu C, Zhang S, Ning B, Huang X, Zhong D, Qi H, Wang J, Huang L. 2016. Midlatitude ionospheric responses to the 2013 SSW under high solar activity. J Geophys Res-Space Phys, 121: 790–803

Dou X K, Qiu S C, Xue X H, Chen T D, Ning B Q. 2013. Sporadic and thermospheric enhanced sodium layers observed by a lidar chain over China. J Geophys Res-Space Phys, 118: 6627–6643

Dungey J W. 1961. Interplanetary magnetic field and the auroral zones. Phys Rev Lett, 6: 47–48

Gao Q, Chu X, Xue X, Dou X, Chen T, Chen J. 2015. Lidar observations of thermospheric Na layers up to 170 km with a descending tidal phase at Lijiang (26.7°N, 100.0°E), China. J Geophys Res-Space Phys, 120: 9213–9220

Geng W, Huang W, Liu G, Aa E, Liu S, Chen Y, Luo B. 2020. Generation of ionospheric scintillation maps over Southern China based on Kriging method. Adv Space Res, 65: 2808–2820

Gerding M, Daly S, Plane J M C. 2019. Lidar soundings of the mesospheric nickel layer using Ni( 3 F) and Ni( 3 D) transitions. Geophys Res Lett, 46: 408–415

Gong S, Yang G, Dou X, Xu J, Chen C, Gong S. 2015. Statistical study of atmospheric gravity waves in the mesopause region observed by a lidar chain in eastern China. J Geophys Res-Atmos, 120: 7619–7634

Hao Y, Li Q, Guo J, Zhang X, Yang G, Zhang D, Xiao Z. 2021. Imaging of the large-scale ionospheric disturbances induced by seismic waves using GPS network in China (in Chinese). Chin J Geophys, 64: 3925–3932

Google Scholar  

Hao Y Q, Xiao Z, Zhang D H. 2012. Multi-instrument observation on coseismic ionospheric effects after great Tohoku earthquake. J Geophys Res, 117: A02305

Hao Y Q, Xiao Z, Zhang D H. 2013. Teleseismic magnetic effects (TMDs) of 2011 Tohoku earthquake. J Geophys Res-Space Phys, 118: 3914–3923

Hu L, Zhao X, Sun W, Wu Z, Zheng J, Xie H, Huang Z, Ning B, Li G. 2020. Statistical characteristics and correlation of low-latitude F region bottom-type irregularity layers and plasma plumes over Sanya. J Geophys Res-Space Phys, 125: e27855

Hu Y, Guo X, Li G, Wang C, Huang Z. 2005. Oscillation of quasi-steady Earth’s magnetosphere. Chin Phys Lett, 22: 2723–2726

Hu Y Q, Guo X C, Wang C. 2007. On the ionospheric and reconnection potentials of the Earth: Results from global MHD simulations. J Geophys Res, 112: A07215

Huang F, Lei J, Zhang R, Li N, Gu S, Yu Y, Liu L, Owolabi C, Ning B, Li G, Zhong J, Dang T, Ren D, Yi W, Luan X, Xue X, Yu T, Li F, Dou X, Yoshikawa A. 2020. Prominent daytime TEC enhancements under the quiescent condition of January 2017. Geophys Res Lett, 47: e88398

Jiao J, Yang G, Wang J, Feng W, Plane J M C. 2017. Observations of dramatic enhancements to the mesospheric K layer. Geophys Res Lett, 44: 12–536

Jimoh O, Lei J, Huang F. 2020. Investigation of daytime total electron content enhancements over the Asian-Australian sector observed from the Beidou geostationary satellite during 2016–2018. Remote Sens, 12: 3406

Jin H, Zou S, Yan C, Yang G, Chen G, Zhang S, Yan J, Yi S, Wang Z. 2021. A statistical study of F -Region 3.2-m-scale field-aligned irregularities occurrence and vertical plasma drift over Hainan: Solar activity, season, and magnetic activity dependences. J Geophys Res-Space Phys, 126: e28932

Kelley M C. 2009. The Earth’s ionosphere: Plasma physics and electrodynamics. Int Geophys Ser, 43. San Diego, CA: Academic Press

Knight S. 1973. Parallel electric fields. Planet Space Sci, 21: 741–750

Kuai J, Liu L, Lei J, Liu J, Zhao B, Chen Y, Le H, Wang Y, Hu L. 2017. Regional differences of the ionospheric response to the July 2012 geomagnetic storm. J Geophys Res-Space Phys, 122: 4654–4668

Kuai J, Liu L, Liu J, Sripathi S, Zhao B, Chen Y, Le H, Hu L. 2016. Effects of disturbed electric fields in the low-latitude and equatorial ionosphere during the 2015 St. Patrick’s Day storm. J Geophys Res-Space Phys, 121: 9111–9126

Lei J, Huang F, Chen X, Zhong J, Ren D, Wang W, Yue X, Luan X, Jia M, Dou X, Hu L, Ning B, Owolabi C, Chen J, Li G, Xue X. 2018. Was magnetic storm the only driver of the long-duration enhancements of daytime total electron content in the Asian-Australian sector between 7 and 12 September 2017? J Geophys Res-Space Phys, 123: 3217–3232

Lei J, Zhong J, Mao T, Hu L, Yu T, Luan X, Dou X, Sutton E, Yue X, Lin J, Batista I S. 2016. Contrasting behavior of the F 2 peak and the topside ionosphere in response to the 2 October 2013 geomagnetic storm. J Geophys Res-Space Phys, 121: 10,549–10,563

Li G, Ning B, Otsuka Y, Abdu M A, Abadi P, Liu Z, Spogli L, Wan W. 2021. Challenges to equatorial plasma bubble and ionospheric scintillation short-term forecasting and future aspects in east and southeast Asia. Surv Geophys, 42: 201–238

Li N, Luan X, Lei J, Bolaji O S, Owolabi C, Chen J, Xu Z, Li G, Ning B. 2020. Variations of mesospheric neutral winds and tides observed by a meteor radar chain over China during the 2013 sudden stratospheric warming. J Geophys Res-Space Phys, 125: e27443

Li Q, Huang F, Zhong J, Zhang R, Kuai J, Lei J, Liu L, Ren D, Ma H, Yoshikawa A, Hu L, Guo J, Li W, Zhou X, Cui J. 2020. Persistence of the long-duration daytime TEC enhancements at different longitudinal sectors during the August 2018 geomagnetic storm. J Geophys Res-Space Phys, 125: e28238

Li Q, Xu J, Liu X, Yuan W, Chen J. 2016. Characteristics of mesospheric gravity waves over the southeastern Tibetan Plateau region. J Geophys Res-Space Phys, 121: 9204–9221

Li Q, Xu J, Yue J, Liu X, Yuan W. 2019. Evolution of a mesospheric bore in a duct observed by ground-based double-layer imagers and satellite observations over the Tibetan Plateau region. J Geophys Res-Space Phys, 124: 1377–1388

Li W, Chen Y, Liu L, Le H, Zhang R, Li J, Huang C. 2020. A statistical study on the winter ionospheric nighttime enhancement at middle latitudes in the Northern Hemisphere. J Geophys Res-Space Phys, 125: e27950

Liu G, Huang W, Shen H, Aa E, Li M, Liu S, Luo B. 2019. Ionospheric response to the 2018 sudden stratospheric warming event at middle-and low-latitude stations over China sector. Space Weather, 17: 1230–1240

Liu H, Ding F, Zhao B, Li J, Hu L, Wan W, Ning B. 2017. Ionospheric response following the M w 7.8 Gorkha earthquake on 25 April 2015. J Geophys Res-Space Phys, 122: 6495–6507

Liu J, Qian L, Maute A, Wang W, Richmond A D, Chen J, Lei J, Zhang Q, Xing Z. 2021a. Electrodynamical coupling of the geospace system during solar flares. J Geophys Res-Space Phys, 126: e28569

Liu J, Wang W, Qian L, Lotko W, Burns A G, Pham K, Lu G, Solomon S C, Liu L, Wan W, Anderson B J, Coster A, Wilder F. 2021b. Solar flare effects in the Earth’s magnetosphere. Nat Phys, 17: 807–812

Liu J, Zhang D H, Hao Y Q, Xiao Z. 2019. The comparison of lunar tidal characteristics in the low-latitudinal ionosphere between East Asian and American sectors during stratospheric sudden warming events: 2009–2018. J Geophys Res-Space Phys, 124: 7013–7033

Liu J, Zhang D, Hao Y, Xiao Z. 2020. Multi-instrumental observations of the quasi-16-day variations from the lower thermosphere to the topside ionosphere in the low-latitude eastern Asian sector during the 2017 sudden stratospheric warming event. J Geophys Res-Space Phys, 125: e27505

Liu L, Chen Y, Le H, Ning B, Wan W, Liu J, Hu L. 2013. A case study of postmidnight enhancement in F -layer electron density over Sanya of China. J Geophys Res-Space Phys, 118: 4640–4648

Liu L, Chen Y, Zhang R, Le H, Zhang H. 2021. Some investigations of ionospheric diurnal variation. Rev Geophys Planet Phys, 52: 647–661

Liu L, Ding Z, Le H, Chen Y, Zhang H, Wu J, Li G, Wan W. 2020a. New features of the enhancements in electron density at low latitudes. J Geophys Res-Space Phys, 125: e27539

Liu L, Ding Z, Zhang R, Chen Y, Le H, Zhang H, Li G, Wu J, Wan W. 2020b. A case study of the enhancements in ionospheric electron density and its longitudinal gradient at Chinese low latitudes. J Geophys Res-Space Phys, 125: e27751

Liu W, Blanc M, Wang C, Donavan E, Foster J, Lester M, Opgenoorth H, Ren L. 2021. Scientific challenges and instrumentation for the International Meridian Circle Program. Sci China Earth Sci, 64: 2090–2097

Lyons L R. 1981. The field-aligned current versus electric potential relation and auroral electrodynamics. In: Kan J R, Akasofu S I, eds. Physics of Auroral Arc Formation. Washington DC: Geophysical Monograph, American Geophysical Union. 252

Qiu S, Tang Y, Jia M, Xue X, Dou X, Li T, Wang Y. 2016. A review of latitudinal characteristics of sporadic sodium layers, including new results from the Chinese Meridian Project. Earth-Sci Rev, 162: 83–106

Ren D, Lei J, Zhou S, Li W, Huang F, Luan X, Dang T, Liu Y. 2020. High-speed solar wind imprints on the ionosphere during the recovery phase of the August 2018 geomagnetic storm. Space Weather, 18: e02480

Sun L, Xu J, Wang W, Yuan W, Li Q, Jiang C. 2016. A statistical analysis of equatorial plasma bubble structures based on an all-sky airglow imager network in China. J Geophys Res-Space Phys, 121: 11,495–11,517

Sun L, Xu J, Xiong C, Zhu Y, Yuan W, Hu L, Liu W, Chen G. 2019. Midlatitudinal special airglow structures generated by the interaction between propagating medium-scale traveling ionospheric disturbance and nighttime plasma density enhancement at magnetically quiet time. Geophys Res Lett, 46: 1158–1167

Sun W, Zhao X, Hu L, Yang S, Xie H, Chang S, Ning B, Li J, Liu L, Li G. 2021. Morphological characteristics of thousand-kilometer-scale Es structures over China. J Geophys Res-Space Phys, 126: e28712

Wang C. 2010. New chains of space weather monitoring stations in China. Space Weather, 8: 08001

Wang C, Zhang Q, Chi P J, Li C. 2013. Simultaneous observations of plasmaspheric and ionospheric variations during magnetic storms in 2011: First result from Chinese Meridian Project. J Geophys Res-Space Phys, 118: 99–104

Wang G, Luo B, Liu S, Gong J. 2016. An improved algorithm for Nowcast of Kp index. Chin J Space Sci, 36: 153–166

Wang G J, Shi J K, Reinisch B W, Wang X, Wang Z. 2015. Ionospheric plasma bubbles observed concurrently by multi-instruments over low-latitude station Hainan. J Geophys Res-Space Phys, 120: 2288–2298

Wang J, Yang Y, Cheng X, Yang G, Song S, Gong S. 2012. Double sodium layers observation over Beijing, China. Geophys Res Lett, 39: L15801

Wang Y, Zhang Q H, Jayachandran P T, Lockwood M, Zhang S R, Moen J, Xing Z Y, Ma Y Z, Lester M. 2016. A comparison between large-scale irregularities and scintillations in the polar ionosphere. Geophys Res Lett, 43: 4790–4798

Wu F, Zheng H, Yang Y, Cheng X, Li F, Du L, Wang J, Jiao J, Plane J M C, Feng W, Yang G. 2021. Lidar observations of the upper atmospheric nickel layer at Beijing (40°N, 116°E). J Quant Spectrosc Ra, 260: 107468

Wu K, Xu J, Wang W, Sun L, Liu X, Yuan W. 2017. Interesting equatorial plasma bubbles observed by all-sky imagers in the equatorial region of China. J Geophys Res-Space Phys, 122: 10,596–10,611

Wu K, Xu J, Xiong C, Yuan W. 2018. Edge plasma enhancements of equatorial plasma depletions observed by all-sky imager and the C/NOFS satellite. J Geophys Res-Space Phys, 123: 8835–8849

Xiao Z, Xiao S G, Hao Y Q, Zhang D H. 2007. Morphological features of ionospheric response to typhoon. J Geophys Res, 112: A04304

Xu J, Li Q, Sun L, Liu X, Yuan W, Wang W, Yue J, Zhang S, Liu W, Jiang G, Wu K, Gao H, Lai C. 2021. The ground-based airglow imager network in China: Recent observational results. In: Wang W, Zhang Y, Paxton L J, eds. Upper Atmosphere Dynamics and Energetics. 365–394

Xu J, Li Q, Yue J, Hoffmann L, Straka III W C, Wang C, Liu M, Yuan W, Han S, Miller S D, Sun L, Liu X, Liu W, Yang J, Ning B. 2015. Concentric gravity waves over northern China observed by an airglow imager network and satellites. J Geophys Res-Atmos, 120: 11,058–11,078

Xun Y, Yang G, She C Y, Wang J, Du L, Yan Z, Yang Y, Cheng X, Li F. 2019. The first concurrent observations of thermospheric Na layers from two nearby central midlatitude lidar stations. Geophys Res Lett, 46: 1892–1899

Yu B, Xue X, Lu G, Kuo C L, Dou X, Gao Q, Qie X, Wu J, Qiu S, Chi Y, Tang Y. 2017. The enhancement of neutral metal Na layer above thunderstorms. Geophys Res Lett, 44: 9555–9563

Yu B, Xue X, Scott C J, Wu J, Yue X, Feng W, Chi Y, Marsh D R, Liu H, Dou X, Plane J M C. 2021. Interhemispheric transport of metallic ions within ionospheric sporadic E layers by the lower thermospheric meridional circulation. Atmos Chem Phys, 21: 4219–4230

Yu B, Xue X, Yue X, Yang C, Yu C, Dou X, Ning B, Hu L. 2019. The global climatology of the intensity of the ionospheric sporadic E layer. Atmos Chem Phys, 19: 4139–4151

Yu F R, Huang K M, Zhang S D, Huang C M, Yi F, Gong Y, Wang R, Li G, Ning B. 2019. Quasi 10-and 16-day wave activities observed through meteor radar and MST radar during stratospheric final warming in 2015 spring. J Geophys Res-Atmos, 124: 6040–6056

Yu Y, Wan W, Ning B, Liu L, Wang Z, Hu L H, Ren Z. 2013. Tidal wind mapping from observations of a meteor radar chain in December 2011. J Geophys Res-Space Phys, 118: 2321–2332

Yu Y, Wan W, Reid I M, Chen J, Vincent R A, Ning B, Murphy D J, Yang G, Xue X, MacKinnon A D, Li N, Yan C, Liu L, Hu L, Ren Z, Zhang Y. 2017. Global tidal mapping from observations of a radar campaign. Adv Space Res, 60: 130–143

Yu Y, Wan W, Ren Z, Xiong B, Zhang Y, Hu L, Ning B, Liu L. 2015. Seasonal variations of MLT tides revealed by a meteor radar chain based on Hough mode decomposition. J Geophys Res-Space Phys, 120: 7030–7048

Yue X, Wan W, Liu L, Liu J, Zhang S, Schreiner W S, Zhao B, Hu L. 2016a. Mapping the conjugate and corotating storm-enhanced density during 17 March 2013 storm through data assimilation. J Geophys Res-Space Phys, 121: 12,202–12,210

Yue X, Wang W, Lei J, Burns A, Zhang Y, Wan W, Liu L, Hu L, Zhao B, Schreiner W S. 2016b. Long-lasting negative ionospheric storm effects in low and middle latitudes during the recovery phase of the 17 March 2013 geomagnetic storm. J Geophys Res-Space Phys, 121: 9234–9249

Zhang J J, Yu Y Q, Wang C, Du D, Wei D, Liu L G. 2020. Measurements and simulations of the geomagnetically induced currents in low-latitude power networks during geomagnetic storms. Space Weather, 18: e2020SW002549

Zhang Q H, Lockwood M, Foster J C, Zhang S R, Zhang B C, McCrea I W, Moen J, Lester M, Ruohoniemi J M. 2015. Direct observations of the full Dungey convection cycle in the polar ionosphere for southward interplanetary magnetic field conditions. J Geophys Res-Space Phys, 120: 4519–4530

Zhang Q, Moen J, Lockwood M, McCrea I, Zhang B, McWilliams K, Zong Q, Zhang S, Ruohoniemi J M, Thomas E G, Dunlop M W, Liu R, Yang H, Hu H, Lester M. 2016a. Polar cap patch transportation beyond the classic scenario. J Geophys Res-Space Phys, 121: 9063–9074

Zhang Q H, Zhang B C, Lockwood M, Hu H Q, Moen J, Ruohoniemi J M, Thomas E G, Zhang S R, Yang H G, Liu R Y, McWilliams K A, Baker J B H. 2013. Direct observations of the evolution of polar cap ionization patches. Science, 339: 1597–1600

Zhang Q H, Zhang Y L, Wang C, Lockwood M, Yang H G, Tang B B, Xing Z Y, Oksavik K, Lyons L R, Ma Y Z, Zong Q G, Moen J I, Xia L D. 2020. Multiple transpolar auroral arcs reveal insight about coupling processes in the Earth’s magnetotail. Proc Natl Acad Sci USA, 117: 16193–16198

Zhang Q H, Zhang Y L, Wang C, Oksavik K, Lyons L R, Lockwood M, Yang H G, Tang B B, Moen J I, Xing Z Y, Ma Y Z, Wang X Y, Ning Y F, Xia L D. 2021. A space hurricane over the Earth’s polar ionosphere. Nat Commun, 12: 1207

Zhang Q, Zong Q, Lockwood M, Heelis R A, Hairston M, Liang J, McCrea I, Zhang B, Moen J, Zhang S, Zhang Y, Ruohoniemi J M, Lester M, Thomas E G, Liu R, Dunlop M W, Liu Y, Ma Y. 2016b. Earth’s ion upflow associated with polar cap patches: Global and in situ observations. Geophys Res Lett, 43: 1845–1853

Zhang W, Chen G, Zhang S, Gong W, Chen F, He Z, Huang K, Wang Z, Li Y. 2020. Statistical study of the midlatitude mesospheric vertical winds observed by the Wuhan and Beijing MST radars in China. J Geophys Res-Atmos, 125: e2020JD032776

Zhao B, Hao Y. 2015. Ionospheric and geomagnetic disturbances caused by the 2008 Wenchuan earthquake: A revisit. J Geophys Res-Space Phys, 120: 5758–5777

Zhou C, Tang Q, Song X, Qing H, Liu Y, Wang X, Gu X, Ni B, Zhao Z. 2017. A statistical analysis of sporadic E layer occurrence in the mid-latitude China region. J Geophys Res-Space Phys, 122: 3617–3631

Download references

Acknowledgements

This paper was supported by the Chinese Meridian Project .

Author information

Authors and affiliations.

State Key Laboratory of Space Weather, National Space Science Center, Chinese Academy of Sciences, Beijing, 100190, China

Chi Wang, Jiyao Xu, Hui Li, Yajun Zhu & Jiangyan Wang

University of Chinese Academy of Sciences, Beijing, 100049, China

Chi Wang, Jiyao Xu, Libo Liu, Hui Li & Bingxian Luo

Key Laboratory of Earth and Planetary Physics, Institute of Geology and Geophysics, Chinese Academy of Sciences, Beijing, 100029, China

Libo Liu & Guozhu Li

School of Earth and Space Sciences, University Science and Technology of China, Hefei, 230026, China

Xianghui Xue

Shandong Provincial Key Laboratory of Optical Astronomy and Solar-Terrestrial Environment, Institute of Space Sciences, Shandong University, Weihai, 264209, China

Qinghe Zhang

School of Earth and Space Sciences, Peking University, Beijing, 100191, China

Yongqiang Hao

Department of Space Physics, School of Electronic Information, Wuhan University, Wuhan, 430072, China

Key Laboratory of Science and Technology on Environmental Space Situation Awareness, National Space Science Center, Chinese Academy of Sciences, Beijing, 100190, China

Bingxian Luo

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Chi Wang or Jiyao Xu .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Wang, C., Xu, J., Liu, L. et al. Contribution of the Chinese Meridian Project to space environment research: Highlights and perspectives. Sci. China Earth Sci. 66 , 1423–1438 (2023). https://doi.org/10.1007/s11430-022-1043-3

Download citation

Received : 09 May 2022

Revised : 30 November 2022

Accepted : 09 December 2022

Published : 14 June 2023

Issue Date : July 2023

DOI : https://doi.org/10.1007/s11430-022-1043-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Chinese Meridian Project
• Ground-based observation network
• Space weather
• Solar-terrestrial physics
• Find a journal
• Publish with us
• Track your research

NTRS - NASA Technical Reports Server

Available downloads, related records.

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.

• Great Plains Research

The Archaeology of Space and Place in the West ed. by Emily Dale and Carolyn L. White (review)

• John D. Seebach
• University of Nebraska Press
• Volume 33, Number 2, Fall 2023
• pp. 139-140
• 10.1353/gpr.2023.a925341
• View Citation

Additional Information

Project MUSE Mission

Project MUSE promotes the creation and dissemination of essential humanities and social science resources through collaboration with libraries, publishers, and scholars worldwide. Forged from a partnership between a university press and a library, Project MUSE is a trusted part of the academic and scholarly community it serves.

2715 North Charles Street Baltimore, Maryland, USA 21218

+1 (410) 516-6989 [email protected]

©2024 Project MUSE. Produced by Johns Hopkins University Press in collaboration with The Sheridan Libraries.

Now and Always, The Trusted Content Your Research Requires

Built on the Johns Hopkins University Campus

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Kareem A. Wahid
• For correspondence: [email protected]
• ORCID record for Amy C Moreno
• ORCID record for Clifton David Fuller
• Info/History
• Preview PDF

Background/purpose: The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods: We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results: We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion: Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Competing Interest Statement

KAW serves as an Editorial Board Member for Physics and Imaging in Radiation Oncology. CDF has received travel, speaker honoraria and/or registration fee waiver unrelated to this project from: The American Association for Physicists in Medicine; the University of Alabama-Birmingham; The American Society for Clinical Oncology; The Royal Australian and New Zealand College of Radiologists; The American Society for Radiation Oncology; The Radiological Society of North America; and The European Society for Radiation Oncology.

Funding Statement

KAW was supported by an Image Guided Cancer Therapy (IGCT) T32 Training Program Fellowship from T32CA261856. ZYKs time was supported by a doctoral fellowship from the Cancer Prevention Research Institute of Texas grant #RP210042. MAN receives funding from NIH National Institute of Dental and Craniofacial Research (NIDCR) Grant (R03DE033550). CDF received/receives unrelated funding and salary support from: NIH National Institute of Dental and Craniofacial Research (NIDCR) Academic Industrial Partnership Grant (R01DE028290) and the Administrative Supplement to Support Collaborations to Improve AIML-Readiness of NIH-Supported Data (R01DE028290-04S2); NIDCR Establishing Outcome Measures for Clinical Studies of Oral and Craniofacial Diseases and Conditions award (R01DE025248); NSF/NIH Interagency Smart and Connected Health (SCH) Program (R01CA257814); NIH National Institute of Biomedical Imaging and Bioengineering (NIBIB) Research Education Programs for Residents and Clinical Fellows Grant (R25EB025787); NIH NIDCR Exploratory/Developmental Research Grant Program (R21DE031082); NIH/NCI Cancer Center Support Grant (CCSG) Pilot Research Program Award from the UT MD Anderson CCSG Radiation Oncology and Cancer Imaging Program (P30CA016672); Patient-Centered Outcomes Research Institute (PCS-1609-36195) sub-award from Princess Margaret Hospital; National Science Foundation (NSF) Division of Civil, Mechanical, and Manufacturing Innovation (CMMI) grant (NSF 1933369). CDF receives grant and infrastructure support from MD Anderson Cancer Center via: the Charles and Daneen Stiefel Center for Head and Neck Cancer Oropharyngeal Cancer Research Program; the Program in Image-guided Cancer Therapy; and the NIH/NCI Cancer Center Support Grant (CCSG) Radiation Oncology and Cancer Imaging Program (P30CA016672). ACM received/receives funding and salary support from: NIDCR (K01DE030524, R21DE031082), the NIH National Cancer Institute (K12CA088084), and the University of Texas MD Anderson Cancer Center Charles and Daneen Stiefel Center for Head and Neck Cancer Oropharyngeal Cancer Research Program. DF was supported by R01CA195524 and NSF-2111147. Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

A CSV file containing the final studies and corresponding extracted data for this scoping review are made publicly available through Figshare (doi: 10.6084/m9.figshare.25535017). All Python code used in the analysis can be found on Github (URL: https://github.com/kwahid/RT_UQ_scoping_review/tree/main). Data will be private until formal manuscript acceptance in journal.

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One
• Addiction Medicine (323)
• Allergy and Immunology (627)
• Anesthesia (163)
• Cardiovascular Medicine (2363)
• Dentistry and Oral Medicine (287)
• Dermatology (206)
• Emergency Medicine (378)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (833)
• Epidemiology (11755)
• Forensic Medicine (10)
• Gastroenterology (701)
• Genetic and Genomic Medicine (3722)
• Geriatric Medicine (348)
• Health Economics (632)
• Health Informatics (2388)
• Health Policy (929)
• Health Systems and Quality Improvement (894)
• Hematology (340)
• HIV/AIDS (780)
• Infectious Diseases (except HIV/AIDS) (13298)
• Intensive Care and Critical Care Medicine (767)
• Medical Education (365)
• Medical Ethics (104)
• Nephrology (398)
• Neurology (3482)
• Nursing (197)
• Nutrition (522)
• Obstetrics and Gynecology (672)
• Occupational and Environmental Health (661)
• Oncology (1818)
• Ophthalmology (535)
• Orthopedics (218)
• Otolaryngology (286)
• Pain Medicine (232)
• Palliative Medicine (66)
• Pathology (445)
• Pediatrics (1030)
• Pharmacology and Therapeutics (426)
• Primary Care Research (418)
• Psychiatry and Clinical Psychology (3169)
• Public and Global Health (6128)
• Radiology and Imaging (1275)
• Rehabilitation Medicine and Physical Therapy (743)
• Respiratory Medicine (825)
• Rheumatology (379)
• Sexual and Reproductive Health (372)
• Sports Medicine (322)
• Surgery (400)
• Toxicology (50)
• Transplantation (172)
• Urology (145)

MIT Technology Review

• Newsletters

The way whales communicate is closer to human language than we realized

A wave of new projects are taking us closer to understanding what whales are communicating to each other

• Rhiannon Williams archive page

Sperm whales are fascinating creatures. They possess the biggest brain of any species, six times larger than a human’s, which scientists believe may have evolved to support intelligent, rational behavior . They’re highly social, capable of making decisions as a group, and they exhibit complex foraging behavior.  

But there’s also a lot we don’t know about them, including what they may be trying to say to one another when they communicate using a system of short bursts of clicks, known as codas. Now, new research published in Nature Communications today suggests that sperm whales’ communication is actually much more expressive and complicated than was previously thought. 

A team of researchers led by Pratyusha Sharma at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) working with Project CETI, a nonprofit focused on using AI to understand whales, used statistical models to analyze whale codas and managed to identify a structure to their language that’s similar to features of the complex vocalizations humans use. Their findings represent a tool future research could use to decipher not just the structure but the actual meaning of whale sounds.

The team analyzed recordings of 8,719 codas from around 60 whales collected by the Dominica Sperm Whale Project between 2005 and 2018, using a mix of algorithms for pattern recognition and classification. They found that the way the whales communicate was not random or simplistic, but structured depending on the context of their conversations. This allowed them to identify distinct vocalizations that hadn’t been previously picked up on.

Instead of relying on more complicated machine-learning techniques, the researchers chose to use classical analysis to approach an existing database with fresh eyes.

“We wanted to go with a simpler model that would already give us a basis for our hypothesis,” says Sharma. “The nice thing about a statistics approach is that you do not have to train a model and it’s not a black box, and [the analyses are] easier to perform,”  says Felix Effenberger, a senior AI research advisor to the Earth Species Project, a nonprofit that’s researching how to decode non-human communication using AI. But he points out that machine learning is a great way to speed up the process of discovering patterns in a data set, so adopting such a method could be useful in the future.

The algorithms turned the clicks within the coda data into a new kind of data visualization the researchers call an exchange plot, revealing that some codas featured extra clicks. These extra clicks, combined with variations in the duration of their calls, appeared in interactions between multiple whales, which the researchers say suggests that codas can carry more information and possess a more complicated internal structure than we’d previously believed.

“One way to think about what we found is that people have previously been analyzing the sperm whale communication system as being like Egyptian hieroglyphics, but it’s actually like letters,” says Jacob Andreas, an associate professor at CSAIL who was involved with the project.

Although the team isn’t sure whether what it uncovered can be interpreted as the equivalent of the letters, tongue position, or sentences that go into human language, they are confident that there was a lot of internal similarity between the codas they analyzed, he says. “This in turn allowed us to recognize that there were more kinds of codas, or more kinds of distinctions between codas, that whales are clearly capable of perceiving—[and] that people just hadn’t picked up on at all in this data.”

The team’s next step is to build language models of whale calls and to examine how those calls relate to different behaviors. They also plan to work on a more general system that could be used across species, says Sharma. Taking a communication system we know nothing about, working out how it encodes and transmits information, and slowly beginning to understand what’s being communicated could have many purposes beyond whales. “I think we’re just starting to understand some of these things,” she says. “We’re very much at the beginning, but we are slowly making our way through.”

Gaining an understanding of what animals are saying to each other is the primary motivation behind projects such as these. But if we ever hope to understand what whales are communicating, there’s a large obstacle in the way: the need for experiments to prove that such an attempt can actually work, says Caroline Casey, a researcher at UC Santa Cruz who has been studying elephant seals’ vocal communication for over a decade.

Artificial intelligence

Sam altman says helpful agents are poised to become ai’s killer function.

Open AI’s CEO says we won’t need new hardware or lots more training data to get there.

• James O'Donnell archive page

What’s next for generative video

OpenAI's Sora has raised the bar for AI moviemaking. Here are four things to bear in mind as we wrap our heads around what's coming.

• Will Douglas Heaven archive page

Is robotics about to have its own ChatGPT moment?

Researchers are using generative AI and other techniques to teach robots new skills—including tasks they could perform in homes.

• Melissa Heikkilä archive page

An AI startup made a hyperrealistic deepfake of me that’s so good it’s scary

Synthesia's new technology is impressive but raises big questions about a world where we increasingly can’t tell what’s real.

Stay connected

Get the latest updates from mit technology review.

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at [email protected] with a list of newsletters you’d like to receive.