assignment for problem based learning

Center for Teaching Innovation

Resource library.

Establishing Community Agreements and Classroom Norms
Sample group work rubric
Problem-Based Learning Clearinghouse of Activities, University of Delaware

Problem-Based Learning

Problem-based learning  (PBL) is a student-centered approach in which students learn about a subject by working in groups to solve an open-ended problem. This problem is what drives the motivation and the learning. 

Why Use Problem-Based Learning?

Nilson (2010) lists the following learning outcomes that are associated with PBL. A well-designed PBL project provides students with the opportunity to develop skills related to:

Working in teams.
Managing projects and holding leadership roles.
Oral and written communication.
Self-awareness and evaluation of group processes.
Working independently.
Critical thinking and analysis.
Explaining concepts.
Self-directed learning.
Applying course content to real-world examples.
Researching and information literacy.
Problem solving across disciplines.

Considerations for Using Problem-Based Learning

Rather than teaching relevant material and subsequently having students apply the knowledge to solve problems, the problem is presented first. PBL assignments can be short, or they can be more involved and take a whole semester. PBL is often group-oriented, so it is beneficial to set aside classroom time to prepare students to  work in groups  and to allow them to engage in their PBL project.

Students generally must:

Examine and define the problem.
Explore what they already know about underlying issues related to it.
Determine what they need to learn and where they can acquire the information and tools necessary to solve the problem.
Evaluate possible ways to solve the problem.
Solve the problem.
Report on their findings.

Getting Started with Problem-Based Learning

Articulate the learning outcomes of the project. What do you want students to know or be able to do as a result of participating in the assignment?
Create the problem. Ideally, this will be a real-world situation that resembles something students may encounter in their future careers or lives. Cases are often the basis of PBL activities. Previously developed PBL activities can be found online through the University of Delaware’s PBL Clearinghouse of Activities .
Establish ground rules at the beginning to prepare students to work effectively in groups.
Introduce students to group processes and do some warm up exercises to allow them to practice assessing both their own work and that of their peers.
Consider having students take on different roles or divide up the work up amongst themselves. Alternatively, the project might require students to assume various perspectives, such as those of government officials, local business owners, etc.
Establish how you will evaluate and assess the assignment. Consider making the self and peer assessments a part of the assignment grade.

Nilson, L. B. (2010). Teaching at its best: A research-based resource for college instructors (2nd ed.).  San Francisco, CA: Jossey-Bass. 

Problem-Based Learning (PBL)

What is Problem-Based Learning (PBL)? PBL is a student-centered approach to learning that involves groups of students working to solve a real-world problem, quite different from the direct teaching method of a teacher presenting facts and concepts about a specific subject to a classroom of students. Through PBL, students not only strengthen their teamwork, communication, and research skills, but they also sharpen their critical thinking and problem-solving abilities essential for life-long learning.

By working with PBL, students will:

Become engaged with open-ended situations that assimilate the world of work
Participate in groups to pinpoint what is known/ not known and the methods of finding information to help solve the given problem.
Investigate a problem; through critical thinking and problem solving, brainstorm a list of unique solutions.
Analyze the situation to see if the real problem is framed or if there are other problems that need to be solved.

How to Begin PBL

Establish the learning outcomes (i.e., what is it that you want your students to really learn and to be able to do after completing the learning project).
Find a real-world problem that is relevant to the students; often the problems are ones that students may encounter in their own life or future career.
Discuss pertinent rules for working in groups to maximize learning success.
Practice group processes: listening, involving others, assessing their work/peers.
Explore different roles for students to accomplish the work that needs to be done and/or to see the problem from various perspectives depending on the problem (e.g., for a problem about pollution, different roles may be a mayor, business owner, parent, child, neighboring city government officials, etc.).
Determine how the project will be evaluated and assessed. Most likely, both self-assessment and peer-assessment will factor into the assignment grade.

Designing Classroom Instruction

How to Orchestrate a PBL Activity

Explain Problem-Based Learning to students: its rationale, daily instruction, class expectations, grading.
Serve as a model and resource to the PBL process; work in-tandem through the first problem
Help students secure various resources when needed.
Supply ample class time for collaborative group work.
Give feedback to each group after they share via the established format; critique the solution in quality and thoroughness. Reinforce to the students that the prior thinking and reasoning process in addition to the solution are important as well.

Teacher’s Role in PBL

The Role of the Students

Robert Gagné’s Taxonomy of Learning

Gagne classified learning outcomes into five major categories: verbal information, intellectual skills, cognitive strategies, motor skills and attitudes. What is learning to Gagné? As outlined in his 9-events of instruction, Gagne believed that learning was…

Open Source Learning Management Systems (LMS)

Learning Management Systems (LMSs) are becoming a vital part of classrooms in the 21th Century. This is a list of open source learning management systems. By open source we mean that source code of…

Adaptive Learning: What is It, What are its Benefits and How Does it Work?

People learn in many different ways. Adaptive learning has sought to address differences in ability by targeting teaching practices. The use of adaptive models, ranging from technological programs to intelligent systems, can be…

Instructional Design

What is Instructional Design The Association for Educational Communications and Technology (AECT) defines instructional design as “the theory and practice of design, development, utilization, management, and evaluation of processes and resources for learning”…

Concept Maps and How To Use Them

Concept maps help our brains take in information, mostly when there is visual information. The maps help us to see the big picture along with the connected and related data. They also help…

What is PBL? & 5 Problem-Based Learning Examples

Julia Francis : Aug 12, 2022 2:00:00 PM

“We cannot solve our problems with the same thinking we used when we created them.” ~Albert Einstein

Problem solving is a life skill that goes far beyond the classroom. The best education is an education that teaches critical and strategic thinking and allows students to meet life’s problems and challenges with an open mind and the confidence to find a solution.

At Alludo, we’re big believers in problem-based learning, an active-learning strategy that prepares students for the realities of life by encouraging them to use strategic thinking to arrive at solutions that work. We’ve included problem-based learning activities in our professional development catalog because we believe that using PBL in the classroom can help teachers help students. Here’s what you need to know about PBL plus five problem-based learning examples to inspire you.

What is problem-based learning.

Choose a Central Concept or Principle

Think of a Real-World Context for the Problem

Introduce the problem in stages, write a teacher's guide, provide students with key resources.

Plan a Road Trip
Create a Sustainable City
Choose and Craft a Voyage Around the World
Plan a Zoo Habitat
Codebreak Math Equations

What Are Challenges in Problem-Based Learning?

Alludo's Take

Encourage Teachers in Your District to Innovate with Problem-Based Learning

Problem-based learning, or PBL, is an inquiry-based learning method that uses complex, real-world problems to help students learn. It stands in contrast to some traditional teaching methods where teachers present facts and concepts directly to students.

The strictest presentation of PBL would involve a teacher using PBL for an entire semester or school year. However, in practice, teachers use it in a variety of ways. It may be most useful when used in lab situations or design projects. It may also be used to initiate discussions.

PBL promotes the development of critical thinking skills, problem solving skills, and communication in students and may be used when students work in groups.

Problems may vary widely depending upon the class or context, but effective PBL problems share the following characteristics:

They must motivate students to understand concepts on a deep level.
They should incorporate content objectives and connect them to previous knowledge.
They should require students to make decisions and defend them using logical reasoning and critical thinking skills.
For group projects, problems need enough complexity to require students to work together and arrive at a solution.
For multi-stage projects, initial steps must be open-ended and engaging to get students invested in solving the problem.

Teachers should tie the material being used to real-world situations and develop a problem that incorporates previous lessons while still challenging students to apply what they have learned. Ideally, complex problems should be introduced in stages and teachers should identify important resources and provide them as a jumping-off point for students.

Teachers may distribute PBL problems using three techniques:

Case study. The problem is submitted to PBL students in writing.
Role playing. Students improvise scenes based on descriptions of key players.
Simulation. Students use a computer-based program to simulate a problem.

The common characteristic is that any problem presented to students must have its roots in a real-world situation.

What Are the Steps in Problem-Based Learning?

Teachers who wish to incorporate problem-based learning in the classroom should follow these steps to create a problem and introduce it to students.

Choose a Central Concept of Principle

The first step is to select a central concept or principle for students to learn. The concept chosen should be one that’s typically included in a given course. The problem should be similar to a typical problem that would be assigned at the end of a chapter to help students learn the concept.

After choosing a central concept, the teacher should develop a list of student learning objectives for students to meet as they research the problem and determine the best way to solve it.

At this stage, teachers need to develop a real-world context that will allow students to work their way through the problem and use appropriate resources to develop a solution. Some options include the following:

Introducing a storytelling aspect to create an example of a real-world problem.
Find an actual, real-world case that can be adapted by adding motivation for students to solve the problem.

Magazines, newspapers, articles, and TV news can all provide ideas for real-world problems, as can talking to professionals in the field to get their ideas.

Introduce the problem in stages to help students identify learning issues and inspire students to research the concepts being targeted. Here are some questions that can help in the development of the stages:

What should the first stage look like and which open-ended questions can be asked? Remember that all questions should be linked to the concepts students are learning.
How will the problem itself be structured?
How long will it take students to solve the problem?
Will students receive additional information at later stages of the problem?
What resources will students need to begin?
What should the end product look like?

Asking these questions can help teachers develop stages that make sense and guide students as they work toward a solution.

The teacher should prepare a guide with detailed plans for instruction related to the problem. The guide should spell out plans to cycle through the problem using different modes of learning. It may also include alternative options.

Any problem being presented in a sizable class may include a combination of whole-class discussions plus small group work and mini-lectures to ensure that all students receive the support they need.

PBL requires teachers to provide students with some (but not all) resources they will need to complete the problem being presented. It is important to leave some of the knowledge resourcing to students, so they learn how to identify good resources and use them independently.

Where students may need help is in understanding offline resources, including the library and how to use it, since many of today’s digital natives may be inclined to rely solely on the internet for research.

K-12 Professional Development Strategy Framework

What Are Examples of Problem-Based Learning?

Now, let’s look at some problem-based learning examples that teachers can use as inspiration to develop new problems to inspire and educate their students.

#1: Plan a Road Trip

Planning a road trip is a real-world problem that students may already have experienced on one level if they’ve traveled with their families.

Using a road trip as a PBL assignment incorporates a variety of disciplines, including geography, social sciences, environmental sciences, and math. Students should plan every aspect of the trip, including the route to be taken, points of interest to be visited along the way, expenses, and fuel consumption.

#2: Create a Sustainable City

It would be hard to imagine a real-world problem more pressing than the issue of figuring out how we can live sustainably and avoid burning through our natural resources.

Students can work together, using their personal observations and research, to think about the problems that cities face regarding sustainability and coming up with ideas to address them.

#3: Choose and Craft a Voyage Around the World

A voyage around the world poses challenges that allow students to tap into a variety of subjects, including geography, world culture, social studies, and even velocity and flotation.

Students should consider modes of transportation, time frames, weather, and more, before they present their results.

#4: Plan a Zoo Habitat

This PBL assignment could begin with a visit to the zoo, where students can observe animals in habitats and speak to zookeepers about what it’s like to care for animals in a zoo.

After that, students should pick an animal, consider where they live naturally and what they eat, and use biology and environmental science to plan a habitat where the animal can live.

#5: Codebreak Math Equations

Instead of solving the usual equations, this PBL example puts students in the roles of professional codebreakers.

They will use logic, critical thinking skills, and mathematics to decrypt a code and craft a response to it, using a code of their own.

Implementing a problem-based learning process in the classroom does have some challenges and teachers must work to overcome these to make sure that students get the most of the problems they work on.

Students may not have prepared for PBL in their past studies. They may need some hand-holding and guidance if they’ve never worked on a real-world problem before.
PBL can be time-consuming and requires a significant amount of prep for teachers.
Since students work in groups, there may be group dynamic issues to address and teachers must keep an eye on students.
PBL requires buy-in and support from staff and educational leaders. Without full support, it can be difficult to implement PBL in the classroom.

Teachers and administrators should work together before the PBL process is implemented to brainstorm ideas and identify potential issues.

Alludo’s Take

Alludo partners with school districts around the country to provide them with a dynamic professional learning environment that drives engagement and supports teachers. Because we know that teachers are always looking for ways to engage their students, we have included PBL missions in the Alludo content catalog .

By completing our missions, teachers learn the pros and cons of PBL and how to implement it. The result is that teachers are encouraged to innovate in the classroom. In other words, engaged teachers are likely to have engaged students, too!

Bringing PBL into the classroom gives teachers an engaging framework to help students learn. When teachers innovate, students are more likely to be engaged in their schoolwork and the result is improved student achievement and better outcomes.

Experience personalized learning for all levels of educators with a free trial of Alludo’s professional development platform. You’ll enjoy:

Hundreds of core topics
Asynchronous microlearning activities
Timely and specific feedback
Analytics that show learning impact
Access anytime, anywhere

What's New in the Alludo Catalog ?

“It would take us years to roll out all the PD that we can on Alludo." - Kathy Jackson, Director of Teaching and Learning for K-12, YCJUSD

Analytic vs. Holistic Rubrics: Which Type of Rubric Should You Use?

“One test of the correctness of educational procedure is the happiness of the child.” ~Maria Montessori

The Art of Rubric Design: Crafting Effective Assessment Tools

As educators, we've all experienced the challenge of evaluating student assignments filled with scattered ideas and jumbled thoughts, leaving us...

18 Problem-Based Learning Examples

problem-based learning examples and definition, explained below

Problem-based learning (PBL) is a student-centered teaching method where students are given the opportunity to solve open-ended real-world problems. The teacher provides limited guidance and is usually referred to as a “facilitator”.

The burden of responsibility for the majority of the work rests squarely on the shoulders of the students.

Problem-Based Learning Examples

Broad problem posing: A teacher writes the question on the board: “Are organic fertilizers better than commercial fertilizers?” The question is purposively broad and requires student teams to clarify the question before even beginning to address it.
Solving problems through inquiry: Problem-based learning has strong overlaps with inquiry based learning, where the teacher presents a problem and the students must develop a study to inquire about answers.
Divergent thinking problems: Students in the first grade have to create a way to communicate with another group without speaking if both are lost in a forest.
Product development: The professor of a Design course has student teams create product packaging that complies with rigorous environmental standards.
Real-life problem solving: Students in second grade are given the task of studying the causes of potholes and creating ways to fill them.
Role playing a problem: IT majors play the role of government compliance officers and evaluate various social media apps regarding free-speech and privacy regulations.
Solving real-life mathematical problems: Fourth graders use math to estimate crop yields of a hypothetical farm and then represent the results graphically.
Multidisciplinary problem solving: Business majors work with students in a nutrition course to create a pizzeria franchise.
Authentic learning scenarios: Medical students are given a clinical scenario that includes the patient’s chart, X-rays and results of various tests. They work in groups to diagnosis the patient’s disease and design a treatment program.
Solving hypothetical problems: Anthropology students create a public holiday for an underserved class or people in a foreign country.
Solving social problems: Students in a Civil Engineering course have to design housing for the poor in an isolated region using only local materials.
Escape rooms: The popular trend of escape rooms can be seen as a form of problem-based learning. Learners must solve the problem of ‘how to escape’.
Solving a riddle: The teacher presents students with a riddle, which they must work together to solve. This may require the application of curriculum-based outcomes like using certain math equations.
Situated learning: Students work on problems in the workplace or ‘real life’ rather than in the classroom, helping them to see how the theory gets applied in a real world context.
Turning exams into challenges: Instead of using paper-based exams, the teacher poses a challenge and the students need to present a report on the solution to the challenge.
Creating an app: Students in a university programming class don’t just demonstrate their knowledge of programming; they have to create an app that solves a real-life problem.
Developing an environmental regeneration plan: Students identify problems with the current ecosystem and then create a plan to solve the problem. Next, they can actually put the plan into action and report on results.
Working on a social problem: Students are presented with a social problem that can be solved through policy. Students must come up with a social policy that maximizes benefits while also working through potential side-effects and collateral of an intervention.

Benefits of Problem Based Learning

There are numerous benefits of PBL for students. According to Nilson (2010), PBL promotes:

development of critical thinking skills
problem-solving abilities
communication skills
how to handle project management demands
oral and written communication
researching and information literacy
self-awareness
understanding of group dynamics
leadership and teamwork
self-directed learning

Case Studies

1. invasive species.

Students in environmental studies are given a problem-based assignment on an invasive species. The teacher provides a little early support as possible, simply instructing each group to identify the species and develop an action plan to mitigate its impact.

The students form work teams and conduct a brainstorming session on which invasive species exist in a nearby habitat. Then they examine the impact of the species in great detail, identifying the origin of the species, how it effects other plant and wildlife, human activities connected to the problem, and trajectories of consequences in the future.

Once that thorough analysis has occurred, students then begin exploring possible solutions. They have to construct a detailed plan of action and carefully consider the short and long-term effects of each step.

The plan should include government policy, educational programs, and scientific research programs that should be put in place to monitor their plan’s results.

2. Collaborative PBL: Home for the Handicapped

Real-world problems often require an interdisciplinary approach. That means professionals from different backgrounds and perspectives have to collaborate, which is sometimes easier said than done.

In this project, architecture and product design students have to work together to design a house suitable for the handicapped. This means that the floorplan must be easily navigated and that furniture and appliances have to be modified.

The project can be as demanding as the instructors require, from simply making the plans on paper, to actually constructing mock-ups of products and having them tested by affected individuals.

3. Cybersecurity

Issues related to cybersecurity as a result of globalization and technological dependence continue to escalate. Therefore, in addition to teaching future programmers about how to write gaming code, students also need to develop expertise in more serious issues.

Cybersecurity presents an opportunity for students to work in teams on a real-world issue that can have serious consequences. Students are assigned to develop a protocol to protect a nuclear reactor or financial depository.

The programs they design have to be able to handle a variety of potential threats, both internal and external. To make the assignment more realistic, the instructor will activate several programs designed to attack the organization the students are supposed to protect.

Not only do the students need to create programs that defend the organization, but they also must devise protocols to activate in case of a successful breach.

4. Design a Board Game

Students are eager to express their creativity and enjoy working independent of a lot of rules and restrictions. These characteristics are well-suited for PBL activities and result in greater student engagement and deep learning.

Recognizing these features of PBL has led to one teacher giving the students the task of designing their own board game. The facilitator/teacher leaves everything up to the students, and only supplies a set of dice.

The students then hold a class-wide brainstorming session on possible game themes. Once a list is generated, they divide up into teams based on common interests. The facilitator distributes the dice to each group and then steps aside.

The students then get to work on formulating the rules of the game and working out the process of how to play. Eventually they get to the point of being ready to construct a game prototype.

At the end, each team gets to play each other’s games and then engage in a reflection activity. Reflection can involve a worksheet or class discussion, as students consider their performance in the task and key learning outcomes they may have experienced.

5. Increasing Voter Registration

Voter turnout has been low in the U.S. for quite some time. For a democracy, this is not only a problem of people’s voices not being heard, but it can reflect feelings of disappointment in the political process as well.

To address these issues, students in a political science course must work together to understand the issues impacting low voter turnout and devise an action plan to address those factors.

The students start by researching the causal factors through a variety of methods. They might read the relevant literature on the subject, and/or conduct interviews and surveys involving non-voters.

By thoroughly understanding the issues, they can then formulate a plan to encourage voter turnout. That plan is completely up to them. It is important that the facilitator/course instructor provide as little intervention or assistance as possible.

Problem-based learning is a great way for students to learn. Instead of reading a textbook, writing term papers, or listening to hours of lectures, student take an active role in the learning process.

It starts with the instructor, referred to as a facilitator, simply presenting an open-ended problem in a real-world scenario. The students are then given an opportunity to work collaboratively to examine the problem and develop a solution.

Students benefit from this type of learning activity in numerous ways. They learn how to work with others, gain experience and insights into leadership and group dynamics, and develop critical thinking and problem-solving skills.

But perhaps the most significant benefit, is that students become more engaged and enthusiastic about the learning process.

Ali, S. S. (2019). Problem based learning: A student-centered approach. English language teaching , 12(5), 73-78.

Duch, B. J., Groh, S. E, & Allen, D. E. (Eds.). (2001). The power of problem-based learning . Sterling, VA: Stylus.

Hmelo-Silver, C.E., Eberbach, C. (2012). Learning theories and problem-based learning. In: Bridges, S., McGrath, C., Whitehill, T. (Eds.), Problem-based learning in clinical education (pp. 3-17). Innovation and Change in Professional Education, vol 8. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-2515-7_1

Malmia, W., et al. (2019). Problem-based learning as an effort to improve student learning outcomes. Int. J. Sci. Technol. Res, 8 (9), 1140-1143.

Moust, J., Bouhuijs, P., & Schmidt, H. (2021). Introduction to problem-based learning: A guide for students. London: Routledge.

Nilson, L. B. (2010). Teaching at its best: A research-based resource for college instructors (2nd ed.).  San Francisco, CA: Jossey-Bass. 

Wirkala, C., & Kuhn, D. (2011). Problem-based learning in K–12 education: Is it effective and how does it achieve its effects? American Educational Research Journal, 48 (5), 1157–1186. https://doi.org/10.3102/0002831211419491

Dave Cornell (PhD)

Dr. Cornell has worked in education for more than 20 years. His work has involved designing teacher certification for Trinity College in London and in-service training for state governments in the United States. He has trained kindergarten teachers in 8 countries and helped businessmen and women open baby centers and kindergartens in 3 countries.

Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 25 Positive Punishment Examples
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 25 Dissociation Examples (Psychology)
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 15 Zone of Proximal Development Examples
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ Perception Checking: 15 Examples and Definition

Chris Drew (PhD)

This article was peer-reviewed and edited by Chris Drew (PhD). The review process on Helpful Professor involves having a PhD level expert fact check, edit, and contribute to articles. Reviewers ensure all content reflects expert academic consensus and is backed up with reference to academic studies. Dr. Drew has published over 20 academic articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education and holds a PhD in Education from ACU.

Chris Drew (PhD) #molongui-disabled-link 25 Positive Punishment Examples
Chris Drew (PhD) #molongui-disabled-link 25 Dissociation Examples (Psychology)
Chris Drew (PhD) #molongui-disabled-link 15 Zone of Proximal Development Examples
Chris Drew (PhD) #molongui-disabled-link Perception Checking: 15 Examples and Definition

1 thought on “18 Problem-Based Learning Examples”

Thanks. Most helpful.

Problem-Based Learning: Six Steps to Design, Implement, and Assess

November 30, 2015
Vincent R. Genareo PhD and Renee Lyons

T wenty-first century skills necessitate the implementation of instruction that allows students to apply course content, take ownership of their learning, use technology meaningfully, and collaborate. Problem-Based Learning (PBL) is one pedagogical approach that might fit in your teaching toolbox.

PBL is a student-centered, inquiry-based instructional model in which learners engage with an authentic, ill-structured problem that requires further research (Jonassen & Hung, 2008). Students identify gaps in their knowledge, conduct research, and apply their learning to develop solutions and present their findings (Barrows, 1996). Through collaboration and inquiry, students can cultivate problem solving (Norman & Schmidt, 1992), metacognitive skills (Gijbels et al., 2005), engagement in learning (Dochy et al., 2003), and intrinsic motivation. Despite PBL’s potential benefits, many instructors lack the confidence or knowledge to utilize it (Ertmer & Simons, 2006; Onyon, 2005). By breaking down the PBL cycle into six steps, you can begin to design, implement, and assess PBL in your own courses.

Step One: Identify Outcomes/Assessments

PBL fits best with process-oriented course outcomes such as collaboration, research, and problem solving. It can help students acquire content or conceptual knowledge, or develop disciplinary habits such as writing or communication. After determining whether your course has learning outcomes that fit with PBL, you will develop formative and summative assessments to measure student learning. Group contracts, self/peer-evaluation forms, learning reflections, writing samples, and rubrics are potential PBL assessments.

Step Two: Design the Scenario

Next you design the PBL scenario with an embedded problem that will emerge through student brainstorming. Think of a real, complex issue related to your course content. It’s seldom difficult to identify lots of problems in our fields; the key is writing a scenario for our students that will elicit the types of thinking, discussion, research, and learning that need to take place to meet the learning outcomes. Scenarios should be motivating, interesting, and generate good discussion. Check out the websites below for examples of PBL problems and scenarios.

Problem-Based Learning at University of Delaware

Problem-Based Learning in Biology

Science PBL

Step Three: Introduce PBL

If PBL is new to your students, you can practice with an “easy problem,” such as a scenario about long lines in the dining hall. After grouping students and allowing time to engage in an abbreviated version of PBL, introduce the assignment expectations, rubrics, and timelines. Then let groups read through the scenario(s). You might develop a single scenario and let each group tackle it in their own way, or you could design multiple scenarios addressing a unique problem for each group to discuss and research.

Step Four: Research

PBL research begins with small-group brainstorming sessions where students define the problem and determine what they know about the problem (background knowledge), what they need to learn more about (topics to research), and where they need to look to find data (databases, interviews, etc.). Groups should write the problem as a statement or research question. They will likely need assistance. Think about your own research: without good research questions, the process can be unguided or far too specific. Students should decide upon group roles and assign responsibility for researching topics necessary for them to fully understand their problems. Students then develop an initial hypothesis to “test” as they research a solution. Remember: research questions and hypotheses can change after students find information disconfirming their initial beliefs.

Step Five: Product Performance

After researching, the students create products and presentations that synthesize their research, solutions, and learning. The format of the summative assessment is completely up to you. We treat this step like a research fair. Students find resources to develop background knowledge that informs their understanding, and then they collaboratively present their findings, including one or more viable solutions, as research posters to the class.

Step Six: Assessment

During the PBL assessment step, evaluate the groups’ products and performances. Use rubrics to determine whether students have clearly communicated the problem, background, research methods, solutions (feasible and research-based), and resources, and to decide whether all group members participated meaningfully. You should consider having your students fill out reflections about their learning (including what they’ve learned about the content and the research process) every day, and at the conclusion of the process.

Although we presented PBL as steps, it really functions cyclically. For example, you might teach an economics course and develop a scenario about crowded campus sidewalks. After the groups have read the scenario, they develop initial hypotheses about why the sidewalks are crowded and how to solve the problem. If one group believes they are crowded because they are too narrow and the solution is widening the sidewalks, their subsequent research on the economic and environmental impacts might inform them that sidewalk widening isn’t feasible. They should jump back to step four, discuss another hypothesis, and begin a different research path.

This type of process-oriented, self-directed, and collaborative pedagogical strategy can prepare our students for successful post-undergraduate careers. Is it time to put PBL to work in your courses?

References Barrows, H.S. (1996). Problem-based learning in medicine and beyond: A brief overview. In L. Wilkerson, & W. H. Gijselaers (Eds.), New directions for teaching and learning, No.68 (pp. 3-11). San Francisco: Jossey-Bass.

Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based learning: A meta-analysis. Learning and instruction, 13(5), 533-568.

Ertmer, P. A., & Simons, K. D. (2006). Jumping the PBL implementation hurdle: Supporting the efforts of K–12 teachers. Interdisciplinary Journal of Problem-based Learning, 1(1), 5.

Gijbels, D., Dochy, F., Van den Bossche, P., & Segers, M. (2005). Effects of problem-based learning: A meta-analysis from the angle of assessment. Review of Educational Research, 75(1), 27-61.

Jonassen, D. H., & Hung, W. (2008). All problems are not equal: Implications for problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 2(2), 4.

Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: A review of the evidence. Academic Medicine, 67(9), 557-565.

Onyon, C. (2012). Problem-based learning: A review of the educational and psychological theory. The Clinical Teacher, 9(1), 22-26.

Vincent R. Genareo is a postdoctoral research associate at Iowa State University, Research Institute for Studies of Education (RISE). Renee Lyons is a PhD candidate at Clemson University, Department of Education.

Stay Updated with Faculty Focus!

Get exclusive access to programs, reports, podcast episodes, articles, and more!

Opens in a new tab

Welcome Back

Username or Email

Remember Me

Already a subscriber? log in here.

Illinois Online
Illinois Remote

TA Resources
Teaching Consultation
Teaching Portfolio Program
Grad Academy for College Teaching
Faculty Events
The Art of Teaching
2022 Illinois Summer Teaching Institute
Large Classes
Leading Discussions
Laboratory Classes
Lecture-Based Classes
Planning a Class Session
Questioning Strategies
Classroom Assessment Techniques (CATs)
Problem-Based Learning (PBL)
The Case Method
Community-Based Learning: Service Learning
Group Learning
Just-in-Time Teaching
Creating a Syllabus
Motivating Students
Dealing With Cheating
Discouraging & Detecting Plagiarism
Diversity & Creating an Inclusive Classroom
Harassment & Discrimination
Professional Conduct
Foundations of Good Teaching
Student Engagement
Assessment Strategies
Course Design
Student Resources
Teaching Tips
Graduate Teacher Certificate
Certificate in Foundations of Teaching
Teacher Scholar Certificate
Certificate in Technology-Enhanced Teaching
Master Course in Online Teaching (MCOT)
2022 Celebration of College Teaching
2023 Celebration of College Teaching
Hybrid Teaching and Learning Certificate
Classroom Observation Etiquette
Teaching Philosophy Statement
Pedagogical Literature Review
Scholarship of Teaching and Learning
Instructor Stories
Podcast: Teach Talk Listen Learn
Universal Design for Learning

Sign-Up to receive Teaching and Learning news and events

Problem-Based Learning (PBL) is a teaching method in which complex real-world problems are used as the vehicle to promote student learning of concepts and principles as opposed to direct presentation of facts and concepts. In addition to course content, PBL can promote the development of critical thinking skills, problem-solving abilities, and communication skills. It can also provide opportunities for working in groups, finding and evaluating research materials, and life-long learning (Duch et al, 2001).

PBL can be incorporated into any learning situation. In the strictest definition of PBL, the approach is used over the entire semester as the primary method of teaching. However, broader definitions and uses range from including PBL in lab and design classes, to using it simply to start a single discussion. PBL can also be used to create assessment items. The main thread connecting these various uses is the real-world problem.

Any subject area can be adapted to PBL with a little creativity. While the core problems will vary among disciplines, there are some characteristics of good PBL problems that transcend fields (Duch, Groh, and Allen, 2001):

The problem must motivate students to seek out a deeper understanding of concepts.
The problem should require students to make reasoned decisions and to defend them.
The problem should incorporate the content objectives in such a way as to connect it to previous courses/knowledge.
If used for a group project, the problem needs a level of complexity to ensure that the students must work together to solve it.
If used for a multistage project, the initial steps of the problem should be open-ended and engaging to draw students into the problem.

The problems can come from a variety of sources: newspapers, magazines, journals, books, textbooks, and television/ movies. Some are in such form that they can be used with little editing; however, others need to be rewritten to be of use. The following guidelines from The Power of Problem-Based Learning (Duch et al, 2001) are written for creating PBL problems for a class centered around the method; however, the general ideas can be applied in simpler uses of PBL:

Choose a central idea, concept, or principle that is always taught in a given course, and then think of a typical end-of-chapter problem, assignment, or homework that is usually assigned to students to help them learn that concept. List the learning objectives that students should meet when they work through the problem.
Think of a real-world context for the concept under consideration. Develop a storytelling aspect to an end-of-chapter problem, or research an actual case that can be adapted, adding some motivation for students to solve the problem. More complex problems will challenge students to go beyond simple plug-and-chug to solve it. Look at magazines, newspapers, and articles for ideas on the story line. Some PBL practitioners talk to professionals in the field, searching for ideas of realistic applications of the concept being taught.
What will the first page (or stage) look like? What open-ended questions can be asked? What learning issues will be identified?
How will the problem be structured?
How long will the problem be? How many class periods will it take to complete?
Will students be given information in subsequent pages (or stages) as they work through the problem?
What resources will the students need?
What end product will the students produce at the completion of the problem?
Write a teacher's guide detailing the instructional plans on using the problem in the course. If the course is a medium- to large-size class, a combination of mini-lectures, whole-class discussions, and small group work with regular reporting may be necessary. The teacher's guide can indicate plans or options for cycling through the pages of the problem interspersing the various modes of learning.
The final step is to identify key resources for students. Students need to learn to identify and utilize learning resources on their own, but it can be helpful if the instructor indicates a few good sources to get them started. Many students will want to limit their research to the Internet, so it will be important to guide them toward the library as well.

The method for distributing a PBL problem falls under three closely related teaching techniques: case studies, role-plays, and simulations. Case studies are presented to students in written form. Role-plays have students improvise scenes based on character descriptions given. Today, simulations often involve computer-based programs. Regardless of which technique is used, the heart of the method remains the same: the real-world problem.

Where can I learn more?

PBL through the Institute for Transforming Undergraduate Education at the University of Delaware
Duch, B. J., Groh, S. E, & Allen, D. E. (Eds.). (2001). The power of problem-based learning . Sterling, VA: Stylus.
Grasha, A. F. (1996). Teaching with style: A practical guide to enhancing learning by understanding teaching and learning styles. Pittsburgh: Alliance Publishers.

Center for Innovation in Teaching & Learning

249 Armory Building 505 East Armory Avenue Champaign, IL 61820

217 333-1462

Email: [email protected]

Office of the Provost

Language →

Active learning
The PBL activity
The tutorial sessions
The PBL assessment
Curricular PBL
Implementation chalenges
Project Based Learning
For small groups
For large classes

Active learning for all

Implement group activities with your students and allow for thoughtful assessment by the student himself (self-assessment), by colleagues and by the tutor. establish aspects of learning to be evaluated, each with its own weight in the final grade. promote active learning facilitated by the openpbl platform..

The illustration below shows the OpenPBL platform being used by a student to evaluate a colleague's performance, observing different aspects defined by the tutor.

To best take advantage of this project, please read below for instructions on each major section of the site, accessible them by clicking the icon in the corner of the screen..

Fundamentals

Explore the fundamentals of authentic pbl (problem based learning), learn how to create rubrics to evaluate learning from different aspects, and learn how to differentiate pbl from project-based learning., get a sense of what active learning methodologies are and what pbl represents in this context. understand the purpose of this site, its features, and how you can get the most out of it..

Explore examples of using OpenPBL in different learning scenarios and see how you can use this framework to enhance your classes with active learning methodologies.

The openpbl platform can be applied to both small groups (pbl) and large classes, dividing the class into several groups for the same activity or different activities. the platform simplifies the management of grades given by different parties. see our examples..

Learn how to use the OpenPBL platform to form groups of students, submit activities with rubric-based assessment, self-assessment and peer assessment.

See how to use the openpbl technological tool even in a conventional educational context, with lectures, group or individual work, with or without self-assessment and peer assessment..

Understand how you can share an OpenPBL activity created in the app with other users. Also learn how to get an activity already created and shared into the app, so that you can change, mix, and reuse it together with your students.

Learn how to get openpbl activities ready to accelerate your use of the platform to promote active learning activities with your students..

This section presents the documentation about the application. The images are in English only, but the text is translated into your language. Thus, you have all the instructions for using the application to properly use its features.

Learn in an illustrated way about how authentic pbl (apbl) works, the challenges for using it in "conventional" education contexts, how to implement openpbl activities and how to use the application to make your life easier as a teacher., important, to reach as many people as possible, the content on this site is written in english and translated in real time by artificial inteligence into several other languages. because of this, it is important that you are aware of the following cases:, translation failures may occur, as the process is dynamic. some letters that should be capitalized may appear lowercase, wrong conjugation of tense verbs, and the misrepresentation of the meaning of some expressions.., we still don't have the history to look at translation quality for some languages, especially those of non-western origin. please let us know if you identify serious problems understanding the text..

OpenPBL components

Secondary menu

You can use the qr code below to join our telegram instant messenger channel. you can also access it directly at https://t.me/openpbl, or click on the qr code below, if you are already accessing it on your cell phone..

Don't have the Telegram app yet? Download and install.

Our Mission

Choosing Effective Assessments for PBL

By diversifying the manner of assessment in project-based learning, teachers can get a fuller picture of what students know.

Middle school student working on class engineering project

A common question that emerges as we design and implement problem- and project-based learning (PBL) is how do we effectively assess student learning?

While there are likely many answers to this question, all answers should be anchored to using

a range of assessments to capture student learning in different ways;
multiple assessments over time to ensure that we have multiple snapshots of student learning;
assessments to inform teaching and learning before, during, and after a unit of study; and
assessments across levels of complexity to identify student progress to ensure that quality learning is occurring.

When these ingredients are in place, teachers and students alike are better able to understand the quality of our teaching and the progress of student learning—and, as such, carve a path forward in the teaching and learning process.

Rigorous PBL by Design

Rigorous PBL is defined as an inquiry-based methodology that requires students to solve real-world problems by learning surface, deep, and transfer learning.

Rigor is defined as the equal intensity and integration of surface (“I know” ideas), deep (“I can relate” ideas), and transfer (“I can apply” ideas).

For instance, a student may know the names and specific properties of elements on the periodic table (surface), compare and contrast different elements based on their behavior as represented on the periodic table (deep), and apply their learning in an experiment when making a case for refining strategies for extracting oil (transfer).

In PBL we want to find a way to provide a range of these assessments across the unit of study. In rigorous PBL, we divide PBL into four phases:

Phase 1: Students encounter a real-world problem requiring them to determine the expectations of solving the problem.

Phase 2: Students build new knowledge and skills that are critical to meet curriculum and project expectations.

Phase 3: Students deepen their knowledge and skills by engaging early and often in classroom discussions to solidify understanding of core principles and practices.

Phase 4: Students learn transfer-level skills to solve the problem(s) presented during Phase 1.

Diversifying Our Assessments

We can assess student learning in a number of ways. Here are three approaches to assessment:

1. Stop and assess. One way is to interrupt students’ learning and provide them with an assessment. This could take the form of a test, quiz, or formal presentation. This is likely the most common form of assessment in classrooms.

2. Assess in action. Another option is assessing students as they engage in the learning, and we simply assess their performance without stopping them in the moment. This could take the form of watching students debate, solve a math problem during independent practice, or redraft a paper in class.

3. Have students construct the assessment. The final option is for students to construct the way in which they want to be assessed. In this process, students and teachers work together to identify the areas that need to be assessed, and students devise a way to show that practice. Suppose a student wanted to showcase their understanding of geometric angles: They may propose drawing and labeling a number of angles in front of the teacher.

When we design assessments, we want to ensure that they meet each level of surface, deep, and transfer. As such, it’s helpful to create a simple grid that lays out each level of complexity and each type of assessment.

Diversifying Our Use of Assessments With Students

We use assessments for a number of reasons. One is to assess the end of a learning sequence and report on a student’s learning. We call this a summative assessment. We also use formative assessment, or the collection of data along the course of a student’s learning journey to appraise current performance and make adjustments.

In both uses of assessment, we want to think about how we can support students in being actively involved in knowing their performance and planning next steps. Here are three ways to support students in being actively involved in the assessment process:

Pre-assessment. Ask students to jot down where they are in their learning, how they will perform in their learning, and what next steps they will likely need to take.
Post assessment. Ask students to evaluate any and all discrepancies between their pre-assessment and their current performance. Ask students to share out next steps they can take to improve.
Weekly/daily check-ins. Ask students to appraise their current performance using tools such as success criteria and current assessment information and discuss potential discrepancies in that appraisal. Work with students to develop next steps to improve or enhance their learning.

Putting It Together

How do we put all of this together to enhance student learning in the PBL environment? Let’s look at an example to solidify this new understanding of diversifying our assessment portfolio.

Phase 1: During the entry launch, we may use “stop and assess” as a way to identify student prior knowledge. We may “assess in action” by listening to students as they work on initial tasks or complete a need-to-know list.

Phase 2: Teachers and students are engaging in daily 2-to-3-minute check-ins on performance. Teachers are weaving a number of “stop and assess” and “assess in action” assessments at the surface level.

Phase 3: As students deepen their learning, teachers may capture summative assessment data of student surface and deep knowledge. This is a great opportunity for students to engage in a pre-/post-assessment reflection and determine next steps. In addition, this process may serve as fodder for students to develop a student-constructed assessment for showcasing all levels of learning during phase 4.

Phase 4: During this phase, the teacher is relying more on “assess in action” at the transfer level as well as student-constructed assessments to make judgments on student learning and to feed forward their thinking to students.

Project-Based Learning

This teaching guide explores the different types of project-based learning (PBL), its benefits, and tips for implementation in your classes.

Introduction

Project-based learning (PBL) involves students designing, developing, and constructing hands-on solutions to a problem. The educational value of PBL is that it aims to build students’ creative capacity to work through difficult or ill-structured problems, commonly in small teams. Typically, PBL takes students through the following phases or steps:

Identifying a problem
Agreeing on or devising a solution and potential solution path to the problem (i.e., how to achieve the solution)
Designing and developing a prototype of the solution
Refining the solution based on feedback from experts, instructors, and/or peers

Depending on the goals of the instructor, the size and scope of the project can vary greatly. Students may complete the four phases listed above over the course of many weeks, or even several times within a single class period.

Because of its focus on creativity and collaboration, PBL is enhanced when students experience opportunities to work across disciplines, employ technologies to make communication and product realization more efficient, or to design solutions to real-world problems posed by outside organizations or corporations. Projects do not need to be highly complex for students to benefit from PBL techniques. Often times, quick and simple projects are enough to provide students with valuable opportunities to make connections across content and practice.

Implementing project-based learning

As a pedagogical approach, PBL entails several key processes:

Defining problems in terms of given constraints or challenges
Generating multiple ideas to solve a given problem
Prototyping — often in rapid iteration — potential solutions to a problem
Testing the developed solution products or services in a “live” or authentic setting.

Defining the problem

PBL projects should start with students asking questions about a problem. What is the nature of problem they are trying to solve? What assumptions can they make about why the problem exists? Asking such questions will help students frame the problem in an appropriate context. If students are working on a real-world problem, it is important to consider how an end user will benefit from a solution.

Generating ideas

Next, students should be given the opportunity to brainstorm and discuss their ideas for solving the problem. The emphasis here is not to generate necessarily good ideas, but to generate many ideas. As such, brainstorming should encourage students to think wildly, but to stay focused on the problem. Setting guidelines for brainstorming sessions, such as giving everyone a chance to voice an idea, suspending judgement of others’ ideas, and building on the ideas of others will help make brainstorming a productive and generative exercise.

Prototyping solutions

Designing and prototyping a solution are typically the next phase of the PBL process. A prototype might take many forms: a mock-up, a storyboard, a role-play, or even an object made out of readily available materials such as pipe cleaners, popsicle sticks, and rubber bands. The purpose of prototyping is to expand upon the ideas generated during the brainstorming phase, and to quickly convey a how a solution to the problem might look and feel. Prototypes can often expose learners’ assumptions, as well as uncover unforeseen challenges that an end user of the solution might encounter. The focus on creating simple prototypes also means that students can iterate on their designs quickly and easily, incorporate feedback into their designs, and continually hone their problem solutions.

Students may then go about taking their prototypes to the next level of design: testing. Ideally, testing takes place in a “live” setting. Testing allows students to glean how well their products or services work in a real setting. The results of testing can provide students with important feedback on the their solutions, and generate new questions to consider. Did the solution work as planned? If not, what needs to be tweaked? In this way, testing engages students in critical thinking and reflection processes.

Unstructured versus structured projects

Research suggests that students learn more from working on unstructured or ill-structured projects than they do on highly structured ones. Unstructured projects are sometimes referred to as “open ended,” because they have no predictable or prescribed solution. In this way, open ended projects require students to consider assumptions and constraints, as well as to frame the problem they are trying to solve. Unstructured projects thus require students to do their own “structuring” of the problem at hand – a process that has been shown to enhance students’ abilities to transfer learning to other problem solving contexts.

Using Design Thinking in Higher Education (Educause)

Design Thinking and Innovation (GSM SI 839)

Project Based Learning through a Maker’s Lens (Edutopia)

You may also be interested in:

Case-based learning, game-based learning & gamification, creativity & innovation hub guide, udl learning community 2023, safety, curiosity, and the joy of learning, student engagement part 2: ensuring deep learning, assessing learning, jump-starting discussion using images (part 2).

Publications > Medical Science Educator > Volume 15: No. 2

Assessing Students During the Problem-Based Learning (PBL) process

Phyllis blumberg, ph.d..

University of the Sciences in Philadelphia Philadelphia, PA 19104 U.S.A.

This manuscript describes assessment processes that can be used during the normal conduct of Problem-based learning. The recent focus on assessment causes educators to consider different types of assessment methods designed for the purposes of improving students’ performance. Scoring rubrics and Likert scale assessment forms can be used in both formative and summative assessments. Formative assessment can be an on-going and integral part of the learning and improvement cycle. Summative assessment can be based upon patterns of performance toward the end of the course. Students’ written reports on their research on learning issues and integrative in-class discussions that follow after their research offer especially rich opportunities to assess students on many types of learning. Repeated observations of individuals working together throughout the in-class steps of the PBL process provide further opportunities for assessment. Appropriate assessors include faculty members, student peers and the students can assess themselves.

INTRODUCTION

Problem Based Learning (PBL) has been used in medical education for over forty years. With the recent focus on assessment, educators have become more attuned to how to assess students in PBL. Assessment should be designed for the purposes of improving students’ and instructors’ performance leading to further student improvement. 1 Feedback is an essential component of assessment. (Assessment can be contrasted with evaluation which is often done for the purposes of making pass- fail type judgments about students). 1 This paper will discuss different ways students can be assessed on their learning while engaged in the PBL process.

Current trends in assessment emphasize using different types of assessments including embedded and authentic assessments .1 Embedded assessment means that assessment of student progress and performance are integrated into the regular teaching/learning activities, whereas non-embedded assessments occur outside of the usual learning process. 2 Use of embedded assessments of actual performance takes little additional class time and inherently has content validity. 2 Embedded assessments assess student progress and performance during the regular PBL session. An embedded assessment tool documents what took place during the learning process for the purposes of assessment. These tools often reflect what the assessors observed while the students were engaged in their learning or performing a task. Non-embedded assessments often take the form of tests. Authentic assessments mimic what is actually done in practice by asking students to engage in real tasks. 1 For student these tasks may be simplified, but the assessment should involve real performance. In authentic assessments, students are asked to apply what they learned to real situations, which for medical students would involve doing something that is similar to what physicians actually do. Contrasting these types of assessments (embedded versus non-embedded with authentic versus non-authentic), one could form a 2x 2 matrix to see that assessments can be either 1) embedded and authentic (the most desired assessments), 2) embedded and non-authentic, 3) non-embedded and authentic 4) or non-embedded and non-authentic (e.g., a multiple choice test is generally both non-embedded and non-authentic).

PBL is an instructional method involving different types of active learning opportunities. Because the students are actively engaged in learning in the classroom and demonstrate their progress as they master the content, or problem solving skills, this method provides numerous opportunities for authentic, embedded assessment that do not take away time from instruction. Since PBL employs many different types of learning, what students can be assessed on can vary. The purposes of this manuscript are to 1.) describe some specific assessment tools that can be used in PBL, moved comma and 2.) describe ways to assess students during the PBL process using embedded and authentic assessments.

Most assessments relating to specific steps in the PBL process, that will be discussed later in this paper, use the same types of embedded assessment tools, i.e., scoring rubrics 3 and Likert scale assessment forms, because they are very time efficient and yield equitable grading. 4 Likert scales usually have 5 points ranging from 1= not at all to 5= consistently demonstrates this trait or very much. The middle category is neutral. Examples of Likert assessment scales are question 1, 2a and 3 on Table 1. A rubric is a written summary of the criteria and standards that will be applied to assess the student’s work. Rubrics transform informed professional judgment into numerical ratings. These ratings can be communicated rapidly and generally yield more consistent scoring. 3 It is usually constructed as a matrix with the criteria along the vertical axis and a brief description of the different standards of performance or levels of standards along the horizontal axis. An example of a scoring rubric is contained in Table 2. Faculty members, peers, and the learner him/herself can use rubrics and Likert scales to give students formative and summative assessments. Both Likert scales and rubrics are useful for conducting assessments based on in-class activities because they make the criteria clear and explicit in writing. 3 While this paper focuses on assessing students in PBL, it reflects current assessment trends using rubrics, Likert scales, embedded and authentic assessments. These tools and methods are used in primary, secondary and higher education with many different types of instruction. 1-3

While scoring rubrics and Likert scale assessment forms can be used with many types of assessments, the specific assessment criteria employed are different depending upon what is being assessed. Specific scoring rubrics or assessment forms can be constructed along the lines of the examples given later to assess most desired specified learning outcomes. Narrative comments based upon repeated observations of student performance can further support these rubric or Likert scale scores.

Students and instructors can use the critical incident type of observations5 to give specific examples of behaviors demonstrated during the PBL discussions. They can record examples of whatever is being assessed. A critical incident documents only those events that critical, influential, or decisive in the student’s developing learning abilities. Usually there are only a few critical incidents noted for each student throughout a course so it is not an onerous job. While giving feedback, if someone notes an especially excellent or poor performance, the instructor should briefly record the incident so that it can be used as part of narrative comments on the student’s performance written at the end of the course. This record should consist of a few sentences to jog the faculty member’s memory.

Assessment considerations within the PBL process The classical version of PBL is predicated upon the principle that discussion of a problem or case stimulates learning. All material is discussed twice in the PBL discussions, once without prior preparation and then again after researching questions raised (called learning issues). This iterative process is shown in Figure 1. Discussion of what is known, what is unknown, and raising questions can occur simultaneously, not necessarily sequentially as shown in the figure. Because it is assumed that most readers are somewhat familiar with this PBL process I will comment only on specific ways of implementing the PBL process that can foster assessment of student learning. Many different types of assessments can occur through observations of the various steps of the PBL process. During each step or all of the steps together within the PBL process instructors and students can assess more than one type of learning simultaneously.

Many possible assessments are embedded within the regular PBL activities as these assessment flow from repeated observations of the students’ regular performance in the groups. It is valid to use faculty members, peers and oneself as assessors. Peers and instructors complete assessment forms asking for evidence or absence of evidence of specific outcomes in their fellow students. The key is to sample enough observations without overwhelming everyone with the assessment process. Students and faculty members can rotate in and out of the observer-assessment role.

Assessments during specific steps within the PBL process Generation of learning issues. Throughout the course of the discussion, the students naturally raise questions that they would like clarified. Toward the end of the session, the groups refer to their written list of questions to generate and refine learning issues, or topics that the students need to research on their own outside of class for further understanding. The students refine the questions, group and classify then to make the job of searching for answers more manageable. The questions can either require additional information on the specific problem or take the form of more general knowledge questions, which are preferred for promoting student learning. Questions on the specific problem can be transformed into general knowledge questions. A researchable question based upon a specific question relating to the patient would be, “what type of laboratory data would be indicated for patients like this and why?” This type of question can also require students to consider how strong the evidence is to support these laboratory tests in terms of efficacy and cost effectiveness.

The Association of College and Research Libraries defined five information literacy standards for higher education including: the determination of information needs, the acquisition of information effectively and efficiently, critical assessment of information and its sources, the incorporation of selected information into one’s knowledge base, and the use of information legally and ethically. 6 The step of generation of learning issues is ideal for evaluating student’s ability to define an information need.

Evidence-based decision-making has developed criteria to define a searchable question. 7 These criteria should guide the generation of learning issues. First the problem needs to be clearly identified. This problem should be structured as a specific question. The words of the question should be chosen to facilitate the search for information. A suggested format for defining a searchable question in medicine is called the PICO question. PICO is an acronym for P = problem or population to study, I = intervention, C = comparison, and O = outcome. 8 An example of a good PICO question would be: For persons with dementia, will the use of environmental modifications decrease disruptive behaviors? The students’ ability to define an information need can be directly assessed by asking the students individually to write their questions for further study, including asking them to formulate them as PICO questions, and collecting these questions. A scoring rubric would be helpful to efficiently assess these PICO questions and also a way to offer constructive feedback. Each of the letters in the PICO acronym can become a separate criterion for the rubric. Then the faculty members develop 3-4 levels or standards of each criterion giving explicit differences between each level. The levels would reflect the ease of obtaining useful and appropriate information.

Independent study, development of briefs. In between PBL group sessions the students research their own learning issues or questions (see the bottom, not shaded parts of Figure 1). Students then prepare a short (at most one to two pages including graphs or figures, and can be written in bullet points or outline) summary of the information they acquired to answer the learning issue question, and list their information sources, called briefs. 9 This written summary is not required in most PBL groups, but my experience indicates that it greatly increases the level of discussion when the groups reconvene. Without the summaries prepared by the students, they come to class with large piles of photocopied or printed material that they refer to repeatedly in class and spend a great deal of time searching for information. Forcing people to abstract the essential ideas helps them to synthesize their knowledge, fosters reflection on their learning, and serves as a check as to whether or not they indeed did address the learning issues raised. Ideally these summaries should be sent electronically to the other students in their group and the instructor in charge of the course in advance of the next class session, and the students should have read the summaries generated by their peers prior to coming to class. When students compile all of the summaries of the learning issues, they develop a resource for further use that contains much information and appropriate resources.

Attached to each brief should be a brief tracking sheet, as shown in Appendix A, which can be used for documentation and for self-assessment on the acquisition of information effectively and efficiently. It is a good idea for the faculty members to develop a standard form and distribute an electronic copy of the form that the students can download and always attach to their briefs. The student preparing the brief should rate how useful their search strategy was for addressing the learning issue. For example they might write that this was too inclusive of a search strategy and they got several thousand possible citations to consult. This self-assessment can encourage students to seek help to define more effective searches. Librarians are very useful for helping students to formulate search strategies and identifying appropriate places to find information. A universal objective of PBL education is that the students become aware of the most appropriate resources to find the answers to different types of questions in addition to learning the content itself. Becoming aware of appropriate resources should help students to learn other material in the future. The use of tracking sheets helps students to identify appropriate resources for different types of questions.

Assessments from briefs. Instructors and peers can assess the briefs quickly using a standard, generic form such as found in Table 1. The review of the briefs should be done quickly to determine if they are satisfactory overall. Obtaining some feedback on each brief provides opportunities for immediate and continued improvement on the skills assessed. Students do not have to complete a form for every other student in his/her group every time, but each student should receive some feedback for most of the briefs submitted. Faculty members need to monitor the briefs more closely in the beginning of a PBL program or in a new class. As students progress, this type of formative assessment can be done less frequently. Faculty members and peers can use the same assessment forms, used to provide feedback earlier, for summative assessment on specific learning outcomes by reviewing the last few briefs and their tracking sheets the students develop for a course.

A review of the briefs summarizing the student research on learning issues and their tracking sheets are excellent ways to assess students on several information literacy standards6 including acquisition of information effectively and efficiently, critical evaluation of information and its sources, and the use of information legally and ethically. When faculty members or librarians provide formative feedback on the type of searches the students performed, the students can be encouraged to use more than lay search engines on the Internet.

Some topics will lend themselves better to assess of other specific learning outcomes than others. For example, a question that requires a review of different types of literature can be assessed for integrating different ideas or perspectives; whereas the identification of incidence or prevalence rates of a disease might only assess information literacy skills. Since information obtained from print or electronic sources is not directly related to the patient, students can usually be assessed on their ability to apply theoretical knowledge to the specific patient in the problem. Instructors should use their judgment in deciding if a topic can be assessed on specific types of learning outcomes in advance of reading the brief. They or the students might keep a record of what specific learning outcomes individual students have been assessed on and encourage students to take on different kinds of learning issue research throughout the course. Faculty members can also monitor that individual students are not always selecting the more factual issues to research (which are easy to research in textbooks) and encouraging them to also select more integrative learning issues that require more extensive researching or reading different perspectives.

In my opinion, the briefs and the patterns observed from overall group discussions are the most important assessment tools in PBL. Because briefs are written, they can become a portfolio of the students’ work throughout a semester. Further, it is easy to demonstrate to the students their progress over time. If faculty members rotate among groups without spending much time with each group, assessing the briefs can be an excellent way to determine what the individual students are learning and how well each group is adhering to the intended objectives. Briefs are also one of the few individual components of what is normally a group or collaborative experience.

Second discussion of the case. During the second iteration on the material, students reconvene to discuss material after researching their learning issues. During this discussion they should critically assess what they have learned and integrate multi-disciplinary content knowledge while addressing its application to the problem. On the second pass through the material, they follow a similar process of discussing the case only this time they are armed with much more information (see the shaded box in Figure 1). The explanation of new knowledge gained from learning issue research should emerge naturally from the case discussion. If more questions arise, and that often happens, they become learning issues for further study. Thus, the process is iterative. Students prefer to simply report on what they researched in little sequential monologues without an integrated dialogue on the problem; this practice should be avoided to allow for a more meaningful discussion. One mechanism that fosters a rich, multi-disciplinary discussion is to ask the students collectively to construct a concept map10 summarizing what they know about the problem or case during the second pass with this material. Concept maps graphically illustrate the integration of all they know about a problem, showing relationships and hierarchies. 10 The construction of a concept map during the second iteration of the problem discussion integrating all of the group’s collective knowledge, skills in inquiry, analysis and integration can be assessed by the instructors and the students using a rubric such as found in Table 2. 11 Instructors can use concept maps to assess the organization of knowledge into hierarchies, the associations and integrations among separate details. Generally the group would receive a group grade for their concept map. Even if instructors are not present for the discussion, they can assess concepts maps if they are handed in. Students can use computer programs such as “Inspiration” 12 to produce them easily and neatly. “Inspiration” is a computer software package that allows students to construct graphic organizers. Students can use the software to develop ideas and organize their thinking. It can be used to brainstorm, plan, organize, outline, diagram and write. The software uses standard conventions or symbols to show relationships, hierarchies, and consequences and, therefore, is very useful for constructing concept maps. Concept maps made on “Inspiration” 12 are easier to follow than those made by hand or not using the standard notation of the software. Therefore, they are easier to assess. If they are constructed on a white board that has a computer camera attached to it often known as Smart Boards or Smart Sympodium, they can be electronically saved to a computer file.

Feedback phase . Feedback should occur at the end of each session, and groups need to reserve time for this formative assessment to occur before the group recesses for the day. Instructors often need to model how to give constructive feedback in a supportive way. Students might be asked to comment orally or use a written form to provide this feedback. A suggested format might be to give several statements or questions reflecting performance and ask the students to use a Likert scale to rate their assessments and write comments supporting the numbers they assigned.

Feedback on the discussion of the first iteration of the problem may concentrate on the quality of their discussions, their use of evidence-based decision-making, their ability to formulate learning issues, and the group functioning. Appropriate questions for formative feedback based upon the second iteration of the problem might be: 1) How much did you learn from this PBL discussion?; 2) How well did the PBL case discussion address the learning issues from the case?; 3) How well will you be able to apply what you learned from the case discussion to patient care?; and 4) an open ended statement such as please provide suggestions to improve the PBL discussions. The first three questions can use Likert scales and ask for comments. Such feedback assesses students’ ability to become self-directed learners. Different people can be the assessors but peers and self assessments are essential to obtaining a fuller picture.

On a daily or weekly basis, the observations of student performance in the PBL activities or reviews of briefs should offer formative feedback to help students to improve. These types of assessments offer insights into how well the students are learning and are consistent with current accreditation standards such as those in the LCME13 standards. These same assessment forms taken together can be used to look at repeated observations or reviews to determine trends and patterns. Such trends and patterns can become the basis for making summative evaluations. A summary of the narrative comments made throughout the semester should be included in the summative evaluations also. ED-32 of LCME 13 states that, “Narrative descriptions of student performance and of non-cognitive achievement should be included as part of evaluations in all required courses and clerkships where teacher-student interaction permits this form of assessment”.

Faculty members who are accustomed to giving objective tests may be concerned with the reliability and validity of the kinds of measures discussed in this manuscript. The assessment literature indicates that using scoring rubrics or Likert scales yield reliable and valid measurements if the criteria are appropriate and the description of the standard levels of performance are sufficiently grounded in real samples of different performance quality. 1-3 Repeated observations of students can indicate patterns of performance and therefore, are more reliable assessments. Rubrics and Likert scales can also be used to chart student progress over time. 4 Observations from a single class are usually not indicative of true abilities. Rubrics and Likert scales provide equitable measures because the same criteria are used for all students. This perception of equitable grading is especially important to minority students. 4 To further improve the reliability of these assessments, trained assessors who observe a sample of PBL sessions can do these observations. Training assessors, including peer assessors, involves practicing with specific feedback what to observe and how to consistently complete the rubrics or checklists. Since many of the assessments are both embedded and authentic tools, training students to be reliable assessors also serves to improve their learning and performance. The training process itself is instructive for the students as it can show them an excellent level of performance. During the training, students come to understand what is expected of them before they are assessed. Triangulation of data from different sources and collected over time results in valid measures of the student’s learning. In assessment, triangulation, “refers to the attempt to get a fix on a phenomenon or (an interpretation) by approaching it via several independent routes. In short, you avoid dependence on the validity of any one source by the process of triangulation.” 14 Triangulation supports a finding by showing that independent measures agree. 5,14 Triangulation can occur from different data sources, different times, different methods and different types of data. 5 In addition many of the assessments discussed here are authentic measures and as authentic assessments they mimic professional practice, therefore, they should have good predictive validity to future clinical work. These embedded and authentic measurements also have high face validity because they are based upon actual performance.

CONCLUSIONS

This manuscript describes the assessment processes for medical students that can be used during the normal conduct of PBL. Thus, assessment can be part of the learning process and not seen as taking away time from instruction. Formative assessment can be an on-going and integral part of the learning and improvement cycle. Summative assessment can be based upon patterns of performance toward the end of the course. Because students demonstrate the continual mastery of their learning in small groups in PBL, many opportunities for embedded, authentic assessment of student learning readily exist. Specifically, this manuscript describes the use of scoring rubrics, Likert scale assessment forms, and reflective comments in both formative and summative assessments. Appropriate assessors include faculty members, student peers and the student can assess themselves. The summaries of independent research that the students do to address their learning issues, called briefs,9 are very useful for assessing students on their a) developing information literacy skills, b) synthesis of their knowledge, c) application of knowledge to clinical problems, and d) their written communication skills. The briefs can become a portfolio of the students’ work that can be very useful for documenting learning outcomes for LCME13 or other accreditation agencies. When students develop concept maps 10 on the second iteration of the problem, they collaboratively integrate all that they know about a problem These concept maps can be used to assess a) how accurately they have mastered the knowledge, b) how appropriate is their synthesis of their older and newly acquired knowledge, and c) the organization and integration of their knowledge showing linkages among concepts, causes, effects and implications. Repeated observations of individuals working together throughout the in-class steps of the PBL process provide opportunities for assessment of a) knowledge; b) the integration of various theories, explanations, and multi-disciplinary perspectives; c) the individual’s ability to work on teams or groups; and d) professional behaviors. Thus, PBL offers many rich and varied authentic and embedded assessment opportunities.

NOTE: Please refer to the complete PDF file for the referenced Tables and Figures

IAMSE on Twitter

Tweets by @iamse

What is inquiry based learning, and how can it open a world of possibilities for children?

by Freya Lucas

Shape Copy 5 Created with Sketch. Quality

Inquiry based learning is an educational approach that focuses on children being investigators and problem solvers. Central to the approved learning frameworks, inquiry based learning encourages active involvement in learning, builds children’s understanding of concepts, and builds the creative thinking and inquiry processes that are core aspects of lifelong learning.

Active involvement in learning builds children’s understanding of concepts and the creative thinking and inquiry processes that are necessary for lifelong learning.

For many adults, inquiry based learning can be thought of as “learning in reverse”, because it presents a range of scenarios, questions and problems for children to investigate, rather than presenting information or knowledge up front.

‘Problems’ which require critical and creative thinking are particularly suited to inquiry based learning, allowing children to develop their abilities to ask questions, design investigations, interpret evidence, form explanations and arguments, and communicate what they have learnt.

In essence, inquiry based learning in an early childhood context is a child-centred approach to pedagogy, which encourages children to ask questions and investigate answers through real-world experiences.

Children who engage with inquiry based learning are typically curious and enthusiastic participants in their learning, and develop a range of skills and processes such as problem solving, enquiry, experimentation, hypothesising, researching and investigating.

What is the role of the educator?

For educators, particularly those who may have had experiences in their own learning which were highly controlled or managed by teachers, inquiry based learning can require a shift in perspective.

In an inquiry based learning context, the role of the educator is redefined, shifting from ‘one who holds the knowledge’ to ‘creating a positive environment in which children are guided to discover answers to questions which interest them.’

How do educators initiate inquiry based learning?

Typically inquiry based learning, in its most authentic form, will begin with a question or an idea.

In an early childhood context, the possibilities are endless, but may include questions like:

Where do birds go at night? Why can’t we hear them?
How are clouds made?
Who chose what letters mean and how they sound?
Why are baby birds grown in eggs? Why don’t people grow in eggs?

Once a question has been posed, the educator/s can support the children to hypothesise and wonder, offer them opportunities to be autonomous problem solvers, organise for learning resources or experiences which support children’s thinking.

Essentially, inquiry based learning is an invitation for children to actively engage with an idea or topic, and join in a journey to a solution or outcome. Often many other possibilities, leads, or directions will arise from the inquiry, which may then branch off in a number of different directions.

What do children do on this journey?

During an inquiry based learning journey children have many different ways of participating, including:

Asking questions and sharing their ideas
Using scientific thinking elements such as researching, hypothesis testing, making predictions, experimenting, investigating and recording
Using higher order thinking skills like decision making, planning and problem solving
Reflecting on their learning and sharing ideas
Thinking about, and making suggestions on ‘what happens next’.

Resources for learning more

There are a number of resources available to support early childhood education and care (ECEC) professionals to learn more about inquiry based learning including:

Inquiry learning
Inquiry learning is deep learning
Understanding play based and inquiry learning
Inquiry learning in play spaces
Establishing inquiry based learning environments

Download The Sector's new App!

ECEC news, jobs, events and more anytime, anywhere.

Latest in Quality

Victoria opens Early Childhood Provisionally Registered Teacher Grants Program

2024-05-01 09:26:40

Fill 11 Copy 7 Created with Sketch. Provider

Ballina Byron FDC shares experiences of refreshed A&R process in NSW

2024-05-01 09:23:42

Cire connects with nature through Bush Kinder program expansion

2024-04-30 10:12:02

Child attending Mirrabooka ECEC service involved in needle prick incident

2024-04-25 00:11:04

Related News

Children’s sense of agency fostered through self directed play

Education for Sustainability: the role of inquiry based learning for children

Editorial Team
Focus and Scope
Author Guidelines
Publication Ethics
Open Access Policy
Peer Review Process
Online Submission
Author(s) Fee

CERTIFICATE

Announcements

PENERAPAN MODEL PEMBELAJARAN PROBLEM BASED LEARNING TERHADAP KEMAMPUAN BERPIKIR SISWA PADA MATERI PESAWAT SEDERHANA

Tujuan penelitian yaitu mendeskripsikan Penerapan Model Pembelajaran Problem Based Learning terhadap Kemampuan Berpikir siswa pada materi Pesawat Sederhana. Penelitian dilakukan di SMP Negeri 2 Mootilango. Kemampuan peserta didik diukur menggunakan Indikator Taksonomi SOLO. Penelitian ini menggunakan metode kuantitatif, eksprerimen semu. Teknik pengumpulan data yang digunakan adalah lembar validasi dan lembar tes. Hasil penelitian yang didapatkan Kemampuan Berpikir peserta didik setelah diajarkan menggunakan Model Pembelajaran Problem Based Learning pada Materi Pesawat Sederhana menyatakan: Kemampuan berpikir peserta didik pada level Unistruktural dengan persentase 58,75%, termasuk dalam kriteria baik. Kemampuan berpikir peserta didik pada level Multistruktural dengan persentase 40%, termasuk dalam kriteria cukup baik. Kemampuan berpikir peserta didik di level Relasional dengan persentase 1,25%, termasuk dalam kriteria kurang. Dari penelitian yang telah dilakukan dapat disimpulkan kemampuan berpikir peserta didik sebagian besar masih berada pada tingkat rendah, kemudian diikuti peserta didik yang berkemampuan sedang, peserta didik yang berkemampuan tinggi.

Budimansyah, 2012. Dimensi – dimensi Praktik Pendidikan Karakter. Bandung, Widya Aksara

Davidi E. I. Noviyanti dan Berjamai Govan Star, 2020. Kajian faktor – faktor penghambar keterampilan berpikir siswa pada pelajaran Bahasa Indonesia. Jurnal literasi Pendidikan. Santu Paulus Ruteng, Flores. 1(1). ISSN 2721-0295

Dharma, dkk. 2022. Faktor penyebab dan alternatif solusi rendahnya kemampuan berpikir pada siswa. Jurnal Ilmiah Pendidikan Profesi Guru. Universitas Triatma Mulya Indonesia. Volume 5 Nomor 3. ISSN 2621-5713

Hartanti, Djudin, Mursyid, 2021. Analisis Tingkat Berpikir Kritis dalam menyelesaikan Gerak Parabola menggunakan Taksonomi Structure of The Observed Learning Outcomes (SOLO) siswa kelas X MIA SMAN 1 Sungai Raya. Jurnal Inovasi Penelitian dan Pembelajaran Fisika. Universitas Tanjungpura, Kota Pontianak. 2(1), ISSN 2746-3796

Herliani, 2016. Penggunaan taksonomi SOLO (Structure of Observed Learning Outcomes) untuk meningkatkan keterampilan berpikir siswa pada Mata Pelajaran Biolgi SMA. Universitas Mulawarman. 13(1) (232-236) ISSN : 2528-5742

Helmon Arnoldus, 2018. Pengaruh Problem Based Learning (PBL) terhadap kemampuan berpikir kritis. Jurnal Inovasi Pendidikan Dasar, 2(1)

Kristin Firosalia dan Utama K. Hardiana. 2020. Meta-Analisis Pengaruh Model Pembelajaran Problem Based Learning terhadap kemampuan berpikir IPA di Sekolah Dasar. Jurnal BASICEDU. Universitas Kristen Satya Wacana, Jawa Tengah. 4(4) (899 – 898) ISSN 2580-1147

Latjompoh Masra, 2018. Validitas Perangkat Pembelajaran Berorientasi Keterampilan proses sains untuk melatih kemampuan berpikir dan Menanamkan karakter bagi siswa SMA kota Gorontalo. Prosiding Seminar Nasional VI Hayati. (470 – 477)

Rosid Muhammad, 2019. Analisis Kemampuan Berpikir kritis siswa Pada Pembelajaran Reaksi redoks dan elektrokimia menggunakan Model Pembelajaran Problem Based Learning, Jurnal Pendidikan Fisika, 8(4) (303-308). Banyuwangi

Setiawan Dewa G. Eka, Supartin, Hidjrawaty Tantu, dkk. 2022. Pengaruh Penerapan Model Pembelajaran Daring terhadap hasil belajar siswa. Educatio: Jurnal Ilmu Pendidikan. Program Studi Pendidikan Fisika, FMIPA, Universitas Negeri Gorontalo. 17(1) (12-20)

Sitompul Nova, 2021. Pengaruh Model Pembelajaran Problem Based Learning Terhadap Peningkatan Kemampuan Berpikir Kritis Matematis Siswa SMP Kelas IX. Jurnal Pendidikan Matematika. Volume 4(1). ISSN 2620-8067

Susanto, 2013. Teori belajar dan pembelajaran. Jakarta:Kencana Prenada Media Group

Umardiyah F dan Nasrulloh Farid. 2021. Pemberian SCAFFOLDING berdasarkan pelevelan taksonomi SOLO siswa kategori Unistruktural dalam menyelesaikan soal jarak dimensi tiga. EDU-MAT Jurnal Pendidkan Matematika. Jombang, Indonesia. 9(1). (24 – 47) ISSN 2597-9051

Widyawati, dkk. 2018. Analisis kesalahan siswa dalam memecahkan masalah lingkaran berdasarkan Taksonomi SOLO pada kelas VIII. Jurnal Pendidikan Matematika dan Sains. STKIP PGRI Tulungagung. 6(1)

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
My Account Login
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 30 April 2024

Reconstruction of unstable heavy particles using deep symmetry-preserving attention networks

Michael James Fenton ORCID: orcid.org/0009-0007-2746-3813 1 na1 ,
Alexander Shmakov 2 na1 ,
Hideki Okawa ORCID: orcid.org/0000-0002-2548-6567 3 ,
Yuji Li 4 ,
Ko-Yang Hsiao 5 ,
Shih-Chieh Hsu ORCID: orcid.org/0000-0001-6214-8500 6 ,
Daniel Whiteson ORCID: orcid.org/0000-0002-2005-3113 1 &
Pierre Baldi ORCID: orcid.org/0000-0001-8752-4664 2

Communications Physics volume 7 , Article number: 139 ( 2024 ) Cite this article

Metrics details

Experimental particle physics
Phenomenology

Reconstructing unstable heavy particles requires sophisticated techniques to sift through the large number of possible permutations for assignment of detector objects to the underlying partons. An approach based on a generalized attention mechanism, symmetry preserving attention networks (SPA-NET), has been previously applied to top quark pair decays at the Large Hadron Collider which produce only hadronic jets. Here we extend the SPA-NET architecture to consider multiple input object types, such as leptons, as well as global event features, such as the missing transverse momentum. In addition, we provide regression and classification outputs to supplement the parton assignment. We explore the performance of the extended capability of SPA-NET in the context of semi-leptonic decays of top quark pairs as well as top quark pairs produced in association with a Higgs boson. We find significant improvements in the power of three representative studies: a search for \(t\bar{t}H\) , a measurement of the top quark mass, and a search for a heavy \({Z}^{{\prime} }\) decaying to top quark pairs. We present ablation studies to provide insight on what the network has learned in each case.

Introduction

Event reconstruction is a crucial problem at the Large Hadron Collider (LHC), where heavy, unstable particles such as top quarks, Higgs bosons, and electroweak W and Z bosons decay before being directly measured by the detectors. Measuring the properties of these particles requires reconstructing their four-momenta from their immediate decay products, which we refer to as partons . Since many partons leave indistinguishable signatures in detectors, a central difficulty is assigning the observed detector objects to each parton. As the number of partons grows, the combinatorics of the problem becomes overwhelming, and the inability to efficiently select the correct assignment dilutes valuable information.

Previously, methods such as χ 2 fits 1 or kinematic likelihoods 2 have provided analytic approaches for performing this task. These approaches are limited, however, by the requirement of exhaustively building each possible permutation of the event and by the limited amount of kinematic information that can be incorporated. Particularly at high-energy hadron colliders such as the LHC, events often contain many extra objects from additional activity as well as the particles originating from the hard scattering event, which can cause the performance of permutation-based methods to degrade substantially.

In recent years, modern machine learning tools such as graph neural networks and transformers 3 have been broadly applied to many problems in high-energy physics. For example, the problem of identifying the origin of single, large-radius jets has been closely studied 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 using such techniques. Some of these have incorporated symmetry considerations 11 , 12 , 14 to aid performance. Implementations of such strategies to event-level reconstruction have been limited so far to single object permutation assignment 15 , 16 , 17 or direct regression 18 .

This work presents a complete machine learning approach to multi-object event reconstruction and kinematic regression at the LHC, named SPA-NET owing to its use of a symmetry-preserving attention mechanism, designed to incorporate all of the symmetries present in the problem. It was first introduced 15 , 16 in the context of reconstruction of the all-hadronic final state in which only one type of object is present. In this work, we extend and complete the method by generalizing to arbitrary numbers of object types, as well as adding multiple capabilities that can aid the application of SPA-NET in LHC data analysis, including signal and background discrimination, kinematic regression, and auxiliary outputs to separate different kinds of events.

To demonstrate the new capacity of the technique, we study its performance in final states containing a lepton and a neutrino. The method is compared to existing baseline approaches and demonstrated to provide significant improvements in three flagship LHC physics measurements: \(t\bar{t}H\) cross-section, top quark mass, and a search for a hypothetical \({Z}^{{\prime} }\) boson decaying to top quark pairs. These examples demonstrate various additional features, such as kinematic regression and signal versus background discrimination. The method can be applied to any final state at the LHC or other particle collider experiments, and may be applicable to other set assignment tasks in other scientific fields.

SPA-NET extensions

We present several improvements to the base SPA-NET architecture 15 , 16 to tackle the additional challenges inherent to events containing multiple reconstructed object classes and to allow for a greater variety of outputs for an array of potential auxiliary tasks. These modifications allow SPA-NET to be applied to essentially any topology and allow for the analysis of many additional aspects of events beyond the original jet-parton assignment task.

Base SPA-NET overview

For context, we first provide a brief overview of the original SPA-NET architecture 15 , 16 . These components are those which are presented with black boxes and lines in Fig. 1 . The jets, represented by their kinematics, are first embedded into a high dimensional latent space and subsequently processed by a central transformer encoder 3 with the goal of providing contextual information to the jets. We note that the architecture of this transformer encoder follows the original definition 3 , with one major exception: we omit the positional encoding to prevent introducing ordering over our input. As the jets are presented as a set of momentum vectors, with no obvious order, we want the network to remain permutation equivariant with respect to the input order. We replicate the architecture for the particle transformers, now applying individually trained transformers for every resonance particle in our event.

The diagram flows left to right, with inputs denoted by \({{{{{{{{\mathcal{E}}}}}}}}}_{i}\) , assignment outputs denoted by P j , regression outputs η ν and \(m_{t\bar{t}}\) , and classification output \({{{{{{{\mathcal{S/B}}}}}}}}\) . Black blocks show components common to our previous works 15 , 16 , with new components shown in blue.

Finally, to extract the joint distribution over jets for each resonance particle, we apply a symmetric tensor attention layer defined in Section 3 of our previous work 16 . This layer applies a generalized form of attention, modified by a symmetry group over assignments, to produce a symmetric joint distribution over jets describing the likelihood of assigning said jets to the resonance particle. This split architecture, with individual branches for every resonance particle, allows us to avoid computing a full permutation over all possible assignments and reduced the runtime from combinatorial w.r.t the number of jets, \({{{{{{{\mathcal{O}}}}}}}}(N!)\) , to \({{{{{{{\mathcal{O}}}}}}}}({N}^{{k}_{p}})\) where k p is the number of daughter particles produced by a resonance particle.

Input observables

While the original SPA-NET 15 , 16 studies concentrated on examples where all objects have hadronic origins, we focus here on the challenges of semi-leptonic topologies. These events contain several different reconstructed objects, including the typical hadronic jets as well as leptons and missing transverse momentum ( \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) ) typically associated with neutrinos. Unlike jets or leptons, this \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) is a global observable, and its multiplicity does not vary event by event.

We accommodate these additional inputs by training individual position-independent embeddings for each class of input. This allows the network to adjust to the various distributions for each input type, and allows us to define sets of features specific to each type of object. We parameterize jets using the \(\{M,{p}_{{{{{{{{\rm{T}}}}}}}}},\eta ,\sin \phi ,\cos \phi ,b{{{{{{{\rm{-tag}}}}}}}}\}\) representation, where M is the jet mass, p T is the jet momentum transverse to the incoming proton beams, and ϕ is the azimuthal angle around the detector, represented by its trigonometric components to avoid the boundary condition at ϕ = ± π . η is the pseudo-rapidity 19 of the jet, the standard measure of the polar angle between the incoming proton beam and the jet commonly used in particle physics due to its Lorentz-invariant quantities. Leptons are similarly represented using \(\{M,{p}_{{{{{{{{\rm{T}}}}}}}}},\eta ,\sin \phi ,\cos \phi ,{{{{{{{\rm{flavor}}}}}}}}\}\) where flavor is 0 for electrons and 1 for muons. Finally, \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) is represented using two scalar values, the magnitude and azimuthal angle, and is treated as an always-present jet or lepton. The individual embedding layers map these disparate objects with different features into a unified latent space which may be processed by the central transformer.

The global inputs, such as \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) , need to be treated differently than the jets and leptons, as they do not have associated parton assignments. Therefore, after computing the central transformer, we do not include the extra global \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) vector in the particle transformers. This allows the transformer to freely share the \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) information with the other objects during the central transformer step while preventing it from being chosen as a reconstruction object for jet-parton assignment.

Secondary outputs

Beyond jet-parton assignment, we are interested in reconstruction of further observables, such as the unknown neutrino η , or differentiation of signal events from background. These observables are defined at event level, and are independent of the jet multiplicity, so we must construct a way of summarizing the entire event in a single vector to predict these values.

To accomplish this, we add additional output heads to the central transformer, presented with blue boxes and lines on the right in Fig. 1 , which are trained end-to-end simultaneously with the base reconstruction task. We extract an event embedding from the central transformer by including a learnable event vector in the inputs to the transformer. We append this learned event vector \({{{{{{{{\mathcal{E}}}}}}}}}_{E}\in {{\mathbb{R}}}^{D}\) to the list of embedded input vectors: \({{{{{{{\mathcal{E}}}}}}}}=\{{{{{{{{{\mathcal{E}}}}}}}}}_{1},{{{{{{{{\mathcal{E}}}}}}}}}_{2},\ldots ,{{{{{{{{\mathcal{E}}}}}}}}}_{n},{{{{{{{{\mathcal{E}}}}}}}}}_{L},{{{{{{{{\mathcal{E}}}}}}}}}_{G},{{{{{{{{\mathcal{E}}}}}}}}}_{E}\}\) prior to the central transformer (Fig. 1 ). This allows the central transformer to process this event vector using all of the information available in the observables.

We extract the encoded event vector after the central transformer and treat it as a latent summary representation of the entire event z E . We can then feed these latent features into simple feed-forward neural networks to perform signal vs background classification, \({{{{{{{\mathcal{S/B}}}}}}}}({z}_{E})\) , neutrino kinematics regression, η ν ( z E ), or any other downstream tasks. These tasks may additionally be learned after the main SPA-NET training as z E may be computed used a fixed set of SPA-NET weights and then used for other downstream tasks without altering the original SPA-NET.

These additional feed-forward networks are trained using their respective loss, either categorical log-likelihood or mean squared error (MSE). These auxiliary losses are simply added to the total SPA-NET loss, weighted by their respective hyperparameter α i . With the parton reconstruction loss, \({{{{{{{{\mathcal{L}}}}}}}}}_{{{{{{{{\rm{reconstruction}}}}}}}}}\) defined as the masked minimum permutation loss from Equation 6 of our previous work 16 , the SPA-NET loss becomes:

Particle detector

In our previous work 16 , we introduced the ability to reconstruct partial events by splitting the reconstruction task based on the event topology. This is a powerful technique that is particularly useful in complex events, where it is very likely that at least one of the partons will not have a corresponding detector object.

However, the assignment outputs are trained only on examples in which the event contains all detector objects necessary for a correct parton assignment. We refer to the reconstruction target particles in these examples as reconstructable. We must train this way because only reconstructable particles have truth-labeled detector objects, which are required for training, and we ignore non-reconstructable particles via the masked loss defined in Equation 6 of our previous work 16 . As a result of this training procedure, the SPA-NET assignment probability P a only represents a conditional assignment distribution over jet indices j i for each particle p given that the particle is reconstructable:

We use P ( p reconstructable) = P ( p ) and P ( p not reconstructable) = P ( ¬ p ) for conciseness. To construct an unconditional assignment distribution, we need to additionally estimate the probability that a given particle is reconstructable in the event, P d . This additional distribution may be used to produce a pseudo-marginal probability for the assignment. While \({P}_{a}(\,{j}_{1},{j}_{2},\ldots ,{j}_{{k}_{p}}| \neg p)=0\) is not a valid distribution, and therefore this marginal probability is ill-defined, we may still use this pseudo-marginal probability

as an overall measurement of the assignment confidence of the network.

We aim to estimate this reconstruction probability, P d ( p ), with an additional output head of SPA-NET. We will refer to this output as the detection output, because it is trained to detect whether or not a particle is reconstructable in the event. We train this detection output in a similar manner as the classification outputs but at the particle level instead of the event level. That is, we extract a summary particle vector from each of the particle transformer encoders using the same method as the event summary vector from the central transformer. We then feed these particle vectors into a feed-forward binary classification network to produce a Bernoulli probability for each particle. We have to also take into account the potential event-level symmetries in a similar manner to the assignment reconstruction loss from Equation 6 of our previous work 16 . We train this detection output with a cross-entropy loss over the symmetric particle masks:

The complete loss equation for the entire network can now be defined:

Baseline methods

We compare SPA-NET to two commonly used methods, the Kinematic Likelihood Fitter (KLFitter) 2 , and a Permutation Deep Neural Network (PDNN), which uses a fully connected deep neural network similar to existing literature 20 . Both methods are permutation-based, meaning they sequentially evaluate every possible permutation of particle assignments. This results in a combinatorial explosion, with for example 5!/2 = 60 possible assignments of the jets in a semi-leptonically decaying \(t\bar{t}\) + jet event (the reduction by a factor of two comes from the assignment symmetry between the hadronically decaying W boson decay products). That is, there are 60 different possible permutations that must be evaluated per event, even before considering systematic uncertainty evaluation or further additional jets. With typical analyses utilizing MC samples containing \({{{{{{{\mathcal{O}}}}}}}}(1{0}^{6}-1{0}^{8})\) events, which must be evaluated for \({{{{{{{\mathcal{O}}}}}}}}(1{0}^{2})\) systematic variations, complex events quickly become intractable or at least extremely computationally expensive, even before considering the decreasing performance of such methods as a function of object multiplicity. The performance of these algorithms is compared to SPA-NET in all presented results.

KLFitter has been extensively used in top quark analyses 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , especially for semi-leptonic \(t\bar{t}\) events. The method involves building every possible permutation of the event and constructing a likelihood score for each. The permutation with the maximum likelihood is thus taken as the best reconstruction for that event. The likelihood score, which has been updated ( https://github.com/KLFitter/KLFitter ) since the original publication 2 , is defined as

where B represents Breit-Wigner functions, \({m}_{{q}_{1}{q}_{2}{q}_{3}}\) , \({m}_{{q}_{1},{q}_{2}}\) , \({m}_{{q}_{4}\ell \nu }\) , m ℓ ν are invariant masses computed from the final state particle momenta. The variables m t ( W ) and Γ t ( W ) are the masses and decay widths of the top quark ( W boson), respectively. The expressions \({E}_{\ell ,jet}^{(meas)}\) represents the (measured) energy of the leptons or jets, respectively, and the functions W v a r ( v a r A ∣ v a r B ) are the transfer function for the variable v a r A from v a r B .

This method suffers from several limitations. Firstly, the requirement to construct and test every possible permutation leads to a run-time that grows exponentially with the number of jets or other objects in the event. This quickly becomes a limiting factor in large datasets, which at the LHC often contain millions of events that must be evaluated hundreds of times each (once per systematic uncertainty shift). While semi-leptonic \(t\bar{t}\) can largely remain tractable, it can significantly slow down analyses due to the heavy computing cost, and it is typical to limit the evaluation to only a subset of the reconstructed objects in order to reduce this burden, which restricts the number of events that can be correctly reconstructed. More complex final states, for example \(t\bar{t}H\) production, require even more objects to be reconstructed and thus take even longer to compute, severely limiting the usability of the method in such channels.

A second limitation of the method is its treatment of partial events, which the likelihood is not designed to handle, and thus performance in these events is significantly degraded. Finally, the method does not take into account any correlations between the decay products of the target particles and the rest of the event, since only the particles hypothesized as originating from the targets are included in the likelihood evaluation. An advantage of the method is the use of transfer functions to represent detector effects, but these must be carefully derived for each detector to achieve maximum performance, which can be a difficult and time-consuming endeavor.

There are two variations of the KLFitter likelihood of interest in our studies: one in which the top quark mass is given an assumed value, and one in which it is not. Specifying the assumed mass leads to improved reconstruction efficiency at the expense of biasing towards permutations at this mass, causing sculpting of backgrounds and other undesirable effects. In the analyses presented in \(t\bar{t}H\) and \({Z}^{{\prime} }\) analyses, the top quark mass is fixed to a value of 173 GeV, since this biasing is less important than overall reconstruction efficiency. In contrast, the top quark mass measurement must avoid biasing towards a specific mass value, and thus the mass is not fixed in the likelihood for this analysis.

The PDNN uses a fully connected deep neural network that takes the kinematic and tagging information of the reconstructed objects as inputs, similar to the method described in existing literature 20 . Again, each possible permutation of the event is evaluated, and the assignment with the highest network output score is taken as the best reconstruction. Training is performed as a discrimination task, in which the correct permutations are marked as signal, and all of the other permutations are marked as background.

This method also suffers from several limitations, including the same exponentially growing run-time due to the permutation-based approach, the inability to adequately handle partial events, and the lack of inputs related to additional event activity. Further, the method does not incorporate the symmetries of the reconstruction problem due to the way in which input variables must be associated with the hypothesized targets. Recently, message-passing graph neural networks were applied to the all-hadronic \(t\bar{t}\) final state 17 , but as all studies presented here are performed in the lepton+jets channel, no comparison is made to such methods.

Datasets and training

Several datasets of simulated collisions are generated to test a variety of experimental analyses and effects. All datasets are generated at a center-of-mass energy of \(\sqrt{s}=13\) TeV using MADGRAPH_AMC@NLO 30 (v3.2.0, NCSA license) for the matrix element calculation, PYTHIA8 31 (v8.2, GPL-2) for the parton showering and hadronisation, and DELPHES 32 (v3.4.2, GPL-3) using the default CMS detector card for the simulation of detector effects. For all samples, jets are reconstructed using the anti- k T jet algorithm 33 with a radius parameter of R = 0.5, a minimum transverse momentum of p T > 25 GeV, and an absolute pseudo-rapidity of ∣ η ∣ < 2.5. To identify jets originating from b -quarks, a b -tagging algorithm with a p T -dependent efficiency and mis-tagging rate is applied. Electrons and muons are selected with the same p T and η requirements as for jets. No requirement is placed on the missing transverse momentum \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) .

A large sample of simulated Standard Model (SM) \(t\bar{t}\) production is generated with the top quark mass m t = 173 GeV, and used for initial studies as well as the background model in the \({Z}^{{\prime} }\) studies. It contains approximately 11M events after a basic event selection of exactly one electron or muon and at least four jets of which at least two are b -tagged. We further produce samples for the top mass analysis: ~0.2M events each at mass points of m t = 170, 171, 172, 173, 174, 175, 176 GeV in order to build templates, as well as a training sample of ~12M total \(t\bar{t}\) events produced in steps of 0.1 GeV to achieve an approximately flat m t distribution in the 166-176 GeV range. This sample is used for all \(t\bar{t}\) reconstruction studies as well as the top mass analysis. A final sample with m t = 171.9 GeV was produced to be used as pseudo-data for the top mass analysis. The value used was initially known by only one member of the team to avoid bias in the final mass extraction.

A sample of simulated SM \(t\bar{t}H\) production, in which the Higgs boson decays to a pair of b -quarks, is generated to model the signal process for the \(t\bar{t}H\) analysis. This sample has the same event selection as applied to the \(t\bar{t}\) samples, with an additional requirement of at least six jets due to the additional presence of the Higgs boson. Training of SPA-NET is performed using 10M \(t\bar{t}H\) events with at least two b -tagged jet, while the final measurement is performed using a distinct sample where 0.2M of 1.1M events satisfy the more stringent requirement of containing least four b -tagged jets. Training with the two-tag requirement achieved better overall performance than on the tighter four-tag selection, which follows the most recent ATLAS analyses in this channel 34 . The background in this analysis is dominated by \(t\bar{t}+b\bar{b}\) production, which is modeled using a simulated sample in which the top and bottom pairs are explicitly included in the hard process generated by MADGRAPH_AMC@NLO; of the 1.3M events generated, 0.2M survive the event selection.

Finally, we produce Beyond the Standard Model (BSM) events containing a hypothetical \({Z}^{{\prime} }\) boson that decays to a pair of top quarks, using the vPrimeNLO model 35 in MADGRAPH_AMC@NLO. One sample of 0.2M events is produced at each of \({m}_{{Z}^{{\prime} }}=500,700,900\) GeV to evaluate search sensitivity at a range of masses. A sample with an approximately flat \({m}_{{Z}^{{\prime} }}\) distribution is generated for network training by generating events in 1 GeV steps between 400 and 1000 GeV. We match jets to the original decay products of the top quarks and Higgs bosons using an exclusive \(\Delta R=\left.\right(\sqrt{{({\phi }_{j}-{\phi }_{d})}^{2}+{({\eta }_{j}-{\eta }_{d})}^{2}} < 0.4\) requirement, such that only one decay product can be matched to each jet and vice versa, always taking the closest match. This method is adopted both in ATLAS and CMS analyses and allows a crisp definition of the correct assignments as well as categorization of events based upon which particles are reconstructable, as explained in the Particle Detector subsection.

We train all models on a single machine with a AMD EPYC 7502 CPU and 4 NVidia 3090 GPUs for training. Each model was trained for a period of 24 hours on this machine, as we have found that to be sufficient time for models to converge in training and validation loss. We use the same hyperparameters derived in our previous work 16 as each event topology presented here may be interpreted as a variation of the same event topologies.

The data generated for this study is available in the our online repository ( https://mlphysics.ics.uci.edu/data/2023_spanet/ ). The code used for training is available on github ( https://github.com/Alexanders101/SPANet ).

Results and discussion

Reconstruction and regression performance.

We present the reconstruction efficiency for SPA-NET in semi-leptonic \(t\bar{t}\) and \(t\bar{t}H(H\to b\bar{b})\) events, compared to the performance of the benchmark methods KLFitter and PDNN. Efficiencies are presented relative to all events in the generated sample, as well as relative to the subset of events in which all top quark (and Higgs boson in the case of \(t\bar{t}H\) ) daughters are truth-matched to reconstructed jets, which we call Full Events. We also show efficiencies for each type of particle, with t H the hadronically decaying top quark, t L the leptonically decaying top quark, and H the Higgs boson. We present the efficiencies in three bins of jet multiplicity as well as inclusively.

In Table 1 , the efficiencies for accurate reconstruction of semi-leptonic \(t\bar{t}\) events are shown. We find that SPA-NET outperforms both benchmark methods in all categories. The performance of KLFitter is substantially lower than the other two methods everywhere, reaching only 12% for full-event efficiency in full events with ≥6 jets. The PDNN performance is close to SPA-NET in low jet multiplicity events, but the gap grows as the number of jets in the event increases. This is expected due to the encoded symmetries in SPA-NET that allow it to more efficiently learn the high multiplicity, more complex events, as well as the additional permutations that must be considered by the PDNN. SPA-NET is further suited to higher-multiplicity events due to not suffering from the large run-time scaling of the permutation based approaches. Results for \(t\bar{t}H(H\to b\bar{b})\) events, also presented in Table 1 , show similar trends.

Regression performance

In semi-leptonic \(t\bar{t}\) decays, there is a missing degree of freedom due to the undetected neutrino. The transverse component and ϕ angle of the neutrino can be well-estimated from the missing transverse momentum in the event, but the longitudinal component (or equivalently, the neutrino η ) cannot be similarly estimated at hadron colliders due to the unknown total initial momentum along the beam. A typical approach is to assume that the invariant mass of the combined lepton and neutrino four-vectors should be that of the W boson, m W = 80.37 GeV. This assumption leads to a quadratic formula that can lead to an ambiguity if the equation has either zero or two real solutions, and assumes on-shell W bosons and perfect lepton and \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) reconstruction. When the equation has two real solutions, the one with the lower absolute value is adopted. If the solutions are complex, we take the real component.

SPA-NET has been extended to provide additional regression outputs, which can be used to directly estimate such missing components. In Fig. 2 a, b, distributions of truth versus predicted neutrino η show that the SPA-NET regression is more diagonal than the traditional W -mass-constraint method. Figure 2 c, d shows the distributions and residuals of neutrino η , making it clear that SPA-NET regression has improved resolution of this quantity. However, Fig. 2 e, f show that neither method is able to accurately reconstruct the W -mass distribution. This distribution is not regressed directly, but is calculated by combining the \({E}_{{{{{{{{\rm{T}}}}}}}}}^{{{{{{{{\rm{miss}}}}}}}}}\) and lepton information with the predicted value of η . The mass constraint method produces a large peak exactly at the W -mass as expected, with a large tail at high mass coming from events in which the quadratic solutions are complex. In contrast, the SPA-NET regression, which has no information on the expected value of the W -mass, has a similar shape above m W , and a broad shoulder at lower values. It may thus be useful to refine the regression step to incorporate physics constraints, such as the W boson mass, to help the network learn important, complex quantities such as this. Incorporating more advanced regression techniques, such as this or combining with alternative methods such as ν -Flows 36 , 37 , is left to future work.

a , b show the true value on the x -axis versus predicted values from the SPA-NET regression and W -mass constraint respectively on the y -axis, with the one-dimensional distributions shown outside the axes. c compares the neutrino η from SPA-NET regression (blue dotted), W -mass constraint (red dashed), and the true distribution (black solid), with ( d ) showing the residuals between truth and SPA-NET regression (blue dotted) or W -mass constraint (red dashed). e , f show the same distributions, this time for the reconstructed leptonic W boson mass.

Particle presence outputs

The additional SPA-NET outputs, described in the Particle Detector subsection and shown in Fig. 3 , can be very useful in analysis. The KLFitter, PDNN, and SPA-NET event-level likelihoods are shown in Fig. 3 a–c. We note that the permutation methods only provide event-level scores for the entire assignment, and that the scores are highly overlapping with little separation between correctly and incorrectly reconstructed events. Figure 3 d–f shows the SPA-NET per-particle marginal (pseudo)-probabilities, which are summed to calculate the event-level likelihood. The distributions of the assignment probability, separated by events, which SPA-NET has predicted correctly or incorrectly, are shown in Fig. 3 g–i, and Fig. 3 j–l shows the distribution of the detection probability split by whether the particle is reconstructable or not. All of the SPA-NET scores show clear separation between these categories, and this separation can be used in a variety of ways, such as to remove incomplete or incorrectly matched events via direct cuts, separate different types of events into different regions, or provide separation power as inputs to an additional multivariate analysis. The top quark mass and \({Z}^{{\prime} }\) analyses both cut on these scores in order to remove incorrect/non-reconstructable events and improve signal-to-background ratio (S/B). In the \(t\bar{t}H\) analysis, these are used as inputs to a Boosted Decision Tree (BDT) to classify signal and background, and are found to provide a large performance gain.

The KLFitter likelihood is shown in ( a ), the Permutation Deep Neural Network (PDNN) log-likelihood in ( b ), and the SPA-NET event-level log-likelihood in ( c ), split by correctly reconstructed events (blue), incorrect events (orange), and non-reconstructable events (green). Further, the SPA-NET marginal probabilities for leptonic top, hadronic top, and Higgs are shown in ( d – f , respectively, grouped in the same way. g – i ) show the SPA-NET assignment probabilities, grouped by correct (blue) and incorrect (orange) events. Finally, the SPA-NET detection probabilities, split by reconstructable (blue) and non-reconstructable (orange), are shown in ( j – l ).

Computational overhead

Performance tests are performed on an AMD EPYC 7502 CPU with 128 threads and an NVidia RTX 3090 GPU. Including all pre-initialization steps, we evaluate the average run time for the three methods—KLFitter, PDNN, and SPA-NET—for both \(t\bar{t}\) and \(t\bar{t}H\) events. We find that KLFitter averages 24 (2) events per second on \(t\bar{t}\) ( \(t\bar{t}H\) ). The PDNN averages 2626 (51) events per second when run on a CPU, and 3034 (101) events per second on a GPU, with the speed up from GPU hardware minimal due to the fact that permutation building dominates the computation time. In contrast, SPA-NET averages 705 (852) events per second on a CPU, and 4407 (3534) events per second on a GPU, showing reduced scaling with the more complex \(t\bar{t}H\) events as expected. We therefore conclude that inference of SPA-NET should not be a bottleneck to analyses, as is often the case for methods like KLFitter. These numbers are summarized in table form in Supplementary Table 1 .

Ablation studies

In this section, we present several studies designed to reveal what the networks have learned. We find that training is, in general, very robust, showing little dependence on details of inputs or hyperparameters. For example, training performance is unchanged within statistical uncertainties when representing particles using { M , p T , η , ϕ } or { p x , p y , p z , E } 4-vector representations. Reconstruction performance varies by less than 1% if the training sample with a single top mass value is replaced by that with a flat mass spectrum.

In addition, we find that the performance of the network in testing depends on the kinematic range of the training samples in a sensible way. For example, the performance of the network on independent testing events varies with the top quark pair invariant mass, reflecting the mass distribution of the training sample. Figure 4 shows the testing performance versus top quark pair mass for networks trained on the full range of masses, or only events with invariant mass less than 600 GeV. The performance at higher mass is degraded when high-mass samples are not included in the training, as the nature of the task depends on the mass, which impacts the momentum and collimation of the decay products. Furthermore, the network performance is independent of the process (SM \(t\bar{t}\) or BSM \({Z}^{{\prime} }\) ) used to generate the training sample. The performance is reliable in the full range in which training data is present. It is noteworthy that the SM training still achieves similar performance up to ~1 TeV as the network trained on \({Z}^{{\prime} }\) events, despite having fewer events at this value, indicating that the training distribution need not be completely flat so long as some examples are present in the full range.

Shown is the performance for three networks with distinct training samples: \({Z}^{{\prime} }\to t\bar{t}\) events with the full range of invariant masses (blue), \({Z}^{{\prime} }\to t\bar{t}\) events with masses <600 GeV (orange), and SM \(t\bar{t}\) with the full range of invariant masses (green).

To evaluate if the network is learning the natural symmetries of the data, we perform two further tests. The first is to investigate the azimuthal symmetry of the events, which we evaluate by applying the network to events that are randomly rotated in the ϕ plane and/or mirrored across the beam axis, which should have no impact on the nature of the reconstruction task. We find that in 41% of test events, the difference in the marginal probabilities is <1% and 84% of all events have a difference of less than 5%. This implies that the network approximately learns the inherent rotational and reflection symmetries of the task, without explicitly encoding this into the the network architecture. The full residual distributions are shown in Supplementary Fig. 1 .

The impact of adding rotation invariance to the network has been evaluated by employing an explicitly invariant attention architecture which employs a matrix of relative Lorentz-covariant quantities between each pair of particles, similar to existing literature 18 , 38 . We focus specifically on the symmetry induced by rotations along the beam axis. We follow the covariant transformer architecture 18 , and treat the ϕ and η angles as covariant, and compute the difference between these angles for all pairs of jets in the event. The remaining features are treated as invariant and processed normally by the attention. Figure 5 a shows that employing the invariant attention mechanism improves performance for small datasets, but does not lead to higher overall performance. This observation is consistent with the findings of existing literature. 18 , 38 . The explicit invariance does bring visible improvement in training speed as seen in Fig. 5 b. After fully training both networks on various training data sizes, we examine the training log and determine how many batches (gradient updates) were necessary before achieving maximal validation accuracy. We see that the invariant attention significantly reduces the number of updates needed to train the network. The trade-off of this regime is to make each network larger and more memory intensive, as the inputs must now be represented as pairwise matrices of features instead of simple vectors. Since the overall performance in the end is the same, and since we notice that a regular network already learns to approximate this invariance, we proceed using the traditional attention architecture, and this invariant network is not used for any further studies presented here.

Shown are ( a ) reconstruction purity and ( b ) training speed, with the regular transformer shown in dashed orange and the explicitly invariant transformer 18 in solid blue. The uncertainty bars in ( a ) show the variation in reconstruction purity across 16 separate trainings at each dataset size.

Search for \(t\bar{t}H(H\to b\bar{b})\)

While the previous sections have detailed the per-event performance of SPA-NET, in the following sections we demonstrate its expected impact on flagship LHC physics measurements and searches.

The central challenge of measuring the cross-section for \(t\bar{t}H\) production, in which the Higgs boson follows its dominant decay mode to a pair of b -quarks, is separating the \(t\bar{t}H\) signal from the overwhelming \(t\bar{t}\) + \(b\bar{b}\) background. Typically, machine learning algorithms such as deep neural networks or boosted decision trees are trained to distinguish signal and background using high-level event features 34 , 39 . Since the key kinematic difference between the signal and background is the presence of a Higgs boson, the performance of this separation is greatly dependent on the quality of the event reconstruction, where improvements by SPA-NET can make a significant impact on the final result.

Reconstruction and background rejection

Event reconstruction is performed with SPA-NET, KLFitter, and a PDNN. The reconstruction efficiency for each of these methods is shown in Table 1 , where it is already clear that SPA-NET outperforms both of the baseline methods.

The reconstructed quantities and likelihood or network scores are then used to train a classifier to distinguish between signal and background. The full input list is shown in Supplementary Table 2 , with most variable definitions taken from the latest ATLAS result 34 . A BDT is trained for each reconstruction algorithm with the same input definitions and hyperparameters using the XGBoost package 40 . Tests using a BDT trained on lower-level information, i.e., the four-vectors of the predicted lepton and jet assignments, found significantly weaker performance than these high-level BDTs. We also compare the performance of the BDTs to two different SPA-NET outputs that are trained to separate signal and background. The first, which we call SPA-NET Pretraining, is an additional output head of the primary SPA-NET network, which has the objective of separating signal and background events. The second, which we call SPA-NET Fine-tuning, uses the same embeddings and central transformer as the former method, but the signal versus background classification head is trained in a separate second step after the initial training is complete. In this way, the network is able to first learn the optimal embedding of signal events, and utilize this embedding as the inputs to a dedicated signal vs background network. We have implemented in the SPA-NET package an option to output directly the embeddings from the network such that they can be used in this or other ways by the end user.

The receiver operating curve for the various classification networks is shown in Fig. 6 . The best separation performance comes from the fine-tuned SPA-NET model, as expected. The BDT with kinematic variables reconstructed with the SPA-NET jet-parton assignment (SPA-NET+BDT setup) is next, followed by the purely pre-trained model. All of these substantially outperform both the KLFitter+BDT and PDNN+BDT baselines.

Shown is signal efficiency versus background rejection for several SPA-NET based set ups—SPA-NET fine-tuning (solid blue), SPA-NET+Boosted Decision Tree (BDT) (dash-dot pink), and SPA-NET pretraining (dash-dot green)—as well as BDTs based on outputs of traditional reconstruction techniques Permutation Deep Neural Network (PDNN) (dotted red) and KLFitter (dot-dash yellow).

Impact on sensitivity

To estimate the impact of significantly improved signal-background separation from SPA-NET reconstruction, we perform an Asimov fit to the network output distributions with the pyhf package 41 , 42 . The signal is normalized to the SM cross-section of 0.507 pb 43 and corrected for the branching fraction and selection efficiency of our sample. The dominant \(t\bar{t}+b\bar{b}\) background is normalized similarly, using the cross-section calculated by MADGRAPH_AMC@NLO of 0.666 pb. We further multiply the background cross-section by a factor of 1.5, in line with measurements from ATLAS 34 and CMS 39 that found this background to be larger than the SM prediction, rounded up to account also for the LO→NLO cross-section enhancement. We neglect the sub-leading backgrounds. The distributions are binned according to the AutoBin feature 44 preferred by ATLAS in order to ensure no bias is introduced between the different methods due to the choice of binning. Results normalized to 140 fb −1 , the luminosity of Run 2 of the LHC, using 5 bins and assuming an overall systematic uncertainty of 10% are presented in Table 2 . The numbers in the parentheses in Table 2 are results of an LHC Run 3 analysis normalized to 300 fb −1 of data using 8 bins with an overall systematic uncertainty assumption of 7%. Although the Run 3 center-of-mass energy of the LHC is \(\sqrt{s}=13.6\) TeV, all results presented assume \(\sqrt{s}=13\) TeV for simplicity.

In both scenarios, the sensitivity tracks the signal-background separation performance shown in Fig. 6 , with SPA-NET fine-tuning achieving the greatest statistical power. Neither of the benchmark methods is able to reach the 3 σ statistical significance threshold in the Run 2 analysis, while both SPA-NET+BDT and fine-tuning reach this mark. Similarly, these methods both reach the crucial 5 σ threshold normally associated with discovery, with the benchmark methods at only roughly 4 σ .

SPA-NET thus provides a significant expected improvement over the benchmark methods. While the full LHC analysis will require a more complete treatment, including significant systematic uncertainties due to the choice of event generators, previous studies have demonstrated minimal dependence to such systematic uncertainties 16 .

Top mass measurement

The top quark mass m t is a fundamental parameter of the Standard Model that can only be determined via experimental measurement. These measurements are critical inputs to global electroweak fits 45 , and m t even has implications for the stability of the Higgs vacuum potential, which has cosmological consequences 46 , 47 . Precision measurements of the top quark mass are thus one of the most important pieces of the experimental program of the LHC, with the most recent results reaching sub-GeV precision 48 , 49 , 50 . We demonstrate in this section the improvement enabled by the use of SPA-NET in a template-based top mass extraction.

We perform a two-dimensional fit to the invariant mass distributions of the hadronic top quark and W boson as reconstructed by each method, using the basic preselection described in the Datasets and Training subsection. We further truncate the mass distributions to 120 ≤ m t ≤ 230 GeV and 40 ≤ m W ≤ 120 GeV. The fraction of events with correct or incorrect predictions for the top quark jets has a strong impact on the resolution with which the mass can be extracted. Better reconstruction should thus improve the overall sensitivity to the top quark mass.

Incorporation of the W -mass information in the 2D fit allows for a simultaneous constraint on the jet energy scale uncertainty, often a leading contribution to the total uncertainty, by also fitting a global jet scale factor (JSF) to be applied to the p T of each jet. Further, events that do not contain a fully reconstructable top quark are removed by cutting on the various scores from each method. KLFitter events are required to have a log-likelihood score >−70, PDNN events must have a network score of >0.12, and SPA-NET events must have a marginal probability of >0.23, optimized in each case to minimize the uncertainty on the extracted top mass. We additionally compare each method to an idealized perfect reconstruction method, in which all unmatched events are removed, and the truth-matched reconstruction is used for all events. The perfect-matched method provides an indication of the hypothetical limit of improvement achievable through better event reconstruction. In all cases, we neglect background from other processes, since these backgrounds tend to be on the order of a few percent 25 , and would be further suppressed by the network score cuts.

The top quark mass and JSF are extracted using a template fit from Monte Carlo samples which have top quark masses in 1 GeV intervals between 170 and 176 GeV. Templates are constructed for varying mass and JSF hypotheses for both the top and W boson mass distributions. These templates are built separately for each of the correct, incorrect, and unmatched event categories as the sum of a Gaussian and a Landau distribution, with five free parameters: the mean μ and the width σ of each, as well as the relative fraction f . We found an approximately linear relation between the template parameters as a function of the top quark mass and JSF, allowing for linear interpolation between the mass points. Finally, we validate the mass extracted by a template fit in hypothetical similar experiments and find a small bias, for which we derive a correction.

The impact of various reconstruction techniques can be best measured by the resulting uncertainty on the top quark mass and JSF. Figure 7 shows the expected uncertainty ellipses for a dataset with luminosity of 140 fb −1 and assuming a JSF variation of ±4%. The final uncertainty on the top mass is 0.193 GeV for KLFitter, 0.176 GeV for PDNN, and 0.165 GeV for SPA-NET. This indicates a 15% improvement in top quark mass uncertainty when using SPA-NET compared to the benchmark methods. The idealized reconstruction technique achieves an uncertainty of 0.109 GeV, demonstrating how much room for improvement remains. The dominant contribution to the gap between the perfect and SPA-NET reconstruction comes from the perfect removal of all unmatched events.

Shown are results for the KLFitter (blue), permutation deep neural network (PDNN) (yellow), SPA-NET (green), and an idealized perfect reconstruction (red). Also shown are 1 σ (solid) and 3 σ (dashed) uncertainty ellipses.

Search for \({Z}^{{\prime} }\to t\bar{t}\)

Many BSM theories hypothesize additional heavy particles which may decay to \(t\bar{t}\) pairs, such as heavy Higgs bosons or new gauge bosons ( \({Z}^{{\prime} }\) ). We investigate a generic search for such a \({Z}^{{\prime} }\) particle, for which accurate reconstruction of the \(t\bar{t}\) mass peak over the SM background plays a crucial role. We compare the performance of the benchmark reconstruction methods to that of various SPA-NET configurations by assessing the ability to discover a \({Z}^{{\prime} }\) signal.

An important aspect is the selection of training data, due to the unknown mass of the \({Z}^{{\prime} }\) , which strongly affects the kinematics of the \(t\bar{t}\) system. To avoid introducing bias into the network, the training sample is devised to be approximately flat in \({m}_{t\bar{t}}\) . The network training was otherwise identical to that described for the SM \(t\bar{t}\) network, and performance on SM \(t\bar{t}\) events was approximately the same in the mass range covered by both samples.

The basic \(t\bar{t}\) selection described in the Dataset and Training subsection is applied, and all events are reconstructed as described earlier in order to calculate the \(t\bar{t}\) invariant mass, \({m}_{t\bar{t}}\) . The mass resolution of a hypothetical resonance can often be improved by removing poorly- or partially-reconstructed events. In the context of the algorithms under comparison, this corresponds to a requirement on the KLFitter likelihood or network output scores. The threshold is chosen to optimize the analysis with each algorithm, leading to a significant reduction of the SM \(t\bar{t}\) background when using the PDNN and SPA-NET. For SPA-NET we require a marginal probability of >0.078, and for PDNN we require a score of >0.43. For KLFitter, no cut is applied, as no improvement was found. More details on these cuts and the effect on the background distributions are shown in Supplementary Figs. 2 and 3 in Supplementary Note 1 .

We use the pyhf 41 , 42 package to extract the \({Z}^{{\prime} }\) signal and assess statistical sensitivity.

The expected results for a Run 2 analysis, normalized to 140 fb −1 with 20 GeV bins and a systematic uncertainty of 10%, are shown in Table 3 . The discovery significance is improved by SPA-NET compared to the benchmark methods for all masses considered. For example, for a \({Z}^{{\prime} }\) of mass 700 GeV the limit improves from 1.6 σ using KLFitter to 3.1 σ using SPA-NET.

The expected sensitivity in a Run 3 dataset with the integrated luminosity of 300 fb −1 is computed with an optimistic systematic uncertainty of 5% as also shown in Table 3 . For all the three benchmark signals, discovery significance exceeds 5 σ using SPA-NET, while for the baseline methods only the high mass point for the PDNN reaches this threshold. At a \({Z}^{{\prime} }\) mass of 500 GeV, KLFitter does not reach the 3 σ evidence threshold, while SPA-NET is able to make a discovery. It is noteworthy that the neutrino regression does not lead to an improvement on the final sensitivity, despite showing improved resolution compared to the baseline mass constraint method. This is due to the effect on the background shape, which similarly improves in this case.

Improved reconstruction with SPA-NET can therefore greatly boost particle discovery potential. This finding should extend to other hypothetical resonances such as heavy Higgs bosons, \({W}^{{\prime} }\) bosons, or SUSY particles as well as non- \(t\bar{t}\) final states such as di-Higgs, di-boson, t b or any other in which reconstruction is crucial and challenging.

Conclusions

This paper describes significant extensions and improvements to SPA-NET, a complete package for event reconstruction and classification for high-energy physics experiments. We have demonstrated the application of our method to three flagship LHC physics measurements or searches, covering the full breadth of the LHC program; a precision measurement of a crucial SM parameter, a search for a rare SM process, and a search for a hypothetical new particle. In each case, the use of SPA-NET provides large improvements over benchmark methods. We have further presented studies exploring what the networks learn, demonstrating the ability to learn the inherent symmetries of the data and strong robustness to training conditions. SPA-NET is the most efficient, high-performing method for multi-object event reconstruction to date and holds great promise for helping unlock the power of the LHC dataset.

Data availability

Our data is available in an online repository.

Code availability

Our code is available on github ( https://github.com/Alexanders101/SPANet ).

Snyder, S. S. Measurement of the top quark mass at D0. Ph.D. thesis, SUNY, Stony Brook (1995).

Erdmann, J. et al. A likelihood-based reconstruction algorithm for top-quark pairs and the KLFitter framework. Nucl. Instrum. Meth. A 748 , 18–25 (2014).

Article ADS Google Scholar

Vaswani, A. et al. Attention is all you need. In: Advances in neural information processing systems, vol. 30 (2017).

Qu, H. & Gouskos, L. Jet tagging via particle clouds. Phys. Rev. D 101 , 056019 (2020).

Moreno, E. A. et al. JEDI-net: a jet identification algorithm based on interaction networks. Eur. Phys. J. C 80 , 58 (2020).

Mikuni, V. & Canelli, F. ABCNet: an attention-based method for particle tagging. Eur. Phys. J. Plus 135 , 463 (2020).

Article Google Scholar

Lu, Y., Romero, A., Fenton, M. J., Whiteson, D. & Baldi, P. Resolving extreme jet substructure. JHEP 08 , 046 (2022).

Ju, X. & Nachman, B. Supervised jet clustering with graph neural networks for lorentz boosted bosons. Phys. Rev. D 102 , 075014 (2020).

Guo, J., Li, J., Li, T. & Zhang, R. Boosted Higgs boson jet reconstruction via a graph neural network. Phys. Rev. D 103 , 116025 (2021).

Dreyer, F. A. & Qu, H. Jet tagging in the Lund plane with graph networks. JHEP 03 , 052 (2021).

Bogatskiy, A., Hoffman, T., Miller, D. W. & Offermann, J. T. PELICAN: Permutation equivariant and lorentz invariant or covariant aggregator network for particle physics (2022).

Gong, S. et al. An efficient Lorentz equivariant graph neural network for jet tagging. JHEP 07 , 030 (2022).

Article ADS MathSciNet Google Scholar

Qu, H., Li, C. & Qian, S. Particle transformer for jet tagging. In: Proceedings of the 39th International Conference on Machine Learning, 18281–18292 (2022).

Bogatskiy, A., Hoffman, T., Miller, D. W., Offermann, J. T. & Liu, X. Explainable equivariant neural networks for particle physics: PELICAN https://arxiv.org/abs/2307.16506 (2023).

Fenton, M. J. et al. Permutationless many-jet event reconstruction with symmetry preserving attention networks. Phys. Rev. D 105 , 112008 (2022).

Shmakov, A. et al. SPANet: Generalized permutationless set assignment for particle physics using symmetry preserving attention. SciPost Phys. 12 , 178 (2022).

Ehrke, L., Raine, J. A., Zoch, K., Guth, M. & Golling, T. Topological reconstruction of particle physics processes using graph neural networks. Phys. Rev. D 107 , 116019 (2023).

Qiu, S., Han, S., Ju, X., Nachman, B. & Wang, H. Holistic approach to predicting top quark kinematic properties with the covariant particle transformer. Phys. Rev. D 107 , 114029 (2023).

Workman, R. L. et al. Review of particle physics. PTEP 2022 , 083C01 (2022).

Google Scholar

Erdmann, J., Kallage, T., Kröninger, K. & Nackenhorst, O. From the bottom to the top—reconstruction of \(t\bar{t}\) events with deep learning. JINST 14 , P11015 (2019).

ATLAS Collaboration. Measurements of normalized differential cross sections for \(t\bar{t}\) production in pp collisions at \(t\bar{t}\) TeV using the ATLAS detector. Phys. Rev. D 90 , 072004 (2014).

ATLAS Collaboration. Measurement of the top-quark mass in the fully hadronic decay channel from ATLAS data at \(\sqrt{s}=7{{{{{{{\rm{\,TeV}}}}}}}}\) . Eur. Phys. J. C 75 , 158 (2015).

ATLAS Collaboration. Measurements of spin correlation in top-antitop quark events from proton-proton collisions at \(\sqrt{s}=7\) TeV using the ATLAS detector. Phys. Rev. D 90 , 112016 (2014).

ATLAS Collaboration. Search for the Standard Model Higgs boson produced in association with top quarks and decaying into \(b\bar{b}\) in pp collisions at \(b\bar{b}\) = 8 TeV with the ATLAS detector. Eur. Phys. J. C 75 , 349 (2015).

ATLAS Collaboration. Measurements of top-quark pair differential and double-differential cross-sections in the ℓ +jets channel with p p collisions at \(\sqrt{s}=13\) TeV using the ATLAS detector. Eur. Phys. J. C 79 , 1028 (2019). [Erratum: Eur.Phys.J.C 80, 1092 (2020)].

ATLAS Collaboration. Measurement of the charge asymmetry in top-quark pair production in association with a photon with the ATLAS experiment. Phys. Lett. B 843 , 137848 (2023).

CMS Collaboration. Measurement of the top quark forward-backward production asymmetry and the anomalous chromoelectric and chromomagnetic moments in pp collisions at \(\sqrt{s}\) = 13 TeV. JHEP 06 , 146 (2020).

ATLAS & CMS Collaborations. Combination of the W boson polarization measurements in top quark decays using ATLAS and CMS data at \(\sqrt{s}=\) 8 TeV. JHEP 08 , 051 (2020).

ATLAS & CMS Collaborations. Combination of inclusive and differential \({{{{{{{\rm{t}}}}}}}}\overline{{{{{{{{\rm{t}}}}}}}}}\) charge asymmetry measurements using ATLAS and CMS data at \({{{{{{{\rm{t}}}}}}}}\overline{{{{{{{{\rm{t}}}}}}}}}\) and 8 TeV. JHEP 04 , 033 (2018).

Alwall, J. et al. The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations. JHEP 07 , 079 (2014).

Sjöstrand, T. et al. An introduction to PYTHIA 8.2. Comput. Phys. Commun. 191 , 159–177 (2015).

de Favereau, J. et al. DELPHES 3, A modular framework for fast simulation of a generic collider experiment. JHEP 02 , 057 (2014).

Cacciari, M., Salam, G. P. & Soyez, G. The anti- k t jet clustering algorithm. JHEP 04 , 063 (2008).

ATLAS Collaboration. Measurement of Higgs boson decay into b -quarks in associated production with a top-quark pair in p p collisions at \(\sqrt{s}=13\) TeV with the ATLAS detector. JHEP 06 , 097 (2022).

Fuks, B. & Ruiz, R. A comprehensive framework for studying W \({}^{{\prime} }\) and Z \({}^{{\prime} }\) bosons at hadron colliders with automated jet veto resummation. JHEP 32 , 5 (2017).

Leigh, M., Raine, J. A., Zoch, K. & Golling, T. ν -flows: conditional neutrino regression. SciPost Phys. 14 , 159 (2023).

Raine, J. A., Leigh, M., Zoch, K. & Golling, T. Fast and improved neutrino reconstruction in multineutrino final states with conditional normalizing flows. Phys. Rev. D 109 , 012005 (2024).

Li, C. et al. Does Lorentz-symmetric design boost network performance in jet physics? https://arxiv.org/abs/2208.07814 (2022).

CMS Collaboration. Measurement of the t \(\overline{{{{{{{{\rm{t}}}}}}}}}\) H and tH production rates in the \({{{{{{{\rm{H}}}}}}}}\to {{{{{{{\rm{b}}}}}}}}\overline{{{{{{{{\rm{b}}}}}}}}}\) decay channel with 138 fb −1 of proton-proton collision data at \(\sqrt{s}=13\,{{{{{{{\rm{TeV}}}}}}}}\) . Tech. Rep., CERN, Geneva. https://cds.cern.ch/record/2868175 . (2023).

Chen, T. & Guestrin, C. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD ’16, 785–794 (ACM, New York, NY, USA, 2016).

Heinrich, L., Feickert, M. & Stark, G. pyhf: v0.7.3. https://github.com/scikit-hep/pyhf/releases/tag/v0.7.3 .

Heinrich, L., Feickert, M., Stark, G. & Cranmer, K. pyhf: pure-python implementation of histfactory statistical models. J. Open Source Softw. 6 , 2823 (2021).

de Florian, D. et al. Handbook of LHC Higgs Cross Sections: 4. Deciphering the Nature of the Higgs Sector 2/2017 (2016).

Calvet, T. P.Search for the production of a Higgs boson in association with top quarks and decaying into a b-quark pair and b-jet identification with the ATLAS experiment at LHC. Ph.D. thesis, Aix-Marseille University, https://cds.cern.ch/record/2296985 . (2017).

ALEPH, CDF, D0, DELPHI, L3, OPAL, SLD, LEP Electroweak Working Group, Tevatron Electroweak Working Group, SLD Electroweak, Heavy Flavour Groups. In : Precision Electroweak Measurements and Constraints on the Standard Model. CERN-PH-EP-2010-095 (2010).

Degrassi, G. et al. Higgs mass and vacuum stability in the Standard Model at NNLO. JHEP 08 , 098 (2012).

Andreassen, A., Frost, W. & Schwartz, M. D. Scale Invariant Instantons and the Complete Lifetime of the Standard Model. Phys. Rev. D 97 , 056006 (2018).

CMS Collaboration. Measurement of the top quark mass using a profile likelihood approach with the lepton + jets final states in proton–proton collisions at \(\sqrt{s}=13\,\,{{\mbox{Te}}}\,\hspace{-0.79982pt}\,{{\mbox{V}}}\,\) . Eur. Phys. J. C 83 , 963 (2023).

CMS Collaboration. Measurement of the differential \(t\overline{t}\) production cross section as a function of the jet mass and extraction of the top quark mass in hadronic decays of boosted top quarks. Eur. Phys. J. C 83 , 560 (2023).

ATLAS Collaboration. Measurement of the top-quark mass using a leptonic invariant mass in pp collisions at \(\sqrt{s}\) = 13 TeV with the ATLAS detector. JHEP 06 , 019 (2023).

Download references

Acknowledgements

We would like to thank Ta-Wei Ho for assistance in generating some of the samples used in this paper. D.W. and M.F. are supported by DOE grant DE-SC0009920. The work of A.S. and P.B. in part supported by ARO grant 76649-CS to P.B. H.O. and Y.L. are supported by NSFC under contract no. 12075060, and SCH is supported by NSF under Grant no. 2110963.

Author information

These authors contributed equally: Michael James Fenton, Alexander Shmakov.

Authors and Affiliations

Department of Physics and Astronomy, University of California, Irvine, Irvine, 92607, CA, USA

Michael James Fenton & Daniel Whiteson

Department of Computer Science, University of California, Irvine, Irvine, 92607, CA, USA

Alexander Shmakov & Pierre Baldi

Institute of High Energy Physics, Chinese Academy of Sciences, Shijingshan, 100049, Beijing, China

Hideki Okawa

Institute of Modern Physics, Fudan University, Yangpu, 200433, Shanghai, China

Department of Physics, National Tsing Hua University, Hsingchu City, 30013, Taiwan

Ko-Yang Hsiao

Department of Physics and Astronomy, University of Washington, Seattle, 98195-4550, WA, USA

Shih-Chieh Hsu

You can also search for this author in PubMed Google Scholar

Contributions

Michael Fenton: conception, direction, supervision of all students, manuscript preparation. Alexander Shmakov: development, implementation, and training of SPA-NET and PDNN, manuscript preparation. Hideki Okawa: MC production, \({Z}^{{\prime} }\) analysis lead, manuscript preparation, supervision of Y. Li. Yuji Li: \(t\bar{t}H\) analysis lead Ko-Yang Hsiao: top mass analysis lead Shih-Chieh Hsu: supervision of K-Y Hsiao Daniel Whiteson: manuscript preparation, supervision of A. Shmakov. Pierre Baldi: manuscript editing, machine learning developments, supervision of A. Shmakov.

Corresponding authors

Correspondence to Michael James Fenton or Alexander Shmakov .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Communications Physics thanks Daniel Murnane and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Peer review file, supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Fenton, M.J., Shmakov, A., Okawa, H. et al. Reconstruction of unstable heavy particles using deep symmetry-preserving attention networks. Commun Phys 7 , 139 (2024). https://doi.org/10.1038/s42005-024-01627-4

Download citation

Received : 08 November 2023

Accepted : 11 April 2024

Published : 30 April 2024

DOI : https://doi.org/10.1038/s42005-024-01627-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

Explore articles by subject
Guide to authors
Editorial policies

File(s) under embargo

Reason: Pending publication involved

until file(s) become available

Optimizations on Estimation and Positioning Techniques in Intelligent Wireless Systems

Wireless technologies across various applications aim to improve further by developing intelligent systems, where the performance is optimized through adaptive policy selections that efficiently adjust to the environment dynamics. As a result, accurate observation on the surrounding conditions, such as wireless channel quality and relative target location, becomes an important task. Although both channel estimation and wireless positioning problems have been well studied, with advanced wireless communications relying on complex technologies and being applied to diverse environments, optimization strategies tailored to their unique architectures and scenarios need to be further investigated. In this dissertation, four key research problems related to channel estimation and wireless positioning tasks for intelligent wireless systems are identified and studied. First, a channel denoising problem in multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) systems is addressed, and a Q-learning-based successive denoising scheme, which utilizes a channel curvature magnitude threshold to recover unreliable channel estimates, is proposed. Second, a pilot assignment problem in scalable open radio access network (O-RAN) cell-free massive MIMO (CFmMIMO) systems is studied, where a low-complexity pilot assignment scheme based on a multi-agent deep reinforcement learning (MA-DRL) framework along with a codebook search strategy is proposed. Third, sensor selection/placement problems for wireless positioning are addressed, and dynamic and robust sensor selection schemes that minimize the Cramér-Rao lower bound (CRLB) are proposed. Lastly, a feature selection problem for deep learning-based wireless positioning is studied, and a unique feature size selection method, which weights over the expected information gain and classification capability, along with a multi-channel positioning neural network is proposed.

Degree Type

Doctor of Philosophy
Electrical and Computer Engineering

Campus location

West Lafayette

Advisor/Supervisor/Committee Chair

Advisor/supervisor/committee co-chair, additional committee member 2, additional committee member 3, usage metrics.

Electrical engineering not elsewhere classified

IMAGES

18 Problem-Based Learning Examples (2023)
Problem Based Learning
Problem Based Learning ppt
1.5: Problem Based Learning
Creating Effective Learning Experiences: Problem-Based Learning
Strategies for Implementing Problem-based Learning in Classrooms

VIDEO

Assignment Problem ( Brute force method) Design and Analysis of Algorithm
Problem Based Learning: Group Review Task- Math-2110-141W Spring 2024
ASSIGNMENT PROBLEM: meaning, formulation, Hungarian method
Balanced assignment problem in Operations Research
PROBLEM BASED LEARNING AND PROJECT BASED LEARNING
Mathematical formulation of Assignment problem and Assignment problem as a particular case of LPP

COMMENTS

Problem-Based Learning
Nilson (2010) lists the following learning outcomes that are associated with PBL. A well-designed PBL project provides students with the opportunity to develop skills related to: Working in teams. Managing projects and holding leadership roles. Oral and written communication. Self-awareness and evaluation of group processes. Working independently.
Problem-Based Learning (PBL)
PBL is a student-centered approach to learning that involves groups of students working to solve a real-world problem, quite different from the direct teaching method of a teacher presenting facts and concepts about a specific subject to a classroom of students. Through PBL, students not only strengthen their teamwork, communication, and ...
What is PBL? & 5 Problem-Based Learning Examples
Problem-based learning, or PBL, is an inquiry-based learning method that uses complex, real-world problems to help students learn. It stands in contrast to some traditional teaching methods where teachers present facts and concepts directly to students. The strictest presentation of PBL would involve a teacher using PBL for an entire semester ...
18 Problem-Based Learning Examples (2024)
Problem-based learning (PBL) is a student-centered teaching method where students are given the opportunity to solve open-ended real-world problems. The teacher provides limited guidance and is usually referred to as a "facilitator". The burden of responsibility for the majority of the work rests squarely on the shoulders of the students.
Problem-Based Learning: Six Steps to Design, Implement, and Assess
Step One: Identify Outcomes/Assessments. PBL fits best with process-oriented course outcomes such as collaboration, research, and problem solving. It can help students acquire content or conceptual knowledge, or develop disciplinary habits such as writing or communication. After determining whether your course has learning outcomes that fit ...
What is Problem-Based Learning (PBL)
Problem-based learning (PBL) is a teaching style that pushes students to become the drivers of their learning education. Problem-based learning uses complex, real-world issues as the classroom's subject matter, encouraging students to develop problem-solving skills and learn concepts instead of just absorbing facts.
Problem-Based Learning (PBL)
Problem-Based Learning (PBL) is a teaching method in which complex real-world problems are used as the vehicle to promote student learning of concepts and principles as opposed to direct presentation of facts and concepts. In addition to course content, PBL can promote the development of critical thinking skills, problem-solving abilities, and ...
Grappling With Real-World Problems
Overview. Problem-based learning (PBL) is integrated at Two Rivers Public Charter School in Washington, DC, at every grade level—pre-K through eighth grade. Students are presented with a real-world problem, undertake a series of investigations, and create a product that they present to an authentic audience as part of the Expeditionary ...
OpenPBL
Active learning for all. Implement group activities with your students and allow for thoughtful assessment by the student himself (self-assessment), by colleagues and by the tutor. Establish aspects of learning to be evaluated, each with its own weight in the final grade. Promote active learning facilitated by the OpenPBL platform.
Problem-based Learning
Overview. Problem-based learning is a method of teaching where students are presented with a real or realistic problem, such as a case study or hypothetical situation, and use inductive reasoning to learn both information about the topic and how to think critically about it. Instead of passively listening to lectures or even being led through ...
Problem-Based Assignments Solve Real-World Issues
In her book, Students Taking Charge: Inside the Learner-Active, Technology-Infused Classroom, she offers the following clear, step-by-step guidelines for creating problem-based assignments. Start with the standards. Identify the content, skills and concepts you plan to cover in a specified unit of time, such as a three-week period.
Problem-Based Learning: What and How Do Students Learn?
Problem-based learning (PEL) is. an instructional method in which students learn through facilitated problem. solving. In PBL, student learning centers on a complex problem that does not have a single correct answer. Students work in collaborative groups to identify what they need to learn in order to solve a problem.
Problem Library
The PBL (Problem Based Learning) Cookie Chemistry activity was designed for the first two class periods of General . . . Amy Greene. Chemistry and Biochemistry. 1-3 Hours. Introductory (majors and non-majors) DISCIPLINEAdd Filter »Collapse » Biological Sciences Biotechnology Chemistry Accounting Aviation Science Business Administration ...
PDF 7 Key Elements of Problem-based Learning (PBL) Classrooms
Introduction 2. These elements were created by the Sammamish Collaborative, a 5-year collaboration between Sammamish High School teachers, teacher leaders, school leaders, and our partners at the Institute for Science and Math Education (ISME) at the University of Washington and Knuth Research, Inc. Their development was part of an effort to ...
PDF Problem Based Learning: A Student-Centered Approach
2. Traditional Versus Problem-Based Learning In traditional approaches to learning based on traditional way, data is provided first in traditional way of learning, but in PBL problems are presented to the students' to solve the problem. Figure 1 represents the process of learning and showing how each step is inter-connected to each other ...
PDF Teaching Project-based Assignments
Teaching Project-based Assignments. Oftentimes, assigned research papers and other longer projects fail because students do not understand the underlying problems their work addresses or how they can authentically contribute to solving them. Project-based learning (PBL) emphasizes the problem throughout the writing process so that students ...
Choosing Effective Assessments for PBL
Here are three approaches to assessment: 1. Stop and assess. One way is to interrupt students' learning and provide them with an assessment. This could take the form of a test, quiz, or formal presentation. This is likely the most common form of assessment in classrooms. 2. Assess in action.
Project-Based Learning
Introduction. Project-based learning (PBL) involves students designing, developing, and constructing hands-on solutions to a problem. The educational value of PBL is that it aims to build students' creative capacity to work through difficult or ill-structured problems, commonly in small teams. Typically, PBL takes students through the ...
Project-Based Learning and Problem-Based Learning (x-BL)
Project-based Learning begins with the assignment of tasks that will lead to the creation of a final product or artefact. The emphasis is on the end product. Students work on open-ended assignments. These could be more than one problem. Students analyse the problems and generate solutions. Students design and develop a prototype of the solution.
Assessing Students During the Problem-Based Learning (PBL) process
Problem-based learning is an iterative process. Evidence-based decision-making has developed criteria to define a searchable question. 7 These criteria should guide the generation of learning issues. First the problem needs to be clearly identified. This problem should be structured as a specific question.
Interdisciplinary Journal of Problem-Based Learning
Interdisciplinary Journal of Problem-Based Learning Volume 11 Issue 2 Article 2 Published online: 5-11-2017 Problem-Based Assignments as a Trigger for Developing Ethical and Reflective Competencies Dieter Euler University of St. Gallen, Switzerland, [email protected] Patrizia Kühner
Task Structuring for On-line Problem Based Learning: A Case Study
The decision to use the WBC tool to mediate, at least in part, collaborative problem-based learning assignments was based on a few factors. First, the instructor had noted that students often had difficulty finding time to meet outside of class to complete group projects. Second, the instructor felt that from previous semesters using a
Assessment for problem-based learning
Assessment for learning provides information about the competency level and competence development of a student, obtained with various instruments at different assessment moments (Schuwirth & Van der Vleuten, 2011b). Peer and self-assessment is considered to be an essential element in the process of problem-based learning (Dochy, Segers ...
The Micro Assignment Guided Inquiry and Collaboration (MAGIC) Method: A
In this article, we discuss Micro Assignment Guided Inquiry and Collaboration (MAGIC), an active learning method that draws on the merits of inquiry-based learning in STEM courses. We describe the use of Micro Assignments (MAs) consisting of a series of short, instructive guiding questions that scaffold the course material.
Sustainability
These methods include discussion-based learning, problem-based learning, case studies, and interdisciplinary projects . Through these interactive and student-centered learning approaches, students can apply their thinking skills in real-world contexts, leading to deeper understanding and improved skills [ 30 ].
What is inquiry based learning?
Inquiry based learning is an educational approach that focuses on children being investigators and problem solvers. Central to the approved learning frameworks, inquiry based learning encourages active involvement in learning, builds children's understanding of concepts, and builds the creative thinking and inquiry processes that are core aspects of lifelong learning.
Penerapan Model Pembelajaran Problem Based Learning Terhadap Kemampuan
Herliani, 2016. Penggunaan taksonomi SOLO (Structure of Observed Learning Outcomes) untuk meningkatkan keterampilan berpikir siswa pada Mata Pelajaran Biolgi SMA. Universitas Mulawarman. 13(1) (232-236) ISSN : 2528-5742. Helmon Arnoldus, 2018. Pengaruh Problem Based Learning (PBL) terhadap kemampuan berpikir kritis. Jurnal Inovasi Pendidikan ...
Reconstruction of unstable heavy particles using deep symmetry ...
Reconstructing unstable heavy particles is a crucial aspect of many analyses at the Large Hadron Collider (LHC). We introduce SPA-Net, a machine-learning approach to this problem which outperforms ...
Optimizations on Estimation and Positioning Techniques in Intelligent
Second, a pilot assignment problem in scalable open radio access network (O-RAN) cell-free massive MIMO (CFmMIMO) systems is studied, where a low-complexity pilot assignment scheme based on a multi-agent deep reinforcement learning (MA-DRL) framework along with a codebook search strategy is proposed.

Center for Teaching Innovation

Problem-Based Learning

Why Use Problem-Based Learning?

Considerations for Using Problem-Based Learning

Getting Started with Problem-Based Learning

Problem-Based Learning (PBL)

By working with PBL, students will:

How to Begin PBL

Designing Classroom Instruction

How to Orchestrate a PBL Activity

Teacher’s Role in PBL

The Role of the Students

Similar Posts

Robert Gagné’s Taxonomy of Learning

Open Source Learning Management Systems (LMS)

Adaptive Learning: What is It, What are its Benefits and How Does it Work?

Instructional Design

Concept Maps and How To Use Them

What is PBL? & 5 Problem-Based Learning Examples

Table of Contents

Think of a Real-World Context for the Problem

What Are Challenges in Problem-Based Learning?

Encourage Teachers in Your District to Innovate with Problem-Based Learning

What Are the Steps in Problem-Based Learning?

Choose a Central Concept of Principle

What Are Examples of Problem-Based Learning?

#1: Plan a Road Trip

#2: Create a Sustainable City

#3: Choose and Craft a Voyage Around the World

#4: Plan a Zoo Habitat

#5: Codebreak Math Equations

Alludo’s Take

What's New in the Alludo Catalog ?

Analytic vs. Holistic Rubrics: Which Type of Rubric Should You Use?

The Art of Rubric Design: Crafting Effective Assessment Tools

18 Problem-Based Learning Examples

Problem-Based Learning Examples

Benefits of Problem Based Learning

Case Studies

2. Collaborative PBL: Home for the Handicapped

3. Cybersecurity

4. Design a Board Game

5. Increasing Voter Registration

1 thought on “18 Problem-Based Learning Examples”

Leave a Comment Cancel Reply

Problem-Based Learning: Six Steps to Design, Implement, and Assess

Step One: Identify Outcomes/Assessments

Step Two: Design the Scenario

Step Three: Introduce PBL

Step Four: Research

Step Five: Product Performance

Step Six: Assessment

Stay Updated with Faculty Focus!

Welcome Back

Already a subscriber? log in here.

Where can I learn more?

Language →

Active learning for all

The illustration below shows the OpenPBL platform being used by a student to evaluate a colleague's performance, observing different aspects defined by the tutor.

Fundamentals

Explore examples of using OpenPBL in different learning scenarios and see how you can use this framework to enhance your classes with active learning methodologies.

Learn how to use the OpenPBL platform to form groups of students, submit activities with rubric-based assessment, self-assessment and peer assessment.

Understand how you can share an OpenPBL activity created in the app with other users. Also learn how to get an activity already created and shared into the app, so that you can change, mix, and reuse it together with your students.

This section presents the documentation about the application. The images are in English only, but the text is translated into your language. Thus, you have all the instructions for using the application to properly use its features.

OpenPBL components

Secondary menu

Don't have the Telegram app yet? Download and install.

Choosing Effective Assessments for PBL

Rigorous PBL by Design

Diversifying Our Assessments

Diversifying Our Use of Assessments With Students

Putting It Together

Project-Based Learning

Introduction

Implementing project-based learning

Defining the problem

Generating ideas

Prototyping solutions

Unstructured versus structured projects

You may also be interested in: