ap research statistical analysis

Find what you need to study

Academic Paper: Discussion and Analysis

5 min read • march 10, 2023

Dylan Black

Introduction

After presenting your data and results to readers, you have one final step before you can finally wrap up your paper and write a conclusion: analyzing your data! This is the big part of your paper that finally takes all the stuff you've been talking about - your method, the data you collected, the information presented in your literature review - and uses it to make a point!

The major question to be answered in your analysis section is simply "we have all this data, but what does it mean?" What questions does this data answer? How does it relate to your research question ? Can this data be explained by, and is it consistent with, other papers? If not, why? These are the types of questions you'll be discussing in this section.

Source: GIPHY

Writing a Discussion and Analysis

Explain what your data means.

The primary point of a discussion section is to explain to your readers, through both statistical means and thorough explanation, what your results mean for your project. In doing so, you want to be succinct, clear, and specific about how your data backs up the claims you are making. These claims should be directly tied back to the overall focus of your paper.

What is this overall focus, you may ask? Your research question ! This discussion along with your conclusion forms the final analysis of your research - what answers did we find? Was our research successful? How do the results we found tie into and relate to the current consensus by the research community? Were our results expected or unexpected? Why or why not? These are all questions you may consider in writing your discussion section.

You showing off all of the cool findings of your research! Source: GIPHY

Why Did Your Results Happen?

After presenting your results in your results section, you may also want to explain why your results actually occurred. This is integral to gaining a full understanding of your results and the conclusions you can draw from them. For example, if data you found contradicts certain data points found in other studies, one of the most important aspects of your discussion of said data is going to be theorizing as to why this disparity took place.

Note that making broad, sweeping claims based on your data is not enough! Everything, and I mean just about everything you say in your discussions section must be backed up either by your own findings that you showed in your results section or past research that has been performed in your field.

For many situations, finding these answers is not easy, and a lot of thinking must be done as to why your results actually occurred the way they did. For some fields, specifically STEM-related fields, a discussion might dive into the theoretical foundations of your research, explaining interactions between parts of your study that led to your results. For others, like social sciences and humanities, results may be open to more interpretation.

However, "open to more interpretation" does not mean you can make claims willy nilly and claim "author's interpretation". In fact, such interpretation may be harder than STEM explanations! You will have to synthesize existing analysis on your topic and incorporate that in your analysis.

Liam Neeson explains the major question of your analysis. Source: GIPHY

Discussion vs. Summary & Repetition

Quite possibly the biggest mistake made within a discussion section is simply restating your data in a different format. The role of the discussion section is to explain your data and what it means for your project. Many students, thinking they're making discussion and analysis, simply regurgitate their numbers back in full sentences with a surface-level explanation.

Phrases like "this shows" and others similar, while good building blocks and great planning tools, often lead to a relatively weak discussion that isn't very nuanced and doesn't lead to much new understanding.

Instead, your goal will be to, through this section and your conclusion, establish a new understanding and in the end, close your gap! To do this effectively, you not only will have to present the numbers and results of your study, but you'll also have to describe how such data forms a new idea that has not been found in prior research.

This, in essence, is the heart of research - finding something new that hasn't been studied before! I don't know if it's just us, but that's pretty darn cool and something that you as the researcher should be incredibly proud of yourself for accomplishing.

Rubric Points

Before we close out this guide, let's take a quick peek at our best friend: the AP Research Rubric for the Discussion and Conclusion sections.

Source: CollegeBoard

Scores of One and Two: Nothing New, Your Standard Essay

Responses that earn a score of one or two on this section of the AP Research Academic Paper typically don't find much new and by this point may not have a fully developed method nor well-thought-out results. For the most part, these are more similar to essays you may have written in a prior English class or AP Seminar than a true Research paper. Instead of finding new ideas, they summarize already existing information about a topic.

Score of Three: New Understanding, Not Enough Support

A score of three is the first row that establishes a new understanding! This is a great step forward from a one or a two. However, what differentiates a three from a four or a five is the explanation and support of such a new understanding. A paper that earns a three lacks in building a line of reasoning and does not present enough evidence, both from their results section and from already published research.

Scores of Four and Five: New Understanding With A Line of Reasoning

We've made it to the best of the best! With scores of four and five, successful papers describe a new understanding with an effective line of reasoning, sufficient evidence, and an all-around great presentation of how their results signify filling a gap and answering a research question .

As far as the discussions section goes, the difference between a four and a five is more on the side of complexity and nuance. Where a four hits all the marks and does it well, a five exceeds this and writes a truly exceptional analysis. Another area where these two sections differ is in the limitations described, which we discuss in the Conclusion section guide.

You did it!!!! You have, for the most part, finished the brunt of your research paper and are over the hump! All that's left to do is tackle the conclusion, which tends to be for most the easiest section to write because all you do is summarize how your research question was answered and make some final points about how your research impacts your field. Finally, as always...

Key Terms to Review ( 1 )

Research Question

Stay Connected

© 2024 Fiveable Inc. All rights reserved.

AP® and SAT® are trademarks registered by the College Board, which is not affiliated with, and does not endorse this website.

The Ultimate Guide to the AP Statistics Exam

The AP Statistics Exam is a comprehensive assessment that evaluates students' understanding of statistical concepts and their ability to apply statistical methods to analyze data. This guide provides detailed information to help students prepare for the exam and maximize their chances of success. Here are the key components of the ultimate guide to the AP Statistics Exam:

A. Exam Format and Content: 

The Exam Format and Content section is an essential component of the ultimate guide to the AP Statistics Exam. It provides a comprehensive overview of the structure, format, and content of the exam, ensuring that students have a clear understanding of what to expect on test day. Here are the key details covered in this section:

1. Exam Structure: The AP Statistics Exam consists of two sections: Section I - Multiple Choice and Section II - Free Response. Section I comprises 40 multiple-choice questions, while Section II consists of six free-response questions. The exam duration is 3 hours.

2. Multiple-Choice Questions: The multiple-choice section tests students' knowledge and understanding of statistical concepts and their ability to apply statistical methods. The questions may require students to interpret graphs, perform calculations, analyze data sets, and answer conceptual questions. Students must carefully read and analyze each question before selecting the best answer from the provided options.

3. Free-Response Questions: The free-response section assesses students' skills in applying statistical concepts to real-world scenarios. The questions are typically presented in a scenario-based format, requiring students to analyze data, design experiments, construct confidence intervals, perform hypothesis tests, and provide written explanations of their methods and conclusions. Students should demonstrate their ability to communicate statistical concepts effectively and justify their reasoning.

4. Investigative Task: One of the free-response questions in Section II is an investigative task, which requires students to design and implement a statistical investigation. They must formulate a research question, design a data collection plan, analyze the collected data, and draw conclusions based on their findings. The investigative task assesses students' ability to apply the entire statistical process, from formulating a question to making inferences and drawing conclusions.

5. Content Areas: The AP Statistics Exam covers a wide range of statistical topics, including data collection methods, exploratory data analysis, probability, random variables, sampling distributions, confidence intervals, hypothesis testing, regression analysis, and chi-square analysis. Students should have a solid understanding of these topics and be able to apply statistical methods and reasoning to solve problems and draw conclusions.

6. Calculator Usage: The AP Statistics Exam allows the use of graphing calculators. Students should be familiar with the functions and capabilities of their calculator and know how to use it effectively to perform calculations, create graphs, and analyze data. However, it is important to note that calculators cannot be used on all parts of the exam, particularly in certain sections of the free-response questions.

Understanding the exam format and content is crucial for effective exam preparation. By familiarizing themselves with the structure, question types, and content areas of the AP Statistics Exam, students can better focus their studies and develop appropriate strategies for success. This knowledge enables students to allocate their time effectively during the exam, apply the appropriate statistical techniques, and respond confidently to each question.

B. Course Content Review: 

The "Course Content Review" section is a fundamental part of the ultimate guide to the AP Statistics Exam. This section provides a comprehensive review of the key concepts and topics covered in the AP Statistics course, ensuring that students have a solid understanding of the content necessary to excel on the exam. Here is a detailed breakdown of what this section entails:

1. Data Collection Methods: This topic covers various methods used to collect data, such as surveys, experiments, observational studies, and sampling techniques. Students learn about the strengths and limitations of each method, as well as how to identify and address potential sources of bias.

2. Exploratory Data Analysis: Exploratory data analysis involves summarizing and visualizing data to identify patterns, relationships, and distributions. Students learn about measures of central tendency, measures of dispersion, histograms, box plots, scatterplots, and other graphical representations of data. They also gain an understanding of how to interpret and draw conclusions from these visualizations.

3. Probability: Probability is a fundamental concept in statistics. Students learn about the rules of probability, including addition, multiplication, and complementary probabilities. They study the concept of independence and conditional probability, as well as the application of probability in solving real-world problems.

4. Random Variables and Probability Distributions: This topic focuses on discrete and continuous random variables and their probability distributions. Students learn about probability mass functions and probability density functions, as well as expected value, variance, and standard deviation. They also explore common probability distributions, such as the binomial, geometric, normal, and exponential distributions.

5. Sampling Distributions: Sampling distributions involve understanding the behavior of sample statistics when repeatedly sampling from a population. Students learn about the sampling distribution of the mean and the sampling distribution of the proportion. They explore concepts such as the Central Limit Theorem and the standard error of a statistic.

6. Confidence Intervals: Confidence intervals provide a range of plausible values for population parameters. Students learn how to construct confidence intervals for means, proportions, and the difference between means or proportions. They also understand the interpretation of confidence intervals and how to interpret them in the context of a problem.

7. Hypothesis Testing: Hypothesis testing is used to make inferences and draw conclusions about population parameters. Students learn about null and alternative hypotheses, test statistics, p-values, and significance levels. They develop the ability to conduct hypothesis tests for means, proportions, and the difference between means or proportions.

8. Regression Analysis: Regression analysis involves examining the relationship between two or more variables. Students learn how to perform linear regression, interpret regression coefficients, and assess the goodness of fit of a regression model. They also explore concepts such as correlation, residual analysis, and the interpretation of regression output.

9. Chi-Square Analysis: Chi-square analysis is used to test the independence or association between categorical variables. Students learn about contingency tables, expected frequencies, chi-square test statistics, and the interpretation of chi-square results. They also explore the application of chi-square analysis in various contexts.

Throughout the Course Content Review section, students are provided with explanations, examples, and practice problems to reinforce their understanding of each topic. It is crucial for students to engage actively with the material, ask questions, and seek clarification on any challenging concepts. A strong grasp of the course content will not only facilitate success on the AP Statistics Exam but also provide a solid foundation for future statistical analysis and understanding.

C. Study Strategies and Tips: 

The "Study Strategies and Tips" section is a crucial part of the ultimate guide to the AP Statistics Exam. It provides valuable guidance on how to effectively prepare for the exam and optimize study efforts. Here are some detailed study strategies and tips to help students excel in their AP Statistics preparations:

1. Create a Study Schedule: Establish a study schedule that suits your learning style and commitments. Set aside dedicated study time each day or week, ensuring you allocate ample time for each topic and allow for review sessions closer to the exam date. A well-structured schedule helps you stay organized and focused throughout your study period.

2. Review Course Materials: Start by reviewing your class notes, textbooks, and other course materials to refresh your memory on key concepts and topics. Pay attention to areas that you find challenging and prioritize them in your study plan. Ensure you have a good understanding of the fundamental concepts before moving on to more complex topics.

3. Utilize Online Resources: Take advantage of online resources specific to AP Statistics, such as educational websites, video tutorials, and practice exams. These resources can provide alternative explanations, additional examples, and interactive tools to reinforce your understanding of statistical concepts. Khan Academy and College Board's official AP Statistics resources are excellent starting points.

4. Practice with Past Exams: Familiarize yourself with the structure and format of the AP Statistics Exam by practicing with past exams. The College Board website offers a collection of past AP Statistics exams and scoring guidelines. Work through the questions under timed conditions to simulate the exam experience and assess your progress. Analyze your answers and review the scoring guidelines to identify areas for improvement.

5. Work on Sample Problems: In addition to past exams, practice sample problems and questions to strengthen your problem-solving skills. Seek out a variety of practice problems that cover different concepts and question types. Practice interpreting graphs, performing calculations, and applying statistical methods to real-world scenarios. This helps reinforce your understanding and builds confidence in tackling similar problems on the exam.

6. Form or Join Study Groups: Collaborate with fellow students by forming or joining study groups. Engaging in group discussions allows you to share insights, clarify doubts, and reinforce your understanding through peer teaching. Work on practice problems together, explain concepts to each other, and discuss challenging topics. Teaching others can deepen your understanding of the material.

7. Seek Help from Teachers or Tutors: If you encounter difficulties in certain areas, don't hesitate to seek help from your teacher or a tutor. They can provide additional explanations, clarify concepts, and offer personalized guidance. They may also provide additional resources or practice materials to supplement your studies.

8. Take Breaks and Maintain Balance: While it's important to dedicate focused study time, remember to take regular breaks to rest and recharge. This helps prevent burnout and promotes better retention of information. Maintain a healthy balance by incorporating physical activity, relaxation techniques, and hobbies into your study routine.

9. Review and Self-Assess: As you progress through your studies, regularly review the topics you've covered. Summarize key concepts in your own words to reinforce your understanding. Self-assess your knowledge by periodically testing yourself with practice questions or creating flashcards for important definitions and formulas.

10. Simulate Exam Conditions: Prior to the actual exam, simulate exam conditions by taking practice exams under timed conditions. This helps you develop time management skills and familiarize yourself with the pacing required to complete the exam within the allocated time. Practicing under exam-like conditions can also help reduce anxiety and build confidence.

Remember, everyone's study style and preferences may differ, so adapt these strategies to suit your needs. Experiment with different techniques to identify what works best for you. Consistent and focused effort, coupled with effective study strategies, will enhance your understanding of AP Statistics and increase your chances of success on the exam

D. Problem-Solving Techniques: 

Mastering problem-solving techniques is essential for success in the field of statistics and on the AP Statistics Exam. This section will delve into the various problem-solving techniques that can help you tackle complex statistical problems effectively. Here are some key techniques to consider:

1. Understand the Problem: Begin by thoroughly understanding the problem statement. Identify the key variables, data sets, and any specific conditions or constraints mentioned. Clarify any ambiguities or uncertainties to ensure you have a clear understanding of what is being asked.

2. Break Down the Problem: Break down the problem into smaller, manageable parts. Identify the specific concepts or statistical methods that are relevant to each part. This helps you approach the problem systematically and ensures that you address all necessary components.

3. Visualize the Data: Use visualizations, such as graphs or charts, to gain insights into the given data. Visual representations can help identify patterns, trends, or outliers that are crucial for understanding the problem and determining appropriate statistical approaches.

4. Choose the Right Method: Select the appropriate statistical method or technique to solve the problem. This requires a solid understanding of the different statistical concepts and methods covered in the AP Statistics course. Consider the nature of the data, the research question, and the assumptions underlying each method to make an informed choice.

5. Apply Formulas and Procedures: Once you have identified the relevant statistical method, apply the appropriate formulas or procedures. Ensure that you understand how to use the formulas correctly and know when and where to apply them. Pay attention to any assumptions or conditions that must be met for the method to be valid.

6. Show Your Work: When solving statistical problems, it's crucial to show your work step by step. Clearly outline the formulas used, provide intermediate calculations, and explain the rationale behind each step. This not only helps you organize your thoughts but also enables the grader to understand your reasoning and award partial credit, if applicable.

7. Interpret Your Results: After obtaining the numerical solution, interpret your findings in the context of the problem. Clearly explain the implications and draw meaningful conclusions based on the statistical analysis. Consider the practical significance of the results and discuss any limitations or potential sources of error.

8. Practice with a Variety of Problems: The key to becoming proficient in problem-solving is consistent practice. Work on a wide range of statistical problems that encompass different concepts, methods, and scenarios. Practice helps you develop problem-solving intuition, strengthens your analytical skills, and builds confidence in approaching unfamiliar problems.

9. Review Mistakes and Learn from Them: Analyze your mistakes and learn from them. If you encounter difficulties or errors, take the time to understand where you went wrong and why. Review the correct solution and identify any gaps in your understanding. This iterative process helps refine your problem-solving skills and prevents similar mistakes in the future.

10. Seek Assistance and Collaboration: Don't hesitate to seek help or collaborate with others when encountering challenging problems. Engage in discussions with classmates, teachers, or online forums to gain alternative perspectives and insights. Exploring different approaches to problem-solving can deepen your understanding and broaden your problem-solving toolkit.

Remember, problem-solving in statistics requires a combination of conceptual understanding, mathematical skills, and logical reasoning. By practicing these techniques, developing your problem-solving abilities, and applying them to a variety of statistical problems, you will become more confident and proficient in tackling the challenges presented by the AP Statistics Exam and real-world statistical scenarios.

Final Thoughts

By following the ultimate guide to the AP Statistics Exam, students can gain a comprehensive understanding of the exam format, review the necessary content, practice problem-solving techniques, and develop effective study strategies. With thorough preparation and diligent practice, students can approach the AP Statistics Exam with confidence and maximize their chances of earning a high score.

You Might Also Like

Guidelines To Write Impressive High School Resume

Know some important guidelines will help you write an effective high school resume that will stand out in the crowd. Check out resume sample here

Post Scholarship Application Steps to Follow

So what Happens post Submission? What are the things and factors to keep in mind. This Guide covers all the factors in and around the scholarship

The Secret Behind Early Applications

If you are serious about taking admission to your dream university/college, early decision admission program is best option for you, have a look! - AP Guru

Free Resources

Choose Your Test

Sat / act prep online guides and tips, the ultimate guide to the ap statistics exam.

Advanced Placement (AP)

Are you taking AP Statistics? If so, you're likely wondering what to expect from the AP Statistics exam. Before you sit down to take the final test, it's important to understand how the AP Stats test is formatted, what topics it will cover, and how it'll be scored.

This guide will explain all of that information, show you official sample problems and give you tips on the best way to prepare for the AP Statistics test.

In 2023, the AP Statistics exam will take place on Thursday, May 5th at 12:00pm.

How Is the AP Statistics Exam Structured?

How long is the AP Statistics exam? The test is a total of three hours long and contains two sections: multiple choice and free response. You're allowed a graphing calculator for the entire exam.

Multiple-Choice Section

• 40 multiple-choice questions
• 90 minutes long
• Worth 50% of exam score
• You can spend an average of a little more than two minutes on each multiple-choice question and finish the section in time.

Free-Response Section

• 5 short-answer questions
• 1 Investigative Task
• The five short-answer questions are meant to each be solved in about 12 minutes, and the Investigative Task is meant to be solved in about 30 minutes.

What Does the AP Statistics Exam Test You On?

The content of the AP Stats exam and course is centered around nine units. Below are the nine units, along with what percentage of the exam will be on them and all the topics that fall beneath each of them. Each unit starts with an "introducing statistics" question that'll be answered throughout the unit. The list below covers every single topic that the AP Statistics exam could test you on.

Unit 1: Exploring One-Variable Data (15-23% of exam)

• Introducing statistics: What can we learn from data?
• Representing a categorical variable with tables
• Representing a categorical variable with graphs
• Representing a quantitative variable with tables
• Describing the distribution of a quantitative variable
• Summary statistics for a quantitative variable
• Graphical representations of summary statistics
• Comparing distributions of a quantitative variable
• The normal distribution

Unit 2: Exploring Two-Variable Data (5-7% of exam)

• Introducing statistics: Are variables related?
• Representing two categorical variables
• Statistics for two categorical variables
• Representing the relationship between two quantitative variables
• Correlation
• Linear regression models
• Least squares regression
• Analyzing departures from linearity

Unit 3: Collecting Data (12-15% of exam)

• Introducing statistics: Do the data we collected tell the truth?
• Introduction to planning a study
• Random sampling and data collection
• Potential problems with sampling
• Introduction to experimental design
• Selecting an experimental design
• Inference and experiments

Unit 4: Probability, Random Variables, and Probability Distributions (10-20% of exam)

• Introducing statistics: Random and non-random patterns?
• Estimating probabilities using simulation
• Introduction to probability
• Mutually exclusive events
• Conditional probability
• Independent events and unions of events
• Introduction to random variables and probability distributions
• Mean and standard deviation of random variables
• Combining random variables
• Introduction to the binomial distribution
• Parameters for a binomial distribution
• The geometric distribution

Unit 5: Sampling Distributions (7-12% of exam)

• Introducing statistics: Why is my sample not like yours?
• The normal distribution, revised
• The Central Limit Theorem
• Biased and unbiased point estimates
• Sampling distributions for sample proportions
• Sampling distributions for differences in sample proportions
• Sampling distributions for sample means
• Sampling distributions for differences in sample means

Unit 6: Inference for Categorical Data: Proportions (12-15% of exam)

• Introducing statistics: Why be normal?
• Constructing a confidence interval for a population proportion
• Justifying a claim based on a confidence interval for a population proportion
• Setting up a test for a population proportion
• Interpreting p- values
• Concluding a test for a population proportion

Unit 7: Inference for Quantitative Data: Means (10-18% of exam)

• Introducing statistics: Should I worry about error?
• Constructing a confidence interval for a population mean
• Justifying a claim about a population mean based on a confidence interval
• Setting up a test for a population mean
• Carrying out a test for a population mean

Unit 8: Inference for Categorical Data: Chi-Square (2-5% of exam)

• Introducing statistics: Are my results unexpected?
• Setting up a chi-square goodness of fit test
• Carrying out a chi-square test for goodness of fit
• Expected counts in two-way tables
• Setting up a chi-square test for homogeneity or independence
• Carrying out a chi-square test for homogeneity or independence
• Skills focus: Selecting an appropriate inference procedure for categorical data

Unit 9: Inference for Quantitative Data: Slopes (2-5% of exam)

• Introducing statistics: Do those points align?
• Confidence intervals for the slope of a regression model
• Justifying a claim about the slope of a regression model based on a confidence interval
• Setting up a test for the slope of a regression model
• Carrying out a test for the slope of a regression model
• Skills focus: Selecting an appropriate inference procedure

AP Statistics Sample Questions

As we mentioned above, there are three types of questions on the AP Stats exam: multiple choice, short answer, and investigative task. Below are examples of each question type. You can see more sample questions and answer explanations in the AP Statistics Course Description.

Multiple-Choice Sample Question

There are 40 multiple-choice questions on the exam. Each has five answer options. Some questions will be accompanied by a chart or graph you need to analyze to answer the question.

Short-Answer Sample Question

There are five short-answer questions on the AP Stats test. Each of these questions typically includes several different parts you need to answer. You're expected to spend about 12 minutes on each short-answer question.

Investigative Task Sample Question

The final question on the exam is the Investigative Task question. This is the most in-depth question on the test, and you should spend about 30 minutes answering it. It will have multiple parts you need to answer and require multiple statistics skills. You'll also need to provide a detailed explanation of your answers that shows the strength of your statistics skills. Be sure to show all your work as you'll be graded on the completeness of your answer.

How Is the AP Statistics Test Graded?

For the multiple-choice part of the exam, you earn one point for each question you answer correctly. There are no point deductions for incorrect answers or questions you leave blank. Official AP graders will grade your free-response questions. Each of the six free-response questions is scored on a scale of 0 to 4 points, so the total section is out of 24 points.

The free-response questions are graded holistically , which means, instead of getting a point or half a point for each bit of correct information you include, graders look at your answer to each question as a "complete package," and your grade is awarded on the overall quality of your answer. The grading rubric for each free-response question is:

• 4: Complete Response: Shows complete understanding of the problem's statistical components
• 3: Substantial Response: May include arithmetic errors, but answers are still reasonable and show substantial understanding of the problem's statistical components
• 2: Developing Response: May include errors that result in some unreasonable answers, but shows some understanding of the problem's statistical components
• 1: Minimal Response: Misuses or fails to use appropriate statistical techniques and shows only a limited understanding of statistical components by failing to identify important components
• 0: No Response: Shows little or no understanding of statistical components

What does holistic grading mean for you? Basically, you can't expect to earn many points by including a few correct equations or arithmetic answers if you're missing key statistical analysis. You need to show you understand how to use stats to get a good score on these questions.

Estimating Your AP Statistics Score

If you take a practice AP Stats exam (which you should!) you'll want to get an estimate of what your score on it is so you can get an idea of how well you'd do on the real exam. To estimate your score, you'll need to do a few calculations.

#1: Multiply the number of points you got on the multiple-choice section by 1.25

#2: For free-response questions 1 through 5, add the number of points you got together and multiply that sum by 1.875 (don't round). If you need help estimating your score, the official free-response questions we linked to above include sample responses to help you get an idea of the score you'd get for each question.

#3: For free-response question #6, multiply your score by 3.125.

#4: Add the scores you got in steps 1-3 together to get your Composite Score.

For example, say you got 30 questions correct on the multiple-choice section, 13 points on questions 1-5, and 2 points on question 6. Your score would be (30 x 1.25) + (13 x 1.875) + (2 x 3.125) = 68.125 which rounds to 68 points. By looking at the chart below, you can see that'd get you a 4 on the AP Statistics exam.

Below is a conversion chart so you can see how raw score ranges translate into final AP scores. I've also included the percentage of students who earned each score in 2021 to give you an idea of what the score distribution looks like:

Source: The College Board

Where Can You Find Practice AP Stats Tests?

Practice tests are an important part of your AP Stats prep. There are official and unofficial AP Stats practice tests available, although we always recommend official resources first. Below are some of the best practice tests to use.

Official Practice Tests

• 2012 Complete Practice Test
• 1997 Complete Practice Test
• Free-Response Questions 1998-2021

To learn more about where to find AP Statistics practice tests and how to use them, check out our complete guide to AP Statistics practice exams.

3 Tips for the AP Statistics Exam

In this section we go over three of the most useful tips you can use when preparing for and taking the AP Statistics test. Follow these and you're more likely to get a great score on the exam.

#1: For Free Response, Answer the Entire Question

As we mentioned earlier, free-response questions on AP Stats are graded holistically, which means you'll get one score for the entire question. This is different from many other AP exams where each correct component you include in a free-response question gets you a certain number of points, and those points are then added up to get your total score for that question.

The Stats free-response questions are graded holistically because there are often multiple correct answers in statistics depending on how you solve the problem and explain your answer. This means you can't just answer part of the question and expect to get a good score, even if you've answered that part perfectly. If you've ignored a large part of the problem, your score will be low no matter what.

So instead of trying to get a point here and there by including a correct formula or solving one part of a question, make sure you're looking at the entire problem and answering it as completely as possible. Also, if you need to include an explanation, be sure it explains your thought process and the steps you took. If your explanation shows you understand important stats concepts, it could help you get a higher score even if your final answer isn't perfect.

Aiming for the most complete response possible is also important if you can't answer one part of a question that's needed to answer other parts. For example, if you can't figure out what the answer to part A is, but you need to use that answer for parts B and C, just make up an answer (try to keep it logical), and use that answer to solve the other parts, or explain in detail how you'd solve the problem if you knew what the answer to part A was. If you can show you know how to solve the latter problems correctly, you'll likely get some credit for showing you understand the stats concepts being tested.

#2: Know How to Use Your Calculator

You'll need a graphing calculator to answer pretty much every question on the Stats exam, so make sure you know how to use it. Ideally, the calculator you use on test day will be the same one you've been doing homework and taking tests with throughout the school year so you know exactly how to use it.

Knowing how to solve common stats functions on your calculator and interpret the answers you get will save you a lot of time on the exam. Your calculator will likely be most useful on the multiple-choice section where you don't need to worry about showing work. Just plug the data you're given into your calculator, and run the right equations. Then you'll have your answer!

#3: Know Your Vocabulary

You may think that since AP Stats is a math course, vocab won't be an important part of the test, but you need to know quite a few terms to do well on this exam. Confusing right- and left-skewed or random sampling and random allocation, for example, could lead to you losing tons of points on the test.

During the school year, stay on top of any new terms you learn in class. Making flashcards of the terms and quizzing yourself regularly is a great way to stay up-to-date on vocab. Many AP Stats prep books also include a glossary of important terms you can use while studying.

Before the AP Stats exam, you should know all important terms like the back of your hand. Having a general idea isn't good enough. A big part of stats is being able to support your answers, and to do this you'll often need to use stats vocab in your explanations. Just stating the term won't earn you nearly as many points as being able to explain what the term is and how it supports your answer, so make sure you really know your vocab well.

Summary: Statistics AP Exam

The AP Statistics exam is three hours long and consists of 40 multiple-choice questions and six free-response questions. To prepare well for AP Stats exam questions, it's important to take practice exams and know how to grade them so you can estimate how well you'd do on the actual test. When studying for the AP exam, remember to answer the entire question for free response, know how to use your calculator, and be on top of stats vocabulary.

What's Next?

Feel the need to do some quick reviewing after looking through what'll be covered on the AP Stats exam? Take a spin through our guide to statistical significance to refresh yourself on how to run a t-test .

How difficult is AP Stats compared to other AP classes? Get the answer by reading our guide to the hardest AP exams and classes .

Wondering which other math classes you should take besides statistics? Math is often the trickiest subject to choose classes for, but our guide will help you figure out exactly which math classes to take for each year of high school.

A prep book can be one of your best study resources for the AP Stats exam. But which prep book should you choose? Check out our guide to AP Stats prep books to learn which is the best and which you should avoid .

Christine graduated from Michigan State University with degrees in Environmental Biology and Geography and received her Master's from Duke University. In high school she scored in the 99th percentile on the SAT and was named a National Merit Finalist. She has taught English and biology in several countries.

Student and Parent Forum

Our new student and parent forum, at ExpertHub.PrepScholar.com , allow you to interact with your peers and the PrepScholar staff. See how other students and parents are navigating high school, college, and the college admissions process. Ask questions; get answers.

Ask a Question Below

Have any questions about this article or other topics? Ask below and we'll reply!

Improve With Our Famous Guides

• For All Students

The 5 Strategies You Must Be Using to Improve 160+ SAT Points

How to Get a Perfect 1600, by a Perfect Scorer

Series: How to Get 800 on Each SAT Section:

Score 800 on SAT Math

Score 800 on SAT Reading

Score 800 on SAT Writing

Series: How to Get to 600 on Each SAT Section:

Score 600 on SAT Math

Score 600 on SAT Reading

Score 600 on SAT Writing

Free Complete Official SAT Practice Tests

What SAT Target Score Should You Be Aiming For?

15 Strategies to Improve Your SAT Essay

The 5 Strategies You Must Be Using to Improve 4+ ACT Points

How to Get a Perfect 36 ACT, by a Perfect Scorer

Series: How to Get 36 on Each ACT Section:

36 on ACT English

36 on ACT Math

36 on ACT Reading

36 on ACT Science

Series: How to Get to 24 on Each ACT Section:

24 on ACT English

24 on ACT Math

24 on ACT Reading

24 on ACT Science

What ACT target score should you be aiming for?

ACT Vocabulary You Must Know

ACT Writing: 15 Tips to Raise Your Essay Score

How to Get Into Harvard and the Ivy League

How to Get a Perfect 4.0 GPA

How to Write an Amazing College Essay

What Exactly Are Colleges Looking For?

Is the ACT easier than the SAT? A Comprehensive Guide

Should you retake your SAT or ACT?

When should you take the SAT or ACT?

Stay Informed

Get the latest articles and test prep tips!

Looking for Graduate School Test Prep?

Check out our top-rated graduate blogs here:

GRE Online Prep Blog

GMAT Online Prep Blog

TOEFL Online Prep Blog

Holly R. "I am absolutely overjoyed and cannot thank you enough for helping me!”

Analytical Psychology

AP Psychology

Research Methods

CollegeBoard

University/Undergrad

AP Psychology - Research Methods & Statistical Analysis 

● Types of Research

○ Experiments

○ Correlational studies

■ Relationship between two events

■ It is not enough to simply find a relationship

■ Significance is when the relationship is not due to chance (see below) ■ Advantage: they happen outside of the lab

■ Disadvantage: they do not prove causality (x causes y).

■ Usually showed in scatterplots

● If the line goes up and to the right, the relationship is positive ● If the points are completely random, there is no relationship ● If the line goes down and to the right, the relationship is negative ● Correlation coefficient : a statistical measure of how strongly related any two sets of scores are (aka x and y)

○ Between +1.0 and -1.0

○ +1.0, one set of scores increases in direct proportion to

increases in the other set (positive relationship)

○ 0, there is no relationship

○ -1.0, one set of scores goes up precisely as the other goes

○ The stronger the relationship, the closer to -1 or 1. The

weaker it is, the closer it is to 0.

○ There is a perfect correlation for coefficients of either 1 or

○ Survey Research

■ Includes giving participants lists of questions to be answered, which can be delivered as paper-and-pencil questionnaires, administered

electronically, or conducted verbally.

■ Advantages: cheap because of free software and websites make distributing these easily, they’re easy because people without training can

make surveys, and, finally, multiple variables can be examined at the same time

■ Disadvantaged: badly written/biased questions, people might misinterpret questions, it only provides “shallow” information (broad but not deep), and can not prove causality

○ Naturalistic Observations

■ Observing behavior in its natural setting where observers are normally unnoticed

■ Advantage: real behavior (not affected by lab settings), no reduced Hawthorne Effect or Demand Characteristic

■ Disadvantage: Might not be studying what you think you’re studying (if someone comes to your school, they’re not really going to get a valid picture because every day is different), there’s no control over the experiment, and you cannot prove causality

○ Case Studies

■ A study that focuses on one person or simply a few individuals ■ Examples include Phineas Gage (the guy with a rod in his head in the 1800s) or Clive Wearing (the guy who forgets things a lot)

■ Advantages: researchers can dig deep to find relationships or other ideas to make a hypothesis, and it’s the opposite of a survey

■ Disadvantages: time-consuming, subjects to not represent all of society, and these cannot prove causality

○ Longitudinal Studies

■ When the researcher follows the same participant or same group (Cohort) of participants for an extended period

■ Advantages: you get to find new confounding variables, can be used in a variety of other research methods (case studies, surveys [2 surveys over four years])

■ Disadvantages: time-consuming, expensive, difficult to control (odd variables may show up — maybe a car accident), does not prove causality ○ Cross-Sectional Studies

■ When a researcher compares multiple segments of the population at the same time (Example: online vs face to face education)

■ Advantages: you gather data immediately, can be used in other research methods (2 opposing case studies)

■ Disadvantages: only a snapshot (example: the one day your parents check your grade may be the day your teacher accidentally put in a missing

assignment)

● Operational Definition (for this study, we define sadness as the number of tears you cry ) and Measurement (FRQ’s will use these words: operationally defining, operationalize)

○ You must define your hypothesis

■ Example: how effective is online school?

■ What is an online school? Is it using Zoom? Google Classroom?

■ How do you measure it? GPA? Student feelings?

○ Empiricism : measurable and observable

■ Your hypothesis should have empiricism

● Validity and Research Design

○ You must have facts

○ Everything must be defined (sadness = the number of tears shed)

■ Advantages: allows for scientific measurement

■ Disadvantage: oversimplifies the issue (sadness isn’t just the number of tears you cry )

Experimental Method and Ethical Guidelines

● Independent Variable (IV) — Given to the Experimental (Treatment) Group ○ Also called “the treatment”

○ Think of the question as a hypothesis…

■ If _____ then _____

■ The “if” is the IV

● Dependent Variable (DV)

○ Data the researcher receives (the results)

○ You’re looking for what changes

○ There are two possible errors with DV’s

■ False alarm, there really wasn’t actually a relationship

■ There was some sort of change, but you don’t notice (example: side

effects that you don’t notice until later cause you weren’t looking for

● Control Variables : steps taken by researchers to eliminate any confounding variables from the experimental design so that the only difference between the control and experimental groups is the independent variable (NOT THE CONTROL GROUP) ● Confounding Variables (lurking variable, research bias, bias)

○ Variables that may affect the result of DV that aren’t the IV

○ The more control you have, the less bias comes through (you eliminate every other variable; yay!). However, if there is too much control, people will not behave normally (increase the Hawthorne Effect: people act differently when being watched)

○ Self-Report Bias (surveys only): people may have one idea about themselves, but that’s actually false (you think you’re good at singing but you REALLY

■ When people's expectations or beliefs influence or determine their

experience in a given situation

■ Placebo is good for the individual, the placebo effect is bad for research, but the placebo treatment is necessary (given to control group)

■ Example: if studying the effect of caffeine, give experimental group coffee and the control group decaf — they both think they got coffee, but how to actions change? Make sure the coffees have the same temp, taste, etc.

○ Hawthorne Effect (Demand Characteristic)

■ To eliminate, use the single-blind technique (the participant is deceived about something: usually, if they got the placebo or IV)

● Deception is unethical, however

● Participant doesn’t know the hypothesis either

○ Experimenter Bias

■ To eliminate, use the double-blind technique

● The researcher also doesn’t know which participant is in which

○ Representation (Representative) Bias (Participants are not representative of the larger population, think demographics

● Random Sample vs Representative Sample

○ Random Sample

■ Everyone in the population has the same chance to be chosen for the sample group (usually impossible, because if you ask for volunteers from your social media, you’ll be limited in that, or if you pass out flyers to random students, some students might be sick)

○ Representative Sample

■ The same proportion of demographic characteristics

■ Example: the average age of the US population is 38, but does your sample have the same average age?

● Random Selection vs Random Assignment

○ Random selection

■ See above

○ Random assignment

■ Those participants have an equal chance of being in either the

experimental group or the control group

■ This helps reduce experimenter bias

● Ethical Issues in Research

○ All experiments cause some anxiety or stress (maybe somebody is late for work but agreed to do this)

○ Some studies need to cause stress or anxiety

■ Is the information worth the pain?

■ Who benefits from this?

○ Institutional Review Board (I.R.B.)

■ Experiments must be proposed to this board to make sure ethics are

○ Placebo Effect: is it okay to lie to people to look at the results?

○ Fiduciary Responsibility

■ Researchers must put the participant first, even if it means stopping an experiment

○ Informed Consent

■ The subject understands what is going on

● Minors may be too young, people with Alzheimer’s may not

● However, this institutes the Hawthorne Effect

○ Debriefing

■ After the study, the researcher must tell everything to the participant and offer counseling (if stress was involved). Includes information that the

participant may not enjoy, like that the researcher lied.

○ Right to Decline

■ Subjects pay quit participation at any time

○ Confidentiality

■ the researcher/therapist cannot disclose confidential communications to any third party unless mandated or permitted by law to do so

● Considering the Strength of an Experiment

○ Possible ideas: is it easy to replicate (i.e. duplicating an experiment to strengthen confidence in the findings by obtaining the same or similar results), what about ethicality (did they use animals so no humans were harmed and no deception was needed?), or goal (could it lead to the development of a medication?)

Statistical Analysis in Psychology

● Statistics

○ Data exploration techniques designed to aid in the comprehension of large amounts of data

● Inferential Statistics

○ When a researcher analyzes numerical data in order to determine the likelihood that the given findings are a result of systematic (aka apply to a larger population) rather than random fluctuations or events.

○ Let’s say P represents an error, chance, confounding variables, everything that makes an experiment messy

○ If P = .05, there is a 95% chance that your experiment is correct. Anything below this makes the difference not significant and the experiment is not good. ■ P is significant when <.05

○ Null Hypothesis

■ A prediction that there will not be a significant relationship, or that you cannot take a small sample and apply it to a larger population

■ The goal is to disprove the null hypothesis, which, in inferential statistics, predicts the results cannot be applied to a larger population (basically, your P will be big)

○ Inferential Errors

■ When a researcher links a smaller group to a larger population when a link does not exist.

● Descriptive Statistics

○ Numerical analysis that summarizes quantitative information about a population (usually the smaller one)

○ Mean : the arithmetic average of scores summed and divided

○ Median : the score that appears exactly in the middle of the whole set ○ Mode : the most frequently occurring score

○ The median and mode are relatively unaffected by the presence of a few extreme scores (outliers), but one extreme score can throw off the mean

○ Skewed Data : occurs when an outlier disproportionately affects the mean ■ Skewed negatively : there’s an extreme outlier that’s really low and pulls the score down more than it should

■ Skewed positively : there’s an extreme outlier that’s really high and pulls the score up more that it should

■ IV on x, DV on y

■ Bar charts

● Categorical data (each bar is a category, vanilla ice cream,

chocolate ice cream)

● Columns don’t touch!

■ Histograms

● Orginal (data can be put

● Columns touch

● Numerical

○ Dispersion (Measures of Variability)

■ Measures how far the scores differ from the mean

■ Low variability means the scores are close together (high amplitude on graphs)

■ High variability means the scores are spread out (low amplitude on graphs)

■ The range is the highest score minus the lowest

■ Standard deviation: people usually vary about x amount from the average ● The further from the mean you are, the rarer that score is

■ A set of scores with a small range has a small standard deviation ○ Z-Scores

■ Same thing as a standard deviation

■ Helps researchers compare scores from different studies with different standard deviations (used usually when there is a different number of participants)

■ Scores below the mean have a negative Z-score and vice versa ● Example: if Chuck scored a 40 on a test with a mean score of 38 and the standard deviation was 4, what is his z-score? Answer: .5

AP ® Research Handbook

Research philosophy & ethics.

• No Plagiarism
• Institutional Review Board
• Data privacy standards

This AP ® Research Handbook 1 is still a work in progress. Please excuse any blank sections and filler text.

*AP is a registered tradebook of the College Board, which does not endorse and is not involved in the ongoing production of this handbook. ↩

How to conduct a meta-analysis in eight steps: a practical guide

• Open access
• Published: 30 November 2021
• Volume 72 , pages 1–19, ( 2022 )

Cite this article

You have full access to this open access article

• Christopher Hansen 1 ,
• Holger Steinmetz 2 &
• Jörn Block 3 , 4 , 5  

146k Accesses

45 Citations

158 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

1 Introduction

“Scientists have known for centuries that a single study will not resolve a major issue. Indeed, a small sample study will not even resolve a minor issue. Thus, the foundation of science is the cumulation of knowledge from the results of many studies.” (Hunter et al. 1982 , p. 10)

Meta-analysis is a central method for knowledge accumulation in many scientific fields (Aguinis et al. 2011c ; Kepes et al. 2013 ). Similar to a narrative review, it serves as a synopsis of a research question or field. However, going beyond a narrative summary of key findings, a meta-analysis adds value in providing a quantitative assessment of the relationship between two target variables or the effectiveness of an intervention (Gurevitch et al. 2018 ). Also, it can be used to test competing theoretical assumptions against each other or to identify important moderators where the results of different primary studies differ from each other (Aguinis et al. 2011b ; Bergh et al. 2016 ). Rooted in the synthesis of the effectiveness of medical and psychological interventions in the 1970s (Glass 2015 ; Gurevitch et al. 2018 ), meta-analysis is nowadays also an established method in management research and related fields.

The increasing importance of meta-analysis in management research has resulted in the publication of guidelines in recent years that discuss the merits and best practices in various fields, such as general management (Bergh et al. 2016 ; Combs et al. 2019 ; Gonzalez-Mulé and Aguinis 2018 ), international business (Steel et al. 2021 ), economics and finance (Geyer-Klingeberg et al. 2020 ; Havranek et al. 2020 ), marketing (Eisend 2017 ; Grewal et al. 2018 ), and organizational studies (DeSimone et al. 2020 ; Rudolph et al. 2020 ). These articles discuss existing and trending methods and propose solutions for often experienced problems. This editorial briefly summarizes the insights of these papers; provides a workflow of the essential steps in conducting a meta-analysis; suggests state-of-the art methodological procedures; and points to other articles for in-depth investigation. Thus, this article has two goals: (1) based on the findings of previous editorials and methodological articles, it defines methodological recommendations for meta-analyses submitted to Management Review Quarterly (MRQ); and (2) it serves as a practical guide for researchers who have little experience with meta-analysis as a method but plan to conduct one in the future.

2 Eight steps in conducting a meta-analysis

2.1 step 1: defining the research question.

The first step in conducting a meta-analysis, as with any other empirical study, is the definition of the research question. Most importantly, the research question determines the realm of constructs to be considered or the type of interventions whose effects shall be analyzed. When defining the research question, two hurdles might develop. First, when defining an adequate study scope, researchers must consider that the number of publications has grown exponentially in many fields of research in recent decades (Fortunato et al. 2018 ). On the one hand, a larger number of studies increases the potentially relevant literature basis and enables researchers to conduct meta-analyses. Conversely, scanning a large amount of studies that could be potentially relevant for the meta-analysis results in a perhaps unmanageable workload. Thus, Steel et al. ( 2021 ) highlight the importance of balancing manageability and relevance when defining the research question. Second, similar to the number of primary studies also the number of meta-analyses in management research has grown strongly in recent years (Geyer-Klingeberg et al. 2020 ; Rauch 2020 ; Schwab 2015 ). Therefore, it is likely that one or several meta-analyses for many topics of high scholarly interest already exist. However, this should not deter researchers from investigating their research questions. One possibility is to consider moderators or mediators of a relationship that have previously been ignored. For example, a meta-analysis about startup performance could investigate the impact of different ways to measure the performance construct (e.g., growth vs. profitability vs. survival time) or certain characteristics of the founders as moderators. Another possibility is to replicate previous meta-analyses and test whether their findings can be confirmed with an updated sample of primary studies or newly developed methods. Frequent replications and updates of meta-analyses are important contributions to cumulative science and are increasingly called for by the research community (Anderson & Kichkha 2017 ; Steel et al. 2021 ). Consistent with its focus on replication studies (Block and Kuckertz 2018 ), MRQ therefore also invites authors to submit replication meta-analyses.

2.2 Step 2: literature search

2.2.1 search strategies.

Similar to conducting a literature review, the search process of a meta-analysis should be systematic, reproducible, and transparent, resulting in a sample that includes all relevant studies (Fisch and Block 2018 ; Gusenbauer and Haddaway 2020 ). There are several identification strategies for relevant primary studies when compiling meta-analytical datasets (Harari et al. 2020 ). First, previous meta-analyses on the same or a related topic may provide lists of included studies that offer a good starting point to identify and become familiar with the relevant literature. This practice is also applicable to topic-related literature reviews, which often summarize the central findings of the reviewed articles in systematic tables. Both article types likely include the most prominent studies of a research field. The most common and important search strategy, however, is a keyword search in electronic databases (Harari et al. 2020 ). This strategy will probably yield the largest number of relevant studies, particularly so-called ‘grey literature’, which may not be considered by literature reviews. Gusenbauer and Haddaway ( 2020 ) provide a detailed overview of 34 scientific databases, of which 18 are multidisciplinary or have a focus on management sciences, along with their suitability for literature synthesis. To prevent biased results due to the scope or journal coverage of one database, researchers should use at least two different databases (DeSimone et al. 2020 ; Martín-Martín et al. 2021 ; Mongeon & Paul-Hus 2016 ). However, a database search can easily lead to an overload of potentially relevant studies. For example, key term searches in Google Scholar for “entrepreneurial intention” and “firm diversification” resulted in more than 660,000 and 810,000 hits, respectively. Footnote 1 Therefore, a precise research question and precise search terms using Boolean operators are advisable (Gusenbauer and Haddaway 2020 ). Addressing the challenge of identifying relevant articles in the growing number of database publications, (semi)automated approaches using text mining and machine learning (Bosco et al. 2017 ; O’Mara-Eves et al. 2015 ; Ouzzani et al. 2016 ; Thomas et al. 2017 ) can also be promising and time-saving search tools in the future. Also, some electronic databases offer the possibility to track forward citations of influential studies and thereby identify further relevant articles. Finally, collecting unpublished or undetected studies through conferences, personal contact with (leading) scholars, or listservs can be strategies to increase the study sample size (Grewal et al. 2018 ; Harari et al. 2020 ; Pigott and Polanin 2020 ).

2.2.2 Study inclusion criteria and sample composition

Next, researchers must decide which studies to include in the meta-analysis. Some guidelines for literature reviews recommend limiting the sample to studies published in renowned academic journals to ensure the quality of findings (e.g., Kraus et al. 2020 ). For meta-analysis, however, Steel et al. ( 2021 ) advocate for the inclusion of all available studies, including grey literature, to prevent selection biases based on availability, cost, familiarity, and language (Rothstein et al. 2005 ), or the “Matthew effect”, which denotes the phenomenon that highly cited articles are found faster than less cited articles (Merton 1968 ). Harrison et al. ( 2017 ) find that the effects of published studies in management are inflated on average by 30% compared to unpublished studies. This so-called publication bias or “file drawer problem” (Rosenthal 1979 ) results from the preference of academia to publish more statistically significant and less statistically insignificant study results. Owen and Li ( 2020 ) showed that publication bias is particularly severe when variables of interest are used as key variables rather than control variables. To consider the true effect size of a target variable or relationship, the inclusion of all types of research outputs is therefore recommended (Polanin et al. 2016 ). Different test procedures to identify publication bias are discussed subsequently in Step 7.

In addition to the decision of whether to include certain study types (i.e., published vs. unpublished studies), there can be other reasons to exclude studies that are identified in the search process. These reasons can be manifold and are primarily related to the specific research question and methodological peculiarities. For example, studies identified by keyword search might not qualify thematically after all, may use unsuitable variable measurements, or may not report usable effect sizes. Furthermore, there might be multiple studies by the same authors using similar datasets. If they do not differ sufficiently in terms of their sample characteristics or variables used, only one of these studies should be included to prevent bias from duplicates (Wood 2008 ; see this article for a detection heuristic).

In general, the screening process should be conducted stepwise, beginning with a removal of duplicate citations from different databases, followed by abstract screening to exclude clearly unsuitable studies and a final full-text screening of the remaining articles (Pigott and Polanin 2020 ). A graphical tool to systematically document the sample selection process is the PRISMA flow diagram (Moher et al. 2009 ). Page et al. ( 2021 ) recently presented an updated version of the PRISMA statement, including an extended item checklist and flow diagram to report the study process and findings.

2.3 Step 3: choice of the effect size measure

2.3.1 types of effect sizes.

The two most common meta-analytical effect size measures in management studies are (z-transformed) correlation coefficients and standardized mean differences (Aguinis et al. 2011a ; Geyskens et al. 2009 ). However, meta-analyses in management science and related fields may not be limited to those two effect size measures but rather depend on the subfield of investigation (Borenstein 2009 ; Stanley and Doucouliagos 2012 ). In economics and finance, researchers are more interested in the examination of elasticities and marginal effects extracted from regression models than in pure bivariate correlations (Stanley and Doucouliagos 2012 ). Regression coefficients can also be converted to partial correlation coefficients based on their t-statistics to make regression results comparable across studies (Stanley and Doucouliagos 2012 ). Although some meta-analyses in management research have combined bivariate and partial correlations in their study samples, Aloe ( 2015 ) and Combs et al. ( 2019 ) advise researchers not to use this practice. Most importantly, they argue that the effect size strength of partial correlations depends on the other variables included in the regression model and is therefore incomparable to bivariate correlations (Schmidt and Hunter 2015 ), resulting in a possible bias of the meta-analytic results (Roth et al. 2018 ). We endorse this opinion. If at all, we recommend separate analyses for each measure. In addition to these measures, survival rates, risk ratios or odds ratios, which are common measures in medical research (Borenstein 2009 ), can be suitable effect sizes for specific management research questions, such as understanding the determinants of the survival of startup companies. To summarize, the choice of a suitable effect size is often taken away from the researcher because it is typically dependent on the investigated research question as well as the conventions of the specific research field (Cheung and Vijayakumar 2016 ).

2.3.2 Conversion of effect sizes to a common measure

After having defined the primary effect size measure for the meta-analysis, it might become necessary in the later coding process to convert study findings that are reported in effect sizes that are different from the chosen primary effect size. For example, a study might report only descriptive statistics for two study groups but no correlation coefficient, which is used as the primary effect size measure in the meta-analysis. Different effect size measures can be harmonized using conversion formulae, which are provided by standard method books such as Borenstein et al. ( 2009 ) or Lipsey and Wilson ( 2001 ). There also exist online effect size calculators for meta-analysis. Footnote 2

2.4 Step 4: choice of the analytical method used

Choosing which meta-analytical method to use is directly connected to the research question of the meta-analysis. Research questions in meta-analyses can address a relationship between constructs or an effect of an intervention in a general manner, or they can focus on moderating or mediating effects. There are four meta-analytical methods that are primarily used in contemporary management research (Combs et al. 2019 ; Geyer-Klingeberg et al. 2020 ), which allow the investigation of these different types of research questions: traditional univariate meta-analysis, meta-regression, meta-analytic structural equation modeling, and qualitative meta-analysis (Hoon 2013 ). While the first three are quantitative, the latter summarizes qualitative findings. Table 1 summarizes the key characteristics of the three quantitative methods.

2.4.1 Univariate meta-analysis

In its traditional form, a meta-analysis reports a weighted mean effect size for the relationship or intervention of investigation and provides information on the magnitude of variance among primary studies (Aguinis et al. 2011c ; Borenstein et al. 2009 ). Accordingly, it serves as a quantitative synthesis of a research field (Borenstein et al. 2009 ; Geyskens et al. 2009 ). Prominent traditional approaches have been developed, for example, by Hedges and Olkin ( 1985 ) or Hunter and Schmidt ( 1990 , 2004 ). However, going beyond its simple summary function, the traditional approach has limitations in explaining the observed variance among findings (Gonzalez-Mulé and Aguinis 2018 ). To identify moderators (or boundary conditions) of the relationship of interest, meta-analysts can create subgroups and investigate differences between those groups (Borenstein and Higgins 2013 ; Hunter and Schmidt 2004 ). Potential moderators can be study characteristics (e.g., whether a study is published vs. unpublished), sample characteristics (e.g., study country, industry focus, or type of survey/experiment participants), or measurement artifacts (e.g., different types of variable measurements). The univariate approach is thus suitable to identify the overall direction of a relationship and can serve as a good starting point for additional analyses. However, due to its limitations in examining boundary conditions and developing theory, the univariate approach on its own is currently oftentimes viewed as not sufficient (Rauch 2020 ; Shaw and Ertug 2017 ).

2.4.2 Meta-regression analysis

Meta-regression analysis (Hedges and Olkin 1985 ; Lipsey and Wilson 2001 ; Stanley and Jarrell 1989 ) aims to investigate the heterogeneity among observed effect sizes by testing multiple potential moderators simultaneously. In meta-regression, the coded effect size is used as the dependent variable and is regressed on a list of moderator variables. These moderator variables can be categorical variables as described previously in the traditional univariate approach or (semi)continuous variables such as country scores that are merged with the meta-analytical data. Thus, meta-regression analysis overcomes the disadvantages of the traditional approach, which only allows us to investigate moderators singularly using dichotomized subgroups (Combs et al. 2019 ; Gonzalez-Mulé and Aguinis 2018 ). These possibilities allow a more fine-grained analysis of research questions that are related to moderating effects. However, Schmidt ( 2017 ) critically notes that the number of effect sizes in the meta-analytical sample must be sufficiently large to produce reliable results when investigating multiple moderators simultaneously in a meta-regression. For further reading, Tipton et al. ( 2019 ) outline the technical, conceptual, and practical developments of meta-regression over the last decades. Gonzalez-Mulé and Aguinis ( 2018 ) provide an overview of methodological choices and develop evidence-based best practices for future meta-analyses in management using meta-regression.

2.4.3 Meta-analytic structural equation modeling (MASEM)

MASEM is a combination of meta-analysis and structural equation modeling and allows to simultaneously investigate the relationships among several constructs in a path model. Researchers can use MASEM to test several competing theoretical models against each other or to identify mediation mechanisms in a chain of relationships (Bergh et al. 2016 ). This method is typically performed in two steps (Cheung and Chan 2005 ): In Step 1, a pooled correlation matrix is derived, which includes the meta-analytical mean effect sizes for all variable combinations; Step 2 then uses this matrix to fit the path model. While MASEM was based primarily on traditional univariate meta-analysis to derive the pooled correlation matrix in its early years (Viswesvaran and Ones 1995 ), more advanced methods, such as the GLS approach (Becker 1992 , 1995 ) or the TSSEM approach (Cheung and Chan 2005 ), have been subsequently developed. Cheung ( 2015a ) and Jak ( 2015 ) provide an overview of these approaches in their books with exemplary code. For datasets with more complex data structures, Wilson et al. ( 2016 ) also developed a multilevel approach that is related to the TSSEM approach in the second step. Bergh et al. ( 2016 ) discuss nine decision points and develop best practices for MASEM studies.

2.4.4 Qualitative meta-analysis

While the approaches explained above focus on quantitative outcomes of empirical studies, qualitative meta-analysis aims to synthesize qualitative findings from case studies (Hoon 2013 ; Rauch et al. 2014 ). The distinctive feature of qualitative case studies is their potential to provide in-depth information about specific contextual factors or to shed light on reasons for certain phenomena that cannot usually be investigated by quantitative studies (Rauch 2020 ; Rauch et al. 2014 ). In a qualitative meta-analysis, the identified case studies are systematically coded in a meta-synthesis protocol, which is then used to identify influential variables or patterns and to derive a meta-causal network (Hoon 2013 ). Thus, the insights of contextualized and typically nongeneralizable single studies are aggregated to a larger, more generalizable picture (Habersang et al. 2019 ). Although still the exception, this method can thus provide important contributions for academics in terms of theory development (Combs et al., 2019 ; Hoon 2013 ) and for practitioners in terms of evidence-based management or entrepreneurship (Rauch et al. 2014 ). Levitt ( 2018 ) provides a guide and discusses conceptual issues for conducting qualitative meta-analysis in psychology, which is also useful for management researchers.

2.5 Step 5: choice of software

Software solutions to perform meta-analyses range from built-in functions or additional packages of statistical software to software purely focused on meta-analyses and from commercial to open-source solutions. However, in addition to personal preferences, the choice of the most suitable software depends on the complexity of the methods used and the dataset itself (Cheung and Vijayakumar 2016 ). Meta-analysts therefore must carefully check if their preferred software is capable of performing the intended analysis.

Among commercial software providers, Stata (from version 16 on) offers built-in functions to perform various meta-analytical analyses or to produce various plots (Palmer and Sterne 2016 ). For SPSS and SAS, there exist several macros for meta-analyses provided by scholars, such as David B. Wilson or Andy P. Field and Raphael Gillet (Field and Gillett 2010 ). Footnote 3 Footnote 4 For researchers using the open-source software R (R Core Team 2021 ), Polanin et al. ( 2017 ) provide an overview of 63 meta-analysis packages and their functionalities. For new users, they recommend the package metafor (Viechtbauer 2010 ), which includes most necessary functions and for which the author Wolfgang Viechtbauer provides tutorials on his project website. Footnote 5 Footnote 6 In addition to packages and macros for statistical software, templates for Microsoft Excel have also been developed to conduct simple meta-analyses, such as Meta-Essentials by Suurmond et al. ( 2017 ). Footnote 7 Finally, programs purely dedicated to meta-analysis also exist, such as Comprehensive Meta-Analysis (Borenstein et al. 2013 ) or RevMan by The Cochrane Collaboration ( 2020 ).

2.6 Step 6: coding of effect sizes

2.6.1 coding sheet.

The first step in the coding process is the design of the coding sheet. A universal template does not exist because the design of the coding sheet depends on the methods used, the respective software, and the complexity of the research design. For univariate meta-analysis or meta-regression, data are typically coded in wide format. In its simplest form, when investigating a correlational relationship between two variables using the univariate approach, the coding sheet would contain a column for the study name or identifier, the effect size coded from the primary study, and the study sample size. However, such simple relationships are unlikely in management research because the included studies are typically not identical but differ in several respects. With more complex data structures or moderator variables being investigated, additional columns are added to the coding sheet to reflect the data characteristics. These variables can be coded as dummy, factor, or (semi)continuous variables and later used to perform a subgroup analysis or meta regression. For MASEM, the required data input format can deviate depending on the method used (e.g., TSSEM requires a list of correlation matrices as data input). For qualitative meta-analysis, the coding scheme typically summarizes the key qualitative findings and important contextual and conceptual information (see Hoon ( 2013 ) for a coding scheme for qualitative meta-analysis). Figure  1 shows an exemplary coding scheme for a quantitative meta-analysis on the correlational relationship between top-management team diversity and profitability. In addition to effect and sample sizes, information about the study country, firm type, and variable operationalizations are coded. The list could be extended by further study and sample characteristics.

Exemplary coding sheet for a meta-analysis on the relationship (correlation) between top-management team diversity and profitability

2.6.2 Inclusion of moderator or control variables

It is generally important to consider the intended research model and relevant nontarget variables before coding a meta-analytic dataset. For example, study characteristics can be important moderators or function as control variables in a meta-regression model. Similarly, control variables may be relevant in a MASEM approach to reduce confounding bias. Coding additional variables or constructs subsequently can be arduous if the sample of primary studies is large. However, the decision to include respective moderator or control variables, as in any empirical analysis, should always be based on strong (theoretical) rationales about how these variables can impact the investigated effect (Bernerth and Aguinis 2016 ; Bernerth et al. 2018 ; Thompson and Higgins 2002 ). While substantive moderators refer to theoretical constructs that act as buffers or enhancers of a supposed causal process, methodological moderators are features of the respective research designs that denote the methodological context of the observations and are important to control for systematic statistical particularities (Rudolph et al. 2020 ). Havranek et al. ( 2020 ) provide a list of recommended variables to code as potential moderators. While researchers may have clear expectations about the effects for some of these moderators, the concerns for other moderators may be tentative, and moderator analysis may be approached in a rather exploratory fashion. Thus, we argue that researchers should make full use of the meta-analytical design to obtain insights about potential context dependence that a primary study cannot achieve.

2.6.3 Treatment of multiple effect sizes in a study

A long-debated issue in conducting meta-analyses is whether to use only one or all available effect sizes for the same construct within a single primary study. For meta-analyses in management research, this question is fundamental because many empirical studies, particularly those relying on company databases, use multiple variables for the same construct to perform sensitivity analyses, resulting in multiple relevant effect sizes. In this case, researchers can either (randomly) select a single value, calculate a study average, or use the complete set of effect sizes (Bijmolt and Pieters 2001 ; López-López et al. 2018 ). Multiple effect sizes from the same study enrich the meta-analytic dataset and allow us to investigate the heterogeneity of the relationship of interest, such as different variable operationalizations (López-López et al. 2018 ; Moeyaert et al. 2017 ). However, including more than one effect size from the same study violates the independency assumption of observations (Cheung 2019 ; López-López et al. 2018 ), which can lead to biased results and erroneous conclusions (Gooty et al. 2021 ). We follow the recommendation of current best practice guides to take advantage of using all available effect size observations but to carefully consider interdependencies using appropriate methods such as multilevel models, panel regression models, or robust variance estimation (Cheung 2019 ; Geyer-Klingeberg et al. 2020 ; Gooty et al. 2021 ; López-López et al. 2018 ; Moeyaert et al. 2017 ).

2.7 Step 7: analysis

2.7.1 outlier analysis and tests for publication bias.

Before conducting the primary analysis, some preliminary sensitivity analyses might be necessary, which should ensure the robustness of the meta-analytical findings (Rudolph et al. 2020 ). First, influential outlier observations could potentially bias the observed results, particularly if the number of total effect sizes is small. Several statistical methods can be used to identify outliers in meta-analytical datasets (Aguinis et al. 2013 ; Viechtbauer and Cheung 2010 ). However, there is a debate about whether to keep or omit these observations. Anyhow, relevant studies should be closely inspected to infer an explanation about their deviating results. As in any other primary study, outliers can be a valid representation, albeit representing a different population, measure, construct, design or procedure. Thus, inferences about outliers can provide the basis to infer potential moderators (Aguinis et al. 2013 ; Steel et al. 2021 ). On the other hand, outliers can indicate invalid research, for instance, when unrealistically strong correlations are due to construct overlap (i.e., lack of a clear demarcation between independent and dependent variables), invalid measures, or simply typing errors when coding effect sizes. An advisable step is therefore to compare the results both with and without outliers and base the decision on whether to exclude outlier observations with careful consideration (Geyskens et al. 2009 ; Grewal et al. 2018 ; Kepes et al. 2013 ). However, instead of simply focusing on the size of the outlier, its leverage should be considered. Thus, Viechtbauer and Cheung ( 2010 ) propose considering a combination of standardized deviation and a study’s leverage.

Second, as mentioned in the context of a literature search, potential publication bias may be an issue. Publication bias can be examined in multiple ways (Rothstein et al. 2005 ). First, the funnel plot is a simple graphical tool that can provide an overview of the effect size distribution and help to detect publication bias (Stanley and Doucouliagos 2010 ). A funnel plot can also support in identifying potential outliers. As mentioned above, a graphical display of deviation (e.g., studentized residuals) and leverage (Cook’s distance) can help detect the presence of outliers and evaluate their influence (Viechtbauer and Cheung 2010 ). Moreover, several statistical procedures can be used to test for publication bias (Harrison et al. 2017 ; Kepes et al. 2012 ), including subgroup comparisons between published and unpublished studies, Begg and Mazumdar’s ( 1994 ) rank correlation test, cumulative meta-analysis (Borenstein et al. 2009 ), the trim and fill method (Duval and Tweedie 2000a , b ), Egger et al.’s ( 1997 ) regression test, failsafe N (Rosenthal 1979 ), or selection models (Hedges and Vevea 2005 ; Vevea and Woods 2005 ). In examining potential publication bias, Kepes et al. ( 2012 ) and Harrison et al. ( 2017 ) both recommend not relying only on a single test but rather using multiple conceptionally different test procedures (i.e., the so-called “triangulation approach”).

2.7.2 Model choice

After controlling and correcting for the potential presence of impactful outliers or publication bias, the next step in meta-analysis is the primary analysis, where meta-analysts must decide between two different types of models that are based on different assumptions: fixed-effects and random-effects (Borenstein et al. 2010 ). Fixed-effects models assume that all observations share a common mean effect size, which means that differences are only due to sampling error, while random-effects models assume heterogeneity and allow for a variation of the true effect sizes across studies (Borenstein et al. 2010 ; Cheung and Vijayakumar 2016 ; Hunter and Schmidt 2004 ). Both models are explained in detail in standard textbooks (e.g., Borenstein et al. 2009 ; Hunter and Schmidt 2004 ; Lipsey and Wilson 2001 ).

In general, the presence of heterogeneity is likely in management meta-analyses because most studies do not have identical empirical settings, which can yield different effect size strengths or directions for the same investigated phenomenon. For example, the identified studies have been conducted in different countries with different institutional settings, or the type of study participants varies (e.g., students vs. employees, blue-collar vs. white-collar workers, or manufacturing vs. service firms). Thus, the vast majority of meta-analyses in management research and related fields use random-effects models (Aguinis et al. 2011a ). In a meta-regression, the random-effects model turns into a so-called mixed-effects model because moderator variables are added as fixed effects to explain the impact of observed study characteristics on effect size variations (Raudenbush 2009 ).

2.8 Step 8: reporting results

2.8.1 reporting in the article.

The final step in performing a meta-analysis is reporting its results. Most importantly, all steps and methodological decisions should be comprehensible to the reader. DeSimone et al. ( 2020 ) provide an extensive checklist for journal reviewers of meta-analytical studies. This checklist can also be used by authors when performing their analyses and reporting their results to ensure that all important aspects have been addressed. Alternative checklists are provided, for example, by Appelbaum et al. ( 2018 ) or Page et al. ( 2021 ). Similarly, Levitt et al. ( 2018 ) provide a detailed guide for qualitative meta-analysis reporting standards.

For quantitative meta-analyses, tables reporting results should include all important information and test statistics, including mean effect sizes; standard errors and confidence intervals; the number of observations and study samples included; and heterogeneity measures. If the meta-analytic sample is rather small, a forest plot provides a good overview of the different findings and their accuracy. However, this figure will be less feasible for meta-analyses with several hundred effect sizes included. Also, results displayed in the tables and figures must be explained verbally in the results and discussion sections. Most importantly, authors must answer the primary research question, i.e., whether there is a positive, negative, or no relationship between the variables of interest, or whether the examined intervention has a certain effect. These results should be interpreted with regard to their magnitude (or significance), both economically and statistically. However, when discussing meta-analytical results, authors must describe the complexity of the results, including the identified heterogeneity and important moderators, future research directions, and theoretical relevance (DeSimone et al. 2019 ). In particular, the discussion of identified heterogeneity and underlying moderator effects is critical; not including this information can lead to false conclusions among readers, who interpret the reported mean effect size as universal for all included primary studies and ignore the variability of findings when citing the meta-analytic results in their research (Aytug et al. 2012 ; DeSimone et al. 2019 ).

2.8.2 Open-science practices

Another increasingly important topic is the public provision of meta-analytical datasets and statistical codes via open-source repositories. Open-science practices allow for results validation and for the use of coded data in subsequent meta-analyses ( Polanin et al. 2020 ), contributing to the development of cumulative science. Steel et al. ( 2021 ) refer to open science meta-analyses as a step towards “living systematic reviews” (Elliott et al. 2017 ) with continuous updates in real time. MRQ supports this development and encourages authors to make their datasets publicly available. Moreau and Gamble ( 2020 ), for example, provide various templates and video tutorials to conduct open science meta-analyses. There exist several open science repositories, such as the Open Science Foundation (OSF; for a tutorial, see Soderberg 2018 ), to preregister and make documents publicly available. Furthermore, several initiatives in the social sciences have been established to develop dynamic meta-analyses, such as metaBUS (Bosco et al. 2015 , 2017 ), MetaLab (Bergmann et al. 2018 ), or PsychOpen CAMA (Burgard et al. 2021 ).

3 Conclusion

This editorial provides a comprehensive overview of the essential steps in conducting and reporting a meta-analysis with references to more in-depth methodological articles. It also serves as a guide for meta-analyses submitted to MRQ and other management journals. MRQ welcomes all types of meta-analyses from all subfields and disciplines of management research.

Gusenbauer and Haddaway ( 2020 ), however, point out that Google Scholar is not appropriate as a primary search engine due to a lack of reproducibility of search results.

One effect size calculator by David B. Wilson is accessible via: https://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-Home.php .

The macros of David B. Wilson can be downloaded from: http://mason.gmu.edu/~dwilsonb/ .

The macros of Field and Gillet ( 2010 ) can be downloaded from: https://www.discoveringstatistics.com/repository/fieldgillett/how_to_do_a_meta_analysis.html .

The tutorials can be found via: https://www.metafor-project.org/doku.php .

Metafor does currently not provide functions to conduct MASEM. For MASEM, users can, for instance, use the package metaSEM (Cheung 2015b ).

The workbooks can be downloaded from: https://www.erim.eur.nl/research-support/meta-essentials/ .

Aguinis H, Dalton DR, Bosco FA, Pierce CA, Dalton CM (2011a) Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact. J Manag 37(1):5–38

Google Scholar  

Aguinis H, Gottfredson RK, Joo H (2013) Best-practice recommendations for defining, identifying, and handling outliers. Organ Res Methods 16(2):270–301

Article   Google Scholar  

Aguinis H, Gottfredson RK, Wright TA (2011b) Best-practice recommendations for estimating interaction effects using meta-analysis. J Organ Behav 32(8):1033–1043

Aguinis H, Pierce CA, Bosco FA, Dalton DR, Dalton CM (2011c) Debunking myths and urban legends about meta-analysis. Organ Res Methods 14(2):306–331

Aloe AM (2015) Inaccuracy of regression results in replacing bivariate correlations. Res Synth Methods 6(1):21–27

Anderson RG, Kichkha A (2017) Replication, meta-analysis, and research synthesis in economics. Am Econ Rev 107(5):56–59

Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM (2018) Journal article reporting standards for quantitative research in psychology: the APA publications and communications BOARD task force report. Am Psychol 73(1):3–25

Aytug ZG, Rothstein HR, Zhou W, Kern MC (2012) Revealed or concealed? Transparency of procedures, decisions, and judgment calls in meta-analyses. Organ Res Methods 15(1):103–133

Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50(4):1088–1101. https://doi.org/10.2307/2533446

Bergh DD, Aguinis H, Heavey C, Ketchen DJ, Boyd BK, Su P, Lau CLL, Joo H (2016) Using meta-analytic structural equation modeling to advance strategic management research: Guidelines and an empirical illustration via the strategic leadership-performance relationship. Strateg Manag J 37(3):477–497

Becker BJ (1992) Using results from replicated studies to estimate linear models. J Educ Stat 17(4):341–362

Becker BJ (1995) Corrections to “Using results from replicated studies to estimate linear models.” J Edu Behav Stat 20(1):100–102

Bergmann C, Tsuji S, Piccinini PE, Lewis ML, Braginsky M, Frank MC, Cristia A (2018) Promoting replicability in developmental research through meta-analyses: Insights from language acquisition research. Child Dev 89(6):1996–2009

Bernerth JB, Aguinis H (2016) A critical review and best-practice recommendations for control variable usage. Pers Psychol 69(1):229–283

Bernerth JB, Cole MS, Taylor EC, Walker HJ (2018) Control variables in leadership research: A qualitative and quantitative review. J Manag 44(1):131–160

Bijmolt TH, Pieters RG (2001) Meta-analysis in marketing when studies contain multiple measurements. Mark Lett 12(2):157–169

Block J, Kuckertz A (2018) Seven principles of effective replication studies: Strengthening the evidence base of management research. Manag Rev Quart 68:355–359

Borenstein M (2009) Effect sizes for continuous data. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis. Russell Sage Foundation, pp 221–235

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. John Wiley, Chichester

Book   Google Scholar  

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2010) A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 1(2):97–111

Borenstein M, Hedges L, Higgins J, Rothstein H (2013) Comprehensive meta-analysis (version 3). Biostat, Englewood, NJ

Borenstein M, Higgins JP (2013) Meta-analysis and subgroups. Prev Sci 14(2):134–143

Bosco FA, Steel P, Oswald FL, Uggerslev K, Field JG (2015) Cloud-based meta-analysis to bridge science and practice: Welcome to metaBUS. Person Assess Decis 1(1):3–17

Bosco FA, Uggerslev KL, Steel P (2017) MetaBUS as a vehicle for facilitating meta-analysis. Hum Resour Manag Rev 27(1):237–254

Burgard T, Bošnjak M, Studtrucker R (2021) Community-augmented meta-analyses (CAMAs) in psychology: potentials and current systems. Zeitschrift Für Psychologie 229(1):15–23

Cheung MWL (2015a) Meta-analysis: A structural equation modeling approach. John Wiley & Sons, Chichester

Cheung MWL (2015b) metaSEM: An R package for meta-analysis using structural equation modeling. Front Psychol 5:1521

Cheung MWL (2019) A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychol Rev 29(4):387–396

Cheung MWL, Chan W (2005) Meta-analytic structural equation modeling: a two-stage approach. Psychol Methods 10(1):40–64

Cheung MWL, Vijayakumar R (2016) A guide to conducting a meta-analysis. Neuropsychol Rev 26(2):121–128

Combs JG, Crook TR, Rauch A (2019) Meta-analytic research in management: contemporary approaches unresolved controversies and rising standards. J Manag Stud 56(1):1–18. https://doi.org/10.1111/joms.12427

DeSimone JA, Köhler T, Schoen JL (2019) If it were only that easy: the use of meta-analytic research by organizational scholars. Organ Res Methods 22(4):867–891. https://doi.org/10.1177/1094428118756743

DeSimone JA, Brannick MT, O’Boyle EH, Ryu JW (2020) Recommendations for reviewing meta-analyses in organizational research. Organ Res Methods 56:455–463

Duval S, Tweedie R (2000a) Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56(2):455–463

Duval S, Tweedie R (2000b) A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 95(449):89–98

Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315(7109):629–634

Eisend M (2017) Meta-Analysis in advertising research. J Advert 46(1):21–35

Elliott JH, Synnot A, Turner T, Simmons M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review: 1. Introduction—the why, what, when, and how. J Clin Epidemiol 91:2330. https://doi.org/10.1016/j.jclinepi.2017.08.010

Field AP, Gillett R (2010) How to do a meta-analysis. Br J Math Stat Psychol 63(3):665–694

Fisch C, Block J (2018) Six tips for your (systematic) literature review in business and management research. Manag Rev Quart 68:103–106

Fortunato S, Bergstrom CT, Börner K, Evans JA, Helbing D, Milojević S, Petersen AM, Radicchi F, Sinatra R, Uzzi B, Vespignani A (2018) Science of science. Science 359(6379). https://doi.org/10.1126/science.aao0185

Geyer-Klingeberg J, Hang M, Rathgeber A (2020) Meta-analysis in finance research: Opportunities, challenges, and contemporary applications. Int Rev Finan Anal 71:101524

Geyskens I, Krishnan R, Steenkamp JBE, Cunha PV (2009) A review and evaluation of meta-analysis practices in management research. J Manag 35(2):393–419

Glass GV (2015) Meta-analysis at middle age: a personal history. Res Synth Methods 6(3):221–231

Gonzalez-Mulé E, Aguinis H (2018) Advancing theory by assessing boundary conditions with metaregression: a critical review and best-practice recommendations. J Manag 44(6):2246–2273

Gooty J, Banks GC, Loignon AC, Tonidandel S, Williams CE (2021) Meta-analyses as a multi-level model. Organ Res Methods 24(2):389–411. https://doi.org/10.1177/1094428119857471

Grewal D, Puccinelli N, Monroe KB (2018) Meta-analysis: integrating accumulated knowledge. J Acad Mark Sci 46(1):9–30

Gurevitch J, Koricheva J, Nakagawa S, Stewart G (2018) Meta-analysis and the science of research synthesis. Nature 555(7695):175–182

Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods 11(2):181–217

Habersang S, Küberling-Jost J, Reihlen M, Seckler C (2019) A process perspective on organizational failure: a qualitative meta-analysis. J Manage Stud 56(1):19–56

Harari MB, Parola HR, Hartwell CJ, Riegelman A (2020) Literature searches in systematic reviews and meta-analyses: A review, evaluation, and recommendations. J Vocat Behav 118:103377

Harrison JS, Banks GC, Pollack JM, O’Boyle EH, Short J (2017) Publication bias in strategic management research. J Manag 43(2):400–425

Havránek T, Stanley TD, Doucouliagos H, Bom P, Geyer-Klingeberg J, Iwasaki I, Reed WR, Rost K, Van Aert RCM (2020) Reporting guidelines for meta-analysis in economics. J Econ Surveys 34(3):469–475

Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, Orlando

Hedges LV, Vevea JL (2005) Selection methods approaches. In: Rothstein HR, Sutton A, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment, and adjustments. Wiley, Chichester, pp 145–174

Hoon C (2013) Meta-synthesis of qualitative case studies: an approach to theory building. Organ Res Methods 16(4):522–556

Hunter JE, Schmidt FL (1990) Methods of meta-analysis: correcting error and bias in research findings. Sage, Newbury Park

Hunter JE, Schmidt FL (2004) Methods of meta-analysis: correcting error and bias in research findings, 2nd edn. Sage, Thousand Oaks

Hunter JE, Schmidt FL, Jackson GB (1982) Meta-analysis: cumulating research findings across studies. Sage Publications, Beverly Hills

Jak S (2015) Meta-analytic structural equation modelling. Springer, New York, NY

Kepes S, Banks GC, McDaniel M, Whetzel DL (2012) Publication bias in the organizational sciences. Organ Res Methods 15(4):624–662

Kepes S, McDaniel MA, Brannick MT, Banks GC (2013) Meta-analytic reviews in the organizational sciences: Two meta-analytic schools on the way to MARS (the Meta-Analytic Reporting Standards). J Bus Psychol 28(2):123–143

Kraus S, Breier M, Dasí-Rodríguez S (2020) The art of crafting a systematic literature review in entrepreneurship research. Int Entrepreneur Manag J 16(3):1023–1042

Levitt HM (2018) How to conduct a qualitative meta-analysis: tailoring methods to enhance methodological integrity. Psychother Res 28(3):367–378

Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C (2018) Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA publications and communications board task force report. Am Psychol 73(1):26

Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage Publications, Inc.

López-López JA, Page MJ, Lipsey MW, Higgins JP (2018) Dealing with effect size multiplicity in systematic reviews and meta-analyses. Res Synth Methods 9(3):336–351

Martín-Martín A, Thelwall M, Orduna-Malea E, López-Cózar ED (2021) Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations. Scientometrics 126(1):871–906

Merton RK (1968) The Matthew effect in science: the reward and communication systems of science are considered. Science 159(3810):56–63

Moeyaert M, Ugille M, Natasha Beretvas S, Ferron J, Bunuan R, Van den Noortgate W (2017) Methods for dealing with multiple outcomes in meta-analysis: a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. Int J Soc Res Methodol 20(6):559–572

Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 6(7):e1000097

Mongeon P, Paul-Hus A (2016) The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics 106(1):213–228

Moreau D, Gamble B (2020) Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychol Methods. https://doi.org/10.1037/met0000351

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev 4(1):1–22

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A (2016) Rayyan—a web and mobile app for systematic reviews. Syst Rev 5(1):1–10

Owen E, Li Q (2021) The conditional nature of publication bias: a meta-regression analysis. Polit Sci Res Methods 9(4):867–877

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E,McDonald S,McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372. https://doi.org/10.1136/bmj.n71

Palmer TM, Sterne JAC (eds) (2016) Meta-analysis in stata: an updated collection from the stata journal, 2nd edn. Stata Press, College Station, TX

Pigott TD, Polanin JR (2020) Methodological guidance paper: High-quality meta-analysis in a systematic review. Rev Educ Res 90(1):24–46

Polanin JR, Tanner-Smith EE, Hennessy EA (2016) Estimating the difference between published and unpublished effect sizes: a meta-review. Rev Educ Res 86(1):207–236

Polanin JR, Hennessy EA, Tanner-Smith EE (2017) A review of meta-analysis packages in R. J Edu Behav Stat 42(2):206–242

Polanin JR, Hennessy EA, Tsuji S (2020) Transparency and reproducibility of meta-analyses in psychology: a meta-review. Perspect Psychol Sci 15(4):1026–1041. https://doi.org/10.1177/17456916209064

R Core Team (2021). R: A language and environment for statistical computing . R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ .

Rauch A (2020) Opportunities and threats in reviewing entrepreneurship theory and practice. Entrep Theory Pract 44(5):847–860

Rauch A, van Doorn R, Hulsink W (2014) A qualitative approach to evidence–based entrepreneurship: theoretical considerations and an example involving business clusters. Entrep Theory Pract 38(2):333–368

Raudenbush SW (2009) Analyzing effect sizes: Random-effects models. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis, 2nd edn. Russell Sage Foundation, New York, NY, pp 295–315

Rosenthal R (1979) The file drawer problem and tolerance for null results. Psychol Bull 86(3):638

Rothstein HR, Sutton AJ, Borenstein M (2005) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester

Roth PL, Le H, Oh I-S, Van Iddekinge CH, Bobko P (2018) Using beta coefficients to impute missing correlations in meta-analysis research: Reasons for caution. J Appl Psychol 103(6):644–658. https://doi.org/10.1037/apl0000293

Rudolph CW, Chang CK, Rauvola RS, Zacher H (2020) Meta-analysis in vocational behavior: a systematic review and recommendations for best practices. J Vocat Behav 118:103397

Schmidt FL (2017) Statistical and measurement pitfalls in the use of meta-regression in meta-analysis. Career Dev Int 22(5):469–476

Schmidt FL, Hunter JE (2015) Methods of meta-analysis: correcting error and bias in research findings. Sage, Thousand Oaks

Schwab A (2015) Why all researchers should report effect sizes and their confidence intervals: Paving the way for meta–analysis and evidence–based management practices. Entrepreneurship Theory Pract 39(4):719–725. https://doi.org/10.1111/etap.12158

Shaw JD, Ertug G (2017) The suitability of simulations and meta-analyses for submissions to Academy of Management Journal. Acad Manag J 60(6):2045–2049

Soderberg CK (2018) Using OSF to share data: A step-by-step guide. Adv Methods Pract Psychol Sci 1(1):115–120

Stanley TD, Doucouliagos H (2010) Picture this: a simple graph that reveals much ado about research. J Econ Surveys 24(1):170–191

Stanley TD, Doucouliagos H (2012) Meta-regression analysis in economics and business. Routledge, London

Stanley TD, Jarrell SB (1989) Meta-regression analysis: a quantitative method of literature surveys. J Econ Surveys 3:54–67

Steel P, Beugelsdijk S, Aguinis H (2021) The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews. J Int Bus Stud 52(1):23–44

Suurmond R, van Rhee H, Hak T (2017) Introduction, comparison, and validation of Meta-Essentials: a free and simple tool for meta-analysis. Res Synth Methods 8(4):537–553

The Cochrane Collaboration (2020). Review Manager (RevMan) [Computer program] (Version 5.4).

Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C, Glasziou P, Shemilt I, Synnot A, Turner T, Elliot J (2017) Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol 91:31–37

Thompson SG, Higgins JP (2002) How should meta-regression analyses be undertaken and interpreted? Stat Med 21(11):1559–1573

Tipton E, Pustejovsky JE, Ahmadi H (2019) A history of meta-regression: technical, conceptual, and practical developments between 1974 and 2018. Res Synth Methods 10(2):161–179

Vevea JL, Woods CM (2005) Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychol Methods 10(4):428–443

Viechtbauer W (2010) Conducting meta-analyses in R with the metafor package. J Stat Softw 36(3):1–48

Viechtbauer W, Cheung MWL (2010) Outlier and influence diagnostics for meta-analysis. Res Synth Methods 1(2):112–125

Viswesvaran C, Ones DS (1995) Theory testing: combining psychometric meta-analysis and structural equations modeling. Pers Psychol 48(4):865–885

Wilson SJ, Polanin JR, Lipsey MW (2016) Fitting meta-analytic structural equation models with complex datasets. Res Synth Methods 7(2):121–139. https://doi.org/10.1002/jrsm.1199

Wood JA (2008) Methodology for dealing with duplicate study effects in a meta-analysis. Organ Res Methods 11(1):79–95

Download references

Open Access funding enabled and organized by Projekt DEAL. No funding was received to assist with the preparation of this manuscript.

Author information

Authors and affiliations.

University of Luxembourg, Luxembourg, Luxembourg

Christopher Hansen

Leibniz Institute for Psychology (ZPID), Trier, Germany

Holger Steinmetz

Trier University, Trier, Germany

Erasmus University Rotterdam, Rotterdam, The Netherlands

Wittener Institut Für Familienunternehmen, Universität Witten/Herdecke, Witten, Germany

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jörn Block .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table 1 .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Hansen, C., Steinmetz, H. & Block, J. How to conduct a meta-analysis in eight steps: a practical guide. Manag Rev Q 72 , 1–19 (2022). https://doi.org/10.1007/s11301-021-00247-4

Download citation

Published : 30 November 2021

Issue Date : February 2022

DOI : https://doi.org/10.1007/s11301-021-00247-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

AP®︎/College Statistics

Unit 1: exploring categorical data, unit 2: exploring one-variable quantitative data: displaying and describing, unit 3: exploring one-variable quantitative data: summary statistics, unit 4: exploring one-variable quantitative data: percentiles, z-scores, and the normal distribution, unit 5: exploring two-variable quantitative data, unit 6: collecting data, unit 7: probability, unit 8: random variables and probability distributions, unit 9: sampling distributions, unit 10: inference for categorical data: proportions, unit 11: inference for quantitative data: means, unit 12: inference for categorical data: chi-square, unit 13: inference for quantitative data: slopes, unit 14: prepare for the 2022 ap®︎ statistics exam.