statistical issues

Which statement about the relationship between statistical power and statistical probability is true?

A statistical test having high power also has high probability for finding significant support.

lecture

Section 1 00:00:01 PROFESSOR: Just as archers have to be on target to hit their mark, a researcher must design a valid and reliable study to research the subject or construct they wish to know more about. In this section, we're going to answer the question, how do researchers create a reliable and valid study? This is section one of Statistical Issues. Our objectives are to define and differentiate between 00:00:25 reliability and validity, to explain issues influencing statistical significance, and to examine benefits and limitations of using statistics to support psychological claims. In this section, we'll be defining and differentiating between reliability and validity. Reliability is the extent to which the components and results of a research study are stable and consistent. A statistical measure can still be reliable 00:00:49 even if it's invalid. You may have a reliable test that consistently yields similar results across studies. But it also has to be valid if you want your results to be accurate. To establish reliable data, a test must demonstrate consistent results across studies. Related questions on a survey should be answered in a 00:01:08 similar way. For example, if you have similar questions on a text that deal with low anxiety but that are worded differently, students should answer similarly on all of the questions. Finally, observed behavior should be interpreted in a consistent manner. If one researcher rates someone high in anxiety, then 00:01:25 another researcher should draw the same conclusion. An unreliable study can occur if survey or test questions are unclear or confusing. It can also occur if test administration is inconsistent. For example, if you are giving the students a survey at lunch and after school, it might yield an unreliable study. Reliable test data means that the test needs to be 00:01:47 administered to all subjects in the same way. An unreliable study can occur if the test measure or survey yield inconsistent results. Or it can occur if participants guess on test questions. If the questions are confusing or hard to understand, participants might guess, which will give you an unreliable test and inconsistent information. 00:02:08 Here are some examples of unreliable measures. If researchers were giving a survey that attempted to measure student engagement in school, one statement in the survey might try to measure students reasons for completing homework. Then the student would write whether they agree or disagree with the statement. The statement, "I don't not do the work in this class, so no 00:02:27 one will know I'm having trouble with it" is hard to understand, because it's a double negative. So it may need to be reworded. The second statement, "I do not like it when I am not allowed to wear hats to school," is worded in a way that focuses on what someone does not like rather than what they do like. So it's a little confusing. 00:02:44 It should be something like, "I like it when I can wear hats to school," so that it'll be easier for the participant to understand. One behavior can be perceived differently by multiple researchers. One researcher might say, "The subject demonstrates excessive nervousness," while another would say, "The subject demonstrates heightened excitability." This creates a 00:03:05 shaky foundation for your study. And it could result in an unreliable study. Validity is the extent to which the components and results of a research study are logical, meaningful, and relevant to target variables. If a measure is unreliable, it is not valid. So ask yourself, do your test questions assess what you want them to? 00:03:26 Are they on target? A valid study allows the researcher to draw good conclusions or inferences about the topic being studied. How do we know the research measure is valid? It depends on how the construct is defined. For example, if you're talking about how happiness is measured, can happiness be measured by the number of times someone smiles during the day? 00:03:48 To establish valid data, test questions should relate to the subject or construct. For example, all the questions in a study about personal freedom should accurately represent feeling free. Scores should adequately represent or predict a specific outcome. For example, ACT and SAT scores are supposed to adequately reflect college performance. 00:04:08 If they do, then the ACT or SAT is a valid representation of college performance. Finally, scores should adequately represent the theory about subject or construct. Many researchers will consult experts when determining the validity of their questions. An invalid study can occur if survey or test questions are poorly designed or misinterpreted. 00:04:30 It can occur if insufficient data is collected or if the sample selection is not representative of the population. Finally, they can occur if unrecognized variables interfere with study results, for example, prior knowledge of an individual or influence of unknown, extraneous variables. Here are some examples. 00:04:50 If you have the statement, "When I study, I underline main ideas in my notes," this is a survey statement that attempts to represent a shallow learning strategy. However, many students who try to understand subjects at a deep or shallow level underline their notes. So it's not a good question to distinguish between the two types of learning. The statement, "I smile at strangers often," is a survey 00:05:11 statement that attempts to represent friendliness. However, it's not a good statement, because someone may be considered friendly but not smile at strangers often. Here are some more examples. If you have the statement, "I like the color of my car," and it's a survey statement that attempts to represent self-esteem, it's probably not going to be a good statement. The color of the car has nothing to do 00:05:31 with one's self esteem. Finally, the statement, "School rules are stupid," is a statement that attempts to represent a student's opinion on a specific school rule. However, one may not think that all school rules are stupid and still not agree with one specific school rule. So the question may need to be reworded and more specific if you're trying to learn a student's opinion on just one 00:05:54 particular school rule. This is an image of this representative data that's unreliable and invalid, because the data points that you can see here aren't close to one another, and they're not close to the target results. This is an example of a reliable research study, because the data points are close together. However, it's not valid, because they're still not 00:06:18 close to the target. Finally, this is an example of a study that's reliable and valid. You can see that the data points are close together and close to the target. So to review, we've talked about how using reliable and valid measures helps to assure that a research study accurately represents the target population behavior. 00:06:41 Next, we're going to discuss various factors that influence the significance and value of statistical findings. Section 4 00:00:01 PROFESSOR: When you hear a psychologist, doctor, or newscaster use the phrase, "statistics show," they're essentially making the claim that statistical findings significantly support what they're about to say. As researchers, we can make claims about the relationship between variables and the differences between groups all day long. But whether or not our discoveries are significant 00:00:23 helps to determine how true our claim or discovery really is. In this section, we're going to answer the question, is your discovery significant? This is section two of Statistical Issues. Our objectives are to define and differentiate between reliability and validity, to explain issues influencing statistical significance, and to examine the benefits and 00:00:45 limitations of using statistics to support psychological claims. In this section, we'll be explaining issues influencing statistical significance. Statistical significance is the likelihood that a statistical finding is the result of something other than chance or coincidence. When something is found to be statistically significant in research, it means that the difference between groups, 00:01:08 such as boys and girls, or the relationship between variables, such as amount of sleep and happiness, is occurring because of something significant, that it's more than just coincidence. Statistical significance is influenced by level of probability and power. Let's discuss each. Probability is the likelihood of a 00:01:28 circumstance or event occurring. So how certain is the claim? And to what extent can the results be generalized, meaning, is the study a good representation of what I would find in the larger, target population? Is it a strong or weak claim? Statistical power is the probability of correctly finding adequate support for a research hypothesis. 00:01:50 For example, if a research study demonstrates there's a significant connection between my GPA and the number of hours I spend studying, then I also have to look at the power of my statistical test that represents this. Is the power weak or strong? Greater power means my claim is more likely true. There's a direct relationship between statistical probability and statistical power. 00:02:15 Greater power means greater probability of finding significant support. There are different influences on statistical power, such as critical value level, significance error, and sample size. Let's discuss each. A critical value is a statistical value that establishes the boundary between finding significant 00:02:35 support and lacking significant support when testing a research hypothesis. It's used in calculating statistical results by providing a cutoff focal point that determines if a test is considered statistically significant. Here is a normal distribution representation of our sample data. The average scores are in the middle. 00:02:56 Where we establish critical cutoff values in our test determines whether or not we can claim significant support. On a normal distribution, if a test value or number falls outside of our critical values, then you have found statistical support. If it's inside the values, then there's not enough support for your claim. Here's an example. 00:03:16 Say you want to test if playing sports makes one more popular in school. In your study, you compare the level of popularity with number of sports played and establish a specific cutoff or critical value to compare your results. Can you see from this distribution if our study has support for our claim? The answer is no, because the test value is within the 00:03:37 critical values. Significance error is error that occurs from drawing an incorrect conclusion about significant statistical support. The more probability you have in making an error in claiming support, then a researcher lacks the power to claim significant results. There are different types of error. 00:03:58 Type one is claiming significant support for the research hypothesis when there isn't enough support to establish significance. Type two is claiming no significant support for the research hypothesis when there is enough support to establish significance. As a probability of making one type of error increases, the probability of making the other type decreases. 00:04:20 A simple size is the number of participants a researcher includes in their study. The statistical sample influences the power of the statistical test. A larger sample size increases power. The small sample size decreases power. This is because a larger sample is more representative of the target population. So you have a better chance of representing the 00:04:41 group you're studying. So to review in this section, we talked about the three main factors in a statistical test that influence whether or not our statistical results have enough support or are considered significant. Those are critical value level, significance errors, and sample size. Next, we'll discuss whether or not the significant claims 00:05:02 that researchers make really matter, as we learn about the benefits and limitations of statistics. Section 7 00:00:01 PROFESSOR: Sometimes we wonder, do statistical findings really matter, especially when we know that research does not always tell us everything. In this section, we'll explore the benefits and limitations to research discoveries. This is Section Three of Statistical Issues. Our objectives are to define and differentiate between reliability and validity, to explain issues influencing 00:00:24 statistical significance, and to examine benefits and limitations of using statistics to support psychological claims. In this section, we'll be examining benefits and limitations of using statistics to support psychological claims. The benefits of statistics are that they promote positive change, increase knowledge, and improve clarity. Statistics improve clarity by providing a basis for 00:00:50 understanding the world. Research findings discovered through statistics help us in understanding our world more clearly. For example, in our health, life experiences, or relationships, like if an individual is confused about why he or she is tired so often. They also help us establish practical guidelines for improvement. 00:01:09 Statistics helps us to increase knowledge by addressing gaps in knowledge. Researchers may observe variations in behavior, and as a result, test existing theories from different perspectives. This helps to increase existing knowledge and provide more knowledge for concepts that are not fully understood. Statistics also helps to improve social practices, such 00:01:30 as the education system or medical care by providing more research support for suggesting improvements in each field as more knowledge is gained. Statistics helps to promote positive change. They broaden our perspectives and provide a voice to minorities, underrepresented cultures, or others with diverse opinions. They instigate improvement. 00:01:50 For example, we can use research to support an argument as to why funding is needed for a specific purpose, or we can gather enough evidence to support a claim that there are no significant intellectual differences between racial groups or sexes and then help to improve the educational options for those races or genders. There are limitations of statistics, however. Those include providing limited information, that they 00:02:14 can be influenced by inaccurate assumptions, that they can promote limitations in perception, and that results can be misrepresented. Let's take a look at each of these ideas. Although statistical findings are sometimes used in arguments to support individual beliefs, one limitation is that statistical findings only offer collective information, meaning that the findings apply to the 00:02:35 population average and that they are not applicable to all individuals within a defined population. Researchers sometimes make inaccurate assumptions when measuring data. Those assumptions could include assuming that variable relationships are linear. For example, a researcher may look at the relationship between the success rates of a counselor and the years of 00:02:56 practice of that counselor, but they may fail to recognize other variables like influence of job satisfaction, emotional stability of the counselor, office atmosphere, and so forth. Other inaccurate assumptions including assuming that statistical measures are reliable and valid or that research data is normally distributed. For example, height and weight are normally distributed, but 00:03:17 other variables are not but could still be measured by researchers as if they are. Finally, one could assume that statistical findings are solutions. So these are suggestions, not solutions. For example, when they assume that eating apples actually reduces depressive symptoms, this does not mean that apples can cure depression, only that it is a suggestion that could 00:03:38 help with depressive symptoms. The media is known for reporting or implying causal relationships without support. The media can also reduce statistical information before sufficient data is established to support the claim. Let's take a look at some examples of how the media can draw causal relationships with their headlines. One example is "Eating Breakfast Makes Girls 00:04:00 Thinner." This tries to draw a relationship between eating breakfast and being thin. Or "Recession Causes an Increase in Teen Dating Violence." This is trying to draw a relationship between dating violence and the recession while there may only be a correlation. Research processes are subjectively influenced even when there are attempts to keep the 00:04:23 experiences free from influence. Some of the ways that they're subjectively influenced are by culture. Culture influences how our thoughts, beliefs, and concepts are interpreted. Also, theories can influence our observations. Theories influence what we observe and how we interpret it. 00:04:41 The limits in perception also include stereotypes and biases. These are concerns. There are concerns that statistics have led to stereotypes and biases by producing labels for individuals. For example, statistics have helped to make distinctions between those who are considered intelligent and 00:04:57 those who are seen as below average in their learning capabilities. Personal labels are also involved with stereotypes and biases. Labels can be positive or negative depending on how they influence a person to behave. For example, someone labeled as bipolar may use their label to excuse demonstrations of uncontrollable anger. 00:05:16 An individual viewing this behavior may judge it as unacceptable until they learn that the individual is bipolar, and then they could excuse behavior that they once found unacceptable. So to review in this section, we've talked about the benefits of statistics including improving clarity, increasing knowledge, and promoting change. We've also talked about the limitations of statistics like 00:05:37 that they can lead to inaccurate assumptions, they can lead to restrictions in information and perception, and that results can sometimes be misrepresented. To review in this lesson, we've talked about creating reliable and valid studies, which help to increase the chance of finding significant and powerful statistical results, which should be the main goal of researchers. While statistics has had societal limitations, it is 00:06:00 also produce many societal benefits.

A study must be valid to be considered reliable

false

Establishing a high critical value when calculating the results of a statistical test means that a researcher will have more confidence in finding significance than when a lower critical value is established.

false

One benefit of statistics is that it can limit an individual's perception.

false

The news media is a valid source for learning about variable relationships

false

To produce a valid test, the survey questions do not have to relate to the subject of the study.

false

An inaccurate assumption often made in a research study is that __________.

research data is normally distributed among the population

A test that is administered in a consistent manner increases the reliability of a study.

true

Statistics assists in establishing practical guidelines for improvement.

true

The number of participants within a study influences the amount of statistical power a test attains.

true