Research Methods in Psychology Chapter 2

Abstract

-Is a concise summary of the article. -It briefly describes the study's hypotheses, method, and major results. -When you are collecting articles for a project, the abstracts can help you quickly decide whether each article describes the kind of research you are looking for, or whether you should move on to the next article.

Availability Heuristic: Being persuaded by that comes easily to mind

-A bias in intuition, in which people incorrectly estimate the frequency of something, relying predominantly on instances that easily come to mind rather than using all possible evidence in evaluating a conclusion. -Things that pop up easily in our mind tend to guide our thinking. -When events or memories are vivid, recent, or memorable, they come to mind more easily, leading us to overestimate how often things happen. -Might lead us to wrongly estimate the number of something or how often something happens. -Our attention can be inordinately drawn to certain instances, leading to overestimation.

Present/Present Bias

-A bias in intuition, in which people incorrectly estimate the relationship between an event and its outcome, focusing on times the event and outcome are present, while failing to consider evidence that is absent and harder to notice. -EX: In the context of managing anger, the present/present bias means we will easily notice the times we did express frustration at the gym, at the dog, or in an email and subsequently felt better. -We notice the times when both the treatment (venting) and the desired outcome (feeling better) are present but are less likely to notice the times when we didn't express our anger and just felt better anyway. -The treatment was absent but the outcome was still present. -When thinking intuitively, we tend to focus only on experiences that fall in the present/present cell. -But if we think harder and look at the whole picture, we might conclude expressing frustration doesn't work well at all.

Confounds

-A general term for a potential alternative explanation for a research finding -A threat to internal validity -Occurs when you think one thing caused an outcome but in fact other things changed, too, so you are confused about what the cause really was. -EX: You might think online brain-training exercises are making your grades improve, but because you are also taking different classes and have gained experience as a student, you can't determine which of these factors caused the improvement

Comparison Group

-A group in an experiment whose level on the independent variable differs from those of the treatment group in some intended and meaningful way -Enables us to compare what would happen both with and without the thing we are interested in

Disinformation "fake news"

-A news story, info, photo, or video deliberately created and shared known to be false or misleading. -Those who spread it include hate groups who have cloaked false, racist stories in websites disguised to look real -They include false foreign social media accounts aimed at suppressing African American votes in the USA -People cannot always tell when news is fake -EX: One poll found the majority of Americans from both political parties believed fake news headlines they had seen -It has made some people disengage from voting. -It has made others act drastically -EX: In 2016, after reading unfounded conspiracy stories, a man fired a rifle into a pizza restaurant he read was a sex-trafficking site.

Open Access

-A peer-reviewed academic journal that anyone, even the general public, can read without paying for access. -

Paywalled

-A peer-reviewed academic journal that the general public must pay to access -Only people who are members of subscribing institutions can access the content.

Example of Review Journal Article, Meta-Analysis, & Effect Size: Nazareth Review

-A review article by Alina Nazareth and her colleagues, summarized 266 studies on human sex differences in navigation skills. -The authors computed the average effect size across all 266 studies. -Meta-analysis is valued by psychologists because it weighs each study proportionately and does not allow cherry-picking particular studies. -Before being published in a journal, both empirical articles and review articles must be peer-reviewed -Both types are considered the most prestigious forms of publication, in part because of this peer review process.

Empirical Journal Articles

-A scholarly article that reports for the first time the results of a research study. -Report, for the first time, the results of an empirical research study. -Contain details about the study's method, the statistical tests used, and the results of the study.

Review Journal Articles

-An article summarizes and integrate all the studies that have been published in one research area. -Sometimes a review article uses a quantitative technique called meta-analysis , which combines the results of many studies and gives a number that summarizes the magnitude, or the effect size.

TRUSTING AUTHORITIES ON THE SUBJECT

-An authority with a scientific degree may be better able to accurately understand and interpret scientific evidence. -If an authority refers to research evidence in their own area of expertise, their advice might be worthy of attention. -However, authorities can also base their advice on their own experience or intuition, just like the rest of us. -And they, too, might present only the studies that support their own side. -Not all research is equally reliable. -The research experts use to support their arguments might have been conducted poorly. -The research someone cites to support an argument may not accurately and appropriately support that particular argument.

Summary: TRUSTING AUTHORITIES ON THE SUBJECT

-Authorities may attempt to convince us to accept their claims. -If their claims are based on their own experience or intuition, we should probably not accept them. -If they use well conducted studies to support their claims, we can be more confident about taking their advice.

The Intuitive Thinker Versus the Scientific Reasoner

-Because of our biases, we tend to notice and actively seek info that confirms our ideas. -To counteract biases, adopt the empirical mindset of a researcher. -Empiricism involves basing beliefs on systematic info from the senses. -Now we have an additional nuance for what it means to reason empirically: To be an empiricist, you must also guard against common biases when you look at the data. -Researchers ask: Compared to what? Rather than base their beliefs on their personal conviction, researchers collect data on a comparison group. -Knowing they should not simply go along with the story everyone believes, they train themselves to test their intuition with systematic, empirical observations. -They strive to collect potentially disconfirming evidence, not just evidence that confirms their hypotheses. -Keenly aware that they have biases, scientific reasoners allow the data to speak more loudly than their own confidently held but possibly faulty ideas. -While researchers are not perfect reasoners themselves, they strive to guard against the many pitfalls of relying on intuition and they draw more accurate conclusions.

THE RESEARCH VERSUS YOUR EXPERIENCE

-Beliefs based on personal experience may not be accurate. -One reason is that personal experience usually does not involve a comparison group. -In contrast, research explicitly asks: Compared to what? -In addition, personal experience is often confounded. -In daily life, many things are going on at once, and it is impossible to know which factor is responsible for a particular outcome. -In contrast, researchers can closely control for confounding factors. -Research has an advantage over experience because researchers design studies that include appropriate comparison groups. -Conclusions based on research are probabilistic. -Research findings cannot predict or explain all cases all the time; instead, they aim to predict or explain a high proportion of cases. -Individual exceptions to research findings do not nullify the results.

Adjust your search using "or" and "and."

-Broaden with OR: "anorexia" or "bulimia" or "eating disorder." Narrow with AND: "anorexia" and "adolescence"

References

-Contains a full bibliographic listing of all the sources the authors cited in writing their article, enabling interested readers to locate these studies. -When conducting a literature search, reference lists are excellent places to look for additional articles on a given topic. -Once you find one relevant article, the reference list for that article will contain a treasure trove of related work.

Basing conclusions on personal experience is problematic because

-Daily life usually doesn't include comparison experiences -Even if a change has occurred, we often can't be sure what caused it. In everyday life, too much is going on at once. -EX: When you noticed a difference in your well-being after buying a salt lamp, maybe you were also getting regular massages or practicing yoga. Which one caused your positive mood?

Results

-Describes the quantitative and, as relevant, qualitative results of the study, including the statistical tests the authors used to analyze the data. It -Provides tables and figures that summarize key results. -Although you may not understand all the statistics used in the article, you might still be able to understand the basic findings by looking at the tables and figures.

Probablistic

-Describing the empirical method, stating that science is intended to explain a certain proportion (but not necessarily all) of the possible cases. -The conclusions of research are meant to explain a certain proportion (preferably a high proportion) of the possible cases. -In practice, this means scientific conclusions are based on patterns that emerge only when researchers set up comparison groups and test many people.

Introduction

-Is the first section of regular text, and the first paragraphs typically explain the topic of the study. -The middle paragraphs lay out the background for the research. -What theory is being tested? -What have past studies found? -Why is the present study important? -Pay special attention to the final paragraph, which states the specific research questions, goals, or hypotheses for the current study.

READING WITH A PURPOSE: EMPIRICAL JOURNAL ARTICLES

-Don't read every word of every article, from beginning to end. -Instead, read with a purpose. -This means asking two questions as you read: 1) What is the argument? 2) What is the evidence to support the argument? -The first step toward answering these questions is to read the abstract, which provides an overview of the study. -Empirical articles are stories from the theory data cycle -An empirical article reports on data that are generated to test a hypothesis, and the hypothesis is framed as a test of a particular theory. -After reading the abstract, go to the end of the introduction, where you'll find the primary goals and hypotheses of the study. -After reading the goals and hypotheses, read the rest of the introduction to learn more about the theory that the hypotheses are testing. -Another place to find the argument of the paper is the first paragraph of the Discussion section, where most authors summarize the key results and state how well the results supported their hypotheses. -Once you have a sense of what the argument is, look for the evidence. -In an empirical article, the evidence is contained in the Method and Results sections. -What did the researchers do, and what results did they find? -How well do these results support their argument (hypotheses)?

Method

-Explains in detail how the researchers conducted their study. -Contains subsections such as Participants, Materials, Procedure, and Apparatus. -An ideal section gives enough detail that if you wanted to repeat the study, you could do so without having to ask the authors any questions.

Basing conclusions on systematic data collection

-Has the simple but tremendous advantage of providing a comparison group. -Only a systematic comparison can show you whether your well-being improves when you use a salt lamp (compared with when you do not) or whether your anger goes away when you play a violent online game (compared with doing nothing).

Ways that Intuition is Biased

-Humans are not natural scientific thinkers. -We might be aware of our potential to be biased, but we may be too busy, or not motivated enough, to correct and control for these biases. -Fortunately, the formal processes of scientific research help prevent these biases from affecting our decisions. -Five examples of biased reasoning 1) Being swayed by a good story 2) Being persuaded by that comes easily to mind 3) Failing to think about what we cannot see 4) Focusing on the evidence we like best 5) Biased about being biased

THE RESEARCH VERSUS YOUR INTUITION

-Intuition is a flawed source of information because it is affected by biases in thinking. -People are likely to accept an explanation that makes sense intuitively, even if it is not true. -People can overestimate how often something happens if they consider only readily available thoughts, those that come to mind most easily. -People find it easier to notice what is present than what is absent. -When people forget to look at the information that would falsify their belief, they may see relationships that aren't there. -Intuition is also subject to confirmation bias. -We seek out evidence that confirms our initial ideas and fail to seek out evidence that can disconfirm them. -We all seem to have a bias blind spot and believe we are less biased than everyone else. -Scientific researchers are aware of their potential for biased reasoning, so they create special situations in which they can systematically observe behavior. -They create comparison groups, consider all the data, and allow the data to change their beliefs.

PsycINFO

-Is a comprehensive tool for sorting through the vast amount of psychological research. -Searches only sources in psychology. -Has many advantages. 1) It can show you all the articles written by a single author ("Brad Bushman") or under a single keyword ("autism"). 2) It tells you whether each source was peer-reviewed. 3) The tool "Cited by" links to other articles that have cited each target article 4) The tool "References" links to the other articles each target article used. -That means if you've found a great article for your project, the "Cited by" and "References" lists can be helpful for finding more papers just like it. -One disadvantage is that your college or university library must subscribe; the general public cannot use it. -One challenge for students is translating curiosity into the correct search terms.

Science in Popular Media

-Journalists play an important role in telling the public about findings in psychological science. -Psychological research may be covered in specialized outlets such as Psychology Today and the Hidden Brain podcast, which are devoted exclusively to covering social science research for a popular audience. -Research makes its way to wider news feeds and online newspapers. -Most science journalists do an excellent job. -They read the original research, interview multiple experts, and fact-check.

Research is an iterative process. Once you find one suitable article in PsycINFO, do another search

-Look at the PsycINFO "Subject Terms" associated with it and search them in the next attempt. -Check whether PsycINFO suggests "Related Items." -Use that article's References to find similar work. -In PsycINFO, click on Cited by to find other researchers who have used this article.

COMPONENTS OF AN EMPIRICAL JOURNAL ARTICLE

-Most empirical journal articles are written in a standard format, as recommended by the Publication Manual of the American Psychological Association (APA). -They include certain sections in the same order: 1) Abstract 2) Introduction, 3) Method 4) Results 5) Discussion, 6) References. -Each section contains a specific kind of info

Being swayed by a good story

-One example of a bias in our thinking is accepting a conclusion just because it makes sense or feels natural. -We tend to believe good stories, even ones that are false. -Sometimes a good story will turn out to be accurate, of course, but it's important to be aware of the limitations of intuition. -When empirical evidence contradicts what your common sense tells you, be ready to adjust your beliefs on the basis of the research. -Automatically believing a story that seems to make sense can lead you astray.

EX: How Good Is the Study Behind the Story?

-One summer the headlines read, "Horns are growing on young people's skulls. Phone use is to blame, research suggests." -This headline shocked readers and was covered by multiple news outlets. -The "horns" referred to bone growths of 10mm or more that were visible via X-ray at the back of people's skulls. -Researchers reported that 41% of participants ages 18 to 30 had one of these enlargements on their skulls. -The journal publication behind this journalism had been peer-reviewed, a sign of the study's quality. -As headlines went viral, other journalists began writing about the study's flaws. -These reviews revealed that the study did not measure phone use. -Therefore, it could not establish a link between phone use and bone growth. -The X-rays that they analyzed were from people who had visited a chiropractor for neck pain. -Therefore, it was impossible to know if the reported rates of bone growth would apply to the general population. -Researchers failed to disclose a conflict of interest, one author was a chiropractor who sells a pillow designed to improve people's posture.

Books & Edited Books

-One way to get an overview of a body of research is to read a scholarly book or an edited book. -Compared with scholars in other disciplines, such as art history or English, psychologists do not write many full-length scientific books for an audience of other psychologists. -It is more common to contribute a chapter to an edited book. -An edited book is a collection of chapters on a common topic, each chapter of which is written by a different contributor. -EX: The edited book The Handbook of Emotion Regulation contains more than 30 chapters, all written by different researchers. -Edited book chapters are a good place to find a summary of a set of research a particular psychologist has done. -Chapters are not peer reviewed as rigorously as empirical journal articles or review articles. -However, the editor of the book is careful to invite only expert researchers who are intimately familiar with the empirical evidence on a topic to write the chapters. -The audience is usually other psychologists and psychology students.

Open VS Paywalled Access

-Open-access publication of science supports Merton's norm of communality science should be available for everyone, including the taxpaying public. -If you're blocked by a paywall, try the scientist's personal website or the repositories PsyArXiv or PubMed Central (keeping in mind that some manuscripts on PsyArXiv have not yet passed through peer review).

Motives of Disinformation

-People spread deliberately false info for several reasons 1) Propaganda 2) Passion 3) Politics motivate 4) Profit -Disinformation can drive votes and enhance political support. -Provocation motivates people who want to "punk" others into emotional reactions. -Profit motivates false scientific claims about salt lamps, herbal supplements, or crystals to accompany a shopping website. -Also the chance that it's parody sites that create false stories only to make us laugh.

Summary

-People's beliefs can be based on their own experience, on their intuition, on authorities, or on controlled research. -Of these, research info is the most accurate source of knowledge

Conclusions Based on Research

-Peoples beliefs can come from several sources. -You should base your beliefs about psychological phenomena on research, rather than experience, intuition, or authority. -Research can be found in a variety of sources, some more dependable than others. -Ways of knowing that appear in outlined boxes are more trustworthy.

Consulting Scientific Sources

-Psychological scientists usually publish their research in three kinds of sources. 1) Most often, research results are published as articles in scholarly journals. 2) Psychologists may describe their research in single chapters within edited books 3) Some researchers write full-length scholarly books.

Legitimate Journalism Versus Disinformation

-Reading about research in its original form is the best way to get a thorough, accurate, and peer reviewed report of scientific evidence. -Journalists can provide good overviews of psychological research, but choose and read sources carefully and vigilant for disinformation

JOURNAL ARTICLES: PSYCHOLOGY'S MOST IMPORTANT SOURCE

-Scientific journals come out monthly or quarterly, as magazines do. -Unlike magazines, however, scientific journals do not have glossy, colorful covers or advertisements. -Journal articles are written for an audience of other psychological scientists and psychology students.

Types of Disinformation

-Some disinformation is completely false. -Other disinformation is more subtle. -It might attribute false quotes to real people or use a real quote in a false context. -Photos and videos can be especially provocative and convincing. -Disinformation might involve manipulating photos or videos or pasting real images into false contexts.

Figure out the best search terms for your question

-Textbooks, topical encyclopedias, and topical dictionaries can help you find the right terms and give you an overview of the field. -An initial Google search on your question can also help. EX: Do eating disorders happen more frequently in families that eat dinner together? -Instead of "eating disorders," you may need to be more specific using "anorexia nervosa" or "bulimia nervosa." -Instead of "eating dinner together," you may need to be broader like "family meal" or "home environment."

Present/Present Bias: Failing to think about what we cannot see

-The availability heuristic leads us to overestimate events, such as how frequently people encounter red lights or die from violence. -A related problem prevents us from seeing the relationship between an event and its outcome. -EX: When deciding if there's a pattern, between a radical mastectomy and full recovery, or between using a salt lamp and feeling calm, people forget to seek out the information that isn't there. -When testing relationships, we often fail to look for absences -In contrast, it is easy to notice what is present. This tendency, referred to as the present/present bias, reflects our failure to consider appropriate comparison groups

Availability Heuristic & Present/Present Bias

-The availability heuristic plays a role in the present/present bias because instances in the "present/present" cell of a comparison stand out. -But the present/present bias adds the tendency to ignore "absent" cells, which are essential for testing relationships. -To avoid the present/present bias, scientists train themselves always to ask: Compared to what?

Google Scholar

-The free tool works like the regular Google search engine, except the search results will all be in the form of empirical journal articles and scholarly books. -By visiting the user profile for a particular scientist, you can see all of that research team's publications. -One disadvantage is that it doesn't let you limit your search terms to specific fields (such as the abstract or title). -It doesn't categorize the articles it finds. -EX: As peer-reviewed or not, whereas PsycINFO does. -Contains articles from all scholarly disciplines, it may take more time for you to sort through the results.

Discussion

-The opening paragraph of this section summarizes the study's research question and methods and indicates how well the results of the study supported the hypotheses. -Next, the authors usually discuss the study's importance: A) Perhaps their hypothesis was new B) The method they used was a creative and unusual way to test a familiar hypothesis C) The participants were unlike others who had been studied before. -The authors may discuss alternative explanations for their data and pose interesting questions raised by the research.

Bias Blind Spot: Biased about being biased

-The tendency for people to think that compared to others, they themselves are less likely to engage in biased reasoning. -The belief that we are unlikely to fall prey to the other biases. -Most of us think we are less biased than others, so when we notice that our own view of a situation is different from that of somebody else -EX: We conclude that "I'm the objective one here" and "you are the biased one." -It makes us trust our faulty reasoning even more. -It can make it difficult for us to initiate the scientific theory-data cycle. -We might say, "I don't need to test this conclusion; I already know it is correct." -Part of learning to be a scientist is learning not to use feelings of confidence as evidence for the truth of our beliefs. Rather than thinking what they want to, scientists use data.

Confirmation Bias: Focusing on the evidence we like best

-The tendency to consider only the evidence that supports a hypothesis, including asking only the questions that will lead to the expected answer. -The tendency to look only at info that agrees with what we want to believe. - We "cherry-pick" the info we take in, seeking and accepting only the evidence that supports what we already think. -Psychological research has repeatedly found that when people are asked to test a hypothesis, they tend to seek only the evidence that supports their expectations and forget to ask questions that would disconfirm their initial view. -As a result, people tend to gather only a certain kind of info, and then they conclude that their beliefs are supported. -This bias is one reason clinical psychologists and other therapists are required to get a research methods education. -Merton's norm of organized skepticism means that scientists should make a habit of questioning everything and seek evidence both for and against their ideas.

"Predatory" Journals

-Their names sound legitimate -EX: The Journal of Science or Psychiatry and Mental Disorders -But they publish almost any submission they receive, even fatally flawed studies -They exist to make money by charging fees to scientists who want to publish their work.

FINDING AND READING THE RESEARCH

-Tools for finding research in psychology include the online database PsycINFO and Google Scholar. -Journal articles, chapters in edited books, and full-length books should be read with a purpose by asking: What is the theoretical argument? What is the evidence? What do the data say? -Popular media articles and books can be good sources of information about psychology research, as long as you think critically. -Journalists might not always cover psychology research accurately. -You can compare a popular media story to the original empirical article to be sure. -Disinformation is news that is deliberately created to mislead or provoke. -Many people believe news that is demonstrably false.

Is the Story Accurate?

-When journalists report on reliable, well-conducted research, they don't always get the story right. -Sometimes a journalist: 1) Does not have the scientific training, motivation, or time before deadline to understand the original science very well. 2) Sands down the details to make it more accessible to a general audience 3) Wraps up the details of a study with a more dramatic headline than the research can support. -EX: The bone growth study, the original researchers did not use the term "horns." That term was added by a journalist, perhaps to make the story more clickable.

READING WITH A PURPOSE: CHAPTERS AND REVIEW ARTICLES

-While empirical journal articles use predetermined headings such as Method, Results, and Discussion, authors of chapters and review articles create their own headings for the topic. -You can use these headings to get an overview before you start reading in detail. -As you read these sources, ask: -What is the argument? -What is the evidence? -The argument will be the purpose of the chapter or review article, the author's stance on the issue. -In a review article or chapter, the argument often presents an entire theory (whereas an empirical journal article tests only 1 part of a theory). -The evidence is the research that the author reviews.

How can you judge journal quality?

-Your professors and librarians can tell you the names of legitimate journals in psychology. -You can use an online tool such as Cabell's blacklist of predatory journals. -Another rough guide is whether the journal is listed in Journal Citation Reports, which calculates "impact factor." -This metric tells you how often, on average, papers in that journal have been cited. -The impact factor of the highly respected Psychological Science is 6.1, which means that on average its papers are cited 6 times. Impact factor isn't perfect; it cannot tell you whether an article's science is actually sound. -But if a journal has an impact factor of at least 1.0, it is more likely to be legitimate

Popular Articles (Journalism)

1) EX Sources: New York Times, Vox, CNN, Time, Scientific American, Wall Street Journal 2) Purpose: Summarize research that may be of interest to the general public 3) Authors: Journalists, who are often unnamed 4) Audience: The general public 5) Language: Can be understood by most people 6) Sources: Typically do not include footnotes or a list of sources, though they may mention the original researchers and include links to their published journal articles

Scholarly Articles (Journals)

1) EX Sources: Psychological Science, Child Development, Journal of Experimental Psychology 2) Purpose: Report the results of research after it has been peer-reviewed Discuss ongoing research in detail 3) Authors: Scholars, always named, and often identified by the institution at which they work 4) Audience: Scholars and researchers within a specific field of study 5) Language: Highly specialized and/or technical, and often includes professional jargon not easily understood by the general public 6) Sources: Always include sources and full Reference list

2 Types of Journal Articles

1) Empirical Journal Articles 2) Review Articles

3 Tips for Turning Your Question into a Successful

1) Figure out the best search terms for your question 2) Research is an iterative process. Once you find one suitable article in PsycINFO, do another search 3) Adjust your search using "or" and "and."

As you read about science in popular sources, it's worth thinking critically about two issues.

1) Journalists may select a more sensational, clickable story while overlooking its flaws. 2) Even when studies are conducted well, journalists may not describe them accurately.

2 Legitimate Scientific Sources to Conduct Searches

1) PsycINFO 2) Google Scholar

3 Examples of Review Article Arguments

1) Using cell phones impairs driving, even when phones are hands-free. 2) While speed reading is possible, it comes at the cost of comprehension of the text. 3) "Prolonged exposure therapy" is effective for treating most people who suffer from PTSD, though many therapists do not yet use this therapy with their clients.

Meta-Analysis

A way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports.

Confederate

An actor who is directed by the researcher to play a specific role in a research study

Effect Size

The magnitude, or strength, of a relationship between two or more variables.

Intuition

Using our hunches about what seems "natural," or attempting to think about things "logically."

Variety of Scientific Sources

You can read about research in: 1) Empirical journal articles 2) Review journal articles 3) Edited books 4) Full-length books.