MGT 357 Introduction to Evidence-based decision making Class Exam 1

Persuasion

These manager called on experts to search out options offered by vendors or being used by competitors, or to devise novel options, carefully evaluating the merits of the proposed solution. Low success rates. Managers mistakenly saw persuasion as low risk. If the expert could convince them, they believed they could convince other. Used to convince others. Not getting people to participate, not showing end results.

Construct

an abstract concept that is specifically chosen or created to explain a given phenomenon. ranges from simple and unidimensional (weight) to complex and multidimensional (employee satisfaction).

Secondary Data

"Data collected for another purpose" Examples of secondary data: 1. Data from another department at your org. 2. Marketing data

What is Evidence-based management

"making decisions through the conscientious, explicit and judicious use of the beast available evidence from multiple sources..." From the Center of Evidence-Based Management

6 Steps of Evidence-based management

1. Ask: Translate a practical issue into an answerable question 2. Acquire: Systematically search for and retrieve the evidence 3. Appraise: critically judge the trustworthiness of the evidence. 4. Aggregate: Weigh and pull together the evidence 5. Apply: Incorporate the evidence into the decision-making problem 6. Access: Evaluate the outcome of the decision taken, was that the right decision?

How to encourage people to use evidence-based management?

1. Demand evidence: Encourage managers and departments to use data and information to guide their decisions. 2. Examine logic: Carefully consider results from "non-experimental" research, and question the source of the data & results. Do they apply to your organization's context ? 3. Treat the organization as an unfinished prototype: Use trial programs, small experiments, and pilot studies. Pilot study is a study from one department. 4. Embrace the attitude of wisdom: Encourage people to explore what the do not know and ask questions.

Half the decisions in organizations fail: 4 Tactics Used to Implement Decision Stages

1. Intervention 2. Participation 3. Persuasion 4. Edicts

4 Sources of Evidence

1. Scientific literature - studies 2. Organizations - internal data 3. Practitioners - expertise 4. Stakeholders - values and concerns

Reading research articles: helpful tips

1. Title 2. abstract 3. introduction 4. Background/review of literature 5. Methodology 6. Results 7. Discussion Wording of sections may vary based on author, journal, etc...

If evidence is so important and useful, why don't managers use it more?/Deterents

1. Want a quick fix problem: Drink coffee and lose weight FAST! Coffee and cookie diets. Relates back to the business world: Lean six sigma, toyota way, getting lean. 2. Management fashion/fad problem: Read something new and put it into place. "If everybody is doing it, the pressure to it too is immense. It makes the job a bit easier to follow these trends, managers everywhere drag themselves trying not to get fired." 3. Perceived lack of knowledge about statistics, numbers. Managers feel they don't have the necessary background to analyze data.

Misconceptions about Evidence-Based Management

1. all about numbers and statistics 2. Ignores professional experience 3. Evidence has to be high quality 4. Not enough time 5. Evidence will give you the right answer 6. Every Organization is unique so evidence doesn't apply

Types of search strategies

1. citation pearl growing 2. briefsearch 3. building blocks 4. successive fractions

Initial assessment of an article (can also be applied to other sources)

1. currency 2. relevance 3. Authority 4. Accuracy 5. Purpose What are you trying to avoid? CRAAP!

Four validities of accuracy (assessing methods and results)

1. external validity: extent to which a study's findings can generalize to other populations and settings. Environment, demographics, are these results generalizable? Bigger sample, acknowledging limitations. 2. Internal validity: Ability of a study to rule out alternative explanations and support a causal claim. Does the study rule out any alternative conclusions, does it use controls? Experiment is considered the gold standard. 3. Construct validity: Quality of the study's measures and manipulations, how well a measure captures the intended theoretical construct. 4. Statistical Validity: Appropriateness of the study's conclusions based on statistical analysis. How big was the effect? is the effect significant?

Five steps in creating a literature review

1. scanning documents 2. making notes 3. structuring the literature review 4. writing the literature review 5. building the bibliography

What is a literature review?

A summary of a subject field that supports the identification of specific research questions. Draws on and evaluates a range of different types of sources (academic and professional journal articles, books, web-based resources. The literature search helps in the identification and location of relevant documents and other sources.. Goal: finding out what has been done before, what works and what does not. Literature is finding scientific evidence.

6 Steps of Evidence-based management: Asking the right questions

Asking the right questions to get good information to make better decisions: Situational analysis, creating a focused question. Ask: The first step of the EBM approach. Ask 1st What is your focus? Can be a non-effect of an effect. Effect: Need data, comparison. Does it work? Does it work better than? Will it do more good than harm? Non-effect: What do employees want or need? 1. Needs- What do employees need? 2. Attitude - How do __ feel about this? 3. Experience - What are past experiences? 4. Prevalence 5. Procedure 6. Process 7. Explanation 8. Economics

halo effect

Can you see a halo effect? attribute success and failures of firms to the personalities of their leaders. Are comparisons justified if based on success? How much of that success is attributable to chance events such as lucky timing?

concept map

Concept mapping is a useful way of identifying key concepts in a collection of documents or a research area. a picture of the territory under the study, represents concpets and relationships between them. Concepts: labeled with circles or boxes, relationships are represented by arrows. No correct answer for a concept map, their purely to assist understanding. Maps used to: • identify additional search terms during the literature search • clarify thinking about the structure of the literature review in preparation for writing the review • understand theory, concepts and the relationships between them.

Qualitative data

Deals with characteristics and descriptors that can't be easily measured, but can be observed subjectively- such as smells, tastes, textures, attractiveness, and color. Specific to business context: Employee's verbal/written explanation of their job satisfaction, supervisor's comments on an employee's performance, employee gender.

Where to search?

Free searches: Scholar.google.com. Public Library of Science, National Bureau of Economic Research, Institutional Repositories, national bureau of economic research.

Edicts

Here is the way we are going to do things. Use their power to issue a directive that announces a decision. This is done without consulting with people who have stakes in the changes the decision would bring. Managers know that they must rely on their power to issue an edict, but believe that their prerogatives and the need for timely action make this justifiable. They seem unaware of the high rate of failure of an edict. Failure can be traced to under estimating that amount of power needed because people resist the appearance of being forced to comply. Social credit: People need to respect you, have a good relationship with you, therefore you can make an edict. An edict can work if you have enough social credit. If you are making a big change, need social credit to draw from if you want sustainable following. Problem: Social credits get exhausted and need to be replenished.

affect heuristic

If you already like an idea, you are going to find information that is going to support that. Things you do not like, be biased towards what it refutes that.

Acquiring evidence: conducting literature reviews

Looking for the scientific literature, is there information that can help us? Search for and retrieve evidence: involves thinking about where and how to get the information you need. Recall the four different sources of information.

Latham

Management is both an art and a science. It results from using solid...

Participation

Managers who give up control through participation actually get more control. People are more apt to ask for help when they need it and be candid about the decision situation when asked to participate in the decision-making effort. Unilateral action will close off this type of information. Types of participation: 1. Token participation: Have people participate but in a limited way. May not actual use information from participation- creates frustration. 2. Delegated participation: Assign people to participate. limits the involvement of stakeholders, but asks the task force to do more, such as offer a solution. 3. Complete participation: People come up with solution, give opportunity to fully participate. May not include everyone though. 4. Comprehensive participation: Not only solutions but full participation. People fill fully involved. Requires that all stakeholders be involved

Picoc Method: situational analysis

P = Population, who is involved and impacted. Who is on the teams? i= Intervention (success factor), What you are proposing as the change. c= comparison, Did that change make a difference? o= outcome/objectives, what is our ultimate goal? (c)= context, where is this taking place?

Limitation of google scholar

Pretty simple. One box. Does not allow restriction to peer reviewed journals.

Quantitative data

Quantitative data deals with numbers and things you can measure objectively. Height, width, length, temperature, humidity, scale.

Does team-building work?

What questions would you ask about this? 1. What context? 2. How big are the teams? 3. What is team-building? 4. What kind of teams? 5. What counts as team building? 6. What does work mean? How to find out more about questions: Picoc Method

JMU databases

allows restriction to Peer reviewed articles, they are journals. Much more credible and relevant results.

What is a focus group?

a form of group interview where the aim is to understand the social dynamic and interaction between the participants through the collection of verbal and observational data. Type of qualitative data collection They are looking to capture the social dynamics involved with reactions and opinions. Often used early in the process of gathering information. Often used early in the collection process, best when there is little info about the topic. requires a skilled moderator. Can be time-consuming. planning is key. Produces non-numeric (qualitative) data do a focus group BEFORE a survey because you can narrow down questions!

variable

a measurable representation of an abstract construct.

what is the purpose of a lit review?

a summary of a subject field that supports the identification of specific research questions. Needs to draw on and evaluate a range of different types of sources including academic and professional journal articles, books, and web-based resources. Helps in the identification and location of relevant documents and other sources. Literature reviews involves: scanning, making notes, structuring the literature review, writing the literature review, and building bibliography. Use this to make a dissertation. Negative: messy nature of knowledge. Found in a wide range of sources.

Importance of context

an assessment of the situation is critical- there is not necessarily a one size fits all approach Context matters: you should always conduct an analysis of the situation before choosing a course of action. Decisions are made with limited information. Questions made by unclear. Important of appraising the quality of the information gathered. Importance of appraising the source of the information. To make decisions, you have to have as much RELEVANT AND TRUSTWORTHY information as possible.

Longitudinal

an observational study that involves repeated observations (measurements) of the same variable(s) over long periods of time (sometimes years or even decades)

Case example- Merger and cultural differences

canadian firm: plan for a merger from smaller firm. Plan: Integrate IT finance and facility of the two firms 'back offices' to create economies of scale. Front and legal practices remain seperate. Cultures differ widely, create problem? Asked people who had experienced it and said it would cause serious problems and clashes. How did scientific literature help? online database: The metaanalysis confirmed the partners' judgment that there was a negative association between cultural differences and the effectiveness of the post-merger integration. But this only involved high integration, not low. In mergers that required a low level of integration, cultural differences were found to be positively associated with integration benefits. In case of the two law firms, the planned integration concerned only back office functions, making the likelihood of a positive outcome higher __________ Will cultural differences impact a successful merger? p= What kind of population are we talking about? O= What kind of outcome are we aiming for? P/C= How is the assumed cultural differences assessed? Is it personal views or valid? P= What is the population? The back offices of the 2 orgs I= What is the intervention? Merging, changing, integration of back office. C= What is the comparison? does merging create economy of scale or not? o= What is the outcome? Economy of scale c= What is the context? Canadian health care organization and smaller, different organizations, both in healthcare, different cultures. Focused question: Does a difference in organizational culture affect a successful integration of back-office functions during a merger between two healthcare organizations of unequal size?

Conducting your search

choose your keywords, search using appropriate boolean phrases Narrow down your initial list: peer reviewed, date of publication, specific journals. Create additional keywords based on your result; use these to refine your search` Review at least two pages of search results (resist short cuts) Use a variety of databases; also remember to check for relevant books using the JMU library catalog.

primary data

collecting myself from students. The data the researcher collects to address the specific probkem at hand-- the research question." Examples of primary data collection methods: 1. Focus groups 2. Interviews 3. Surveys

operational definition

defines a construct in terms of how it will be empirically measured

relevance

determine how applicable the information is for your purpose

currency

determine if the date of publication of the information is suitable

Authority

determine if the source author, creator, or publisher of the information is knowledgeable.

purpose

determine the reason why the information exists

Accuracy

determine the reliability, truthfulness and correctness of the content

In general

don't get overwhelmed by scientific jargon, even authoritative, peer-reviewed journals contain bad articles and vice versa. Focus on research questions, be critical, ask yourself if it makes sense?

system two thinking

eflective: thinking is slow, effortful, deliberate, learn to drive, when the stakes are high.

What is an interview

face-to-face, verbal exchanges in which one person, the interviewer, attempts to acquire information from and gain an understanding of another person, the interviewee. An Interview may focus on attitudes, beliefs, behaviors or experiences. Produces non-numeric (qualitative) data. Time-consuming, not guaranteed to get the info you seek, response rate may be better, allows for follow-up questions, may offer more insight/detail. Type of qualitative data collection.

Ultimate goal of a literature review

gather credible existing, scientific evidence about our specific focused question. Create a summary of what is known about the topic (concept map). Want to take all of the information to find the big picture. Draw from a lot of literature. any information gathered during the lit review should be scrutinized.

Research design

how you collect the data, the quality of a research design is based on the four vailidities: 1. Internal, external, construct, and statistical conclusion. 2. This is why we focused on these when learning how to "appraise" empirical articles- essentially, we are assessing the quality of the study's research design.

JMU library

http://guides.lib.jmu.edu/MGT357

Intervention

intervention the most successful, and the least frequently used, approach to implementation. middle managers can dramatically increase their prospects of success by using intervention. After a solution was found, these managers intervened again to facilitate implementation by showing how performance could be improved. Using information to show that this had an impact, hoping that the implemented decision will continue because people see the value. Yet only used 7%. Cons: More effort, need data to compare and do not always have that

system one thinking

intuitive: thinking and impressions for action flow effortlessly, produces a constant representation of the world around us, not consciously focusing on how to do things, we just do them. Most of the time, this determines our thoughts. As system one makes sense of visual cues, memories and association, it suppresses alternative stories. We have no way of knowing if we have these errors and what they are. But the fact that people are not aware of their own biases does NOT mean they cannot be neutralized at the organizational level. People are infleunced by the collective- we can check others thoughts and improve their judgement.

successive fractions

is an approach that can be used to reduced a large or too large set of documents. Searching within an already retrieved set of documents can be used to eliminate less relevant or useful documents

What is a survey

measuring instruments that ask individuals to answer a set of questions A survey may also be called a questionnaire. Surveys can be developed using existing questions/scales, or questions can be developed for the present needs. Questions should be carefully formed. Can be a type of quantitative or qualitative data collection (depending on questions used) Basic considerations: require careful planning, should not be quickly pulled together. Requires knowledge of topics (in order to choose questions). Can be distributed online, on paper, or face-to-face, each has pros and cons. Can allow for larger-scale data collection. Pilot testing is important. Can use close-ended and/or open-ended questions. Produces both numeric and non-numeric data.

Effective use of Boolean Operators

not, and, or

confirmation bias

people ignore evidence that contradicts their proconceived notions

What is scientific evidence?

research published in academic journals, findings from empirical studies.

With misconceptions and deterrents managers may approach EBM with...

reservations.

briefsearch

retrieves a few documents crudely and quickly. A briefsearch is often a good starting point, for further work.

citation pearl growing

starts from one or a few documents and uses any suitable terms in those documents to retrieve other documents. This is a relatively easy approach for a newcomer to a topic, or even indeed, research, to use.

Cross-sectional

study in which data of a statistically significant sample of a population (managers, CEO's, employees) is gathered at one point in time. It provides a snapshot of the current condition but does not explain cause and effect.

building blocks

takes the concepts in search statement and extends them by using synonyms and related terms. A thorough, but possibly lengthy search is then conducted seeing all of the terms to create a comprehensive set of documents.

Loss aversion

too cautious

difference between system on and system two thinking: how each influence our decision making

two modes of thinking: 1. intuitive: thinking and impressions for action flow effortlessly, produces a constant representation of the world around us, not consciously focusing on how to do things, we just do them. Most of the time, this determines our thoughts. As system one makes sense of visual cues, memories and association, it suppresses alternative stories. We have no way of knowing if we have these errors and what they are. But the fact that people are not aware of their own biases does NOT mean they cannot be neutralized at the organizational level. People are infleunced by the collective- we can check others thoughts and improve their judgement. 2. reflective: thinking is slow, effortful, deliberate, learn to drive, when the stakes are high.

Triangulation

we should be using more than one method to collect data before making a decision, look at more than one source. Involves the attempt to combine multiple methods... to cross-check findings, often on the assumption that the weaknesses of any single such element will be compensated by the strengths of others. Used to cancel out weaknesses of different methods.

Anchoring

weigh one piece of information too heavil in making decisions

sunk-cost fallacy

when considering new investments, we should disregard past expenditures that don't affect future costs or revenues.