ISR Module 1

framing effect

One of the common types of cognitive biases where the analyst has the tendency to draw different conclusions from the same data if it is presented differently

clustering illusion

One of the common types of cognitive biases where the analyst has the tendency to overvalue small runs, streaks, or clusters in large data sets.

insensitivity to sample size

One of the common types of cognitive biases where the analyst has the tendency to underestimate the variation in small data samples

value

One of the five V's of Big Data operations and is how relevant or appropriate the data is to a particular question.

volume

One of the five V's of Big Data operations and is the amount of raw data produced and stored.

variety

One of the five V's of Big Data operations and is the number of different types and sources of data.

velocity

One of the five V's of Big Data operations and is the speed at which data is produced and changed.

veracity

One of the five V's of Big Data operations and is the trustworthiness, quality, timeliness, etc of data.

visuals

One of the several forms of communication where the analyst can "tell" the story. This method uses graphics to make relationships between data more easily understood.

briefings

One of the several forms of communication where the analyst can foster two-way communications to ensure that customer intelligence questions are addressed. They consist of a well-organized, clear message relevant to the target audience; appropriate, tailored visual aids to reinforce the message; and a presentational style that keeps the audience focused on the message.

Project Maven

____ focuses on computer vision -- an aspect of machine learning and deep learning -- that autonomously extracts objects of interest from moving or still imagery.

ability

____ in creativity is to imagine or invent something new

machine vision

____ incorporates other AI concepts such as pattern recognition and object detection to identify people in photographs.

abductive reasoning

____ is the process of generating a novel hypothesis to explain given evidence that does not readily suggest a familiar explanation. It starts from a set of accepted facts and creatively proposes their best explanation.

spatial analysis

____ is the process of using location information to organize data spatially and identify the resultant patterns and relationships.

machine hearing

_____ and natural language processing allows users to ask their phones questions and even text messages.

assumptions

_____ are beliefs you take for granted

concepts

_____ are ideas, theories, laws, principles, or hypotheses we use in thinking to make sense of things

social network analysis

_____ is a process that applies graph and network theory to analyze the structures of social networks.

bias

A _____ is an unconscious belief that conditions, governs, and compels our behavior.

intelligence question

A well-framed _____ is the most important factor in assuring the overall quality of intelligence analysis.

drive collection

Analyst airmen need to understand how to ____. This includes understanding how information is collected and processed in the AF and more broadly in the IC, so they can answer intelligence questions, assess quality and credibility, and task collectors to collect information to fill intelligence gaps.

psychological blocks

As a result of _____, some solutions are not considered or are rejected simply because our reaction to them is "yuck"

vividness bias

Examples include: case histories and anecdotes over statistical data; words are easier to remember than numbers; and data summaries have less impact that inferior, vivid evidence. What type of bias is this?

oversensitivity to consistency

Consistency can be deceptive; information may be consistent only because it is highly correlated or redundant. It may be consistent only because information is drawn from a very small sample or a biased sample. What type of bias is this?

similarity of cause and effect

Examples include: economic events have economic causes, big events have important consequences and little events don't etc. What type of bias is this?

absence of evidence

Examples include: key information is often lacking in intelligence problems and experiments show that humans have a very difficult time assessing missing data and estimating its potential importance/ impact on the problem. What type of bias is this?

Mercury program

IARPA is developing the _____ to determine methods for continuous, automated analysis of SIGINT in order to anticipate and/or political crises, disease outbreaks, terrorist activity, and military actions. This project utilizes machine hearing and natural language processing.

Biases in Evaluation of Evidence

Information presented in vivid and concrete detail often have unwarranted impact, and people tend to disregard abstract or statistical information that may have greater evidential value. What type of bias is this?

bias in favor of causal explanations

It is rooted in our need to form a coherent narrative. When no pattern is apparent our first reaction is that we lack understanding. We should consider that there is no pattern. What type of bias is this?

1. Develop the focal issue 2. Convene a group of experts to brainstorm about the forces and factors that could affect the issue 3. Select by consensus, the two most critical and uncertain forces 4. Establish the most relevant endpoint for each factor 5. Form a Futures Matrix by crossing the two chosen axes 6. Create bumper sticker phrases to capture the future in each quadrant 7. Generate sign post or indicators for each quadrant that reflect the potential events and activities that could occur.

List all the steps for the alternative futures method.

1. Clearly define the intelligence question to be answered 2. List all significant evidence and assumptions relevant to the question. 3. Prepare a matrix with hypotheses across the top and evidence along the side 4. Refine matrix and reconsider the hypotheses delete evidence and arguments that have no diagnostic value. 5. Draw tentative conclusions about the relative likelihood of each hypothesis. 6. Analyze how sensitive your conclusion is to a few critical items of evidence 7. Report conclusions 8. Identify milestones for future observation that may indicate events are taking a different course than expected

List all the steps of ACH

1. Create your claim 2. Support the claim 3. Objections to claim 4. Rebuttals to objections

List all the steps of argument mapping method

1. Write down the agreed upon focal issue or question 2. Create a matrix with the following headers: OPTIONS, PROS, CONS, FIXES, INTENDED RESULTS OF ADOPTING, BROADER IMPLICATIONS, INDICATORS OF ADOPTED OPTION

List steps of adversary intent matrix method

mirror imaging

One of the common types of cognitive biases where the analyst has the assumption that others see events and react to them just as we do.

illusory correlation

One of the common types of cognitive biases where the analyst has the belief that two events are related when they might not be.

bandwagon effect

One of the common types of cognitive biases where the analyst has the tendency to believe something because many others believe it.

analytic writing

One of the several forms of communication where the analyst not only reports events but explains the significance of these events. Consumers can quickly grasp the main message and underlying reasoning, and analysts can minimize the chance for misinterpretation with this tool.

multimedia products

One of the several forms of communication where the analyst uses either interactive or full-motion video or animated web graphics to display content. They are particularly useful for depicting changes in data over time or comparing complex or multivariate data by allowing users to choose which variable they want to see.

scientific

One of the six modes of reasoning where the analyst draws casual inferences using facts, premises, and logic

market

One of the six modes of reasoning where the analyst draws conclusions from a structured consensus of persons

comparative

One of the six modes of reasoning where the analyst draws conclusions from examples, cases, or analogies

empirical

One of the six modes of reasoning where the analyst draws conclusions from personal interaction with real evidence

authoritative

One of the six modes of reasoning where the analyst draws inferences from expert testimony or strong belief

status quo

One of the six modes of reasoning where the analyst makes immediate judgments and inferences

bias in perception of cause and effect

People overestimate the extent to which other countries are pursuing a coherent, coordinated, rational plan, and thus also overestimate their own ability to predict future events in those nations. What type of bias is this?

adversary intent matrix

The _____ analyzes options that a decision-maker would likely consider in response to a certain situation. Analysis is done from the point of the view of the decision-maker

Mental shortcut

The _____ reduces the need to re-learn lessons, thus promoting efficient thinking. They are the basis of expertise.

17

The intelligence community (IC) is a coalition of ___ agencies and organizations.

perception

The process of _____ links people to their environment and is critical to accurate understanding of the world around us.

Garbage In, Garbage Out (GIGO)

The term _____ is often used here to describe situations where poor data yields poor results

big data

The term _____ refers to exceptionally large data collections and the process of extracting value from them.

information base

To effectively marshal information needed to answer intelligence questions, analysts require a solid understanding of the available _____. This includes data and information requirements, availability, location, strengths and weaknesses, process and dissemination, and value.

False

True/False? There is only one solution to every problem.

illusory correlation

Two events are correlated when the existence of one implies the existence of the other. What type of bias is this?

HUMINT

Type of intelligence that deals with clandestine and covert collection of documents, knowledge, etc.

SIGINT

Type of intelligence that deals with intercepted communications, emissions, and signals.

OSINT

Type of intelligence that deals with mass media, professional and academic publications etc.

GEOINT

Type of intelligence that deals with satellite imagery, geodesy data, GPS waypoints, etc.

MASINT

Type of intelligence that deals with scientific measurements, chemical analysis, etc.

availability bias

Unconscious cue when judging the probability of the event is the ease in which you can imagine the event and the number of instances is easily remembered. It's at its strongest when we make quick "gut" judgments without really analyzing the situation. What type of bias is this?

functional fixation

Under _____, we sometimes see an object only in terms of its name rather than in terms of what it can do.

Bounded rationality

We construct a simplified mental model of reality and then work within this model. This allows us to quickly and effectively operate in a complex world with limited knowledge and resources. This model is ______.

Emotions

When _____ rise, rationality becomes secondary.

expression of uncertainty

When no guide is given the words are given meaning by the context in which they are used and what the reader already perceives about the context. What type of bias is this?

Implications for Analysts

_____ are incomplete and/or ambiguous information causes difficulties in accurate perception. Analysts need understand that their perception is flawed based on their background, life experience, education, etc.

inferences

_____ are interpretations or conclusions you come to

prejudice

_____ are preconceptions that often prevent us from seeing beyond what we already know or believe to be possible. They also inhibit us from accepting change and progress.

information

_____ are the facts, data, evidence, or experiences we use to figure things out.

alternative futures

_____ are used when a situation is too complex or the situation outcome is too uncertain to trust a single outcome assessment. This SAT is a sharp contrast to contrarian techniques and, instead, is a divergent thinking technique that tries to use the complexity and uncertainty of a situation to describe multiple outcomes or futures that the analyst and policymaker should consider, rather than to predict one outcome.

breadth

_____ encompasses multiple view points; comprehensive

analysis of competing hypothesis

_____ ensures all information and arguments are evaluated and given equal treatment when considering hypotheses. It is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions.

Consequences

_____ follow from actions

Implications

_____ follow from thoughts

Patterns

_____ impose the order our minds seek and can blind us to new explanations.

attitude

_____ in creativity is accepting change and newness.

process

_____ in creativity is working hard to continually improve ideas and solutions by making gradual alterations and refinements to their works.

point of view

_____ is "the place" from which you view something

customer engagement

_____ is 1 of the 2 exceptions to the Air Force's 10 standards of measuring and assessing analytic excellence and fully addresses customer requirements within time constraints and in a manner most useful.

timeliness

_____ is 1 of the 2 exceptions to the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and timely to impact planning, mission objectives, operations and otherwise aid in commanders' decisions.

appropriate sourcing

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and cites all sources used, when feasible, and includes an objective assessment of the quality, credibility, and reliability of the underlying sources

assumptions vs judgments

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and clearly distinguishes among underlying intelligence, analyst assumptions, and analyst judgments.

level of confidence

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and helps qualify accuracy and to present analysis in a uniform and consistent manner across the AF, analysts will indicate the degree of confidence/certainty associated with analytic judgments and conclusions to the best of their ability.

utility

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and includes using AF analytic products and services that best enables the customer's understanding.

relevance

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and key to decision making, warfighting, and acquisition efforts. Analysts should make every effort to ensure products and services fulfill the customer's intent and request, while also ensuring products and services are available to others for use beyond the original intent.

logical argumentation

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will employ coherent and logical reasoning techniques, be supported by all key relevant information, and be internally consistent

alternative analysis

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will incorporate plausible alternative assessments, judgments, or hypotheses, particularly when major judgments contend with significant uncertainties, complexity, or a lack of available intelligence, or when low probability events could produce high impact results.

accuracy

_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will make the most accurate judgments and assessments possible, based on the intelligence available and in light of known information gaps.

synthesis

_____ is a creative method that combines two or more existing ideas into a third, new idea.

revolution

_____ is a creative method that completely changes an old idea to a new one; a marked change from the previous version.

evolution

_____ is a creative method that is incremental improvement. Every problem that as been solved can be resolved again in a better way.

re-application

_____ is a creative method that looks at something old in a new way.

changing direction (creative insight)

_____ is a creative method that shifts attention from one angle of a problem to another. The goal is to solve the problem, not to implement a particular solution.

learned helplessness

_____ is a feeling that you don't have the tools, knowledge, materials, ability, to do anything, so you might as well not try.

natural language processing

_____ is a process that applies artificial intelligence, computational linguistics, and computer science in an attempt to enable computers to understand human languages.

collaboration

_____ is a skill that extends across the entire analysis process. It consists of strong interpersonal relationships among analysts within and across intelligence agencies and the way the analysts relate to collectors, customers, and other experts within and outside the IC.

artificial intelligence

_____ is a sub-discipline of cognitive science that integrates with other disciplines such as computer science and engineering with the purpose of producing machines capable of replicating human intelligence and human senses. These machines seek to imitate human intelligence such as perception, learning, understanding, problem solving, communication, consciousness, and human senses such as vision, hearing, and natural behavior.

creativity

_____ is an ability, an attitude. and a process

relevance

_____ is bearing upon or relating to the matter at hand; important to the matter at hand

hindsight bias

_____ is bias that is presented in 3 areas of intelligence evaluations: analysts typically overestimate the quality of their analytic performance; intelligence consumers typically underestimate how much they learned from intelligence reports; and overseers of intelligence production normally judge events that readily foreseeable than was in fact the case. Human mental processes are primarily responsible for this bias.

logical reasoning

_____ is defined as the process of establishing a justified belief by using premises, evidence, and argument.

Mirror Imaging

_____ is filing gaps in the analyst's own knowledge by assuming that the other side is likely to act in a certain way because that is how the US would act under similar circumstances. It also leads to dangerous assumption, because people in other cultures do not think the way that we do.

fairness

_____ is free from bias, dishonesty, favoritism, deception etc.

accuracy

_____ is free from errors, mistakes, or distortions; true, correct

intelligence gap

_____ is information that is needed to appropriately conduct intelligence analysis and help answer an intelligence question but is absent from reporting- what we know, we don't know.

Devil's Advocacy

_____ is most effective when used to challenge an analytical consensus or a key assumption regarding a critically important intelligence question. On those issues that one cannot afford to get wrong, it can provide further confidence that the current analytic line will hold up to close scrutiny.

inductive reasoning

_____ is observation based. It is the intellectual process of drawing generalizations on the basis of observations or other evidence (aka learning from previous experience).

Groupthink

_____ is one of a number of terms that we use without truly realizing what it is, why it occurs, and how we can mitigate it. It is "a mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when members' strivings for unanimity override their motivation to realistically appraise alternative courses if action."

Critical thinking

_____ is self-guided, self-disciplined thinking which attempts to reason at the highest level of quality in a fair-minded way.

precision

_____ is specific, delivered to the necessary level of detail

machine learning

_____ is the modern science of finding patterns and making predictions from data based on work in multivariate statistics, data mining, pattern recognition, and advanced/predictive analytics. Basically, it replicates the human ability to learn and make decisions based on past experience.

logic

_____ is the parts that make sense together; it doesn't contradict itself; it shows sound reasoning

question

_____ is the problem or issue and guides our thinking

deductive reasoning

_____ is theory based. It is a reasoning in which a conclusion is a logical consequence of the premise. It moves from general rules and some evidence to particular cases.

clarity

_____ is understandable, the meaning can be grasped; free from confusion or ambiguity

argument mapping

_____ is used to test a single hypothesis or tentative analytical judgement and then uses a box-and-arrow diagram to visually lay out the argument with the evidence both for and against the hypothesis or analytical judgement.

purpose

_____ is your goal, objective, what you are trying to accomplish

significance

_____ means having importance; having considerable or substantial meaning

cognitive bias

_____ results from subconscious mental procedures for processing information. It is a mental error that is consistent and predictable regardless of cultural differences. Much like an optical illusion, they remain compelling even when we know they exist. They don't result from any emotional or intellectual predisposition toward a certain judgement, but rather from subconscious mental procedures for processing information.

depth

_____ thoroughly covers all the complexities of an issue; considers the many variables in context, situation, idea, etc.