ISR Module 1
framing effect
One of the common types of cognitive biases where the analyst has the tendency to draw different conclusions from the same data if it is presented differently
clustering illusion
One of the common types of cognitive biases where the analyst has the tendency to overvalue small runs, streaks, or clusters in large data sets.
insensitivity to sample size
One of the common types of cognitive biases where the analyst has the tendency to underestimate the variation in small data samples
value
One of the five V's of Big Data operations and is how relevant or appropriate the data is to a particular question.
volume
One of the five V's of Big Data operations and is the amount of raw data produced and stored.
variety
One of the five V's of Big Data operations and is the number of different types and sources of data.
velocity
One of the five V's of Big Data operations and is the speed at which data is produced and changed.
veracity
One of the five V's of Big Data operations and is the trustworthiness, quality, timeliness, etc of data.
visuals
One of the several forms of communication where the analyst can "tell" the story. This method uses graphics to make relationships between data more easily understood.
briefings
One of the several forms of communication where the analyst can foster two-way communications to ensure that customer intelligence questions are addressed. They consist of a well-organized, clear message relevant to the target audience; appropriate, tailored visual aids to reinforce the message; and a presentational style that keeps the audience focused on the message.
Project Maven
____ focuses on computer vision -- an aspect of machine learning and deep learning -- that autonomously extracts objects of interest from moving or still imagery.
ability
____ in creativity is to imagine or invent something new
machine vision
____ incorporates other AI concepts such as pattern recognition and object detection to identify people in photographs.
abductive reasoning
____ is the process of generating a novel hypothesis to explain given evidence that does not readily suggest a familiar explanation. It starts from a set of accepted facts and creatively proposes their best explanation.
spatial analysis
____ is the process of using location information to organize data spatially and identify the resultant patterns and relationships.
machine hearing
_____ and natural language processing allows users to ask their phones questions and even text messages.
assumptions
_____ are beliefs you take for granted
concepts
_____ are ideas, theories, laws, principles, or hypotheses we use in thinking to make sense of things
social network analysis
_____ is a process that applies graph and network theory to analyze the structures of social networks.
bias
A _____ is an unconscious belief that conditions, governs, and compels our behavior.
intelligence question
A well-framed _____ is the most important factor in assuring the overall quality of intelligence analysis.
drive collection
Analyst airmen need to understand how to ____. This includes understanding how information is collected and processed in the AF and more broadly in the IC, so they can answer intelligence questions, assess quality and credibility, and task collectors to collect information to fill intelligence gaps.
psychological blocks
As a result of _____, some solutions are not considered or are rejected simply because our reaction to them is "yuck"
vividness bias
Examples include: case histories and anecdotes over statistical data; words are easier to remember than numbers; and data summaries have less impact that inferior, vivid evidence. What type of bias is this?
oversensitivity to consistency
Consistency can be deceptive; information may be consistent only because it is highly correlated or redundant. It may be consistent only because information is drawn from a very small sample or a biased sample. What type of bias is this?
similarity of cause and effect
Examples include: economic events have economic causes, big events have important consequences and little events don't etc. What type of bias is this?
absence of evidence
Examples include: key information is often lacking in intelligence problems and experiments show that humans have a very difficult time assessing missing data and estimating its potential importance/ impact on the problem. What type of bias is this?
Mercury program
IARPA is developing the _____ to determine methods for continuous, automated analysis of SIGINT in order to anticipate and/or political crises, disease outbreaks, terrorist activity, and military actions. This project utilizes machine hearing and natural language processing.
Biases in Evaluation of Evidence
Information presented in vivid and concrete detail often have unwarranted impact, and people tend to disregard abstract or statistical information that may have greater evidential value. What type of bias is this?
bias in favor of causal explanations
It is rooted in our need to form a coherent narrative. When no pattern is apparent our first reaction is that we lack understanding. We should consider that there is no pattern. What type of bias is this?
1. Develop the focal issue 2. Convene a group of experts to brainstorm about the forces and factors that could affect the issue 3. Select by consensus, the two most critical and uncertain forces 4. Establish the most relevant endpoint for each factor 5. Form a Futures Matrix by crossing the two chosen axes 6. Create bumper sticker phrases to capture the future in each quadrant 7. Generate sign post or indicators for each quadrant that reflect the potential events and activities that could occur.
List all the steps for the alternative futures method.
1. Clearly define the intelligence question to be answered 2. List all significant evidence and assumptions relevant to the question. 3. Prepare a matrix with hypotheses across the top and evidence along the side 4. Refine matrix and reconsider the hypotheses delete evidence and arguments that have no diagnostic value. 5. Draw tentative conclusions about the relative likelihood of each hypothesis. 6. Analyze how sensitive your conclusion is to a few critical items of evidence 7. Report conclusions 8. Identify milestones for future observation that may indicate events are taking a different course than expected
List all the steps of ACH
1. Create your claim 2. Support the claim 3. Objections to claim 4. Rebuttals to objections
List all the steps of argument mapping method
1. Write down the agreed upon focal issue or question 2. Create a matrix with the following headers: OPTIONS, PROS, CONS, FIXES, INTENDED RESULTS OF ADOPTING, BROADER IMPLICATIONS, INDICATORS OF ADOPTED OPTION
List steps of adversary intent matrix method
mirror imaging
One of the common types of cognitive biases where the analyst has the assumption that others see events and react to them just as we do.
illusory correlation
One of the common types of cognitive biases where the analyst has the belief that two events are related when they might not be.
bandwagon effect
One of the common types of cognitive biases where the analyst has the tendency to believe something because many others believe it.
analytic writing
One of the several forms of communication where the analyst not only reports events but explains the significance of these events. Consumers can quickly grasp the main message and underlying reasoning, and analysts can minimize the chance for misinterpretation with this tool.
multimedia products
One of the several forms of communication where the analyst uses either interactive or full-motion video or animated web graphics to display content. They are particularly useful for depicting changes in data over time or comparing complex or multivariate data by allowing users to choose which variable they want to see.
scientific
One of the six modes of reasoning where the analyst draws casual inferences using facts, premises, and logic
market
One of the six modes of reasoning where the analyst draws conclusions from a structured consensus of persons
comparative
One of the six modes of reasoning where the analyst draws conclusions from examples, cases, or analogies
empirical
One of the six modes of reasoning where the analyst draws conclusions from personal interaction with real evidence
authoritative
One of the six modes of reasoning where the analyst draws inferences from expert testimony or strong belief
status quo
One of the six modes of reasoning where the analyst makes immediate judgments and inferences
bias in perception of cause and effect
People overestimate the extent to which other countries are pursuing a coherent, coordinated, rational plan, and thus also overestimate their own ability to predict future events in those nations. What type of bias is this?
adversary intent matrix
The _____ analyzes options that a decision-maker would likely consider in response to a certain situation. Analysis is done from the point of the view of the decision-maker
Mental shortcut
The _____ reduces the need to re-learn lessons, thus promoting efficient thinking. They are the basis of expertise.
17
The intelligence community (IC) is a coalition of ___ agencies and organizations.
perception
The process of _____ links people to their environment and is critical to accurate understanding of the world around us.
Garbage In, Garbage Out (GIGO)
The term _____ is often used here to describe situations where poor data yields poor results
big data
The term _____ refers to exceptionally large data collections and the process of extracting value from them.
information base
To effectively marshal information needed to answer intelligence questions, analysts require a solid understanding of the available _____. This includes data and information requirements, availability, location, strengths and weaknesses, process and dissemination, and value.
False
True/False? There is only one solution to every problem.
illusory correlation
Two events are correlated when the existence of one implies the existence of the other. What type of bias is this?
HUMINT
Type of intelligence that deals with clandestine and covert collection of documents, knowledge, etc.
SIGINT
Type of intelligence that deals with intercepted communications, emissions, and signals.
OSINT
Type of intelligence that deals with mass media, professional and academic publications etc.
GEOINT
Type of intelligence that deals with satellite imagery, geodesy data, GPS waypoints, etc.
MASINT
Type of intelligence that deals with scientific measurements, chemical analysis, etc.
availability bias
Unconscious cue when judging the probability of the event is the ease in which you can imagine the event and the number of instances is easily remembered. It's at its strongest when we make quick "gut" judgments without really analyzing the situation. What type of bias is this?
functional fixation
Under _____, we sometimes see an object only in terms of its name rather than in terms of what it can do.
Bounded rationality
We construct a simplified mental model of reality and then work within this model. This allows us to quickly and effectively operate in a complex world with limited knowledge and resources. This model is ______.
Emotions
When _____ rise, rationality becomes secondary.
expression of uncertainty
When no guide is given the words are given meaning by the context in which they are used and what the reader already perceives about the context. What type of bias is this?
Implications for Analysts
_____ are incomplete and/or ambiguous information causes difficulties in accurate perception. Analysts need understand that their perception is flawed based on their background, life experience, education, etc.
inferences
_____ are interpretations or conclusions you come to
prejudice
_____ are preconceptions that often prevent us from seeing beyond what we already know or believe to be possible. They also inhibit us from accepting change and progress.
information
_____ are the facts, data, evidence, or experiences we use to figure things out.
alternative futures
_____ are used when a situation is too complex or the situation outcome is too uncertain to trust a single outcome assessment. This SAT is a sharp contrast to contrarian techniques and, instead, is a divergent thinking technique that tries to use the complexity and uncertainty of a situation to describe multiple outcomes or futures that the analyst and policymaker should consider, rather than to predict one outcome.
breadth
_____ encompasses multiple view points; comprehensive
analysis of competing hypothesis
_____ ensures all information and arguments are evaluated and given equal treatment when considering hypotheses. It is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions.
Consequences
_____ follow from actions
Implications
_____ follow from thoughts
Patterns
_____ impose the order our minds seek and can blind us to new explanations.
attitude
_____ in creativity is accepting change and newness.
process
_____ in creativity is working hard to continually improve ideas and solutions by making gradual alterations and refinements to their works.
point of view
_____ is "the place" from which you view something
customer engagement
_____ is 1 of the 2 exceptions to the Air Force's 10 standards of measuring and assessing analytic excellence and fully addresses customer requirements within time constraints and in a manner most useful.
timeliness
_____ is 1 of the 2 exceptions to the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and timely to impact planning, mission objectives, operations and otherwise aid in commanders' decisions.
appropriate sourcing
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and cites all sources used, when feasible, and includes an objective assessment of the quality, credibility, and reliability of the underlying sources
assumptions vs judgments
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and clearly distinguishes among underlying intelligence, analyst assumptions, and analyst judgments.
level of confidence
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and helps qualify accuracy and to present analysis in a uniform and consistent manner across the AF, analysts will indicate the degree of confidence/certainty associated with analytic judgments and conclusions to the best of their ability.
utility
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and includes using AF analytic products and services that best enables the customer's understanding.
relevance
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and key to decision making, warfighting, and acquisition efforts. Analysts should make every effort to ensure products and services fulfill the customer's intent and request, while also ensuring products and services are available to others for use beyond the original intent.
logical argumentation
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will employ coherent and logical reasoning techniques, be supported by all key relevant information, and be internally consistent
alternative analysis
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will incorporate plausible alternative assessments, judgments, or hypotheses, particularly when major judgments contend with significant uncertainties, complexity, or a lack of available intelligence, or when low probability events could produce high impact results.
accuracy
_____ is 1 of the Air Force's 10 standards of measuring and assessing analytic excellence and readiness and will make the most accurate judgments and assessments possible, based on the intelligence available and in light of known information gaps.
synthesis
_____ is a creative method that combines two or more existing ideas into a third, new idea.
revolution
_____ is a creative method that completely changes an old idea to a new one; a marked change from the previous version.
evolution
_____ is a creative method that is incremental improvement. Every problem that as been solved can be resolved again in a better way.
re-application
_____ is a creative method that looks at something old in a new way.
changing direction (creative insight)
_____ is a creative method that shifts attention from one angle of a problem to another. The goal is to solve the problem, not to implement a particular solution.
learned helplessness
_____ is a feeling that you don't have the tools, knowledge, materials, ability, to do anything, so you might as well not try.
natural language processing
_____ is a process that applies artificial intelligence, computational linguistics, and computer science in an attempt to enable computers to understand human languages.
collaboration
_____ is a skill that extends across the entire analysis process. It consists of strong interpersonal relationships among analysts within and across intelligence agencies and the way the analysts relate to collectors, customers, and other experts within and outside the IC.
artificial intelligence
_____ is a sub-discipline of cognitive science that integrates with other disciplines such as computer science and engineering with the purpose of producing machines capable of replicating human intelligence and human senses. These machines seek to imitate human intelligence such as perception, learning, understanding, problem solving, communication, consciousness, and human senses such as vision, hearing, and natural behavior.
creativity
_____ is an ability, an attitude. and a process
relevance
_____ is bearing upon or relating to the matter at hand; important to the matter at hand
hindsight bias
_____ is bias that is presented in 3 areas of intelligence evaluations: analysts typically overestimate the quality of their analytic performance; intelligence consumers typically underestimate how much they learned from intelligence reports; and overseers of intelligence production normally judge events that readily foreseeable than was in fact the case. Human mental processes are primarily responsible for this bias.
logical reasoning
_____ is defined as the process of establishing a justified belief by using premises, evidence, and argument.
Mirror Imaging
_____ is filing gaps in the analyst's own knowledge by assuming that the other side is likely to act in a certain way because that is how the US would act under similar circumstances. It also leads to dangerous assumption, because people in other cultures do not think the way that we do.
fairness
_____ is free from bias, dishonesty, favoritism, deception etc.
accuracy
_____ is free from errors, mistakes, or distortions; true, correct
intelligence gap
_____ is information that is needed to appropriately conduct intelligence analysis and help answer an intelligence question but is absent from reporting- what we know, we don't know.
Devil's Advocacy
_____ is most effective when used to challenge an analytical consensus or a key assumption regarding a critically important intelligence question. On those issues that one cannot afford to get wrong, it can provide further confidence that the current analytic line will hold up to close scrutiny.
inductive reasoning
_____ is observation based. It is the intellectual process of drawing generalizations on the basis of observations or other evidence (aka learning from previous experience).
Groupthink
_____ is one of a number of terms that we use without truly realizing what it is, why it occurs, and how we can mitigate it. It is "a mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when members' strivings for unanimity override their motivation to realistically appraise alternative courses if action."
Critical thinking
_____ is self-guided, self-disciplined thinking which attempts to reason at the highest level of quality in a fair-minded way.
precision
_____ is specific, delivered to the necessary level of detail
machine learning
_____ is the modern science of finding patterns and making predictions from data based on work in multivariate statistics, data mining, pattern recognition, and advanced/predictive analytics. Basically, it replicates the human ability to learn and make decisions based on past experience.
logic
_____ is the parts that make sense together; it doesn't contradict itself; it shows sound reasoning
question
_____ is the problem or issue and guides our thinking
deductive reasoning
_____ is theory based. It is a reasoning in which a conclusion is a logical consequence of the premise. It moves from general rules and some evidence to particular cases.
clarity
_____ is understandable, the meaning can be grasped; free from confusion or ambiguity
argument mapping
_____ is used to test a single hypothesis or tentative analytical judgement and then uses a box-and-arrow diagram to visually lay out the argument with the evidence both for and against the hypothesis or analytical judgement.
purpose
_____ is your goal, objective, what you are trying to accomplish
significance
_____ means having importance; having considerable or substantial meaning
cognitive bias
_____ results from subconscious mental procedures for processing information. It is a mental error that is consistent and predictable regardless of cultural differences. Much like an optical illusion, they remain compelling even when we know they exist. They don't result from any emotional or intellectual predisposition toward a certain judgement, but rather from subconscious mental procedures for processing information.
depth
_____ thoroughly covers all the complexities of an issue; considers the many variables in context, situation, idea, etc.