SW Testing Quiz 2

known state; acted on; observed

Testability is the degree to which "program elements" can be put into a ____, ____, and ____

specification; test scenario

The ____ and ____ should give the choice of relevant equivalence partitions

code base; test

TDD is focused on ___ development, not ____ development

a development process where we test first

TDD is...

abstraction

State diagrams can be drawn at different levels of ____

provide a number of examples then generalize

Triangulation

if the software is developed so that its behavior can be verified, it's easy to confirm that it supports a certain feature

The testability premise

the spec, the size of a data type, and/or common sense

Where are boundary values and edge cases are derived from?

1. the degree to which a system or component facilitates the establishment of a test criteria and the performance of tests to determine whether those criteria have been met 2. the degree to which "program elements" can be put into a known state, acted on, and observed 3. how hard it is to find faults in the software

What is testability?

when the code is designed with testability in mind from the start and each program element has a single area of responsibility

What is testable software?

isolation

What is the key concept of a unit test?

(1) program elements can be put into a known state, acted on, and observed and (2) this can be done without affecting any other program elements and without them interfering

What is the outcome of testability?

- first, describe a test set/suite for the unit - write what it, the test, does - make the tests fail - implement the unit - verify that the implementation of the unit makes the tests succeed

What is the process of BDD

Unit testing if the foundation of TDD, but TDD is more than just unit testing

What is the relationship between unit testing and TDD?

Provides a rule-of-thumb when establishing a test suite

Why use the test pyramid?

fear; duplication

____ results in ____

- improved regression testing - allows refactoring without the fear of breaking code - builds the automated unit test suite automatically - provides an easy way to test a new unit without the burden of the entire system - demonstrates the development process - produces a short feedback loop and rapid iterations - forces the developer to plan ahead - reduces long-term development time - helps ensure new code works as it is written - tests can serve as detailed specification - allows you to always know a minute ago that everything worked - goes hand-in-hand with other development practices

Benefits of unit testing and TDD

- where to start in the process - what to test and what not to test - how much to test in one go - what to call the tests - how to understand why a test fails

BDD focuses on...

stories and specifications

BDD focuses on...

given-when-then

BDD gives test cases using...

software development; management

BDD provides ____ and ____ teams with shared tools and a shared process to collaborate on software development

ubiquitous (domain specific)

BDD uses ____ language

- encourages collaboration with non-technical team members - conversation focused - driven by business value - defines user stores and acceptance criteria within business/user-focused language

Benefits of BDD

how many of the statements in the program have been executed

statement coverage

portable

unit tests are ____ and must run on multiple development machines within a continuous integration environment

(1) an entire object or part of an object (i.e. method or interacting methods) and (2) the interaction between several objects

xUnit can be used to test

unit and integration testing; not system testing

xUnit is primarily used for...

- people will only aim for the target code coverage - high coverage numbers are too easy to reach with low quality testing - high percent coverage could still be problematic if critical parts are not being tested - doesn't always correlate to quality (i.e. testing from the user perspective)

Code coverage challenges

readable and maintainable code; descriptive test names; organized or grouped logically

Good unit tests are written with ____, ____, and are ____

a test that operates on an empty value like zero, null, empty string, etc.

Degenerate case

equivalent

Members in equivalence classes are treated as ____

white

Sensitize test cases define specific input variables and associated values to exercise certain parts of the program in the ____ box view

black

Sensitize test cases define specific input variables and associated values to perform certain functions in the ____ box view

control flow graph (CFG)

A ____ models all executions of a unit by describing control structures

range

A developer would base equivalence class partitions based on the ____

any

A good unit test can be run in ____ order

automated

A good unit test is able to be fully ____

mocks

A good unit test may use ____ to achieve isolation from the system as needed

a single

A good unit test tests ____ logical concept in the system

no side effects

A good unit tests causes ____

full control

A good unit tests has ____ over all the pieces running

trustworthy results

A good unit tests provides ____ such that referring to the code is unnecessary to confirm it

domain

A tester would base equivalence class partitions based on the ____

a single

A unit tests checks ______ logical operation(s)

a single

A unit tests checks for _____ assumption(s) about the output

True

(T/F) Code coverage can be used as a *part* of an existing criteria along with reliability measures.

False

(T/F) Code coverage indicates how good your tests are.

false

(T/F) a unit test can have external dependencies, such as a filesystem or API

(1) a method to set up the test class, (2) methods to set up the state before each test, (3) a collection of test methods, (4) methods to update the state after each test, (5) a method to update the state after the test class

A test class includes...

state machines

Certain applications or components of a system can be modeled as ____

uniform

Equivalence testing has a ____ coverage of partitions

the simplest way to make a test pass; just return or do whatever the test expects

Faking

More

For code coverage, low numbers are ___ telling than high numbers.

UI/end-to-end tests, service/integration tests, unit tests

From top to bottom, what are the levels of the test pyramid?

lays the foundation for implementation while remaining focused on the core functionality

Happy path tests

unit testing

How can we test one function before it is used elsewhere?

a function is a unit

How do functional or procedural languages define units?

as an object or an individual method

How do object-oriented languages define units?

acceptance criteria

In BDD, developers implement ____

boundaries of an equivalence partition

Many testing problems are commonly observed at ____

(1) degenerate case, (2) happy path tests, (3) try new info/knowledge/angles, (4) error handling and negative tests

Order of writing tests

- takes longer in the short-term - difficult to unit test functions with irreversible/un-mockable side effects or time delays - no tests for the tests; tests can be written wrong - changes in requirements or the system may mess up existing TDD - desire to make all tests "green" can lead to skipping or removing broken tests - requires developing "mocks" for external dependencies, which can be time consuming - doesn't guarantee quality tests or proper code - tedious

Problems with TDD

80%

Reasonable code coverage percent

faster

Tests run ____ at the bottom of the pyramid

slower

Tests run ____ at the top of the pyramid

(1) how to provide the test values to the software and (2) how to observe the results of test execution

The 2 practical problems of testability

the chance that somebody will test it

The more testable the software, the greater ____

isolation

There is more ____ at the bottom of the test pyramid

integration

There is more ____ at the top of the test pyramid

isolation, low level, white box, portable, consistent, fast

Traits of a unit test

a level of testing

Unit testing is...

white; implementation

Unit tests are ____ box in nature an focus on ____ details

low; tiny

Unit tests are ____ level and focused on a ____ part of a software system

testability

We must write code with ____ in mind

code that tests a unit of work; a method, class, or cluster of functions/classes that implement a single logical operation and checks a single assumption about the resulting output or behavior

What is a unit test?

smallest testable cohesive chunk of a system

What is a unit?

A development process that extends TDD by using natural language that non-technical stakeholders can understand

What is behavioral driven development?

A way of finding untested parts of code by measuring the amount of code executed by your test suite

What is code coverage?

Test cases for input values at the edges of the domain of allowed input

What are edge cases?

transfers of control

What are edges in a control flow graph?

statements or sequences of a statement

What are nodes in a control flow graph?

defining specific input variables and associated values to exercise certain parts of the program in the white-box view or to perform certain functions in the black-box view

What are sensitize test cases?

Faking, the obvious implementation, and triangulation

What are the red to green strategies?

(1) Write tests with different granularity and (2) the more high-level you get, the fewer tests you should have

What are the two aspects of the test pyramid?

verify that code behaves correctly with respect to a specification or some other expectations

What does a test do?

when it's difficult to test whether a change is going to have an impact on some other component

What is "brittle" code?

A unit (i.e. single logical operation) that's composed of functions, objects, or methods.

What is a test suite?

how many of the branches of the control structures have been executed

branches coverage

how many of the boolean sub-expressions have been tested for a true/false value

condition coverage

how many of the functions defined have been called

function coverage

how many lines of a source code have been tested

line coverage