SW Testing Quiz 2
known state; acted on; observed
Testability is the degree to which "program elements" can be put into a ____, ____, and ____
specification; test scenario
The ____ and ____ should give the choice of relevant equivalence partitions
code base; test
TDD is focused on ___ development, not ____ development
a development process where we test first
TDD is...
abstraction
State diagrams can be drawn at different levels of ____
provide a number of examples then generalize
Triangulation
if the software is developed so that its behavior can be verified, it's easy to confirm that it supports a certain feature
The testability premise
the spec, the size of a data type, and/or common sense
Where are boundary values and edge cases are derived from?
1. the degree to which a system or component facilitates the establishment of a test criteria and the performance of tests to determine whether those criteria have been met 2. the degree to which "program elements" can be put into a known state, acted on, and observed 3. how hard it is to find faults in the software
What is testability?
when the code is designed with testability in mind from the start and each program element has a single area of responsibility
What is testable software?
isolation
What is the key concept of a unit test?
(1) program elements can be put into a known state, acted on, and observed and (2) this can be done without affecting any other program elements and without them interfering
What is the outcome of testability?
- first, describe a test set/suite for the unit - write what it, the test, does - make the tests fail - implement the unit - verify that the implementation of the unit makes the tests succeed
What is the process of BDD
Unit testing if the foundation of TDD, but TDD is more than just unit testing
What is the relationship between unit testing and TDD?
Provides a rule-of-thumb when establishing a test suite
Why use the test pyramid?
fear; duplication
____ results in ____
- improved regression testing - allows refactoring without the fear of breaking code - builds the automated unit test suite automatically - provides an easy way to test a new unit without the burden of the entire system - demonstrates the development process - produces a short feedback loop and rapid iterations - forces the developer to plan ahead - reduces long-term development time - helps ensure new code works as it is written - tests can serve as detailed specification - allows you to always know a minute ago that everything worked - goes hand-in-hand with other development practices
Benefits of unit testing and TDD
- where to start in the process - what to test and what not to test - how much to test in one go - what to call the tests - how to understand why a test fails
BDD focuses on...
stories and specifications
BDD focuses on...
given-when-then
BDD gives test cases using...
software development; management
BDD provides ____ and ____ teams with shared tools and a shared process to collaborate on software development
ubiquitous (domain specific)
BDD uses ____ language
- encourages collaboration with non-technical team members - conversation focused - driven by business value - defines user stores and acceptance criteria within business/user-focused language
Benefits of BDD
how many of the statements in the program have been executed
statement coverage
portable
unit tests are ____ and must run on multiple development machines within a continuous integration environment
(1) an entire object or part of an object (i.e. method or interacting methods) and (2) the interaction between several objects
xUnit can be used to test
unit and integration testing; not system testing
xUnit is primarily used for...
- people will only aim for the target code coverage - high coverage numbers are too easy to reach with low quality testing - high percent coverage could still be problematic if critical parts are not being tested - doesn't always correlate to quality (i.e. testing from the user perspective)
Code coverage challenges
readable and maintainable code; descriptive test names; organized or grouped logically
Good unit tests are written with ____, ____, and are ____
a test that operates on an empty value like zero, null, empty string, etc.
Degenerate case
equivalent
Members in equivalence classes are treated as ____
white
Sensitize test cases define specific input variables and associated values to exercise certain parts of the program in the ____ box view
black
Sensitize test cases define specific input variables and associated values to perform certain functions in the ____ box view
control flow graph (CFG)
A ____ models all executions of a unit by describing control structures
range
A developer would base equivalence class partitions based on the ____
any
A good unit test can be run in ____ order
automated
A good unit test is able to be fully ____
mocks
A good unit test may use ____ to achieve isolation from the system as needed
a single
A good unit test tests ____ logical concept in the system
no side effects
A good unit tests causes ____
full control
A good unit tests has ____ over all the pieces running
trustworthy results
A good unit tests provides ____ such that referring to the code is unnecessary to confirm it
domain
A tester would base equivalence class partitions based on the ____
a single
A unit tests checks ______ logical operation(s)
a single
A unit tests checks for _____ assumption(s) about the output
True
(T/F) Code coverage can be used as a *part* of an existing criteria along with reliability measures.
False
(T/F) Code coverage indicates how good your tests are.
false
(T/F) a unit test can have external dependencies, such as a filesystem or API
(1) a method to set up the test class, (2) methods to set up the state before each test, (3) a collection of test methods, (4) methods to update the state after each test, (5) a method to update the state after the test class
A test class includes...
state machines
Certain applications or components of a system can be modeled as ____
uniform
Equivalence testing has a ____ coverage of partitions
the simplest way to make a test pass; just return or do whatever the test expects
Faking
More
For code coverage, low numbers are ___ telling than high numbers.
UI/end-to-end tests, service/integration tests, unit tests
From top to bottom, what are the levels of the test pyramid?
lays the foundation for implementation while remaining focused on the core functionality
Happy path tests
unit testing
How can we test one function before it is used elsewhere?
a function is a unit
How do functional or procedural languages define units?
as an object or an individual method
How do object-oriented languages define units?
acceptance criteria
In BDD, developers implement ____
boundaries of an equivalence partition
Many testing problems are commonly observed at ____
(1) degenerate case, (2) happy path tests, (3) try new info/knowledge/angles, (4) error handling and negative tests
Order of writing tests
- takes longer in the short-term - difficult to unit test functions with irreversible/un-mockable side effects or time delays - no tests for the tests; tests can be written wrong - changes in requirements or the system may mess up existing TDD - desire to make all tests "green" can lead to skipping or removing broken tests - requires developing "mocks" for external dependencies, which can be time consuming - doesn't guarantee quality tests or proper code - tedious
Problems with TDD
80%
Reasonable code coverage percent
faster
Tests run ____ at the bottom of the pyramid
slower
Tests run ____ at the top of the pyramid
(1) how to provide the test values to the software and (2) how to observe the results of test execution
The 2 practical problems of testability
the chance that somebody will test it
The more testable the software, the greater ____
isolation
There is more ____ at the bottom of the test pyramid
integration
There is more ____ at the top of the test pyramid
isolation, low level, white box, portable, consistent, fast
Traits of a unit test
a level of testing
Unit testing is...
white; implementation
Unit tests are ____ box in nature an focus on ____ details
low; tiny
Unit tests are ____ level and focused on a ____ part of a software system
testability
We must write code with ____ in mind
code that tests a unit of work; a method, class, or cluster of functions/classes that implement a single logical operation and checks a single assumption about the resulting output or behavior
What is a unit test?
smallest testable cohesive chunk of a system
What is a unit?
A development process that extends TDD by using natural language that non-technical stakeholders can understand
What is behavioral driven development?
A way of finding untested parts of code by measuring the amount of code executed by your test suite
What is code coverage?
Test cases for input values at the edges of the domain of allowed input
What are edge cases?
transfers of control
What are edges in a control flow graph?
statements or sequences of a statement
What are nodes in a control flow graph?
defining specific input variables and associated values to exercise certain parts of the program in the white-box view or to perform certain functions in the black-box view
What are sensitize test cases?
Faking, the obvious implementation, and triangulation
What are the red to green strategies?
(1) Write tests with different granularity and (2) the more high-level you get, the fewer tests you should have
What are the two aspects of the test pyramid?
verify that code behaves correctly with respect to a specification or some other expectations
What does a test do?
when it's difficult to test whether a change is going to have an impact on some other component
What is "brittle" code?
A unit (i.e. single logical operation) that's composed of functions, objects, or methods.
What is a test suite?
how many of the branches of the control structures have been executed
branches coverage
how many of the boolean sub-expressions have been tested for a true/false value
condition coverage
how many of the functions defined have been called
function coverage
how many lines of a source code have been tested
line coverage