Week 2 - What is Testing?
Test Suite
A collection of test cases
Error
A mistake that creates a fault, e.g. a typo or a conceptual misunderstanding.
Testing Framework
A set of testing tools. E.g. Python's unittest module.
Test Runner
A way to execute the tests and report the results
Test Fixture
A way to setup the elements required for a test and then roll back the setup when the test is complete.
Test Case
A way to test a particular unit of the software with a specific input for a given response.
Failure
Deviation from the expected behavior. Failures occur because there's a "bug" in the code: a fault.
Pros of Automated Testing
Easy to repeat, fewer mistakes, very efficient
Verification Phase
Ensure that the implementation meets the requirements. It's possible that the implementation successfully realizes the design, but not the requirements. This would indicate the design is flawed.
Design Phase
Establishes the layout or architecture of the program (e.g. is a database or RESTful API needed?). Classes mapped out. End result is a design document.
Cons of Automated Testing
High upfront cost, not suited for everything, test maintenance costs
Fault
Instance of incorrect code that can lead to failure. A fault is introduced when a programmer makes a mistake: an error.
Maintenance Phase
Occurs after software is released. Focused on fixing bugs and adding features. Non-functional requirement issues can be addressed (e.g. performance or UI).
Types of Non-functional Testing (3 types)
Performance, Scalability, and Usability
What are the five phases of software development?
Requirements, Design, Implementation, Verification, & Maintenance
Why is testing important?
Save time/money in the long run, fewer bugs after release, happier customers, protects from teammate mistakes, security, quality of life, life and death
Functional Testing
Testing that verifies that the software meets the functional requirements (does it do what it's expected to do?).
Non-functional Testing
Testing that verifies that the software performs at the required levels. Can mean performance, but also usability, reliability, and robustness.
Requirements Phase
The phase that determines what the software must do. Stakeholder interviews are conducted to determine needs/expectations. User stories and use cases are created. In the end, lists of functional and non-functional requirements are created.
Cons of Manual Testing
Time consuming, prone to human error, not easily repeatable
Types of Functional Testing (3 types)
Unit, Integration, and Regression
Unit Testing
Verifying that the smallest component of a software system produces the expected behavior.
Implementation Phase
Where most of the coding happens.
Pros of Manual Testing
Intuitive and no upfront cost