Week 2 - What is Testing?

Test Suite

A collection of test cases

Error

A mistake that creates a fault, e.g. a typo or a conceptual misunderstanding.

Testing Framework

A set of testing tools. E.g. Python's unittest module.

Test Runner

A way to execute the tests and report the results

Test Fixture

A way to setup the elements required for a test and then roll back the setup when the test is complete.

Test Case

A way to test a particular unit of the software with a specific input for a given response.

Failure

Deviation from the expected behavior. Failures occur because there's a "bug" in the code: a fault.

Pros of Automated Testing

Easy to repeat, fewer mistakes, very efficient

Verification Phase

Ensure that the implementation meets the requirements. It's possible that the implementation successfully realizes the design, but not the requirements. This would indicate the design is flawed.

Design Phase

Establishes the layout or architecture of the program (e.g. is a database or RESTful API needed?). Classes mapped out. End result is a design document.

Cons of Automated Testing

High upfront cost, not suited for everything, test maintenance costs

Fault

Instance of incorrect code that can lead to failure. A fault is introduced when a programmer makes a mistake: an error.

Maintenance Phase

Occurs after software is released. Focused on fixing bugs and adding features. Non-functional requirement issues can be addressed (e.g. performance or UI).

Types of Non-functional Testing (3 types)

Performance, Scalability, and Usability

What are the five phases of software development?

Requirements, Design, Implementation, Verification, & Maintenance

Why is testing important?

Save time/money in the long run, fewer bugs after release, happier customers, protects from teammate mistakes, security, quality of life, life and death

Functional Testing

Testing that verifies that the software meets the functional requirements (does it do what it's expected to do?).

Non-functional Testing

Testing that verifies that the software performs at the required levels. Can mean performance, but also usability, reliability, and robustness.

Requirements Phase

The phase that determines what the software must do. Stakeholder interviews are conducted to determine needs/expectations. User stories and use cases are created. In the end, lists of functional and non-functional requirements are created.

Cons of Manual Testing

Time consuming, prone to human error, not easily repeatable

Types of Functional Testing (3 types)

Unit, Integration, and Regression

Unit Testing

Verifying that the smallest component of a software system produces the expected behavior.

Implementation Phase

Where most of the coding happens.

Pros of Manual Testing

Intuitive and no upfront cost