Software Engineering 2 Exam Review

Which of the following are examples of Functional Testing? - Usability Testing - Performance Testing - Unit Testing - Regression Testing

- Unit Testing - Regression Testing

What does it mean when one type of coverage subsumes another?

100% branch coverage guarantees 100% statement coverage. In this way we can say branch coverage subsumes statement coverage.

When do we know we are done?

After adding "enough" tests without triggering a new failure.

How does Partition Testing improve a testing suite?

As we saw with random testing, it isn't very smart to just wildly picks from the input domain. There is a theory that errors tend to "cluster" together within the larger domain. The goal of Partition Testing is to identify sub-domains that can allow for more intelligent testing, but with fewer test cases to cover the entire input domain. The idea is that all the inputs within each sub-domain share an equivalence. That is why this approach is often referred to as Equivalence Partitioning.

Each unittest test case must include at least one ____.

Assertion

Apply either manual or automated testing: You want to ensure developers verify their code doesn't break existing features before adding it to the centralized repository.

Automated

Why is 100% statement coverage not enough?

Because other types of coverage are needed as well.

When is black box testing useful?

Black Box Testing is when you write tests based purely on the description provided for the software (a.k.a. the specification). - Focuses on the input domain for the software - Allows targeted testing of possible inputs - There is no need for the actual code - Non-developers can write tests - Tests can be written before the actual code - Allows for unbiased tests by separating the tester from the developer - Can catch logic errors that other types of testing cannot - Can be used at all levels of testing: unit testing, integration testing, and so on

How does random testing help with large input domains?

Can cover large portions of the input domain with very little code: In this way a single test can be run hundreds, even thousands, of times and cover a larger portion of the input domain.

The @classmethod used with setUpClass() and tearDownClass() is called a modifier. (T/F)

FALSE (it's a decorator)

An Error is a deviation from the expected behavior (T/F)

FALSE - An error is a mistake (typo and conceptual misunderstanding, etc.) which leads to a fault.

How can Git help software development?

In its most basic form, Git is a command line program that allows users to designate any folder on their computer as a local Git repository by using git init. This repo can then be stored centrally almost anywhere, but for this class, we will be using the aforementioned GitHub. Users can also clone remote repos and create an exact copy locally.

What happens during the Design phase?

Layout the architecture of the program and Produce a design document.

Apply either manual or automated testing: a game developer and received an angry email from a customer saying they fell through the ground when trying to climb a particular hill.

Manual

Apply either manual or automated testing: Scenario: You are interested in learning how user friendly your software is.

Manual - This type of task really requires a human to interact with the software.

___ simulate the behavior of a service and its actions can be verified.

Mocks

Identify multiple lightweight Code Review methods

Pair programming Over-the-shoulder Change-based Meetings

___ contain predefined data that is returned when called, but do not imitate behavior.

Stubs

What is the difference between a Stub and a Mock?

Stubs contain predefined data that is returned when called, but do not imitate behavior. Mocks simulate the behavior of a service and its actions can be verified.

If we want to override the behavior of an imported item, we need to use patch (T/F)

TRUE

Unit Testing is when the smallest component of a software system is verified to produce the expected behavior.

TRUE

Define TDD

Test Driven Development is the approach where one only writes new code if there exists at least one failing unit test.

Identify the 4 elements to a testing framework.

Test runner, Test fixture, Test case, Test suite RUNNER - FIXTURE - CASE - SUITE (in all caps so you can remember it better)

Define input domain

The input domain is the pool of all possible inputs that a unit/program can take.

The Therac-25 machine was used for radiation therapy in the 1980's. It had two modes of radiation: low and high. During the accidents, the machine would incorrectly display the machine was set to low power when in fact it was configured for high power. What caused this deadly situation?

The technician input commands very quickly

What is unit testing?

Unit Testing is when the smallest component of a software system is verified to produce the expected behavior.

Which of the following are examples of Non-functional Testing? - Performance Testing - Scalability Testing - Unit Testing - Integration Testing

- Performance Testing - Scalability Testing

How can I be a good Code Reviewee?

- Put your best foot forward - Proof read it before sending, make it worth their time, shine a light on things you're not so sure about - Don't panic or take it personally - believe that the feedback was written from a point of caring. - Learn from your mistakes and from their feedback.

How can I be a good Code Reviewer?

- Don't Brush it off: - Read every line you're assigned - Put effort into your feedback - Think about how you would do it - Be respectful and kind - Ask questions and Provide solutions - Don't be reactive

Apply either manual or automated testing: Scenario: You want to verify your system can handle a large range of possible inputs.

Automated - Instead of having a human try to input thousands of possible inputs, automated tests can randomly generate inputs in a fraction of the time. This is called Random Testing.

Describe how Fuzzing is different than the Random Testing discussed earlier in the course

Fuzzing is a form of random testing that can be used in system testing. It aims to discover errors in the program by feeding them random inputs. A Fuzzer is the tool/software used to perform Fuzzing Is usually a black box testing technique, and helps to improve the software's stability. Used to find security exploits often done in penetration testing.

What are the limitations of black box testing?

- It isn't possible to test every possible input, so tests may miss logic branches/program paths untested - There is no way to know why the failure occurs, just that the failure indicates a fault - Poorly written specifications can lead to inaccurate tests

Describe multiple black box testing techniques

Random Testing, Boundary Testing, Partition Testing

Identify the steps of TDD

1. Write a test 2. Run all currently written tests - If the tests all pass, return to Step 1 - If a test fails, proceed to Step 3 3. Write the bare minimum of code to make the test pass 4. Run all the currently written tests - If tests all pass, return to Step 1 - If the failing test is still failing, return to Step 3 5. Occasionally evaluate if the code can be refactored to reduce duplication or eliminate no longer used parts of the code 6. Eventually stop development after adding "enough" tests without triggering a new failure

What is a Fagan Inspection?

A six-step formal code review process. The steps are planning; overview; preparation; inspection; rework and follow-up.

When is white box testing useful?

Advantages: Based on the code so the quality of the tests can be measured objectively Can be used to compare test suites by measuring their quality Can directly test the coded behavior Disadvantages: Cannot discover errors due to missing paths (i.e. an unimplemented specification) Large software systems make it difficult to test every facet of the code (more on this below) Tests must be written by developers

What software development approach is most commonly associated with Continuous Integration?

Agile Extreme Programming?

What software design approach is most directly associated with TDD?

Agile. TDD forces the developers to really think about the specifications and expected behavior for each feature. In order to write the unit tests first, the developer really has to think about appropriate user stories and use cases.

Apply either manual or automated testing: Scenario: You want to make sure your new changes didn't break your existing features.

Automated - Using pre-written tests that were known to pass, you can quickly verify that your changes didn't cause any of them to fail. This is called Regression Testing.

What is the difference between a failure, fault, and error?

Failure - a deviation from the expected behavior These failures occur because there exists a "bug" in the code: a fault. Fault: - an instance of incorrect code that can lead to a failure A fault is introduced to the program when a programmer makes a mistake: an error. Error - a mistake that introduces a fault (e.g. typo and conceptual misunderstanding)

Identify multiple motivations for conducting Code Reviews

Finding errors Improve readability Verify solutions Better initial code quality Knowledge sharing

What role does Code Review play in Continuous Integration?

For teams that require a Code Review before changing the central repository, the developer will issue a pull request. This will trigger a build and the running of the test suite. If everything is successful, then other developers will be notified that changes are ready for review. If everything looks good, those changes will be added to the project.

What is software verification?

Software Testing occurs during the Verification phase. The goal of the Verification phase is to establish that the work done during the Implementation phase satisfies the Design and Requirements. In other words, "Does the code do the things we said we wanted it to do?" There are multiple ways of verifying software, but the one we are going to focus on is Software Testing.

Describe multiple types of coverage

Statement coverage - This is a way of measuring the quality of a testing suite based on the amount of statements the tests execute in the program Branch coverage - How well our tests cover the different branches in our code. Branches occur anywhere in the program where a decision must be made. These decisions occur in statements that contain conditionals and we need to make sure each is tested as evaluating as either True or False. Condition coverage - Condition coverage requires that we have tests that evaluate each individual condition as both True and False. Branch and Condition coverage - This coverage attempts to have 100% branch and 100% condition coverage. Path Coverage - Path coverage is where the tests strive to test every path through the code. A path is a unique series of branches. So, with path coverage you will likely have to traverse each branch multiple times.

How can we make our random testers "smarter"?

Unguided Random Testing: - Inputs are generated relatively evenly throughout the input domain. Guided Random Testing: - Inputs are generated following a heuristic that informs "smarter" input choices.

What is the testing framework we use in this course?

Unittest

What is a VCS?

Version Control Systems (VCS) help with the following problems by creating a centralized repository (repo) for the project files. Multiple programmers need to work on the same codebase at the same time Changes made by multiple programmers need to be combined so other team members can access the new code There needs to be a way to undo changes that introduce bugs into the codebase

Identify the parts of a random testing system

When random testing we need the following: - Random test case generator - Software to test (aka software under test) - Something to monitor for errors (aka an oracle)

When is it appropriate to employ MC/DC?

When working in safety critical industries such as aviation software.

Describe how white box testing can complement black box testing

Whereas black box testing focused on verifying that the software specifications are met, white box testing focuses on verifying that all the code works as intended by trying to have as much of it run as possible during testing. This is done through something called coverage.

Given the following code snippet, what is the output of the print statement? mock = Mock() mock.func("Hello World") mock.func("Greetings Planet") print(mock.func.call_args)

call('Greetings Planet')

Which type of testing requires the input of the client/user?

Acceptance Testing (or System Testing??)

Which of the following are reasons testing is important? Saves time, Saves money, Life and Death, Happier customers

All of those reasons

What is the difference between a centralized and decentralized VCS?

Centralized systems store all the files in a central repo. As developers make changes, they are immediately sent to the centralized repo and made available to the whole team. Decentralized systems also have a central repo with all the files, but changes made by individual developers are not immediately sent to the main repo. Each developer has a mini VCS on his or her local machine where changes can be committed without being made available to the team. This allows for developers to work on features individually and only push the changes to the central repo when ready. For this class we use a decentralized VCS called Git.

Why not just put all the set up steps within each step? It is because we want to keep our code __ ? (3 letters)

DRY (Don't Repeat Yourself)

What is the difference between functional and non-functional testing?

Functional Testing: These types of tests are used to verify that the software meets the requirement specifications when it comes to functionality; does the software do the things it is expected to do? Examples: - Does clicking "save" actually save the changes to the hard drive? - Does the search algorithm actually find the shortest path? - Does the isItPrime function correctly identify prime numbers? - Does the database correctly supply responses to the client? Types of Functional Testing: Unit testing Integration testing Regression testing Non-Functional Testing: These types of tests are used to verify that the software performs at the required levels. This can literally mean performance, but also usability, reliability, and robustness. Examples: - Does the webpage load in less than 2 seconds? - Does the interface provide enough clues for easy navigation? - Does the system successfully recover from a catastrophic failure? - Does the service support 10,000+ active users at a time? Types of Non-functional Testing: - Performance testing - Scalability testing - Usability testing

Describe how MC/DC relates to other types of Coverage

If one recalls, we learned that if you have 100% branch coverage, you are guaranteed to have 100% statement coverage (a.k.a. branch coverage subsumes statement coverage). Branch and Condition Coverage means that every branch of the code is activated AND every condition in every conditional statement is evaluated as both True and False. Yet, branch and condition testing is still very costly to implement so developers have developed a modified version: Modified Condition/Decision Coverage (MC/DC). Modified Condition/Decision Coverage's purpose is to only test the important conditions to limit the number of test cases required.

Describe the difference between Integration, System, and Acceptance Testing

Integration testing takes a broader look on how those units and later entire modules interact. Ex: previously we used mocks to pretend or simulate a query to a database so that the function would pass/fail based purely on itself, but in integration testing we actually want to have this function query a database for real. System testing attempts to verify the entire program working together. Ex: Think about a mobile banking app. In system testing we think more along the lines of user stories. We proceed through the system as if we were real users. This means we need to test different sequences of actions to verify the system behaves as expected no matter what the user attempts to do. One user story could see the user wishing to deposit a check by taking a picture with the phone's camera. This will require a tester to actually attempt these steps to verify the outcome. In acceptance testing, the developers of the software present a version to the cusutomer/client/end user. The purpose is for these stakeholders to "sign off" on if the software meets their expectations.

Define Continuous Integration

It is better to think of CI as a set of guiding principles on how to manage a team working on a shared codebase, instead of thinking of it as a specific tool/framework. - Use a VCS like Git to maintain a central codebase - Building the software should be automated and easily triggered - Once built, the software should be able to test itself against a provided test suite - Everyone needs to commit work to the shared codebase at least once a day - Every commit to master should be built and tested - Mandatory Code Review when requesting changes to be merged into the shared codebase

What does it mean to write the "bare minimum" of code?

Just enough to make the tests pass.

Apply either manual or automated testing: Scenario: You want to test for game breaking bugs in a massively online multiplayer game.

Manual - While automated tests could be used to test components of the game, testing the combined product really requires humans actively trying to break the game while it is running.

What are the differences between a Mutation-Based and Generation-Based Fuzzer?

Mutation-Based fuzzers start off with selecting a valid input. It then mutates the input in some random way and throws it at the software under test. Generation-Based fuzzers do NOT start with valid inputs. These fuzzers use some knowledge of the input domain to create the random inputs. This is similar to how we used 'rules' to generate random inputs to test a password validator.

Describe the pros and cons of automated testing

Pros of Automated Testing: - Easy to repeat - Once the tests are written, they can be run with the push of a button - Fewer mistakes - Unlike humans, the computers are guaranteed to run the tests the same way each time - Very efficient - Compared to humans, computers can simulate tasks at a much higher rate Cons of Automated Testing: - High upfront cost - It takes time to write tests and set up the needed tools - Not suited for everything - Some tasks, such as UI testing, can benefit greatly from the human eye - Test maintenance - Testing suites need to be regularly updated and added to as the software is expanded

Describe the pros and cons of manual testing

Pros of Manual Testing: -It is intuitive - Even beginner programmers instinctively know to verify their programs by running them -No upfront cost - There is no time investment required before you can start testing Cons of Manual Testing: - Time consuming - It takes a large amount of time to verify software by hand (time best spent elsewhere) - We are only human - People make mistakes that could miss software failures - Not easily repeatable - There is no easy way to quickly rerun all the tests after making small changes

Identify scenarios where random testing is well suited

Random testing is often done outside the normal test suite. It can be thought of as a supplemental testing technique. You will find that many implementations of random testing are run over night and the results are inspected in the morning. Random testing is frequently used during system tests, which attempts to verify that the entire piece of software functions as a unit.

You can create a test class by clicking on the name of the function/class. (T/F)

TRUE

A Failure occurs because of a "bug" in the code: a fault (T/F)

TRUE - Failure happens because of faults

What is an oracle?

The oracle's sole job is to watch for error states in the software and save the random inputs that generated those states for later inspection. An oracle can be as simple as displaying error generating input to the screen or it could be an elaborate piece of software that generates formal bug reports.

Given the following code snippet, what goes in the blank? from unittest.mock import Mock mock = Mock() # Set return_value mock.abs.______ = "7"

return_value

Which of the following is called before each test case in a TestCase object?

setUp()

What is the difference between setUp/tearDown and setUpClass/tearDownClass?

setUp/tearDown are called before/after each test case in a TestCase object, whereas setUpClass/tearDownClass are called before/after all of the test cases in a TestCase object.

Given the following snippet. If we wanted to use a mock to trigger an exception, what would we use in the blank? # Mock file reader to control its behavior open = Mock() def load_file(): # Does absolutely nothing other than raise the desired exception content = open('temp_file.txt') if content.length != 0: return content return None class TestCase(unittest.TestCase): def test_read_file(self): # Test for IOError open.______ = IOError # This is a context manager that allows us to test for exceptions with self.assertRaises(IOError): load_file()

side_effect

Which of the following is called after all the test cases in a TestCase Object are run (not after each test)?

tearDownClass()