Week 7: Software Testing (Part 1) - CS 427

D

"Dividing input conditions into groups and test each group of inputs" is describing which of the following techniques for black-box testing? A) Boundary Value Analysis B) Decision Table Testing C) Use case testing D) Equivalence Class Partitioning

B

"Dividing input conditions into groups and test each group of inputs" is describing which of the following techniques for black-box testing? A) Boundary Value Analysis B) Equivalence Class Partitioning C) Decision Table Testing D) Use Case Testing

T (black-box, from specifications, is good at detecting faults of omission such as missing logic; white-box, from code, is good at detecting faults of commission such as extra logic)

(T/F): Both black-box and white-box testing are needed.

T (infeasible coverage)

(T/F): There are parts of code that cannot be covered by ANY test case.

T

(T/F): Validation confirms the new method meets requirements for a specific use.

F (not only do we need to run tests before delivering the software to customers, but we also need to run tests before moving onto the next phase of the project and after we change the software under test)

(T/F): We only need to run test cases before releasing the software to out customers.

failure

A incorrect implementation of an algorithm results in the program crashes by the test oracle. This is an example of a (mistake/failure/error).

test oracle

A mechanism to determine an expected result to compare with the actual result of the system under test.

D

According to the PIE model, what conditions are necessary for a failure to be observed? A) The fault-containing location(s) in the software is executed (execution) B) State of software is infected, i.e., causing error (infection) C) Infected state is propagated to cause some state/output of the software to be observed (ex: with test oracle) (propagation) D) All of the above

C (the opposite is true)

According to the PIE model, which following condition is NOT correct for a failure to be observed? A) The location or locations in the program that contain the fault must be reached. B) The state of the program must be incorrect. C) The infected state must NOT propagate to cause some output of the program to be incorrect.

mistake

Developers mistyped a variable as "bulid" while he was looking for variable "build". This is an example of a (mistake/failure/error).

system (integration), acceptance (done by client)

During the software development life cycle, black-box tests are typically done for __________ and ________ tests.

unit (implementation), regression (maintenance, usually automated)

During the software development life cycle, white-box tests are typically done for __________ and ________ tests.

validation

Process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. (Are we building the right product?)

verification

Process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. (Are we building the product right?)

both

Tests can document (faults/code/both).

loop coverage

This criteria requires sufficient test cases for all program loops to be executed for zero, one, two, and many iterations covering initialization, typical running, and termination (boundary) conditions.

condition (predicate) coverage

This criteria requires sufficient test cases for each condition in a program decision to take on all possible outcomes at least once. It differs from branch coverage only when multiple conditions must be evaluated to reach a decision. 100% of such coverage means that all sub-expression predicates have been true and false.

failure

Unexpected behavior during execution of a program when running test inputs. Results from a fault remaining undetected during testing.

B (see waterfall diagram for software development if refresher needed; we write tests in the implementation phase which is before the integration phase)

When do we usually want to write Unit tests? A) After maintenance phase B) Prior to integration phase C) During requirement phase D) Prior to designing phase

D

When should we write test cases? A) Before code is written B) While writing code C) After code is written D) All of the above

C

When we conduct tests, the procedure of coming up with a set of interesting input values is considered as: A) Test Automation B) Test Evaluation C) Test Design D) Test Execution

B

When we conduct tests, the procedure of understanding the output of execution cases and report or document the results is considered as: A) Test Automation B) Test Evaluation C) Test Design D) Test Execution

A (you should use white-box testing here since unit testing is done at the implementation phase, and you first want to make sure your code written actually works and covers everything first)

Which following software testing level do we typical NOT prefer using black-box Testing? A) Unit testing B) System Testing C) Integration Testing D) Acceptance Testing (done by customers)

A (we should use white-box testing during the implementation phase to see if the implementation actually works, afterwards we can move onto system and eventually acceptance testing, where black-box tests are mainly used)

Which following software testing level do we typical prefer using Whitebox Testing? A) Unit Testing B) System Testing C) Acceptance Testing (done by customers)

C (we do NOT use black-box testing to find infeasible paths because without access to the code, we would have no idea how much of the code is covered by the black-box test)

Which following statement about Infeasible paths is NOT correct? A) Infeasible paths are those path which can't be tested and verified by any set of possible input values. B) Infeasible paths are hard to detect by automation process. C) We prefer using black-box testing to find out infeasible paths. D) It usually requires domain knowledge for programmers to identify an infeasible path.

C

Which following statement about Whitebox testing is NOT correct? A) Methods like Equivalence Class and Boundary Value Analysis can also be applied in Whitebox testing. B) Maintaining white-box test scripts could be very difficult if implementation details keep changing frequently. C) White-box testing is usually very simple so it does not require knowledge of programming and implementation

B

Which following statement about Whitebox testing is correct? A) As a test developer, I do not need to know the details of implementations to develop my Whitebox test suite. B) Methods like Equivalence Class and Boundary Value Analysis can also be applied in Whitebox testing. C) Whitebox testing is typically conducted before conducting black-box testing

B (you do need to know how to program to conduct test automations; specifically need knowledge in scripting)

Which following statement about assigning testing persons is NOT correct? A) Conducting testing design requires the most technical knowledge B) You do not need to know how to program to conduct test automations C) In order to evaluate the test results, domain knowledge is required

C (others can write tests for your program too, even if they didn't develop it)

Which following statement of the Test Case is NOT correct? A) Test cases can document faults B) Test cases can document code C) One can only build test cases for specific program if the program is developed by him/her. D) The process of developing test cases can also help find problems in the requirements or design of an application.

E

Which of the following are part of the guidelines on black-box tests? A) Test cases need to be traceable to a requirement. B) Test cases need to be repeatable so anyone on the team can run the exact test case and get the exact same result/sequence of events. C) The inputs must be very specific (correct: grade = 80, grade = 20; incorrect: "input a passing grade", "input a failing grade"). D) The expected results must be very specific ("pass", "fail"). E) All of the above

C

Which of the following does NOT belong to the types of test activities? A) Test Design B) Test Automation C) Test Implementation D) Test Evaluation

D

Which of the following is NOT a metric for code coverage? A) Method coverage B) Branch coverage C) Statement coverage D) Parameter coverage

B (there is no way to guarantee that there aren't any faults in the software)

Which of the following is NOT a purpose of testing? A) Determine which parts of the software needs more testing. B) Guaranteeing there is NO faults in the software. C) Determine whether the software is sufficiently qualified for the release. D) Finding faults in the software.

D (you cannot determine that code is completely correct from testing)

Which of the following is NOT a purpose of testing? A) To improve code quality B) To spot code faults C) To document code D) To guarantee code is fully correct

A (automated tests don't do this)

Which of the following is NOT an advantage of test automation? A) It can help in identifying high priority test cases. B) It can be ran over and over again. C) It is good for validating functional flows. D) It improves the efficiency of testing.

C (since black-box testing doesn't require/use any knowledge of the code intervals, we would have no idea if all of the potential execution paths are covered)

Which of the following is NOT an advantage of using Blackbox testing? A) Testers do not need to know how the programs are implemented. B) Test cases can be developed as soon as the specifications are ready C) It can cover most of the potential execution paths. D) Tests are done from a user's perspective and will help in finding inconsistencies between the specifications and real behaviors.

C

Which of the following is a correct description of the main difference between Blackbox testing and Whitebox testing? A) Blackbox testing generally requires testers to have domain knowledge (e.g. programming language of the software implementation) B) Whitebox testing cannot be applied during Integration Test. C) The internal structure, design or implementation of the item is given to the test developers.

A (since black-box testing doesn't require/use any knowledge of the code intervals, we would have no idea if all of the potential execution paths are covered)

Which of the following is a disadvantage of using Blackbox testing? A) It is hard to cover all the program execution paths. B) Tester need a thorough knowledge of the implementation details. C) Maintaining test suites become harder if the implementation changes too frequently.

C (since white-box testing involves having access to the code, it's important to know exactly how the implementation works, which is a disadvantage compared to black-box testing since this takes more time)

Which of the following is a disadvantage of using Whitebox testing? A) Many program paths will be left untested B) Without clear specifications, test cases will be hard to design. C) Programmers need a thorough knowledge of implementations to develop a Whitebox test suite.

D

Which of the following is a type of code coverage measurements? A) Statement coverage B) Method coverage C) Loop coverage D) All of the above

D

Which of the following is a type of testing activity? A) Test Evaluation B) Test Design C) Test Execution D) All of them

C

Which of the following is an advantage of using Whitebox testing? A) Testers do not need to know how the programs are implemented. B) Tests are done from a user's perspective and will help in finding inconsistencies between the specifications and real behaviors. C) Tests can be developed at an earlier stage. Test developers do not need to wait for everything finished.

A (because black-box testing signifies that you don't have access to the code)

Which of the following is true about black-box testing? A) It is not feasible to thoroughly exercise all potential tests inputs. B) Tests are written with knowledge of the internal workings to thoroughly test what is happening on the inside of the code. C) Black-box testing is good at finding faults of extra logic. D) Black-box testing is good at better for finding crashes, out of bounds failures, file not closed failures.

B (developers could be biased in favor of their implementations)

Which of the following statement about developers writing tests is NOT correct? A) Developers may have bias on software codes and functions B) Developers can always detect all faults since they understand their implementations. C) More people are encouraged to engage in writing tests in addition to developers.

C

Which of the following statement about the Test Case is correct? A) Test cases cannot be used as documents as they are considered as two very different categories. B) Only developers can construct test case for their programs. C) The process of developing test cases can also help developers to understand the requirements.

D (there can be a lot more groups of people that develop test cases)

Which of the following statement about writing tests as a developers is NOT correct? A) Developers cannot always detect all faults even they fully understand their implementations. B) More people are encouraged to engage in writing tests in addition to developers. C) When developers construct test suite, they may have bias on software code or functions. D) Only QA developers should develop test cases since they are experts in testing area.

D (could take a super long time whether it be actually implementing the code, tracing all possible paths, etc.)

Which of the following statements about testing activities is correct? A) Designing tests requires relatively little knowledge about the domain and programs B) In order to embed test values into executable scripts, one needs to understand the theory very well. C) It is always efficient to assign the same person for all testing activities. D) Test automation is useful to have but could also be boring to construct.

C

Which of the following statements is true about tests? A) Tests must be written in the form of code B) User interfaces cannot be tested C) Projects can have more test code than source code D) Tests can always spot bugs

B

Which of the following test activities embeds test values into executable scripts? A) Test Design B) Test Automation C) Test Implementation D) Test Evaluation

A

Which of the following type of code coverage aims to ensure that each one of the possible branch from each decision point is executed at least once? A) Branch/Decision coverage B) Loop coverage C) Method coverage D) Statement coverage

B

Which of the following type of code coverage measures the lines covered by test cases? A) Method coverage B) Statement coverage C) Branch/Decision coverage D) Condition/Predicate coverage

D

Which statements about manual testing and automated testing is NOT correct? A) When regression tests are conducted very often, we should prefer doing automated testing. B) To test a Graphics Interface, we want to build automated tests after the interface design is stable. C) Tests that don't require re-execution should be done manually. D) Automated tests always find more bugs than manual tests.

equivalence partitioning

A black-box test design technique in which test cases are designed to execute representatives from equivalence partitions. In principle test cases are designed to cover each partition at least once. Example: For tests of "Pass the course" the most important thing is whether the student has a course grade less than 60 points (out of the full 100 points). Test input values clearly in the two partitions: 25 and 80.

boundary value analysis

A black-box test technique in which test values are picked at the boundary of an equivalence class because faults tend to occur at the boundaries of a domain, not its center. Range input, a to b, test with a, b, a-1, a+1, b-1, b+1 if integer range; otherwise, slightly less than a and slightly more than b. If you can only have a certain quantity (q) of something, try to create q-1, q, q+1.

fault

A defect or bug introduced in a program as a result of a developer making a mistake.

A (since errors are outputs printed out as a result of failures and faults, and we are given an output)

A developer called Math.log(5) function that was supposed to return the natural logarithm value of the input, returns 125 instead. This is an example of a: A) Error B) Failure C) Fault

error

A developer called sqrt(5) function that was supposed to return the square root value of the input but developer got 25 instead. This is an example of a (mistake/failure/error)

black-box testing

A testing approach that focuses on the functionality (or non-functionality) of the system and does not require knowledge of the code intervals.

statement coverage

A type of code coverage measurement that determines if all of the lines in a method have been executed.

method coverage

A type of code coverage measurement that determines if each method in the source code has been called by the test code.

branch (decision) coverage

A type of code coverage measurements that keeps track of the percentage of branches that have been exercised by a test suite. 100% of such coverage means that all predicates in the code have been true and false.

test design

A type of testing activity where we design test inputs to satisfy a test objective. Should be assigned to one with technical knowledge of testing/programming.

test automation

A type of testing activity where we embed test inputs into executable scripts. Should be assigned to one with knowledge of scripting.

test evaluation

A type of testing activity where we evaluate the (actual) results/document expected results. Should be assigned to one with domain knowledge.

test execution

A type of testing activity where we run test inputs on the software and record actual results. Should be assigned to the one with the least knowledge in testing.

code coverage

An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed.

error

An incorrect result produced by a computer program as a result from the failure during execution. Result will likely be different from the true, specified, or theoretically correct value.

white-box testing

Given knowledge of internal workings, thoroughly test what is happening on the inside. Exercise ALL the code portions and address extra, unused code (code coverage).

C

Suppose we want to test a new generic sorting algorithm using black-box testing and we have several different sets of testing inputs. Which following set of testing values ONLY contains inputs from an equivalent class? A) { [7,9,5,4,1], ['a', 'b', 'c'], ['a', 'a', 'b', 'b', 'c', 'c']} B) { [4,3,1, 2, 5], [1, 2, 3, 5], [42, 10, 23, -1], ['a', 'b', 'c'] } C) { [1, 2, 3, 4, 5], ['a', 'b', 'c'], ["aa", "ab", "ac", "bd"] }

automated tests

Test cases that are in code/scripts, created by a tool that records user actions, written manually in a test scripting language. Usefulness: -When you run the test over and over (manual regression tests are BORING) -When you want to use them as documentation (example runs)

manual tests

Test cases that are ran without any automated tools. Involves test description, preconditions (if necessary), input, procedure, expected results, and actual output. Tracing path in program done by hand. Usefulness: -Good for exploratory, GUI testing -When you're only going to run the test once -When tester doesn't know how to program -When tests are too expensive to automate or too expensive to maintain automated tests

debugging

The process of finding a fault given a failure.

testing

The process of finding inputs (and test oracles) that cause the software to fail.