Performance testing

Response time

time from when a user enters a request until the first character of the response is received.

Disk queue length

is the avg. no. of read and write requests queued for the selected disk during a sample interval.

CPU interrupts per second

is the avg. number of hardware interrupts a processor is receiving and processing each second.

Rollback segment

the amount of data that can rollback at any point in time.

Types of Stress Testing:

1.Distributed Stress Testing 2.Application Stress Testing: 3.Transactional Stress Testing: 4.Systemic Stress Testing: 5.Exploratory Stress Testing:

Maximum active sessions

the maximum number of sessions that can be active at once.

Hits per second

the no. of hits on a web server during each second of a load test.

Failures (measuring)

1.Failed Connections: Number of failed connections refused by the client (Weak Signal) 2.Failed Rounds: Number of rounds it gets failed 3.Failed Hits: Number of failed attempts done by the system (Broken links or unseen images)

Application Response

1.Hit time: Average time to retrieve an image or a page 2.Time to the first byte: Time is taken to return the first byte of data or information 3.Page Time: Time is taken to retrieve all the information in a page

Amount of connection pooling

the number of user requests that are met by pooled connections. The more requests met by connections in the pool, the better the performance will be.

Scalability testing purpose

1.It is a type of software testing that tests the ability of a system, a network or a process when the size/volume of the system is changed in order to meet a growing need. 2.The idea behind Scalability Testing is to measure at what point the application stops scaling and identify the reason behind it

Metrics for Stress Testing

1.Measuring Scalability & Performance 2.Application Response 3.Failures

Page faults/second

the overall rate in which fault pages are processed by the processor. This again occurs when a process requires code from outside its working set.

Load testing

(Нагрузочное тестирование) checks the application's ability to perform under anticipated user loads. The objective is to identify performance bottlenecks before the software application goes live.

Volume testing

(Объемное тестирование) Under Volume Testing large no. of. Data is populated in a database and the overall software system's behavior is monitored. The objective is to check software application's performance under varying database volumes.

Spike testing

(Спайк тестирование) tests the software's reaction to sudden large spikes in the load generated by users.

Endurance testing

(Тестирование выносливости системы) is done to make sure the software can handle the expected load over a long period of time.

Common issues observed during Soak Testing

1.Memory allocation (memory leaks that would eventually result in a memory crisis or rounding errors that manifest only over time). 2.Database resource utilization (Failure to close database cursors under some conditions which would eventually result in the entire system stalling). 3.It can also lead to performance degradation, i.e. to ensure that the response time after a long period of sustained activity is as good as it was at the beginning of the test. 4.Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system. 5.The gradual degradation of a response time of some functions as internal data-structures become less efficient during a long test.

Measuring Scalability & Performance

1.Pages per Second: Measures how many pages have been requested / Second 2.Throughput: Basic Metric - Response data size/Second 3.Rounds: Number of times test scenarios have been planned Versus Number of times a client has executed

What is the difference between Performance Testing & Performance Engineering

1.Performance Testing is a discipline concerned with testing and reporting the current performance of a software application under various parameters. 2.Performance engineering is the process by which software is tested and tuned with the intent of realizing the required performance. This process aims to optimize the most important application performance trait i.e. user experience.

The key parameters involved in Reliability Testing are

1.Probability of failure-free operation 2.Length of time of failure-free operation 3.The environment in which it is executed

The focus of Performance Testing is checking a software program's

1.Speed - Determines whether the application responds quickly 2.Scalability(Тестирование маштабируемости) - Determines maximum user load the software application can handle. 3.Stability - Determines if the application is stable under varying loads

Example Performance Test Cases

1.Verify response time is not more than 4 secs when 1000 users access the website simultaneously. 2.Verify response time of the Application Under Load is within an acceptable range when the network connectivity is slow 3.Check the maximum number of users that the application can handle before it crashes

Why do Soak Testing?

A system may behave normally when used for 2 hours, but when the same system is used continuously for 10 hours or more than that then it may fail or behave abnormally/randomly/it may crash. To predict such failure Soak Testing is performed.

Thread counts

An applications health can be measured by the no. of threads that are running and currently active.

Need for Stress Testing

Consider the following scenarios - 1.During festival time, an online shopping site may witness a spike in traffic, or when it announces a sale. 2.When a blog is mentioned in a leading newspaper, it experiences a sudden surge in traffic.

Difference between Functional and Load Testing: Executing frequency

Frequency of executing Functional Testing will be high A frequency of executing load testing will be low

Reliability Testing Example

The probability that a PC in a store is up and running for eight hours without crashing is 99%; this is referred as reliability.

Stress Testing is also known as

Endurance Testing

Types of reliability Testing

Feature Testing Load Testing Regression Test Featured Testing checks the feature provided by the software and is conducted in the following steps: Each operation in the software is executed at least once. Interaction between the two operations is reduced. Each operation have to be checked for its proper execution.

Volume Testing example

For example, testing the music site behavior when there are millions of users to download the song.

Distributed Stress Testing:

In distributed client-server systems, testing is done across all clients from the server. The role of stress server is to distribute a set of stress tests to all stress clients and track on the status of the client. After the client contacts the server, the server adds the name of the client and starts sending data for testing. Meanwhile, client machines send signal or heartbeat that it is connected with the server. If the server does not receive any signals from the client machine, it needs to be investigated further for debugging. From the figure, a server can connect with the 2 clients (Client1 and Client2), but it cannot send or receive a signal from Client 3 & 4.

Application Stress Testing:

It does stress testing on one or more transactions between two or more applications. It is used for fine-tuning & optimizing the system.

Garbage collection

It has to do with returning unused memory back to the system. Garbage collection needs to be monitored for efficiency.

Difference between Load and Stress testing:

Load testing identifies the bottlenecks in the system under various workloads and checks how the system reacts when the load is gradually increased Stress Testing determines the breaking point of the system to reveal the maximum point after which it breaks.

When do you use Load testing?

Load testing is performed to determine how many users the system can handle. You can also check out different scenarios that let you focus on different parts of your system. Like the home page or checkout webpage on your website for web load testing. It also helps you to determine how the load builds and sustains in the system.

Software reliability is measured in terms of

Mean to failure (MTTF): It is the difference of time between two consecutive failures Mean time to repair (MTTR): It is the time required to fix the failure. Reliability for good software is a number between 0 and 1

When do you use Performance Testing?

Performance testing is done to check the performance of website servers, database, and network. If you are adopting waterfall methodology, then it is important for you to check each time release of a new version. However, if you are using the agile software development approach, then you need to test application continuously.

What is Response Time Testing?

RESPONSE TIME TESTING measures the time taken for one system node to respond to the request of another. It is the time a system takes to reach a specific input until the process is over. Response time starts when a user sends a request and ends at the time that the application states that the request has completed.

Difference between Functional and Load Testing: Result factor

Results of functional tests are dependent on the test data Load testing depends on the number of users.

Difference between Functional and Load Testing: Result prediction

Results of functional tests are easily predictable as we have proper steps and preconditions defined Results of load tests are unpredictable

Difference between Functional and Load Testing: Result variation

Results of functional tests vary slightly Load test results vary drastically

Spike Testing

Spike Testing is a performance testing type used to test software applications with extreme increments and decrements in load. The goal of Spike testing is to see how the system responds to unexpected rise and fall of the user load.

Stability testing is also referred to as

Stability testing is also referred to as a Load or endurance testing.

Goals of Stress Testing

The goal of stress testing is to analyze the behavior of the system after a failure. For stress testing to be successful, a system should display an appropriate error message while it is under extreme conditions.

Scalability testing

The objective of scalability testing is to determine the software application's effectiveness in "scaling up" to support an increase in user load. It helps plan capacity addition to your software system.

Hit ratios

This has to do with the number of SQL statements that are handled by cached data instead of expensive I/O operations. This is a good place to start for solving bottlenecking issues.

Systemic Stress Testing:

This is integrated stress testing which can be tested across multiple systems running on the same server. It is used to find defects where one application data blocks another application.

Exploratory Stress Testing:

This is one of the types of stress testing which is used to test the system with unusual parameters or conditions that are unlikely to occur in a real scenario. It is used to find defects in unexpected scenarios like 1.A large number of users logged at the same time 2.If a virus scanner started in all machines simultaneously 3.If Database has gone offline when it is accessed from a website, 4.When a large volume of data is inserted to the database simultaneously

Processor Usage

an amount of time processor spends executing non-idle threads.

Volume Testing Vs Load Testing

Volume testing is testing of an application with as large amount of data in database as possible While in load testing, application is subjected to certain level of load to analyze the behavior of the application

Top waits

are monitored to determine what wait times can be cut down when dealing with the how fast data is retrieved from memory

When do you Use Stress Testing?

Website and app stress tests are significant before big events such as ticket selling for a famous concert with high demand of the people. So it is very important to test regularly to the capacity of the system endurance. This also helps you to prepare for unexpected traffic spikes and gives more time and resources to fix any bottlenecks.

Reliability Testing

a software testing process that checks whether the software can perform a failure-free operation for a specified time period in a particular environment.

Memory use

amount of physical memory available to processes on a computer.

Disk time

amount of time disk is busy executing a read or write request.

Committed memory

amount of virtual memory used.

Stress testing

involves testing an application under extreme workloads to see how it handles high traffic or data processing. The objective is to identify the breaking point of an application.

Network output queue length

length of the output packet queue in packets. Anything more than two means a delay and bottlenecking needs to be stopped.

Database locks

locking of tables and databases needs to be monitored and carefully tuned.

Private bytes

number of bytes a process has allocated that can't be shared amongst other processes. These are used to measure memory leaks and usage.

Memory pages/second

number of pages written to or read from the disk in order to resolve hard page faults. Hard page faults are when code not from the current working set is called up from elsewhere and retrieved from a disk.

Throughput

rate a computer or network receives requests per second.

Network bytes total per second

rate which bytes are sent and received on the interface including framing characters.

Bandwidth

shows the bits per second used by a network interface.