Slide1: © Tata Consultancy Services ltd. 9 April 2008 1 Load Testing Web based Applications with LOADRUNNER


Slide2: Course Prerequisites Windows and Windows applications Load Testing concepts


Slide3: Course Objectives Understand the issues involved in load testing Web applications Learn your responsibilities as the LoadRunner expert Learn how to perform successful load tests using LoadRunner


Slide4: Part 1 – Introduction Part 2 – Planning Part 3 – Using VUGEN Part 4 – Controller Part 5 – Analysis Summary References Session Outline


Slide5: Part I – INTRODUCTION


Slide6: The Reality Fact #1 : Most users click away after 8 second delay Fact #2 : $4.4 billion in revenue lost annually due to poor web performance Source: Zona Research …continued


Slide7: Why Load Tests? The failure of a mission-critical web application can be costly Assure performance and functionality under real-world conditions Locate and resolve potential problems before it hits on the users. ….continued


Slide8: Performance is Important Why Load Tests? Source: Presentation by Lloyd Taylor, VP Technology & Operations, Keynote Systems


Slide9: Why Load Tests? Performance is the key to success of any web based application We will get to know the maximum capacity the system can handle for an application We can decide whether we should go for Hardware upgrades or Performance tuning ….continued


Slide10: Objectives of this Course Understand the issues involved in load testing Web applications Learn your responsibilities as the LoadRunner expert Learn how to perform successful load tests using LoadRunner


Slide11: Load Testing Many concurrent users running the same application to see whether a system handles the load without compromising functionality or performance. Test Objectives How many concurrent users can the system handle without increase in the expected response time? Can the system serve x concurrent users without any errors?


Slide12: Important Terminologies System Response Time Think Time Work Load Transaction Mix Throughput


Slide13: The System


Slide14: Response Time App Server DB Server Load Injector Time in which the system responds for a particular Transaction request


Slide15: Think Time Time taken for selecting a new Transaction after the response for the previous Transaction has been received Transaction 1 Transaction 2 Think Time = 10 Seconds


Slide16: Workload Typical Workload No. of Users Transactions Batch Programs Background Load


Slide17: Transaction Mix Varying execution frequency of different Transactions


Slide18: Throughput Transactions per Second Web Interactions per Second Pages per Second Bytes per Second The amount of work that can be performed by a system or component in a given period of time


Slide19: The Steps Involved in Load Testing Planning Load Tests Creating Vusers Creating Scenarios Executing Scenarios Analysis of the system under Load


Slide20: Part II - PLANNING


Slide21: Objectives of this Chapter Information that are required for load testing a system Where to obtain these information from? Organizing the system information collected Using the information effectively to carry out Load Tests.


Slide22: Business Transactions A business transaction is a set of user actions performed with an application to accomplish a business task. Examples: Reserving flight tickets Buying a computer from a online store Logging into your web-based email service


Slide23: Vital Information for any Load Test The architecture of the system A brief idea about the application. The list of business transactions The time at which the system has peak load The acceptable response time for the transactions No. of concurrent users the system is expected to handle during peak load


Slide24: How to Obtain? IT documentation Application users IT Managers System Administrators Server and online statistics Developers


Slide25: Getting Organized Select the Transactions to automate Decide on the Transaction Mix Obtain User details for the transactions The tables these transactions query The web server / application server components invoked by the transactions …continued


Slide26: Getting Organized Select the transactions Give priority to business critical transactions Give priority to transactions that have high database activity Give priority to transactions that have high web server or application server activity Know the peak load time …continued


Slide27: Getting Organized Decide on Transaction Mix Impact of the failure of a particular transaction on business The transactions that are fired during peak load the no. of times they are fired during peak load …continued


Slide28: Getting Organized User details For each of the chosen transactions find Who are the users who fire the transaction? How many concurrent users are there? How often they fire the transactions? …continued


Slide29: Getting Organized For each of the chosen transactions find Tables queried The tables the transaction queries The tables the transaction updates Server components invoked The servlet / jsp that are invoked by the transaction.


Slide30: Part III - Using VUGEN


Slide31: Objectives of this Chapter Introduction to Virtual User Technology Record scripts General options Recording options Runtime options Replay recorded scripts


Slide32: Virtual User Technology Simulates a real user Requires less resources – machines and people Greater control over test execution Can synchronize actions performed by users Collect and analyze results in a better way VUsers can communicate directly with a server by executing calls to the server API-without relying on client software


Slide33: VuGen Capture a business transaction by recording user actions performed Monitors the communication between the application and the server Generates the required function calls Inserts the generated function calls into a Vuser script


Slide34: Recording a Script New VUser Click on New Vuser Select the new Virtual User from the list of protocols Click OK The required header files are automatically included in the script


Slide35: Vuser Script Sections Script View Script Section


Slide36: Recording Options Recording Mode This allows us to specify the information to be recorded and which functions to be used when generating a script HTML Mode URL Mode


Slide37: HTML Mode The HTML mode is based on user actions, and the scripts contain functions that correspond directly to the action taken Example: ... web_link(“Enterprise Systems Performance", "Text=Enterprise Systems Performance ", "Snapshot=t4.inf", LAST);


Slide38: URL Mode The URL mode is based on HTTP requests sent to the server as a result of user actions. Example: . . . web_url(“Enterprise Systems Performance", "URL=http://www.tcs.com/esp.html", "TargetFrame=", "Resource=0", "RecContentType=text/html", "Referer=http://www.tcs.com/atc?. . . , "Snapshot=t4.inf", "Mode=URL", LAST);


Slide39: Deciding on Recording Mode Deciding on Modes For browser applications, use HTML mode. For non-browser applications, use URL mode. Note: The option of mixing recording modes is available for very advanced users for performance tuning


Slide40: Other Recording Options Selecting the Browser …continued


Slide41: Other Recording Options


Slide42: Recording Scripts …continued Click on Record Button ( ) Record dialog box pops up.


Slide43: Recording Scripts Note: Record user transaction is the appropriate sections Do not name a script init, run, or end, as these names are used by VuGen. Click Stop button ( ) to end recording Save the script.


Slide44: While recording, VuGen creates a series of configuration, data, and source code files which contain Vuser run-time and setup information. The results of each iteration of a script are stored separately. The Iteration0 directory is for system use and should be ignored by the tester. Directory Structure


Slide45: Note: LoadRunner Transactions can also be inserted during recording Load Runner Transactions A LoadRunner transaction measures start to end performance of any user action that is made from the browser. Click the Start Transaction button ( ) after selecting the starting point for measurement Click the End Transaction button ( )after selecting the end point for measurement. Select Transaction Status (optional) – LR_AUTO, LR_PASS, LR_FAIL


Slide46: Parameterization Parameter A parameter is a placeholder which re-places a recorded value in a Vuser script. At run time, a value from an external source is substituted for the parameter. Parameterize Input data During a run, VuGen replaces each parameter with a value from a data file or another external source, and inputs it to the Web application …continued


Slide47: Parameterization solves certain problems that may occur during playback Date constraints make recorded date invalid Unique constraints prevent reusing recorded data Data caching Response times are inaccurate Real user activity is not emulated Servers are not exercised Why Parameterization? …continued


Slide48: Date Constraint Problem Business process works only for the current date Vuser works only on the date it was recorded Parameterization …continued Solution Parameterize the input date Use a Date Type parameter to make the Vuser work for any current date


Slide49: Unique Constraint Problem Some fields require a unique ID as in the case of a Primary Key or fields in database with Unique constraint How to measure the performance of the process which creates a new product? Parameterization …continued Solution Parameterize the unique id fields After running, delete the test ID numbers from the system so the Vuser can be run again


Slide50: Data is Cached Problem Vusers all use the same data Data caching makes load test response times shorter than when real users load the system Parameterization Solution Parameterize input data to force the system to get data from a database …continued


Slide51: How to create data pool? Extract master data from database Write queries on tables being accessed by the business transactions. Spool data to a file Write AWK scripts to generate a proper data pool from the spooled file Create valid transactional data Predict according to type of data Obtain from a functional expert Parameterization


Slide52: The Process Determine which fields to parameterize Replace recorded values with parameters Decide which parameter type to use Choose the data access method and the number of iterations Run the Vuser and analyze the results to verify correct execution Parameterization


Slide53: Parameterization – The Process Right click on the data to parameterize select “Replace with a Parameter” The Select or Create Parameter dialog box opens. Type a suitable Parameter Name Select a Parameter Type Right-click the parameter name and select “Replace More Occurrences” to replace the data by the parameter name.


Slide54: Correlations In applications output of the previous transaction is used as an input for the next transaction. For example: System generated session ids URLs that change each time you access the Web page Fields (sometimes hidden) recorded during a form submission The data is only good for the current session Correlate the data Capture output value from one step Use captured value as input to another step


Slide55: Correlations Correlated data is data which is sent to the client from the server, and later sent back to the server by the client Dynamic data is correlated automatically by the client application (running in the browser) while recording a script


Slide56: Reusing recorded dynamic data will cause errors when you replay the script Recorded dynamic data must be recorrelated using parameters Correlations


Slide57: Determine the value to capture Find the right and left text boundaries of the value to capture Find which occurrence of the text boundaries should be used Add a web_reg_save_param function to the script, above the step which requests the page with the value to capture Add the parameter name, left boundary, right boundary, and occurrence to the function Parameterize the dynamic value in the script every time it occurs Verify correct execution Manual Correlation


Slide58: There are three types of automatic recording solutions Auto-Detect Correlation Rule-Based Correlation Correlating All Statements Automatic Correlation


Slide59: Automatic Correlation Auto-Detect Correlation Automatically correlate the recorded statements, for application servers whose contexts Rule-Based Correlation Rule-based correlation requires correlation rules to be defined before recording a session Correlating All Statements VuGen correlates all statements


Slide60: Control the behavior of the user when running the script Configure Web Vusers to emulate real users accessing the Web site Run-Time Settings Important Run-Time Settings Browser Emulation Think Time Logs


Slide61: Run-Time Settings Browser Emulation Specify Browser type Simulate cache Emulate network capacity Emulate modem speed


Slide62: Run-Time Settings Think Time Settings Specified in seconds Ignore recorded think time Replay the recorded think time Randomize think time Multiply the recorded think time by specific values Limit the think time to a maximum value


Slide63: Run-Time Settings Log Settings Can disable logging Log a specific set of function calls and messages – Standard Log Log parameter substitutions Log response data from server All the messages and function calls raised


Slide64: Part IV - Controller


Slide65: Objectives of this Chapter What is a scenario? The different types of scenarios How to create and run a manual scenario Scheduling a Scenario Running a Scenario


Slide66: What is a Scenario? It describes the events that occur during a testing session It is the means by which you emulate a real-life user Scenarios include a list of machines on which Vusers run a list of scripts that the Vusers run a specified number of Vusers or Vuser groups that run during the scenario Scenarios are created using the LoadRunner Controller


Slide67: Types of Scenario Manual Scenario Create a scenario by defining Vuser groups to which are assigned a quantity of individual Vusers, Vuser scripts, and load generators to run the scripts. Goal Oriented Scenario For Web tests, the goals of the tests are defined by us and LoadRunner automatically builds a scenario based on these goals.


Slide68: Manual Scenario Creating a Manual Scenario Create Vuser groups Each script selected in the New Scenario dialog box is assigned to a Vuser group Each Vuser group is assigned a number of virtual users All Vusers in a group can be assigned to run the same script on the same load generator machine OR Different scripts and load generators can be assigned to the various Vusers in a group


Slide69: Creating VUser groups Click the Add Group button on the right of the Scenario groups window. The Add Group dialog box opens. Enter a name for the Vuser group Specify the number of Vusers in the group


Slide70: Creating VUser groups Select a load generator from the Load Generator Name list Select Add from the Load Generator Name list if the load required generator is not listed. Select a script from the script list Click OK to close the Add Group dialog box. The new group's properties appear in the Scenario Groups window


Slide71: Configuring VUsers in Vuser Groups Select the Vuser group whose Vusers are to be modified, and click the Vusers button on the right of the Scenario Groups window. The Vusers dialog box opens. Can change the Script Can change the Load Generator Can Add Vusers to the group


Slide72: Configuring Load Generators Click the Generators button to open the Load Generator dialog box. Name, Status, Platform and Details of the Load Generators are displayed Changes the status from Down to ready Add a new Load Generator Delete a listed Load Generator Display the details of a Load Generator Disable a Load Generator


Slide73: Configuring Scripts Select the Vuser group whose script needs to be modified, and click the Details button on the right of the Scenario Groups window Run-Time Settings from the Controller shows the setting of VuGen. If these settings are modified from the Controller, LoadRunner runs the script using the modified settings Edit scripts by clicking on View Scripts


Slide74: Scheduling a Scenario Can instruct LoadRunner to execute a scenario or Vuser group with a delay. The time duration of a scenario or Vuser group can be limited. The scenario or Vuser group stops running after the elapse of this time. Can stipulate how many Vusers LoadRunner runs within a certain time frame during a scenario or Vuser group. …..continued


Slide75: Scheduling a Scenario Select the Scenario Scheduling option In the Ramp Up tab: Selecting Load all Virtual Users (Vusers) simultaneously will run all the Vusers at once. To gradually run the Vusers, select the number of Vusers to begin running concurrently and the amount of time LoadRunner has to wait between Vuser ramp ups. …..continued


Slide76: Scheduling a Scenario To set the duration of the scenario, click the Duration tab There are 3 choices Run until completion Specify the amount of time for which the scenario will run, once all the Vusers have been ramped up Run indefinitely Selecting Initialize all Vusers before Ramp-Up, instructs LoadRunner to initialize Vusers before beginning to load them


Slide77: Scheduling a Vuser group After creating a Vuser group, the group’s script execution settings can be scheduled Vuser group Execution Settings The amount of time after the start of the scenario that the group must wait before it starts running The number of Vusers that will run within a specified period of time The amount of time the group will run …..continued


Slide78: Scheduling a Vuser group Select Schedule By Group to set the schedule for the Vuser group. The options are similar to that of scheduling the scenario


Slide79: Running a Scenario Two types of execution: Can run all the Vusers and Vuser groups in a scenario OR Can run the specific Vuser groups and Vusers When running the entire scenario, LoadRunner does not begin running Vusers until all of them have reached the ready state If individual groups or Vusers are run, LoadRunner runs the Vusers as soon as they reach the ready state Clicking the Start Scenario in the RUN tab of the Controller starts the execution of the Scenario Can Initialize, Stop, Pause and Reset Vuser groups


Slide80: Monitoring a Scenario We can monitor scenario execution using the LoadRunner monitors. Some of the monitors are Run-time Transaction System resource Web resource Web server resource Web application server resource Database server resource Network delay


Slide81: Part V - Analysis


Slide82: Objectives of this Chapter Understanding the results directory structure Introduction to the Analysis tool Creating Analysis graphs Interpreting Analysis graphs


Slide83: Results Directory Structure In results directory, LoadRunner creates a subdirectory using the results name specified by us All the data LoadRunner gathers is placed in that directory Every set of results contains general information about the scenario in a result file (.lrr) and an event (.eve) file. …..continued


Slide84: During scenario execution, LoadRunner also gathers data from each Vuser and stores it in an event file _t_rep.eve and an output file output.txt LoadRunner creates a directory for each group in the scenario and a subdirectory for each Vuser Results Directory Structure


Slide85: The Analysis is the utility that processes the gathered result information and generates graphs and reports The result file with an .lrr extension is the input to the Analysis tool An Analysis session contains at least one set of scenario results (lrr file) The Analysis The Analysis graphs helps to determine system performance and provides information about transactions and Vusers by combining results from several scenarios or merging several graphs into one.


Slide86: The Analysis can be started as an independent application or directly from the Controller Starting the Analysis


Slide87: The Analysis graphs are divided into the following categories: Vusers Errors Transactions Web Resources Web Page Breakdown User-Defined Data Points System Resources Network Monitor Analysis graphs Firewalls Web Server Resources Web Application Server Resources Database Server Resources Streaming Media ERP Server Resources …continued


Slide88: Vuser Vuser graphs display information about Vuser states and other Vuser statistics Error Error graphs provide information about the errors that occurred during the scenario execution Transaction Transaction graphs and reports provide information about transaction performance and response time Analysis graphs …continued


Slide89: Web Resource These graphs provide information about the throughput, hits per second, HTTP responses per second, and downloaded pages per second for Web Vusers. System Resource These graphs show statistics relating to the system resources that were monitored during the scenario using the online monitor. Network Monitor Network Monitor graphs provide information about the network delays. Analysis graphs


Slide90: By default, LoadRunner displays only the Summary Report in the graph tree view Opening Analysis graphs Select Graph > Add Graph or click <New Graph> in the graph tree view This opens a New Graph dialog box Choose the required graph


Slide91: LoadRunner Analysis graphs present important information about the performance of your scenario. Using these graphs, Can identify and pinpoint bottlenecks in applications Determine changes needed to improve its performance Estimate the maximum load the server can handle Find response time of transactions under varied load Interpreting Analysis graphs


Slide92: Case Study #1 Analyzing Performance under Load Interpreting Analysis graphs


Slide93: Case Study #2 Breaking down Trx Response time Interpreting Analysis graphs


Slide94: Case Study #3 Web page breakdown graphs Interpreting Analysis graphs


Slide95: Case Study #4 Identifying Network Problems Interpreting Analysis graphs


Slide96: Case Study #5 Identifying Server Problems Interpreting Analysis graphs


Slide97: Case Study #5 Identifying Server Problems Interpreting Analysis graphs


Slide98: Case Study #6 Comparing Scenario Results Interpreting Analysis graphs


Summary: Summary Many concurrent users running the same application to see whether a system handles the load without compromising functionality or performance is known as Load Testing. Steps involved in Load Testing: Planning Load Tests Creating Vusers Creating Scenarios Executing Scenarios Analysis of the system under Load


Slide100: References Books : Effective Software Test Automation: Developing an Automated Software Testing Tool - Kanglin Li, Mengqi Wu, Sybex Automated Testing Handbook - Linda G. Hayes