Quality Control – Process Street

All posts in Quality Control


15 Load Testing Tools, Tips and Methods to Protect You from Crashes

Do you find your website users complaining about it being sluggish at times? Does performance degrade when traffic is high? How certain are you that a new feature or build isn’t responsible for your laggy site? These are questions best answered through load testing.

Load testing is a branch of software performance testing. It involves subjecting a website to simulated workloads that stretch its specified operational capacity to its limits in order to assess its performance.

A slow and vulnerable site can affect your search visibility, user experience, and conversion rate — all of which can negatively impact your revenue.

In this article, I’m going to explain the advantages, goals, and metrics of load testing, and then give you 6 tools and 9 tips to try.

WebLOAD’s dashboard

The advantages of load testing

Some benefits that you get from load testing include:

  • Prevent downtime with analysis. It helps you run various scenarios mimicking real-world usage to gain different insights on your users’ behavior and website performance. This is an accurate and quantifiable way to discover new methods to add value to your website and improve your business.
  • Save money long-term. Although a load test can lead you incur costs it helps you save money in the long run by uncovering emerging problems that you can fix for much less. It also helps maximize your website’s efficiency which saves you from over spending on maintenance.

The goals of load testing

At its core load testing is a means of measuring metrics. It therefore means that there are goals that underlie the quantifying of the specified metrics. Some of the things it seeks to find out are:

  • How robust the system is and more so how it recovers from any errors
  • How efficiently the resources the system has at its disposal are being used e.g. if you have a fast processor and low RAM it means that the hardware resources are under-performing. In such a case load testing will reveal where relevant improvement should be done to achieve optimal output. This is also applicable in assessing efficiency of a website’s resources
  • What hampers the system’s ability to fully handle its work load i.e. bottlenecks

It is important to note that you need to differentiate between goals behind load testing and the means by which you arrive at load testing.

Load testing metrics to analyze

Once you have identified your goals for carrying out load testing, you need to know what metrics to measure. A few key things to look at can include:

Throughput: How much bandwidth a website uses up while conducting the test. It indicates the amount of data sent and received from the servers.

Error rate: How frequently errors occur when a website is processing requests and at what stage they do occur.

Response time: How long a website takes to respond during the highest levels of activity (peak time) and as measured over a period of time (average time).

  • Requests per second: How many requests are sent to the server
  • Concurrent users: How many virtual users are active at any given time
  • CPU utilization: How well the CPU is utilized as the website processes a request
  • RAM utilization: How much memory is utilized as the website processes a request
  • Number of failed or passed transactions: The overall amount of successful and unsuccessful transactions
  • Wait time: How long it takes once a request is sent to when the first byte is received in response

Methods of load testing

There are two general approaches one can use when carrying out a load test.

Longevity testing

This measures the capacity of the system to handle a specific work load that simulates the average amount that a live user handles over an extended period of time. A baseline defining what the optimum work efficiency ought to be is used as a reference point. It is also referred to as endurance testing and is used to check for errors in the system that occur after a sustained period of use.

Volume testing

This measures the capacity of the system to handle large volumes of data in a limited period of time. Subjecting the system to an increasing volume of work helps to assess whether it can competently handle the quantity set out by its developers.

6 load testing tools

The Grinder

The Grinder is a free, JAVA-based load testing tool. It creates the load by using load generators, also known as agents, that handle many workers. It is compatible with a BSD-style open source license. It has flexibility in creating parameters. It also has the capacity to handle various protocols. A console is the central interface through which editing is done and tests are developed.

The console is a GUI application by nature that facilitates monitoring of real time results via the several Grinder agents it controls. It has TCP proxy and can handle distributed testing. It provides complete access to the information generated after a test for verification and analytical purposes.

WebLOAD

WebLOAD is a licensed tool with a free version used in testing large-scale loads. It has capacity to handle complex scenarios and the load itself can be generated through the cloud or on-site. It’s used when testing any web and mobile application or API that deploys Adobe Flex, Oracle Forms, HTML5, Ajax, .NET and others.

It enables automatic correlation, DOM-based recording, automatic bottle neck detection and JavaScript language when scripting. There is a free plan with limited features. The paid plans have flexible pricing and a robust licensing mechanism.

LoadView

LoadView is a paid tool whose unique feature is that it tests in real browsers making it more precise as it can more accurately mimic real-world user behavior. It is cloud-based and as a result has the ability to deliver a large scale, distributed load test.It also provides point and click scripting. While it’s not an open source option there is a free trial feature.

Apache JMeter

JMeter is an open source tool that manages multiple load injectors using a single controller. It supports a variety of protocols that include JAVA-based ones. It has a user-friendly GUI which requires less scripting. It can be loaded onto a network or server to assess its performance under various types of loads.

Testing Anywhere

Testing Anywhere is a tool that enables you to create the test criteria that best suits you via its in-built editor. There are five steps involved in creating any test:

  • Object recorder
  • Advanced web recorder
  • SMART test recorder
  • Image recognition
  • The editor

One common use for it is to automatically uncover any bottlenecks in its early stage of development. This helps you resolve it before the users encounter it.

HP LoadRunner

LoadRunner’s unique advantage as a load testing tool is that it enables you to simultaneously create and manage thousands of users. It consists of several tools namely Virtual User Generator, Load Generator and Analysis and Controller.

It creates a scenario that contains the script to be executed while generating the required number of virtual users. It factors in other specified parameters and generates results. A developer is able to get all the information relating to the infrastructure and performance. It is a paid tool.

Factors to consider when selecting a tool

The best tool is the one that most suits your needs to deliver the best results for your specific context. Things to weigh include:

  • The cost of the license if you opt for a paid tool
  • The protocols you will use. You need a tool that supports the relevant protocols involved for it to deliver
  • How often the vendor or developer updates it and whether there is any support offered in case you run into technical difficulties and need assistance
  • How expensive it will be to train any staff on using it, be it monetarily or time wise
  • The software or hardware that is required to run the tool effectively
  • Whether your client has a particular preference for any specific testing tool

9 tips for website load testing

The best load test that will deliver valuable and actionable feedback to help improve your website is one that you prepare for. Load testing goes beyond just simulating a desired scenario. Here are a few things to do to make sure you get value for your money when testing:

Schedule your testing

The value of regular load tests to consistently gauge your system’s capacity is indispensable. Institute a testing calendar that enables you to receive feedback without going for too long. A recommended period would be two weeks between every test. You will be able to adequately study the results in depth, meaningfully iterate and coordinate for the next test. Such a practice will help you uncover bottlenecks that you can then address before your users get affected by them. Your website will be consistently performing above average as a result.

Use real browsers

Aim to use real browsers as much as possible to generate more accurate results as they better mimic real-world users as compared to virtual browsers. If need be you can add virtual browsers to reach your target.

Set accurate benchmarks

A load test is only as good as the information it delivers when compared to a relevant baseline. Look at what your competitors are doing to help you determine what an accurate and realistic benchmark should be. The closer it is to real-world situations the better the insights you will glean.

Create a team

A well-executed load test can deliver high quality insights that significantly impact your website in a meaningful way. Assign specific tasks to a relevant team member and let them know that they are in charge of delivering on it. Coordinate the who-does-what during a test to foster collaboration. The result will be a test that encounters minimal barriers to its smooth operation. Consider a self-service solution if you have a complete team and a full-service tool if you can’t constitute a big enough team.

Critically analyze your data

A well-timed load test is one that is executed when the website is at its busiest. Study your analytics over the last 12 months to pinpoint when traffic is at its highest point. When testing, aim to go around 20 percent higher than your peak traffic volume to ensure that the website can handle its greatest number of users without significant problems.

Record all information

Maintain a record of all the results generated from each test in a system that is easy for all team members to access. It will help narrow down any bottlenecks that crop up to find the root causes without talking too long. Always monitor your overall infrastructure and servers to stay on top of any emergent issues.

Cross reference the results

Ensure that you match up the results of a specific test to its timeline. Cross reference all the resulting timelines and test results to look at the complete picture when analyzing insights. This will give you a well-rounded understanding of any problematic areas than if you individually analyzed the tests for insights in isolation.

Build up with every test

The best way to find out how the website performs at different load levels is to start small and keep increasing with every test. Compare the metrics you are interested in with every increase in the number of users to gain insight.

Free vs. paid tools

Cutting down on operational costs is admirable for a business but not when it affects the quality of services provided. Avoid using the price of a tool as a determinant in whether to adopt a free or paid one.

The goal of a load test is to mimic real-world user conditions as much as possible for best insights. Assess if you can achieve this through a free tool. If not, a paid one may be an option to consider in order to effectively gather the data needed to improve your site’s performance. The more a load test can copy real-world user behavior the better the results for your website.

Get started load testing

Load testing is an important strategy in making your website more user friendly and efficient. The data gathered from testing in different scenarios provides different insights that help point you to improvements that need to be made. There are varied load testing tools to suit various needs and contexts. Planning your load test and understanding what metrics to look for are key in deriving high value results. A test that is well executed carries significant and material impact on your website and business.

This guest post was submitted by Glenn Lee, Marketing Assistant at Dotcom-Monitor

8 Electrical Inspection Checklists to Keep Your Workspaces Safe

Electrical_Inspection_Checklist_headerAccording to the U.S. Bureau of Labor Statistics there were 32,000 dedicated electrical inspectors in the workforce in 2014.

That’s a good number of electrical inspectors and a great deal more electrical inspections!

Yet, much electrical inspection work is still being done with pen and paper, creating mountains of paperwork and making effective sharing of information difficult.

That’s why we’ve created this Process Street electrical inspection checklists process-pack.

We want to make sure the inspectors using our platform can make the most of the software to speed up their processes, record and compile information more easily, automatically generate reports, and ultimately keep making the world a safer space.
Continue Reading

FMEA: The Analysis Method to Prevent the £100m British Airways Catastrophe

fmea failure mode and effects analysis british airways catastropheBritish Airways chief executive described the incident as “catastrophic” as 800 flights were canceled and 75,000 travelers were affected.

Flight compensation website flightright.com estimated that British Airways would have to pay around €61m to passengers for refunds alone under EU legislation. Add to this the cost of reimbursing angry passengers for unexpected hotel stays and other inconveniences, and the total financial damage to British Airways has been estimated at £100m.

Why? Someone turned their data center off and on again.

The entire airline was down for almost 2 days. This wasn’t a natural disaster, it was a process failure.

In this article, we’ll explore how to spot process failures before they occur using a system called Failure Mode and Effects Analysis (FMEA). We’ll assess an overview and then delve in deeper to ground our understanding and include a premade Process Street FMEA template to help you run your own assessments in future.

This will leave us with three questions in regards to the British Airways affair:

  1. What happened?
  2. Why did it happen?
  3. How do we prevent it from happening again?

We’ll address each of these questions and look at how a company could employ FMEA in practice to tackle this kind of problem before it arises.

Continue Reading

16 Essential Steps to Writing Standard Operating Procedures

The_16_Steps_to_Writing_Standard_Operating_ProceduresWhether you’re starting a business or trying to improve an existing one, you have to understand how things are going to get done.

What tasks do you need to do? Who needs to do them? What are the best ways to approach these tasks?

Working out how these questions can be answered is the first step toward systemizing your business. The creation of processes and workflows will define the way in which your day to day activities function.

One popular approach to this is the creation of standard operating procedures. The simple definition provided by IBM for standard operating procedures is:

…a set of instructions that describes all the relevant steps and activities of a process or procedure.

Simple.

But how do we create these SOPs? What we need are standard operating procedures for creating standard operating procedures. And that’s what we’re going to give you.

We’ll look at:

  • How to create a set of standard operating procedures
  • Some advanced techniques for improving your SOPs
  • Why Process Street is a useful tool for both mapping and following your standard operating procedures

Continue Reading

Software Testing Methods: How Google and Facebook Crush Quality Assurance

google-and-facebook-testing

There’s a big difference between successful software companies and those shoddy unverified apps you get off the app store:

Quality assurance.

While small-time apps aren’t heavily used, and the creator won’t receive many complaints or bad press if anything breaks, Google and Facebook are used by billions of people worldwide.

If a bug affects 0.01% of the user base in a small app, it’s not worth the energy. If it affects 0.01% for Google and Facebook, that’s thousands of complaints, and possible media scandal to deal with. And we all know what the price of that can be.

So, when it comes to studying quality assurance there’s no better examples than two of the biggest Internet companies in the world.

I’ve deliberately not chosen to give Microsoft the time of day in this post, because I’d say their QA is pretty damn weak for their size.

…However, read on to pick up tips from Facebook and Google on how to make software that doesn’t break down, cost you more money, and cause frustration for your customers.

Continue Reading

Software Quality Assurance vs. Quality Control: What’s the Difference?

quality-assurance-vs-quality-control

Thinking the terms ‘quality control’ and ‘quality assurance’ can be used interchangeably is a common mistake.

Whoever coined those terms did nothing to clarify their differences, but in fact:

  • Quality control (QC) makes sure that your product isn’t riddled with bugs.
  • Quality assurance (QA) makes sure engineers are following processes to reduce future bugs, and write code more efficiently.

Quality control is something development teams do every day. They squash bugs in the code they wrote, and run tests to catch future errors.

Quality assurance is the overall management of development processes that make sure less testing and QC needs to be done.

At Process Street, we’ve already looked at the wider implications of QC and how to run tests on your software. We’ve even provided you with a bunch of pre-made software development processes you can adjust to your own team.

In this article, I’ll go into an explanation of quality assurance in software teams, and how you can use this information for your own developers.

Continue Reading

4 Types of Software Testing and When You Should Use Them

software-testing-types

The way customers see it, your software release cycle looks like this:

  • Take ages developing the software
  • Beta test
  • Add a few features, fix a few bugs
  • Done!

This isn’t real life. Beta testing is only one type of test your software needs to pass to avoid being a catastrophic failure.

If your software is struggling to get into a usable state, it’s probably because you’re overlooking testing.

Testing is important for two main reasons:

  1. Tests will reveal flaws in your software
  2. Tests will reveal flaws in your software development process

Note: Apart from customer-facing and QA tests, tests are code.

So, keep reading to find the types of software testing you need to make mandatory in your development team, as well as an explanation of how to do it.

Continue Reading

Get Started Free Today

No Credit Card Required