Tine 2.0 net



Automated Load and Function Tests in Amazon Cloud


With the help of Amazon-Cloud technologies and a combination of software solutions load and function test can be widely automated. With the result that the tests can be done in significantly higher rate and are much less expensive.

With load and function tests there used to be two basic problems. On the one hand the needed hardware resources are often not available. On the other hand such tests are usually done manually. That is a highly complex task by nature and thus very costly. The IT service provider Metaways now developed a solution, which allows widely automated load and function tests using the Amazon Cloud technology. The background: The Amazon Web Services (AWS) provide almost unlimited resources, which are fit for service within minutes. The payoff can be carried out by time or consumption, therefore providing a good cost control.

The first step – the preparation of the test plan – however, has to be done manually. Because all applications, databases, websites and web shops are individually composed, all have to be tested individually. For example, every web shop uses different approaches when it comes to logins or order and payment processes. Also the requirements differ from case to case. For instance, it can be that only selected functionalities shall be tested – like the the newsletter landing page of a website, which performance is highly critical.

Combination of Different Software Components

In other cases a comprehensive assessment might have to be divided into several single tests for different components to get reliable results. The management of the test plans with the help of a Version Control System (VCS) assures, that changes can be traced and, if required, a former version can be restored. If more than one person is working on the test plans, it is traceable who did which changes.

After the manual development of the plan the test can run fully automated. This is possible with the combination of several software components. The controlling of the tests is done by a Metaways application based on AWS CLI (Amazon Web Services Command Line Interface). The Open Source tool Apache JMeter is used for testing. A basic Amazon-Machine-Image (AMI) gets the necessary dynamic information for the start of the Amazon-on-demand instances from the VCS. Hence the virtual server in the Amazon Cloud uses the configured software versions of JMeter and the JMeter plugins. The parameter of the test plans can be handed over to JMeter – for example how many users should be simulated and derive the number of servers, that have to be started in the Amazon cloud for the test.

Simulated Load Shows Limits of Capacity

Since JMeter is a shared application the tool can run the tests on several servers at the same time und produce high stress. If, for example, five instances are started, a master routes the other servers (slaves) with commands. This way all five instances can „fire“ on the tested application at once and produce many requests. If a website is tested, JMeter simulates a browser and calls the URLs of the tested site.

The tool then records, if errors appeared and how long it took to answer. But not just the performance of the application is tested. A JMeter plugin also logs the use of hardware resources: How much CPU is needed? How much RAM is used? How much hard drive capacity is used? And how do these values change during the test?

The progress of the test is incremental. First it runs with a initial value below the theoretical maximum and is then increased with every rerun. This way in the course of the test the maximum will be fathomed and the tester can, for instance, mark the number of simultaneous visitors to reach 100 % CPU workload. The results of the tests will also be generated automatically in the Amazon Cloud. Further JMeter plugins will analyze the data, generate CSV files and graphics and present those on a HTML page. The link to that page is sent automatically to the customers' email address, which is given as parameter of the test plan to the system.

Higher Frequency and Lower Costs

The benefits of this solution are obvious: With the automation tests can be done at a much higher rate. Also the costs are lower as there are only the expenditures for the test development and the charges for the Amazon cloud. A further advantage: The solution can be used with Continuous Integration Platforms such as Jenkins and thus be implemented into the development processes of applications.

So when a new release of a software is planned, the developers can simply rerun the test they did with previous versions to check the new release. Exceedingly valuable in the development of the automated load and functions test became the support for the plugins from the JMeter environment. The graphic output of the test results wouldn't have been possible without the commitment of the voluntary supporters.

In the future it is planned to further enhance the solution. One objective is to write the results (checked addresses, response times, errors etc.) into a database instead of a file as it is at the moment. Thereby it would be possible to generate a test history and compare the different test runs. After some time it could be possible to draw conclusions about the correlation of the capacity of the hardware and the performance of the software. Furthermore, it would be possible to predict from the empirical values when new hardware is needed for an application, so that it can be ordered in time.

Writer / editor: Our colleagues Lars Reimann and Marc Heisterkamp as well as Elke Witmer-Goßner

[back to the news list]

What IS Tine 2.0net

Under Tine 2.0net, we provide you with Tine 2.0 groupware and hosting from Metaways under one roof.
» more Information


In the development of our software Tine 2.0 the continuous advancement is very important for us.
» Design Tine 2.0 with us!


  • Packages with 5 users
  • including e-mail account for each user
  • 5 GB Cloud Storage
  • only 3 months minimum term
  • Software Maintenance and Support Manufacturers
» Details


  • groupware
  • CRM
  • time Tracking
  • HR Management
  • webbased
  • Smartphone synchronization