Scalability Testing the Cloud

At CA Labs we’ve been working on a tool – dubbed “Reacto” – which provides a new approach to measuring the scalability of a software component in an enterprise-scale environment.  (From now on, we will refer to the component we are testing as the “component-under-test” or CUT.)  The general approach of the Reacto framework is to create lightweight, but realistic models of the interaction between the CUT and the other components or systems in the environment.  These models are then executed in an emulation engine and the CUT communicates with the emulated components as though it were talking to real systems.  The key aspect of the research is to balance the modelling so that it is rich enough to “fool” an unmodified CUT into thinking that it is talking to real systems, but lightweight enough so that tens of thousands of instances of model systems can be executed simultaneously in the emulation engine.

The upcoming release of CA CloudMinder Identity Management (IM)  was a perfect opportunity to use Reacto to validate IM’s scalability at managing many thousands of endpoints.  The component of IM that communicates directly with the managed endpoint systems is the Java Connector Server (JCS).  Reacto was used to emulate 10,000 mock endpoints for the JCS to manage, enabling us to measure the scalability and performance of the JCS itself.  Each emulated endpoint had its own IP address and an independent state.

Using JMeter, we invoked the JCS to conduct a set of operations on each endpoint which are typical in identity management.  We executed this test using up to 100 concurrent threads, and used CA Application Performance Management to monitor the JCS’ memory usage, CPU usage and response time.  The test confirmed that IM and the JCS satisfactorily scale to manage 10,000 endpoints.  We also passed on the resource consumption information onto the software architects.  One unexpected result of the test was the discovery that the endpoint caching mechanism used by the JCS used more memory than expected.  This information was thus used by the software architects to improve the design of the JCS.

Reacto’s approach correlates very nicely with CA LISA Service Virtualisation, a product from our 2011 acquisition of Interactive TKO, Inc. (ITKO) that automatically creates models from the observed message traces between a CUT and the other services in the environment.  This enables customers to accurately test the impact of new versions of a system without affecting the production environment and without needing to access the real systems in the production environment. 

Further details of the Reacto and the IM scalability test will appear in the May edition of the CA Technology Exchange.

The following two tabs change content below.

Steve Versteeg

Steve Versteeg is a Research Staff Member with CA Labs, based in Melbourne, Australia. His role is to coordinate collaboritive research between universities and CA Technologies. His current projects are in the areas of software engineering, role engineering and anti-malware. Steve's PhD research was in the area of neural simulation. A well studied neural circuit was used as a case study for re-creating robust behaviour in computer systems. From 2004 until early 2008, Steve worked at WMind LLC as a senior developer and researcher on an experimental automated futures trading system. The system searches the history to identify instances in the past most similar to the current scenario and then makes predicts the most likely future movements. Steve holds a PhD in Computer Science from the University of Melbourne.

Leave a Reply