StackSafe Research Report Shows E-Commerce Would Benefit from Greater Testing Maturity

Share Article

Study examines downtime, testing trends within multi-tiered applications and operating systems.

A recent study conducted by StackSafe, a developer of IT operations staging and testing solutions, and Research Edge, Inc., an independent, third-party research firm, revealed that "high stress environments" like e-commerce face more pressure in making changes to multi-tier applications than other environments. Over 400 IT operations professionals participated in the study, which focused on software infrastructure testing differences in multi-tiered applications and operating systems.

The study, titled "IT Operations Research Report: Testing Maturity Part II - Applications and Operating Systems" is the fourth in a series of reports developed by StackSafe and Research Edge that focus on topics of interest to IT Operations teams. Past reports have focused on downtime, change management and testing maturity.

Multi-tier Applications
The study found that applications heavily influence the approach companies adopt for change management and testing. This is in part due to differences in application complexity, change lead time and downtime costs. Applications tend to fall into three profiles with regard to change management and testing:

  •     High Stress Environments - companies supporting E-commerce and Customer Relationship Management
  •     Prudent Planners - companies supporting Enterprise Resource Planning (ERP), Production/Supply Chain Management and Transaction Processing
  •     Laissez-faire Planners - companies supporting Website Hosting

The report indicates that companies supporting high stress environments have difficulty dealing with the rapid pace of change they encounter. Almost 30 percent of changes made to e-commerce applications are unscheduled and considered "emergencies", giving IT less time to determine what impact the changes may have. The report also indicated that IT teams have no confidence in over 15 percent of the changes that they make to e-commerce applications.

While companies that support web hosting applications are the most likely to have testing tools (82 percent), their use is inconsistent, with only 20 percent of organizations testing all changes to production. They are nearly 40 percent less likely to use automated change management solutions than the average company in the survey.

"The traditional rationale for not consistently testing changes to e-commerce, web hosting and CRM applications is that the applications operate in fast-paced and volatile environments where testing changes is very challenging, especially for web hosting applications," said Dennis Powell, senior product manager with StackSafe. "The rise in easy-to-use and inexpensive testing solutions for IT operations teams means that ALL software infrastructure changes can be tested, regardless of environment or usage. Improving change management and testing maturity can also provide significant value to organizations by reducing downtime and improving availability."

Operating Systems
The report also examined testing and change management differences across different operating systems environments. The research indicated that organizations supporting UNIX push through 25 to 33 percent more changes than companies supporting Windows or Linux environments. The greater number of changes results in 60 percent greater costs due to downtime for UNIX shops.

Organizations with multi-tier applications supporting multiple operating systems were also shown to have more expensive downtime, double that of companies running multi-tier applications on a single OS, due to an increase in IT staff time to address unplanned downtime emergencies. The report indicated that companies with multiple operating systems devote 60 percent more staff time to addressing outages and service issues.

"Heterogeneous operating environments are becoming increasingly common, as organizations turn to specific systems for certain tasks, like development or hosting," continued Powell. "While not standardizing on a single operating system can give an organization more flexibility when it comes to service offering, it also means that changes are more likely to cause problems across the board. Because of this, pre-deployment change testing needs to become the norm for organizations utilizing multiple operating systems, regardless of purpose or industry."

"IT Operations Research Report: Testing Maturity Part II - Applications and Operating Systems" is the fourth report in a series of studies published by StackSafe in conjunction with Research Edge. Past research reports can be viewed on the StackSafe Research Reports Resource Page.

About StackSafe, Inc.
StackSafe provides software infrastructure testing solutions that enable IT Operations to significantly improve uptime and resiliency. StackSafe's flagship product, StackSafeā„¢ Test Center, directly addresses the challenge of incomplete and ineffective IT Operations staging and testing of changes to multi-tiered software infrastructure stacks. With Test Center, StackSafe provides the first easy-to-use virtualized staging and testing solution that enables IT Operations to understand the business impact of changes--before they touch production systems. Test Center was awarded the 2008 ITIL Innovation of the Year award at the International IT Service Management Conference and Exhibition. Based in Vienna, Virginia, StackSafe serves IT Operations teams across multiple vertical markets. For more information, visit

NOTE TO EDITORS: If you would like additional information on StackSafe and its products, please visit the StackSafe Press Room at

The trademark "StackSafe" and the StackSafe logo are the property of StackSafe, Inc. and may not be reproduced without written consent of StackSafe, Inc. All other trademarks belong to their respective owners.


Share article on social media or email:

View article via:

Pdf Print

Contact Author

Mary Evans
Visit website