Stale, filthy, or polluted test data is used throughout the application’s lifetime. Because testing failures hinder software delivery, the absence of proper test data is cutting into productivity benefits from agile and DevOps initiatives.
Organizations that have implemented DevOps have made significant progress in many areas, but the quality is not increasing at the rate or scale required, according to my testing peers in various industry associations and organizations.
Perhaps you’re automating testing, completing all needed unit tests, focusing on code coverage, and throwing in a little Selenium for good measure. How is it possible that quality is deteriorating? You can’t automate your way out of quality problems, and it’s tough to reproduce flaws when your entire team doesn’t have access to the same data—or, even worse, when the same test is run with filthy, used data.
You’ll need test data as well as a strategy for utilizing it. Unfortunately, the majority of individuals are completely unaware of this. They’re so focused on speed and everything-as-code that they don’t consider quality, which results in quality failures.
Here’s what’s wrong with your test results and how to solve them.
The infinity loop is wrong
You’ve probably heard of the DevOps infinite loop, but you may have missed the initial source of quality failures in the diagram. Testing is limited to a single step in the process. The answer isn’t as simple as shifting left to test earlier or right to test in production.
To improve quality and testing, you could reconsider the language of the infinity loop. It needs to be weaved into the loop’s basic fabric. To improve testing and quality, you must be more detailed during each phase, stage, and transition.
Holistic testing is more than automated testing
Janet Gregory, co-author of Agile Testing: A Practical Guide for Testers and Agile Teams and founder of the Agile Testing Fellowship, remarked, “The word continuous testing has been used and abused.” The term was originally intended to refer to a focus on testing and quality throughout the entire project, but it has since been misappropriated. As a result, I use the phrase holistic testing to refer to testing that is concerned with all aspects of quality.
The following are some of the elements of that focus:
- Identify risks, test assumptions, and construct prototypes as you discover, plan, and comprehend.
- Build and deploy—Instrument code and automate tests for all aspects of the system, including stories, features, infrastructure, pipeline, and quality attributes.
- Release, observe and learn—Try the product in production and see how customers use it to see what needs to be changed.
Phantom defects abound
Lack of test planning, gaps in skills and training, and too fragmented test cases are the most obvious and talked-about reasons for testing cycle failures. We all know that software is still closely coupled, and when companies adopt agile approaches, it’s easy to fall into the trap of thinking that all architectural design “emerges.”
Nothing could be further from the truth; even with agile approaches, conscious attention to software architecture is still required. Consider the difference between constructing a hut and constructing a skyscraper. You might be able to scrounge together a hut without too much forethought, and the hut’s architecture would “emerge.” That, however, does not apply to skyscrapers.
According to a 2020 analysis by Undo Corporation and Cambridge Judge Business School, failed tests cost the corporate software market $61 billion per year, with approximately 620 million developer hours wasted on troubleshooting software failures. This sum is astonishing, yet it occurs because we are unable to duplicate the problems, which causes them to be labeled as “phantom defects.”
The challenge: Expanding the narrative beyond asking how
“How will we test this?” delivery teams are asking in the midst of the frenetic rush to automate testing. However, if you want to stick to a holistic assessment, you should ask a broader set of questions:
- What is the purpose of this test (why is this feature or skill being tested)?
- How will you put this to the test?
- What information do you require?
The architects, engineers, and businesses make the decision about what to test and how to test it. Having a deliberate focus on defining, gathering, and maintaining the data utilized for testing is the key to effective and fruitful testing.
Data for DevOps
Unit, security, performance, integration, end-to-end, usability, and other forms of data are all needed for testing. Many other factors, in addition to the sorts of data, are frequently forgotten when acquiring test data, such as:
- What is the domain name, and are there any special requirements for industries like aviation, finance, insurance, or healthcare?
- Are there any legal restrictions on the test data you intend to utilize, such as personally identifiable information?
- Is it possible to use official classification levels, such as Impact Level 2 from the Department of Defense?
- Are the architecture, user needs, and quality attributes understood by the entire team?
Who makes the test data?
Test data is frequently created as an afterthought, and accountability is either vaguely stated or left entirely undefined. In most cases, test data is created by a team of developers, quality assurance testers, security testers, and database administrators. Each of these individuals views it from a unique perspective.
- Developers concentrate on white-box testing, which is based on an understanding of the code. Their data usually depicts the best-case scenario, with a focus on swiftly building new features.
- QA testers frequently use spreadsheets to manage their data, and they may even inherit developer data. They’ll flock to use veiled production data as soon as they can.
- Security testers nearly entirely construct their data by hand, on an ad hoc basis. The emphasis on DevSecOps is assisting in a shift toward the deliberate creation of security test data.
- By bringing production data into lower environments, DBAs provide food for the entire team. Masking techniques may be used.
Pitfalls with production data
The ability to access and spread production data into test environments must be a show-stopper. True, production data is an exact representation of the data that works well; nevertheless, it only covers 70 to 75 percent of the scenarios. The edge cases that create the headline problems are also missed by production data.
The fact that production data is rarely masked or de-identified is the most compelling reason to question its use.
According to Redgate’s 2019 State of Database DevOps report (PDF), 65 percent of firms use production data for testing, while just 35 percent use masking strategies. As a result, there is a massive cyber-risk footprint. Because masking and de-identification of data are challenging, delivery teams are advised to handle data with caution.
So, where do you get your test data? You should use a combination of methods: you should build, borrow, and buy.
- Investigate the data
- De-identify
- Validate
- Build with reusability in mind.
- Automate the generation of test data.
Even with a deliberate focus on test data, two significant stumbling blocks must be avoided: conserving test data and loading/reloading it. A test isn’t valid unless it starts with a proper, repeatable state.
Technology is not the issue
What is the connection between all of this and DevOps? Technology is not the issue, nor is it the answer, when it comes to addressing test data in an organization, as is true with many other challenges in using DevOps principles.
Providing value entails providing high-quality service. Quality testing necessitates robust test data management, test data management tools, and integration into the value stream and DevSecOps pipeline, which can only be accomplished with robust test data management, test data management tools, and integration into the value stream and DevSecOps pipeline.
For more info: https://www.qaaas.co.uk/testing-services/
Also Read: https://www.guru99.com/software-testing.html