DATPROF blog

Test data: the silent foundation of software quality

Maarten Urbach — Fri, 06 Mar 2026 10:47:53 GMT

You’re ready to deploy. The code is solid, performance benchmarks look great, security scans passed, and every test in your pipeline shows green. Your team has worked nights to get here, stakeholders are excited, and the deployment window is open. Everyone’s confident this release will be smooth.

Then production happens. Real users with real data patterns you never anticipated. Edge cases your synthetic test data never covered. Legacy integrations behaving differently than your sanitized test environment suggested they would. Suddenly, what seemed bulletproof in testing becomes the source of emergency hotfixes and midnight calls.

The test data reality check

Test data is not just a QA concern. It’s a strategic investment in software reliability, business resilience, and long-term success. Yet in most organizations, it’s treated like an afterthought. Something we’ll “figure out later” or delegate to whoever has five minutes to spare. Here’s the uncomfortable truth:

Your software is only as reliable as the data you test it with.

Why your current approach isn’t working

Here’s what I see happening in organizations everywhere: when it comes to test data, teams get stuck choosing between two seemingly obvious solutions. Both feel logical in the moment, both have clear benefits, and both will eventually create bigger problems than they solve. It’s like being offered a choice between a rock and a hard place, neither option is actually good, but the pressure to move forward forces a decision.

Most teams fall into one of two traps:

The production copy trap

Realistic? Yes.
Legal headaches? Also yes.
Storage costs that make your CFO wince? Absolutely.

The manual generation trap

Full control? Sure.
Realistic scenarios? Not quite.
Time investment that could build actual features? You bet.

What if there was a better way? Well there is! A trhid way where Instead of falling into the two common traps, the third approach treats test data as a strategic asset that gets actively managed and optimized for each use case. This article is about the thrid way.

The right data for the right test
Making test data work for you, not against you
The path forward
Your next step

The right data for the right test

Not all tests need the same data. Think of it like cooking, you wouldn’t use the same ingredients for appetizers and dessert.

Unit testing: keep it simple

When you’re testing a single function, say, an email validator, you don’t need complex customer relationships or historical data. A handful of synthetic test cases does the job: valid emails, invalid formats, edge cases. It’s quick, lightweight, and gets you the feedback you need without unnecessary complexity.

Integration testing: consistency matters

Here’s where systems need to talk to each other properly. Your payment system connects to inventory, which connects to shipping. The key is having the same customer data present across all systems. Without that consistency, you’ll spend hours debugging phantom issues that only exist because your test data doesn’t align.

System testing: the realistic middle ground

At this stage, you’re testing how everything works together under realistic conditions. You need data that behaves like production but doesn’t require copying entire databases. Database virtualization gives you the flexibility to test different scenarios quickly, peak loads, edge cases, and how new features interact with existing data.

User acceptance testing: mirror reality

This is the final checkpoint before going live. Business users need to recognize the workflows and data patterns they deal with daily. Anonymized production data or carefully selected subsets ensure that when users test the system, it feels familiar and representative of their real-world challenges.

Making test data work for you, not against you

The best test data strategy has three pillars:

Automation – Teams trigger processes themselves
Predictability – Know exactly when your data will be ready
Integrity – Clean, uncorrupted, reliable every time

Imagine requesting test data like ordering from Amazon. Click, confirm, deliver. No emails to DBAs, no waiting for manual processes, no wondering if this batch will actually work.

The path forward

Advanced techniques like subsetting, virtualization, and AI-generated test data aren’t just nice-to-haves anymore. They’re competitive advantages.

Organizations that master test data management:

Deploy faster with confidence
Catch issues before customers do
Meet compliance requirements without breaking the bank
Free up developer time for building, not waiting

Your next step

Test data is not a side concern, it’s the foundation that everything else builds on. When managed smartly and distributed efficiently, teams work faster, more reliably, and deliver significantly higher software quality.

The question isn’t whether you can afford to invest in proper test data management. It’s whether you can afford not to.

4 questions to find out if test data management is for you - DATPROF

Maarten Urbach — Fri, 06 Mar 2026 10:40:56 GMT

When could test data management be of help in your software test project? Think about – or answer – the following questions and find the answer.

In this article, the order in which the questions are asked is irrelevant, except for the first question. You should always answer this question first.

That question is: Do you use personal data during software testing?

1. Do you use personal data during software testing?

Actually, this is the simplest question. Because if the answer to this question is yes, then you will have to think about a test data process. There are some ifs and buts, but in 90% of all cases you will have to start a process.

If the answer to this question is “no”, then you can still think about a test data process, but there are other reasons. These are discussed later in this article.

If the answer is yes, then there are some other questions to ask. Important questions are:

In how many systems is personal data stored?
Is there a single core system and peripheral systems?
How does our chain work?
What data do we want to protect?
…..

There are many other questions that can be asked and a look at our data masking project plan is also helpful.

2. Are there many and/or large databases in use?

Many and large are relative terms. However, in larger organizations with 1000 or more employees, there are often multiple (and large) databases. It is interesting to know how many databases are used for non-production purposes and how large these non-production databases are. We often see that in larger organizations there are at least 3 non-production domains, in addition to their production domain: a development, testing and acceptance domain. But we often see for core systems of organizations that more than one test and development environment is made available. Unfortunately for organizations, these core systems are quite large.

Example: an organization has one production environment, one acceptance environment, three test environments and 3 development environments. The production environment is 2.5 terabytes in total. That gives a total of 17.5 terabytes of non-production environments, 700% compared to production!

And this is true for only one core system. In larger organizations there are several core systems with the corresponding size.

This naturally results in high storage costs. But not only storage is a cost item; with various database suppliers you also have to pay database licenses for test environments or higher environments.

If any of these apply to your situation, it’s worth doing further research into test data management. Techniques are available that can ensure that less data is needed in these environments.

3. How easy is test data available?

In general, there are three common situations when testing software and the availability of test data, namely:

No test data is available, so this still needs to be arranged
Test data is available, but it is not representative of production
Test data is available and it is representative (whether anonymized or not)

For point 1 and point 2 there is, in our opinion, the same ‘solution’: generating or refreshing the test data. Looking at point 2, we do see that there are environments available with test data. These environments are not always representative. There is test data, but it is bad quality or not usable. Sometimes an environment has been around for ages (months, if not years) with all the pollution and test data that has become unusable. Or the environment is filled with self-made test data (the Donald Ducks of this world) and this does not match your wishes and needs. Ultimately, the goal is not to have test data, but to have test data that is representative of what should work in production. In short: generating a data set or refreshing your test data set is a useful action in this situation.

If test data has to be made available, you can often choose from two options. The first is to generate a test dataset yourself. The second is to refresh the test database. This can be done by requesting a new copy of production. Depending on what you want to achieve, making an (anonymized) copy of a production database is the best option. After all, then you know for sure that the test data matches the production situation and is therefore representative. The disadvantage of this is that the process to get a copy production has to be started. This is a tedious (mainly procedural) process, because often software testers cannot start this process themselves. Most testers have to submit a request to management, who then transfer this request elsewhere in the organization, who then pass it on to an employee who has to arrange this. Various studies also show that many (3-8) people are involved in this process. As a result, a lot of time is lost on procedural components, while these intermediate steps could be eliminated. However?

And that leaves the technical element of the refresh itself. The nice thing about technology is that it can be automated. In short: with a smart test data portal, these processes can be started up by software testers themselves. Then the duration of the refresh remains. This lead time can be achieved to very acceptable levels with smart subset technologies and a smart architecture, certainly in comparison with the current working method.

An alternative solution is to use (synthetic) test data generation. The advantage of this is that you don’t have to change a major process. The disadvantage is that the representativeness of the test data will always remain questionable.

In summary: if making test data available is tedious or complicated, it is certainly worth assessing the process and investigating where improvements can be made. Sometimes it’s low-hanging fruit, sometimes it’s more impactful. But there is certainly a business case to be made if availability is a challenge.

4. How many software teams work in the organization?

Another interesting question, the answer of which gives an indication of whether test data management provides improvements in the test process. Actually, the question should be more specific: how many software teams are working on one test database at the same time? Does each team have access to its own test environment? Or are there more teams than test environments or databases? The vast majority of organizations have fewer test environments available than the amount of teams they have -resulting in conflicts.

If several teams in different projects need the same application with the associated test data, this leads to problems in all cases. And as a result, the application is insufficiently tested, which ultimately leads to production disruptions.

In an ideal world, each software tester has his or her own system under test and his/her own test database. Why do we see that happening so little? This is often due to technical barriers and the cost-increasing effects. We have already mentioned the most important technical obstacles in the previous section. This includes: the size of the database and the time it takes to make a database available. The cost-increasing effects will not be extremely difficult to explain; if every tester gets his/her own 100% copy production, the costs will be significantly higher.

But then you also have to ask the question, do you need a 100% copy production to be able to test? Ultimately, testing has to do with risk-mitigating measures. You want to be sure that when the software goes into production, the software does what it’s supposed to do. An OTAP architecture is often also set up for this. In every step of the process, we catch possible errors. In that sense, software testing is sometimes not very different from saying something about an entire population by means of a sample. With a sample we can say with a certain degree of certainty (risk – also mitigating, because we can make this risk bigger and smaller) that this is representative of reality.

Imagine you work for a government institution and in a system there are for example 17,000,000 Dutch people. With a margin of error of 1% and a confidence level of 99% you need a test database that is filled with 16,752 records, which is only 0.098%! It can be even less if you increase the margins and the level. For example: if you take a margin of error of 2% and a confidence level of 95%, you are talking about only 2,401.

Check out https://www.checkmarket.com/sample-size-calculator/

In short: if you are able to extract test data from a copy production in a smart way, then there is a huge advantage to be gained. By ‘smart’ we mean that you can’t just select a few rows from your tables and then put them in your test environment. All inter-relationships between tables must of course also be maintained in order to be able to perform your tests properly. In addition, you may also want to have certain edge cases and possible contamination in your test dataset. You have to find them first. Insight into your data (model) is therefore certainly important here.

Conclusion

Test data management is often considered when there is a need for anonymized test data. However, test data management involves much more than just data anonymization. It also helps manage and distribute test data. So not only if you work with personal data, but also if there are fewer test environments or databases than test teams, it is worthwhile to further explore the possibilities of test data management.

The benefits of using test data management tools - DATPROF

Maarten Urbach — Fri, 06 Mar 2026 10:35:46 GMT

Good software testing stands or falls on having the right test data. This is one of those obvious facts about software testing that nonetheless tends to be a big challenge for many testing teams. Test data management tools offer a solution.

They enable the provisioning (and if needed the anonymization) of data used in testing. In this article, real DATPROF users on PeerSpot offer insights into the benefits of using a test data management tool.

Test data management use cases

DATPROF users put their test data management tool to work in a variety of use cases. For example, Arjan S., a Test Manager at CGI Inc, a consultancy, is responsible for protecting his company’s data. He described his use of DATPROF by saying, “Higher management demanded testing to be done with masked data.” Data masking is often required for privacy and compliance purposes. Arjan S. and his team had been testing with what was available in-house but, as he put, “with homemade tooling it wasn’t possible.”

Another real challenge came from “subsetting test data from/to databases with an incomplete relational model.” Subsetting is the process of extracting a smaller sized set of data from a production database, with references intact, to a non-production environment, such as a testing system. With DATPROF they were able to integrate into what existed to solve the issues.

Finding more disk space by means of subsets was a primary requirement for Erik W., a Business Information Analyst at an insurance company. His team uses a standard software solution with some custom adjustments on an IBM DB2 LUW database. DATPROF’s software offered an alternative to a test database where they had previously used a production copy that required 23 TB of disk space. The disk space has now been reduced to only 130 GB. The successful outcome is significant, with Erik W. commenting, “We therefore only use 0.5% of the original amount of disk space.”

Anonymization, also essential for compliance, security and privacy, is top of mind for a staffer at a government department. They’ve adopted DATPROF Privacy and DATPROF Runtime to mask sensitive test and development data on Oracle, Microsoft SQL Server, MySQL and Postgres databases. He remarked, “As compliance is essential, the automated technology from DATPROF is the solution for GDPR (General Data Protection Regulation) and for protecting sensitive business information.”

Frequent test data refreshes

The ability to refresh test data frequently is seen as a major benefit of test data management tools. Different data sets test software in different ways, so it’s advantageous to have multiple data sets to put into testing processes. According to Prakash P., an SAP Platform Architect at a manufacturing company, DATPROF, “enables us to refresh test data frequently and developers don’t see personally identifiable data, only scrambled data.”

For non-SAP applications, Prakash P. and his team, “did not have the ability to scramble or mask data.” He said, “Doing so was only a dream. Production users, even developers, were usually able to see personally identifiable data in test environments because they have super-user access. But now we don’t have that situation anymore. Anybody who logs in to test data only sees scrambled data. They don’t see the real data.”

Previously, they had to be mindful not to copy production data into the test system because there was no way to protect the information. “Today,” Prakash P. said, “because we have a scrambling solution, we are able to do refreshes frequently.”

Getting better at audits

“Before we had the solution, when an audit was done for non-SAP applications, there were always red flags,” said Manoranjan M., a Product Owner at Heineken, the beverage maker. His team did not have the critical tools to protect company data. It was difficult to get buy-in from management and all other departments involved in making decisions until the DATPROF system was proposed. He added, “Once we found DATPROF, the red flag issue was eliminated. Audit-wise, we are now compliant.”

Simplifying test processes

A test data management tool can help simplify and streamline test processes. For instance, being about to utilize test data across applications for more than one customer is very helpful, according to Prakash P. He shared, “I have not seen any other solution that makes it easier for scrambling between SAP and non-SAP applications. This is definitely important. Our testers would go crazy if the data in our SAP application and our non-SAP applications looked different.”

He elaborated, saying, “We were using it for Microsoft Dynamics Navision and at a later point in time we had to use it for JD Edwards, but we didn’t have to go to any other solution. We were able to quickly port a JD Edwards application into the DATPROF solution and we were able to scramble the data in that application. It’s a single tool that helps us to do that across applications.”

For Arjan S., DATPROF simplifies testing through subsetting, templates that are easy to maintain. Erik W. also acknowledged the value of subsetting and anonymization in making his testing process go better. He added, “It is able to generate a model of the database even when our supplier did not want to release the data model. The tooling provides additional insight into how our package works. I also think the possibility to generate synthetic test data through functions is a nice addition since the start of our collaboration.”

Reusing templates

Reusable templates have helped Prakash P. conduct testing for Microsoft Dynamics Navision in more than 19 operating companies within his business. For him, DATPROF eliminates wasting valuable time customizing for different testing situations. He shared, “But if we have one template that can be reused, it becomes fairly simple for us to quickly adapt it. Once you configure it, it becomes a routine. You just press the button.”

Improving test analytics

For some DATPROF users, what matters is better test analytics. As the tech company Test Consultant put it, “The data model intelligence provided by DATPROF gave us a head start in analyzing the needed test data. With basic database knowledge, we were able to create database templates that will filter out the required test data.” He added that the DATPROF data modeling software, “gave additional insights for debugging in the case of circular foreign keys.”

Conclusion

Software testing is not a simple or stress-free workload. With test data management, however, the process does become a bit easier. Issue with privacy and compliance are more manageable. Audits get easier. The process becomes streamlined with reusable templates and disk space is saved thanks to subsetting capabilities. As these factors come together with DATPROF, users become more efficient and achieve better testing outcomes.

A guide to identifying and reducing the costs of your tdm process

Maarten Urbach — Fri, 06 Mar 2026 10:25:51 GMT

Investing in better test data management software, ensuring compliance with regulations, and embracing innovation without delay can help you significantly reduce costs. This guide will take you from uncovering hidden expenses to discovering new opportunities!

In my previous article on the ROI of well-structured test data processes, I highlighted three common hidden costs that I frequently see among customers:

Time-consuming processes

High storage and infrastructure costs

Compliance risks

Now, I want to go even deeper. This two-part guide will help you identify these hidden costs and take actionable steps to minimize them. Below are the five key cost factors we’ll cover throughout this blog series:

In this part:

1. High costs of test data management software

2. Compliance related costs

3. Delaying valuable improvements and the costs involved

In Part 2:

4. The cost of inefficiencies and lost time

5. Infrastructure and storage costs—an often-overlooked expense

In this two part guide, we’ll explore each of these cost drivers and provide practical solutions to help you reduce expenses and optimize your test data management process. Stay tuned for part two, where we’ll dive deeper into time efficiency and infrastructure savings!

After reading this guide you will know how to turn hidden costs into opportunities.

1. The high costs of test data management software

One of the most obvious costs in test data management is the software itself. For companies that have been working with test data for a long time, these expenses are often seen as unavoidable. In reality, however, many organizations overpay—sometimes millions of dollars—without realizing that more cost-effective solutions exist.

Test data management doesn’t have to be as expensive as it seems. If you’re evaluating your current vendor, considering new options or looking to start, here are some key factors to assess in your evaluation or buying process:

Pricing structure:
Are you paying based on database size, or is it a transparent licensing model?
Support costs:
Are customer service fees reasonable?

Technology compatibility:
Do you pay per technology, or is it included in a reasonable package?

Ease of adoption:
How quickly can your team start using the software, and how dependent will you be on the vendor?

Vendor expertise:
Is test data management their core focus, or just an add-on product?

Bug fixes and updates:
How quickly do you receive updates for software issues?

Feature development:
How often does the vendor release new versions?

Capabilities:
Are you only looking for data masking or generation, or do you also need to accelerate processes and gain better control over test data?

Taking the time to compare vendors and pricing models can lead to significant savings while ensuring you get the features and support you truly need.

2. The costs of (not) complying with laws and regulations

Regulatory compliance is an essential part of test data management. Laws such as GDPR and HIPAA require organizations to handle personal data securely, ensuring customer trust and reducing the risk of data breaches.

Complying with these regulations comes with costs, but the financial risks of non-compliance are often far greater. Organizations that fail to protect sensitive data face potential fines, legal fees, and operational disruptions. While it’s challenging to factor potential fines into a business case, one way to estimate the risk is by calculating:

probability × expected fine amount = potential cost impact

Beyond fines, reputational damage can be even more costly. Losing customer trust can lead to contract cancellations, reduced revenue, and even bankruptcy. Here are a few examples of companies that suffered severe consequences due to data breaches:

Code Spaces (2014): A cloud hosting provider lost almost all customer data due to a cyberattack. Without proper backups, the company collapsed (1).

AMCA (2019): A healthcare debt collection agency suffered a breach impacting 25 million patients. Customer confidence plummeted, leading to bankruptcy (2).

TalkTalk (2015): A major UK telecom provider lost 100,000 customers after a data breach, received a £400,000 fine, and saw a 30% stock decline (3).

Epsilon (2011): A marketing firm exposed customer data from major corporations, resulting in reputational damage and millions in lost revenue (4).

Target (2013): A hack affecting 110 million customers cost Target $292 million and led to the resignation of its CEO (5).

The impact varies by industry, but one thing is the same across all industries: a damaged reputation can have long-term financial consequences that exceed the cost of regulatory fines.

Bottom line: Compliance is not an area where you want to cut costs. The true cost of non-compliance extends far beyond legal penalties—it affects your company’s long-term sustainability and customer trust.

3. The costs of delaying valuable improvements

A less obvious but equally important hidden cost is the price of delaying improvements. We continuously make investments to be better prepared for the future. This is no different for test data management.

You might wonder:’preparing for what?’ Well there are three important developments to keep in mind:

The volume of data in enterprise systems is growing exponentially.

The ways in which organizations want to utilize data are becoming more complex.

As such IT infrastructure complexity is increasing.

The secret to getting ahead is getting started.

Proactively investing in test data management to keep modernizing your test data management strategy not only prevents unexpected costs but also increases flexibility—allowing you to respond quickly to market shifts, improve software quality, and maintain a competitive edge.

Coming up next

There’s still more to uncover! In Part 2 of this guide, we’ll dive deeper into:

- The costs of inefficient test data processes
- Infrastructure and storage expenses
- Strategies to future-proof your test data management approach

By identifying these hidden costs, you’ll be better equipped to reduce costs, improve efficiency, and optimize your test data management process for long-term success.

Sources

1. Goldman, J. (2021, 28 januari). Code Spaces Destroyed by Cyber Attack. eSecurity Planet. https://www.esecurityplanet.com/networks/code-spaces-destroyed-by-cyber-attack/

2. Davis, J. (2021, 12 maart). 41 States Settle with AMCA Over 2019 Data Breach Affecting 21M Patients. Healthtech Security. https://www.techtarget.com/healthtechsecurity/news/366595366/41-States-Settle-with-AMCA-Over-2019-Data-Breach-Affecting-21M-Patients

3. BBC News. (2019, 21 mei). TalkTalk data breach customer details found online. https://www.bbc.com/news/business-48351900

4. Throwback Hack: The Epsilon Email Breach of 2011 | Proofpoint US. (2021b, juli 26). Proofpoint. https://www.proofpoint.com/us/blog/insider-threat-management/throwback-hack-epsilon-email-breach-2011

5. Jones, C. (2022, 3 mei). Warnings (& Lessons) of the 2013 Target Data Breach. Red River | Technology Decisions Aren’t Black And White. Think Red. https://redriver.com/security/target-data-breach

5 reasons to start subsetting - DATPROF

Maarten Urbach — Fri, 06 Mar 2026 09:53:14 GMT

People have various reasons for subsetting their test database. Some relate to speed, others to specific test data criteria and performance. Before we explore the most common reasons, let’s establish a shared understanding of what we mean by ‘test data subsets’: A test data subset is a smaller-sized, extracted, referential integer dataset from a live production database to a non-production environment.

Now, let’s proceed with the top 5 reasons for data subsetting.

1. Non-production environments grow 3 times faster than production

Many organizations have decided that ‘lower’ environments such as development, testing, and acceptance should no longer expand. In numerous situations, it’s determined that non-production environments will have limited storage space. As a result, it becomes essential to utilize your data storage and infrastructure more efficiently.

The need for storage is on the rise, particularly with trends like the ‘internet of things’ and big data. Currently, when production data grows by 1 terabyte, non-production databases expand by 3 terabytes because we replicate the database across acceptance, testing, and development environments. To effectively manage and reduce data in non-production environments, you can initiate a data subsetting project.

2. Generated test data doesn’t always yield valid test cases

Some test teams opt to generate or manually create their own test cases or test data. Synthetic test data has its advantages and disadvantages when compared to ‘real’ data. The advantage is that it is very useful for developing new functions or adding new products to an application. The disadvantages include the challenge of generating test data with the same variations as a production database, including all its historical changes, such as telephone numbers and bank account numbers. Moreover, manually creating test data for a data model with over 500 tables is nearly impossible. It’s more valuable to have highly educated developers and testers focus on other tasks.

The most compelling reason for organizations to choose subsetted data over synthetic test data is the need for trustworthy test data. Production-like data, or a selected subset of it, is far more reliable than ‘fake data’.

3. It’s incredibly challenging to generate data for a data model with over 1,000 tables

Having a large data model, with, for example, over 1,000 tables, is a compelling reason to consider using subsetting technology. Why? Because creating useful test data for such a complex data model is, to say the least, arduous.

Generating synthetic test data is feasible when you have fewer than 200 tables. While we wouldn’t recommend it as a stand-alone solution, it’s possible with this level of complexity. However, when you have more than 500 tables, data generation becomes increasingly difficult. As the number of tables grows, generating high-quality test data becomes nearly impossible. Theoretically, it may be achievable, but the results are not trustworthy. For organizations with large data models, subsetting technology can be a game-changer, providing valuable and credible test data.

4. Test automation demands proper test data

Recently, more clients have been requesting test data subsets for test automation. Many teams are already using or considering automated testing, which represents a step forward toward a more mature testing organization.

In many cases, teams choose an automation tool, implement it, and start using it, only to realize later that they lack the knowledge of how to provide suitable data for test automation. They need data, so they often resort to generated test data (despite its disadvantages) or use a full-sized copy of the production data, which is often highly inefficient due to its size. The optimal approach in this situation is to utilize a subset for automation: less test data results in more efficient and quicker testing outcomes.

5. Reducing idle time in batch processes

Another compelling reason why many organizations test batch processes is to address the issue of lengthy processing times. In many cases, batch processes can consume up to 24 hours or even longer. One of the contributing factors to this extended duration is the use of a full-sized copy of production data for testing. The introduction of test data subsets into this process can have an immediate impact on its efficiency. Creating a subset of production data can significantly improve batch processing times.

You might recognize some of these reasons, or perhaps you have other motivations. Regardless, we hope this blog has been helpful to you and your organization! If you have any questions, please don’t hesitate to reach out to us.

Book a demo

Test data in waterfall and agile teams

Maarten Urbach — Thu, 05 Mar 2026 10:28:07 GMT

We love to talk about Agile, DevOps, and short-cycle development. And with good reason: these strategies help organizations build better software, faster. But something often goes unspoken, and that is that your development strategy has a direct and sometimes painful impact on your infrastructure. And especially on your test data.

Because without the right data, in the right place, at the right time, no sprint or pipeline will move as fast as it should.

In this article

From waterfall to agile

What happens when test data lags behind

Three examples of agile ambitions and test data reality

Why waterfall seems simpler

Test data: from afterthought to building block

From waterfall to agile

The classic waterfall model was built for predictability. With neatly sequenced phases: design, build, test, and test data followed suit. A production clone was made, handed over to testing, and that was that.

But Agile changed the game. And DevOps took it even further. Now, development and testing happen simultaneously, often across multiple teams working on interconnected systems. The goal? Deliver faster through shorter feedback loops.

But that’s only possible if your environment supports it. And that brings us to the overlooked foundation of modern development: test data.

What happens when test data falls behind

Agile teams don’t wait for a “test phase.” They build and test continuously. But if test data delivery is still organized the old-fashioned way, via central database administrators. With full production clones, or manual refreshes, bottlenecks are inevitable.

Instead of accelerating delivery, your sprint velocity gets throttled. And ironically, test data, rarely planned for, is often what causes sprint failure.

Three examples of agile ambitions and test data reality

At DATPROF, I work with teams everyday that experience friction between agile ambitions and test data reality. Here are three examples that show how better test data management can lead to agility.

1. Sprint delays due to data wait times

An organization had fully transitioned to Agile, except for test data. Testers still relied on database administrators to refresh environments. Lead times stretched to days. The solution? A self-service test data portal that allowed testers to refresh and provision data independently. The result: sprint durations decreased by 20%, and teams delivered faster.

2. Unnecessary delays from full production clones

A large insurance provider still used full production clones for each CI/CD stage. Spinning up new environments took days and drained cloud storage budgets. Switching to data virtualization and subsetting reduced the time it takes to set up an environment to under an hour and shrank data volumes by 90%.

3. Audit findings due to insecure test data

A DevOps team at a bank was running end-to-end tests using sensitive customer data, without anonymization. A compliance audit uncovered the issue, halting the work.The fix? Data masking and synthetic data generation, enabling secure, GDPR-compliant testing without compromising realism.

Why waterfall seems simpler… at first

Sure, waterfall gives you time to plan. You know when testing starts, so you prepare a test copy in advance. But even there, cracks are showing. SaaS applications and cloud platforms don’t always allow easy cloning. And when they do, storage costs scale rapidly.

Agile methods don’t give you the luxury of planning ahead. Instead, they force you to rethink how test data is handled. But that’s not a downside. It’s an opportunity to build something more resilient.

Test data: from afterthought to building block

Agile transformations often focus on people and processes. But the technical enablers, like test data provisioning, are often overlooked. And yet, this is exactly where agility either takes root or fails. For test data too take root in an agile team test data must be:

Readily available

Realistic

Compliant

Automatable

Scalable across teams

This requires investment in smart test data management: data subsetting, virtualization, masking, and self-service distribution. Because the shorter your sprint, the higher the pressure on your data.

The organizations that succeed are those that no longer treat test data as a dependency, but as an enabler.

Is AI the best method to generate test data?

Bert Nienhuis — Thu, 05 Mar 2026 10:08:57 GMT

At DATPROF, our development team keeps a close eye on new and emerging technologies—always looking for ways to make test data provisioning faster, more compact, and more secure.

One of the most talked-about technologies right now is AI. It’s a powerful, all-encompassing technology that promises to reshape industries at an incredible pace.

The question on everyone’s mind is:’ how exactly will it impact my field?’

Within the test data community, one of the questions that is surfacing is: is AI the best way to generate test data?

In this article, I’ll attempt to contribute to the answer in the following paragraphs:

The most ambitious applications of AI in the test data industry

Generating test data – the best practices of AI as of this moment

The verdict – there’s still a long way to go

The most ambitious uses of AI in test data generation

There are several ways AI is being applied in the world of test data. In my opinion, the two most ambitious approaches are:

Training an AI model on production data and using that model to generate test data.
Using generative AI models—such as large language models—to directly generate synthetic test data.

Current best practices in AI for test data

In this article, I’ll focus on the first use case: training a model on production data and using it to generate test data. I’ve based this exploration on publicly available documentation from two prominent vendors offering this solution.

Today, several companies provide tools that claim to train and generate “tabular data.” But are these solutions truly ready for enterprise-level use? Or even practical at all?

Based on public benchmarks, we can start to understand how feasible and scalable these solutions really are. I’ve chosen not to name the companies directly so we can focus on the content of the examples:

Case 1:
One provider allows you to train a model on production data.

In one test, training two tables—one with 5,000 rows and another linked tablewith 1,037,854 rows—took 15 hours using 64 CPUs and 256 GB of RAM.

When scaled down to 12 CPUs and 128 GB of RAM, the training time ballooned to 90 hours.

Case 2:
Another vendor provides benchmarks for various AI models across datasets of different sizes. Under the “Large Datasets” category, they report:

A 743MB file with 4.9 million rows and 42 columns took 6 hours to train.

A 154MB file with 1.4 million rows and 15 columns required 3 hours.

A 311MB file with 27,000 rows and 1,300 columns took 26 hours.

These figures reflect only the training time. Data generation time would be additional—though likely faster, it still adds overhead.

The verdict: there’s still a long way to go

So, will AI revolutionize test data management? Based on what I’ve seen so far, we’re not there yet. At this point, I wouldn’t say AI is the best way to generate test data.

For now, AI’s role in test data management is more supportive than foundational.

For example, with DATPROF Privacy, synthetic test data can be generated directly in the database based on specific requirements and rules. In a recent benchmark on average hardware, we generated 100 million rows for an Oracle table with five columns in just 17 minutes…

Keep in mind, most enterprise environments involve multiple production systems, large databases, and thousands of tables. While AI-generated data might offer value for smaller or niche datasets, it’s not yet scalable enough to replace established methods like data masking, subsetting, or rule-based generation.

For now, AI’s role in test data management is more supportive than foundational. It can be useful for tackling specific challenges—like analyzing small, complex datasets or accelerating parts of test data workflows—but in my opinion I would not advise to replace the core techniques that enterprises rely on.

Interesting sources

Abhaya. (2024, 14 november). AI-Driven Test Automation: A Comprehensive Guide to Strategically Scaling for Large Applications. Medium. https://medium.com/%40abhaykhs/ai-driven-test-automation-a-comprehensive-guide-to-strategically-scaling-for-large-applications-50e727125f8b

Can AI-generated test data be high-quality, safe and compliant?

Bert Nienhuis — Thu, 05 Mar 2026 09:54:57 GMT

In my most recent post I evaluated AI-generated synthetic test data. This is a follow up post where I want to dive deeper into the appeal and its limitations. For those who are new to the topic, synthetically generated test data created by AI models is a ‘recent’ development in the software testing industry. Here’s a brief explanation.

AI-generated syntehtic test data is generated by AI models that aim to provide an alternative to using production data in testing environments. Generally, there are two ways AI models generate test data:

Training an AI model on production data
Using generative AI to create data that mimics production data

While synthetic test data may sound like the ideal solution—realistic, quick, and cost-effective—it’s far from perfect. The reality is that generating high-quality test data with AI is often complex, time-consuming, and expensive. Moreover, the process involves significant challenges related to transparency, reliability, and compliance with privacy regulations like GDPR and CCPA.

Let’s look why AI-generated test data, at the moment, often falls short of being a truly viable solution.

Breakdown of article:

The appeal of AI models – and their limitations
Why AI falls short in compliance
Are these methods solving the problem – or creating new ones?
In summary
Sources

The appeal of AI models – and their limitations

AI models can indeed produce test data that resembles production data, especially when trained on actual production datasets. But there’s a catch. The best results come from feeding the AI model with real, production data—a direct conflict with privacy laws and data protection standards.

Why this is problematic:

Transparency and reliability issues: these complex AI systems are often “black boxes,” meaning we don’t fully understand how they produce the data. Without this clarity, it’s hard to guarantee quality and consistency.
Legal noncompliance: training AI models on production data violates core principles of privacy legislation. Regulations like GDPR and CCPA demand explicit consent and restrict the use of personal data for secondary purposes.

This fundamental conflict is where the promise of AI-generated test data begins to unravel.

Why AI often falls short in compliance

Let’s address the big question: Can AI-generated test data be compliant?

There are two answers to this question. The first answer is:’ it could’. If a generative AI model produces data that closely resembles production data without ever using actual production data or data containing personal information, it would appear to be compliant under the current legal framework.

But if it is an AI model that is trained on production data containing personal information, the answer is a resounding no. Privacy laws like GDPR and CCPA mandate consent, transparency, and strict limitations on the use of personal data (1,2). Using anonymized production data might seem like a workaround, but isn’t it an added complexity?:

If the data is already anonymized, why not use it directly for testing instead of adding another layer of complexity by generating synthetic data?

So what do we end up with? A method that is most often non-compliant but also less efficient and more labor-intensive than necessary. Even when AI-generated test data avoids production data altogether, such as through user-defined rules or generative AI, significant challenges remain:

Lack of scalability: Generative AI struggles to produce consistent, high-quality test data across complex systems.
Opaque processes: Without understanding how the data is created, it’s impossible to fully trust its accuracy or reliability.

Are these methods solving the problem – or creating new ones?

The goal of synthetic test data is admirable: creating compliant, high-quality data without relying on production data. But the methods we have today fall short:

Training AI models on production data directly conflicts with privacy regulations.

Generative AI lacks the scalability and transparency needed for reliable test data generation.

This raises a critical question:’aren’t these methods worse than the problem we are trying to solve?’

In summary

AI-generated test data may be a step in the right direction, but it’s not the fast, easy, or compliant solution it’s often portrayed to be. Whether through training on production data or using generative AI, the current methods fail to deliver on scalability, compliance, and simplicity.

At DATPROF, we believe in exploring innovative solutions while staying firmly rooted in compliance and practicality. Want to dive deeper into the complexities of generative AI for test data? Check out our detailed article on the topic.

Sources

Regulation – 2016/679 – EN – gdpr – EUR-Lex. (z.d.-b). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng?utm_source=chatgpt.com
Synthetic data. (2025c, januari 28). European Data Protection Supervisor. https://www.edps.europa.eu/press-publications/publications/techsonar/synthetic-data_en

Part 2: Reducing the costs of your TDM process

Maarten Urbach — Tue, 10 Feb 2026 09:33:14 GMT

In the first part of this guide, we looked at three often-overlooked cost drivers in test data management: expensive software licenses, compliance-related risks, and the cost of delayed innovation.

In this second part, we’ll dive into two more hidden cost areas:

Time lost due to inefficient processes
Infrastructure and storage costs

We’ll wrap up with practical ideas you can implement right away to make your TDM more efficient and cost-effective.

Lost time: often the biggest hidden cost

Lost time due to poor test data management might seem like a soft issue, but its impact is anything but. Test teams that wait too long for usable test data, work with incomplete datasets, or manually create data, because of this valuable time and money is lost.

The rule is simple: the later an issue is found in the development process, the more time consuming and thus expensive the issue is to fix. Yet in practice, many test teams still test with unreliable or outdated test data and this can have a big impact.

The impact of poor-quality test data

Organizations that manually create test data or rely on full production copies run into several challenges:

Increasingly complex IT environments

Stricter privacy and compliance requirements

Country-specific tax or regulatory rules that are hard to simulate manually

In highly regulated industries like finance or insurance—where quality, control, and compliance are critical—poor test data causes bugs to surface late in the development cycle. This leads to costly delays, unnecessary rework, and higher overall risk.

What does lost time really cost?

You wont find lost time on the balance sheet any time soon but it shows up in different ways:

Extended sprints: More bug fixes, delayed releases

Additional costs: Overtime, firefighting, external support

Missed opportunities: Less focus on innovation or value creation

Bottom line: if your test data process isn’t under control, you’re wasting time—and that hits both your budget and your time to market.

Infrastructure and storage: a silent killer

Another hidden cost: infrastructure and storage. Many organizations still run their test environments based on a classic “copy production to test” model. But in a world of Agile, DevOps, and CI/CD, that model no longer fits, in the following paragraphs i will explain why.

Legacy infrastructure vs. modern methods

While modern delivery practices have evolved, the underlying infrastructure often hasn’t. The result? Full copies of production environments are still being used in development, test, and acceptance stages. This leads to:

Massive storage use: Dozens of terabytes per environment
Higher license fees: More data = more cost
Slow, error-prone processes: Hard to scale or adapt

Why full production copies are usually unnecessary

Most research shows that only 10–20% of production data is relevant for testing. But many teams copy everything by default, simply because it’s easy. By narrowing your data scope to what’s actually needed, you can:

Cut storage usage by 80–90%

Reduce software license costs

Speed up testing and provisioning

Moving toward flexible test data management

The future lies in small, targeted datasets—subsets designed to match each test purpose and development phase. This enables teams to work faster without compromising on data quality.

Why synthetic data (still) isn’t the answer

Synthetic test data sounds promising, but the reality is: most solutions aren’t mature enough to generate complex, business-relevant datasets. Especially in domains with intricate data dependencies, the result is often unrealistic test coverage. Until synthetic data generation matures, anonymized subsets of production data remain the most effective solution.

A real-world example

A client with 40 TB of production data used to copy everything into lower environments. By switching to smart subsetting—using just 5% of the original data—they reduced storage, licensing, and infrastructure costs significantly, without sacrificing coverage or quality.

Conclusion: your hidden TDM opportunities

Already using a TDM tool? Great—but chances are, there’s still a lot of untapped potential. Many organizations stop at data masking for compliance, while other major cost savings remain untouched.

Where can you start improving—today?

Automate your provisioning: Speed up delivery and reduce wait times

Use targeted subsets: Smaller datasets = faster, cheaper, better

Think team-first: Deliver exactly the data each team needs, when they need it

Hidden costs are real—but they’re also fixable. Whether you’re just getting started or already have advanced tooling, there’s always room for improvement.

Possibilities and limitations of generative AI for test data generation

Bert Nienhuis — Mon, 28 Apr 2025 06:00:00 GMT

In this previous blog post, I discussed the concept of generating test data by training AI on production data. While it’s an intriguing technology, it’s important to recognize that generating entire databases at scale using AI is often a step too far, especially given the existence of far more efficient techniques.

In recent years, Generative AI (GenAI) has rapidly evolved, using models like large language models (LLMs) to create synthetic data. But how suitable are these models for generating test data? In this article, I’ll explore the capabilities and limitations of GenAI in the context of test data generation.

Why Use GenAI for Test Data?
Three challenges
How can GenAI effectively support enterprise test data management?
My conclusion

Why use GenAI for test data?

One of the biggest advantages of GenAI is the ability to generate data using natural language prompts.

For example:

“Generate 100 rows with ID, first name, last name, email, and birthdate in CSV format.”

In about two minutes, you’ll receive a well-structured dataset like this:

This ease of use makes GenAI an attractive tool. However, there are three key challenges to consider when using GenAI for large-scale test data generation:

Performance at scale

Cost at scale

Prompt engineering complexity

Three challenges

The first challange is that of performance at scale. Although LLMs are getting faster, generating large volumes of data remains relatively slow. For example, GPT-4o, one of the faster models, processes around 193 tokens per second. In our earlier example, each row is roughly 20 tokens. To generate 1 million rows would require about 20 million tokens, taking approximately 28 hours.

Even if you self-host an LLM with powerful hardware, say 2000 tokens/second, generating 1 million rows would still take nearly 3 hours.

In comparison, DATPROF Privacy generated 100 million synthetic rows in an Oracle database using mid-range hardware in just 17 minutes. That’s 100x more data in a fraction of the time.

Cost at scale is the second challenge. Generating large datasets using commercial LLM APIs can become prohibitively expensive.

Generating 1 million rows with 5 columns using GPT-4o: ~$200
Using GPT-4.5 for the same: ~$2,000
For a wider table (15 columns, ~50 tokens/row), generating 10 million rows:
GPT-4o: ~$5,000
GPT-4.5: ~$75,000

Alternatively, running an LLM on-premise requires a major hardware investment. A server with 4× NVIDIA A100 80GB GPUs alone costs upwards of €60,000, just for the GPUs.

And keep in mind: many organizations require far more than just a few million rows.

Another challenge is prompt engineering. LLMs are typically controlled via system and user prompts. A simple request like:

“Generate 1,000 rows of customer data with name, address, and email.”

…is easy enough.

But as data requirements become more complex, think data types, formats, domain-specific rules, date ranges, dependencies between attributes, the prompts must become equally complex. You’ll find yourself encoding what is essentially a rule-based system into natural language, which quickly becomes inefficient and unmanageable.

For structured, large-scale test data, you ideally want a system that explicitly defines the constraints your generated data must meet, rather than relying on an increasingly complicated prompt.

How can GenAI effectively support enterprise test data management?

At scale, using LLMs alone to generate test data offers little benefit compared to traditional rule-based generators. It’s slower, costlier, and often less reliable. However, that doesn’t mean GenAI has no role to play.

"Imagine combining the intelligence and flexibility of GenAI with the speed, control and maintainability of rule based test data generators."

At DATPROF, we see GenAI as a productivity tool, an assistant that can streamline and simplify specific tasks for test data engineers.

Imagine combining the best of both worlds:

The intelligence and flexibility of GenAI

The speed, control, and maintainability of rule-based test data generators

That’s where the real value lies.

For example, GenAI can be used to:

Generate seed data like 100 unique Japanese first names, which are then used by a rule-based generator to create millions of customer records

Write complex SQL filters to exclude specific records from anonymization

Assist in setting up a data generation project

In these scenarios, GenAI acts as a smart assistant, augmenting rather than replacing existing tools.

My conclusion

GenAI is not a silver bullet for test data generation, especially not at scale. But when used strategically, it can significantly enhance productivity and flexibility for test data professionals.

Used in tandem with proven test data software, GenAI can help teams move faster, reduce manual effort, and improve test data quality, particularly for as part of a hybrid workflow.

The future of test data management isn’t GenAI, it’s GenAI working alongside subsetting, masking and rule bases synthetic data generation.

About Bert

I write for test managers and test teams about new developments in the test data industry. Want to stay updated? Sign up to our newsletter.