When you buy through affiliate links in our content, we may earn a commission at no extra cost to you. Learn What Does Data Verification Mean?
Data verification is the process of checking data for accuracy after a data migration. There are different types of verification:
- Full verification, where all of the data is checked
- Sampling verification, where a small sample of the data is checked
Data verification can be both expensive and time-consuming to carry out.
Techopedia Explains Data Verification
When data is migrated from a data warehouse for use in a big data processing system, the data needs to be checked to ensure that it is accurate. Everything from spelling errors to inaccurate numbers to data loss could jeopardize a big data project.
One method of verifying the data is comparing data in one system to the migrated data in the other one-to-one, but this can be time-consuming and the costs of running two systems can be expensive.
It is also possible to check just a subset of the data, but a sample cannot possibly represent all of the data. Administrators must weigh the tradeoff between keeping the time and expense of data verification down while ensuring accuracy. Automating the process is one solution.
