Tech moves fast! Stay ahead of the curve with Techopedia!
Join nearly 200,000 subscribers who receive actionable tech insights from Techopedia.
Data verification is the process of checking data for accuracy after a data migration. There are different types of verification:
Data verification can be both expensive and time-consuming to carry out.
When data is migrated from a data warehouse for use in a big data processing system, the data needs to be checked to ensure that it is accurate. Everything from spelling errors to inaccurate numbers to data loss could jeopardize a big data project.
One method of verifying the data is comparing data in one system to the migrated data in the other one-to-one, but this can be time-consuming and the costs of running two systems can be expensive.
It is also possible to check just a subset of the data, but a sample cannot possibly represent all of the data. Administrators must weigh the tradeoff between keeping the time and expense of data verification down while ensuring accuracy. Automating the process is one solution.