Bad Data - what is it and how can my business avoid it?
What is Bad Data? In business, it refers to duplicate, conflicting, incomplete, invalid, or unsynchronised data – all equally damaging to your company. The chart below demonstrates what each is and some of the horrific possible outcomes!
| Bad Data Type | Description | Possible business scenarios |
| Duplicate Data | 2 or more identical records. | Misrepresentation of inventory accounts. Duplication of marketing material/contacts. Unnecessary billing. |
| Conflicting Data | Same records with different attributes. | Imagine two versions of a company in your system with different delivery or billing information. Deliveries going to the wrong location or payments being lost. |
| Incomplete Data | Missing attributes or updates. | Payroll not processed because of missing Tax code or NI numbers. Not able to identify a loyalty customer correctly. |
| Invalid Data | Attributes not conforming to standardisation. | Ability to export data, sort data, filter and analyse data. Automated processes stall or fall over. |
| Unsynchronised Data | information not shared between two systems. | Poor data management leading to silo creation, repeated work. Inaccuracies across business. Replication of work and data bringing increase in likelihood of errors and ‘Duplicate’ and ‘Conflicting Data’. |
Bad data and it’s compound impact on your business
What happens when we look to the bigger picture? Any individual issue outlined above is bad but what happens when we start using this data to drive decision making? What are the negative impacts on a business if we use this data?
Poor decision-making
Bad data can lead to poor decision-making, as decision-makers may rely on inaccurate or incomplete information.
Increased costs
Data issues can result in increased costs for a business. Meaning you may need to spend additional resources to correct or replace the affected data.
Loss of productivity
Using poor data can cause delays and errors in business processes. This subsequently leads to a loss of productivity for the business.
Damage to reputation
The results of bad data can damage a business’s reputation. Imagine incorrect information being communicated to customers or stakeholders, how will you be perceived?
Legal and compliance issues
Poor and inconsistent data can lead to legal and compliance issues. The impact of non-compliant data in regards to regulatory requirements or contractual obligations is severe!
To prevent the negative impacts of bad data, businesses need to ensure that their data is accurate, complete, consistent, and up-to-date. This can be achieved by implementing data quality controls, conducting regular data audits, and investing in data management tools and systems. Additionally, businesses should ensure that their employees are properly trained on data management. With best practices and awareness of the importance of maintaining high-quality data.
The evidence bad data is bad for business
Data issues costs businesses money, time, and customers. Not only is it a huge hassle to recover and fix mistakes, but it reflects poorly on your business as bad customer service in the process. Here are 5 statistics to prove just how much damage poor data can do and what you can do about it:
1. It costs the US economy over $3 Trillion a Year
Back in 2016 the Harvard Business Review article revealed bad data cost the U.S. economy 3 TRILLION dollars in a single year. That trend continues today. 2022 saw Gartner discussing bad data is responsible for $12.9 million loss for each business per year.
2. The business cost may be as high as 10-25% of a business’ revenue
Just in case you were having a hard time applying the first statistic to your business, wrap your head around this one. Monte Carlo, believe data professionals are spending a whopping 40% of their time evaluating or checking data quality. They consistantly find poor data quality impacts 26% of companies’ revenue. Remember if your data is correct, you have more money to put back into the business and further grow your revenue.
3. The Cost to Fix a Single Error Can Be Over $100
There is something called the 1/10/100 rule. Put simply, it costs about $1 to verify a record as it is entered, about $10 dollars to fix it later, or $100 if nothing is done, as the ramifications of the mistakes are felt over and over again.
Lets put this into a real world scenario:
If an error is overlooked (or someone is lazy and doesn’t correct an error), it impacts later on. At some point a colleague will search the CRM for the company and discover the erroneous record which he suspects is inaccurate. First, he will check his notes, then he will call someone else to verify and possibly will change the record. The result of the discovery will be undermined trust of the data and the ten minutes he spent correcting the data entry are wasted.
Furthermore, if someone in sales contacted a lead to discuss a potential sale with incorrect data the situation gets exponentially worse. The result? A missed an opportunity to partner with a great company because of incorrect timing or loss of trust or credibility. The cost to the firm could ultimately be in the tens of millions of dollars in lost revenue alone – if trust is lost this increases even more!
4. Approximately 15-45% of operating expense is wasted on poor data
Data experts, like Thomas Redman, Jack Olson and Larry English all agree that approximately 15-45% of operating expenses of almost all organisations are wasted due to data quality issues.
This is informed by statistics like these: the average company involved in direct mail wastes about $180,000 a year on direct mailings not getting to the intended person. This number doesn’t even include other inaccuracies such as sending mail to the wrong demographic or to current customers. IT departments are also affected with as much as 50% of their IT budget going toward “information scrap and rework” related to dealing with poor data quality (Larry English).
5. Bad Data can be the result of little or no standardisation
So, this is experience talking, more anecdotal that statistical, but we see time and time again it being the the root cause of most bad data. We see time and money being wasted due to small things like data duplicates and data missing certain attributes. The good news is that by applying the right data integration technologies, you can immediately lower the number of data errors especially those that result from duplicate data entry across multiple applications and systems in your company.
There are, however, two caveats:
An integration product can’t just fix your bad data by itself – it’s important that you take the time to design the right data validation rules and checks for duplicates before moving data between systems using these technologies. Without this, you’ll end up creating data duplicates a lot faster than when you were doing manual data entry.
Second, not just any integration product will work. There are popular but simplistic integration products that simply do not have the capability to implement the logic for valid or duplicate data. These products can do basic one-to-one integrations i.e. move data from one app to another regardless of the quality of that data or whether that data, say a customer record, already exists in the target app.
In conclusion
Data is the lifeblood of business systems today, and in the future. But bad data can impact even the best technology and processes. As we see the trend of AI moving across all businesses the data we keep is even more important.
Simple processes and checks can be put in place to reduce the likelihood of bad data. Great company culture can positively impact best practices around preserving good data. Technology can help ensure all these come together to benefit your business and its data.
Goldstar understands the technology, process and culture required to get the most from your data. We work with you to understand your business, its objectives and help you find the recipe for success, ensuring your data is as fresh as it can be to serve up the best results.
Why not get in touch and we can chat about how you can avoid bad data and launch you to increased ROI, streamlined processes and data-driven decision making.