Should You Migrate Your Data, or Start with a Clean Slate?

Data migration is part and parcel of system design and is common practice when moving to new technology. It can also be a difficult, lengthy process that requires careful planning, preparation, and execution.

Many companies are choosing to start from scratch when implementing new systems. In this article I will explore some of the reasons why and ask if you would consider starting with a clean slate, rather than migrating old data when upgrading your systems.

Data clarity is imperative to improving data-driven decision-making at all levels of any company. Could restarting your systems with clean data and avoiding the clutter or confusion of poor-quality data be the right choice for you?

In this article:

• What do you really need?
• The cost of data
• Customer experience and data
• Security and data governance
• Finding your way through a maze of data
• An opportunity for organisational reform
• Can an apple convince you?

Let’s start by looking at the data you currently store.

What do you really need?

Most businesses store far more data than they actually need. Consider a typical customer project in your organisation; no matter the industry, you will likely need:

• Customer name
• Contact phone number
• Contact email address
• Address for billing
• Payment/credit terms
• Details of their order

This is the essential data you need to fulfil the order. However, most organisations will also collect and store significantly more. A good example is a contact’s birthday. It’s a lovely gesture to send a greeting card or small gift, but do you actually do that? If you do, that’s ok, you’re using the data (with permission) for marketing purposes, but if you don’t use the customer’s date of birth you have no legitimate reason for keeping it.

Here’s another example: do you need to know someone’s marital status for selling home furnishings? Probably not. And then there’s the abundance of social channels you might want to connect on – do you need to ask, enter and store every contact’s Facebook, Twitter, LinkedIn, Snapchat handles? You get the picture.

The cost of data

Some organisations do buy data, but in this article I’m focussing on data that you have collected yourself as part of your day-to-day operations. Just because you haven’t received an invoice for the data you’re storing doesn’t mean that data is free.

Collecting data is expensive at an organisational level. Time is quite literally money. Your staff’s time is being used to:

• Collect the data
• Enter it into your system(s)
• Check the data for accuracy
• Maintain the system, deduplicating records, etc.

And then there’s the cost of the system itself and the data storage required.

Here’s a point that drives me crazy. If data storage was expensive, organisations would be more discerning about the data they choose to keep. The reality is, data storage is cheap and the price is declining every day. This is the primary driver in a prevailing attitude of ‘keep everything, you never know when we might need it’.

Here’s a challenge: look at the information you keep as ‘standard’ for your customers. Sit back and ask yourself and your leadership team:

1. What data do we collect?
2. How do we use it?
3. Is the way we use the data needed/are any processes redundant?
4. How does it benefit us/our customer?

Customer experience and data

Have you ever done business with a company and had to endure an interrogation? If you’ve shopped for insurance you’ll know what I mean.

If you’re asking your customers to provide data that isn’t directly attributable to the work you will fulfil for them, it could raise concerns over why you’re collecting it and how you will use it. Could that detract from their experience or, worse, put them off doing business with you altogether?

Today’s consumers are far more data-savvy and protective of their personal information. We have to trust whoever we share our details with to keep them safe and secure.

Security and data governance

Data governance is about managing data to protect privacy, understanding data use cases, data lifecycle considerations and data quality management issues.

In a nutshell, good data governance means you only keep the data you need, process it for the purpose that is necessary to fulfil your obligations, and keep it for as long as it’s needed. All data should be stored securely and only accessed by individuals that require it to do their job.

Additionally, you must consider compliance with relevant regulations. For those operating in the UK you will need to refer to the General Data Protection Regulations (GDPR) and The Data Protection Act (2018). If you have a worldwide customer base there are additional privacy regulations to think about.

Failure to apply good data-management practices can result in legal action. This is a risk that can be easily mitigated, simply by reducing the amount of data that is stored.

Finding your way through a maze of data

I’m sure you’ve picked up a puzzle book at some point and tried your hand at solving a maze. You start at one side and draw a line to find your way through, avoiding the dead ends and never-ending loops. Data flow can be like this.

When we collect data it comes into the organisation in one place, flows through systems and comes out somewhere else. However, sometimes, the data goes into the system and gets stuck in a loop or hits a dead end. It goes nowhere, it just waits for something to happen.

If you’ve been collecting data for years, chances are some of it will be sitting in a dead end or a loop.

An opportunity for organisational reform

Introducing new technology, processes and systems is the ideal time to consider how and why you process data. Taking a bird’s eye view of your process will show you where the dead ends and loops have developed over time.

Eliminating these drains on your resources will give you long-term savings in time and money.

Coming back to my original question – should you migrate your data or start from scratch? – you’re probably seeing some benefits of not transferring your data. So, my question to you is, do you want all your existing data in your new system? Probably not.

Can an apple convince you?

If you’re still unsure, I’d like to share an analogy with you.

I believe the best way to approach data is to treat it as you would fresh food. If you don’t use it, it will go off. When it goes off you need to throw it away. If you don’t dispose of it, the food around it will decline in quality – mould spreads.

If you’re ready to build new systems in your business there’s a sure way to avoid introducing rotten data – start with a clean slate! It might not be possible for all businesses, but it is worthy of careful consideration. And it’s not as difficult as you might think. To explore the options for you and your business, call for an informal chat. My team and I are always happy to share our experience and expertise to help companies find their way through the data maze.

If you enjoyed this article, you may also be interested in