There’s no way to fill the data skills gap, what now?
A recent survey of 100 UK CIOs found that 76% are worried about recruiting the IT staff they need to remain competitive. They’re right to be worried. The European Commission estimates that 100,000 new data-related jobs will be created by the end of next year – and we don’t even have the data scientists to cope with the roles that exist today.
The sheer volume of data enterprises have collected, often over decades, combined with new data protection rules, plus the emergence of new types of data that enterprises aren’t familiar with, have left them struggling to cope.
Regulation—in particular GDPR, CCPA and similar legislation—means that implementing best practice data management is no longer something that enterprises can ignore. By keeping the wrong data or keeping it in insecure ways there’s a risk of fines that could run into thousands, based on a percentage of revenue and potentially the difference between profit and loss. There is no single solution to this issue, and businesses need to think about the short, medium and long term.
Data skills gap: Short term solutions
The first thing that businesses need to realize is that they are unlikely to be able to recruit their way out of this problem. There is a big gap in the labour market around data analysts. Despite the promise that data lakes can be fished for new opportunities and new insights, this is likely to be a long-term goal. Instead, it’s important first to understand where data is held, whether it’s secure, and classify it to discover if any of it shouldn’t be held under data protection and privacy regulation. While there may be some ambiguity here without the right expertise, for many enterprises there will be data that can be clearly marked for erasure.
When data is identified as redundant, obsolete or trivial (ROT), it’s vital that best practice data erasure is followed. Data can be replicated across different systems—disposing of it in one system may not be enough. And when data is erased, it needs to be done with an audit trail and with appropriate certification. Businesses need to be able to prove that it’s gone.
Businesses also need to get rid of the hoarding mentality that may have set in, perhaps due to a lack of data skills. Along with the knowledge that data is valuable, there has been a tendency to collect as much of it as possible. So customer interactions may be farming more data than is necessary as a matter of habit rather than necessity. Businesses need to examine where data is coming from as much as where it is being stored.
In the medium term: Automate processes as much as possible
Automation can help mitigate many skills shortages, and data sanitization is no different. And even if there was no skills gap, it would still be important to automate data processes as much as possible to make sure that the correct processes are being followed.
Many processes involving managing data can be automated. For example, when hardware reaches end-of-life, it can be sanitized and safely disposed of using automated solutions that ensures there is zero risk of that data being recovered. When customers request that data is removed from the system, or customer data is no longer appropriate to keep, this data sanitization should also be automatic.
We’ve found that many businesses, when erasing data to make it unrecoverable, have to do so on an individual drive-by-drive basis that is frankly unsustainable. Only a scalable, automated system will prevent data from hanging around when it should be long gone.
Long term planning: Train the right people
Ultimately, there are only two ways to solve the problem of a skills gap: hire the right people or train them. With the need to fill data analytics roles increasing, businesses need to think about how they solve this in the long term.
One way is to train people from within. If the right people aren’t available on the job market, then training up current employees means that businesses will have those missing skills. Of course, there are a few problems with this approach – it’s expensive, time consuming, and training people in highly-marketable skills means they are more likely to jump ship. For this to be viable, it’s something that lots of businesses need to do.
The other way is to ensure that people are entering the job market with the right skills. Partnering, or at least having conversations with educational establishments can mean that they have a better understanding of industry needs and tailor their programs to better meet them.
Data analytics is often cited as a vital and in-demand skill because of its application in making the most of the data businesses have. But making sure that businesses have the right data is even more important – and they need to start both planning and acting to make sure that they can manage without immediate access to data skills.