Seven Leading Ways Data Lineage Costs Your Company Money Today
Unlocking the keys to data lineage will play a critical role in all data-driven businesses of the future – and is already doing so at the most successful and profitable organizations today.
So how can you leverage data lineage to transform your businesses without incurring unnecessary costs? Based on our experience, here are the 7 most common factors:
1. Too Many Resources Involved
Are there too many people involved in tracing data? What is the opportunity cost associated with tracking your data across your various sources for data lineage?
Does your company conduct exercises to see how many FTE’s are actually involved in putting together some type of end-to-end view of data lineage and data mapping? Do you take developers, IT operations, data analysts and business analysts away from revenue generating and business growth activities to be able to help build data lineage when it is needed? Performing manual processes and coding to create data lineage is wasteful and time-consuming.
2. Outdated Data Lineage
How often do you update your data lineage? Does your lineage reflect what is happening today?
For many companies updates may only be scheduled quarterly or monthly today, but really should be done weekly or perhaps daily. Because of these delays, your company may be relying on outdated lineage resulting in unnecessary delays when acting on exceptions. These hours can add up. In a lean and agile world, exceptions need to be handled faster. Outdated lineage will certainly slow down response times for exceptions or anomalies making efficiency much worse and costing you money.
3. Incomplete Data Lineage
Are there sources like JCL, Cobol, data lakes, or various repositories that you have no visibility into? Does your lineage have lack of transparency and visibility into databases, systems, and repositories, and is it missing data hops?
Many companies struggle with getting to the stored procedures and actual codes and languages within a system or source. Organizations find that they cannot access their legacy systems or their many data repositories to really get a full view of the lay of the land. Having blind spots or gaps in your lineage represents risk and the associated costs could be significant.
4. Regulatory Compliance Delays
Are you unable to give internal and/or external auditors the information they need? Is the organization risking fines due to the inability to provide timely data?
CCPA, GDPR, CCAR, DFAST, CECL, BCBS 239, Ohio’s Data Protection Act, and other legislation has been in existence, or is new to enterprises, causing a need to be able to comb through data lineage. More recently, finding and tracking PII data leads many organizations to gain visibility into all data sources to search through, find the relevant data, and take action upon it. Audit costs will quickly accumulate when auditors are waiting for information. Missing or incomplete information will most likely result in audit exceptions that may be expensive to address.
5. Poor Quality
Do business owners trust the data they rely on to make day-to-day key decisions? Are they constantly questioning its accuracy and do delayed responses due to root cause analysis time have an impact on critical business decisions?
Many organizations’ ETL processes/platforms are often leading suspects for causing improper changes and transformations of the data, leading to skewed metadata. Perhaps the business owners of the data are forced to spend time reviewing data quality issues with IT because they have an incomplete picture of the sources involved. Inaccurate or delayed business decisions mean lost opportunities and mistakes which impact clients.
6. Under-Performing Data Catalog Asset
Are you only receiving a high-level view of your data lineage today, by only seeing a sampling of your sources? Are there gaps in your data repository because lineage is not collected from information in legacy applications for example?
Harvesting and extracting metadata is critical to scour from everywhere, even where it does not exist today. In addition, perhaps you find that various databases make web service calls to other databases and other sources and your company may be missing out on those changes in coding along the way. Strong data quality metrics are driven by solid metadata ingestion that is seamless and timely, staying up to date to the daily business activities. If properly used and populated, your data catalog can be a working asset for your company otherwise it will become just an expense.
7. Lack of Impact Analysis
Does your organization perform useful and complete impact reviews before making application changes? Do your agile initiatives and sprints suffer due to the impact analysis taking too long?
Conducting an impact analysis can be nearly impossible for some organizations. There is a need to figure out upfront what your code is really doing and what will be the impact of your pending changes. Without a proper impact analysis, you may have originally designed the change to do one thing, but it ends up creating problems in downstream applications. Without the right information to conduct an impact analysis, the result of this could become a futile and frustrating activity. Another outcome will be that the task will be bypassed or done haphazardly resulting in extra testing time or clean up time for expenses, once the errors are found in production.
In 2020, Quantify your Overall Risks
It’s important for companies to take each of these factors and try to quantify the overall costs. From there, make sure to evaluate how to reduce the costs as well. If these areas are new to you, leverage a trusted advisers that has been in the data lineage space for many years with experience extracting and harvesting metadata. Tapping into the knowledge of someone who has completed multiple integrations with leading data catalogs and who has a rich history of developing data lineage connections with disparate sources of metadata can make the process to fix these issues a short, painless, and cost-effective process.
What is your current plan at your company to alleviate these 7 unnecessary costs?
Curious to learn how to get more from your current data catalog investment?
Listen to our webinar, “Maximize Your Data Catalog Investment.”
One Lincoln Center
18 West 140th W Butterfield Road;
Oakbrook Terrace, IL 60181