Data Integration

What is Data Integration?

Introduction

Data Integration is a term used to describe a collection of methods, tools and technologies used to connect more than one data source together, normally for the purpose of sharing, consuming or serving information.

The data integration challenge

Data integration must occur any time data needs to move from one system to another for any reason. These days, many processes are automated by computers and serviced by Internet of Things (IoT) devices. It’s not remarkable, therefore, that most businesses have many data integration challenges. There are remarkably few applications use cases that don’t demand the consumption or distribution of data to third-party data sources. For this reason, data integration is a key part of any software development.

Research conducted by Harvard Business Review (HBR) back in 2012 suggested that data integration represented something like a third of all software development project overheads.

The wide adoption of Software-as-a-Service applications over recent years has also done little to help enterprise data management challenges. Demands for data integration have exploded.

The statistics around SaaS are unnerving. According to a recent report by Blissfully, a New-York startup that operates a platform to track SaaS adoption, the average company with 200 to 500 employees uses about 123 Software-as-a-Service (SaaS) applications these days. For mid-sized companies, there are an average of 32 different billing owners for the SaaS subscriptions. On average, an employee uses eight SaaS applications!

Challenges in re-using data

There are a number of common challenges when it comes to re-using data from third-party systems. These include:

Not knowing what data exists, where it’s held, or how it’s held. Rarely do organizations support a detailed master data management taxonomy or record of data.

Complexity and integrity of file formats. Not all file formats structure data in a form that makes it easy to re-use. Sometimes, harvested data needs to be heavily cleansed and restructured in order to be useful.

Composite file-formats trap data. Some applications genres such as geospatial information systems (GIS) and Computer Aided Design (CAD) applications commonly use composite files that blend vector and binary data. This can make it difficult to extract data contained within the composite file structure.

Data quality. Often, when data is re-used from third-party systems, major data integrity and data quality issues can be exposed. This is because many applications used by organizations support data tables and rows in applications that aren’t used by the ‘customer’ organization. Only when data integrations are implemented in new applications are these data quality issues exposed.

Latency issues. Not all tables in all applications across an enterprise are up-to-date all of the time. For example, invoice data might not be accurate until the end of the month if billing cycles are delayed for batch processing. This can cause latency issues when harvesting or re-using data for new purposes as one system will lag behind another.

The absence of entity identifiers in data. Inconsistencies in the design of data structures in enterprise applications can strip them of useful entity identifiers to aid re-use. For example, when trying to re-use customer data across multiple applications, it may be that applications use alternative identifiers to identify ‘a customer’ – or worse still, no identifier at all!

About Encanvas

Encanvas is an enterprise software company that specializes in helping businesses to create above and beyond customer experiences.

From Low Code to Codeless

Better than code-lite and low-code, we created the first no code (codeless) enterprise application platform to release creative minds from the torture of having to code or script applications.

Live Wireframe

Use Encanvas in your software development lifecycle to remove the barrier between IT and the business. Coding and scripting is the biggest reason why software development has been traditionally unpredictable, costly and unable to produce best-fit software results. Encanvas uniquely automates coding and scripting. Our live wireframing approach means that business analysts can create the apps you need in workshops, working across the desk with users and stakeholders.

AppFabric

When it comes to creating apps to create a data culture and orchestrate your business model, there’s no simpler way to instal and operate your enterprise software platform than AppFabric. Every application you create on AppFabric adds yet more data to your single-version-of-the-truth data insights. That’s because, we’ve designed AppFabric to create awesome enterprise apps that use a common data management substrate, so you can architect and implement an enterprise master data management plan.

Customer Data Platform

Encanvas supplies a private-cloud Customer Data Platform that equips businesses with the means to harvest their customer and commercial data from all sources, cleanse and organize it, and provide tooling to leverage its fullest value in a secure, regulated way. We provide a retrofittable solution that bridges across existing data repositories and cleanses and organizes data to present a useful data source. Then it goes on to make data available 24×7 in a regulated way to authorized internal stakeholders and third parties to ensure adherence to data protection and FCA regulatory standards.

Encanvas Secure&Live

Encanvas Secure and Live (‘Secure&Live’) is a High-Productivity application Platform-as-a-Service. It’s an enterprise applications software platform that equips businesses with the tools they need to design, deploy applications at low cost. It achieves this by removing coding and scripting tasks and the overheads of programming applications. Unlike its rivals, Encanvas Secure&Live is completely codeless (not just Low-Code), so it removes the barriers between IT and the business. Today, you just need to know that it’s the fastest (and safest) way to design, deploy and operate enterprise applications.

Learn more by visiting www.encanvas.com.

The Author

Erica Tomlin is a senior consultant specializing in helping organizational leadership teams to grow by implementing enterprise software platforms that improve data visibility, process agility; and organizational learning – creating an enterprise that learns and adapts faster. She writes on subjects of change management, organizational design, rapid development applications software, and data science. She can be contacted via her LinkedIn profile.

Further reading