Why Integrations are still hard in 2019

We were promised personalised medicines, jet packs and flying cars by now..  why is integration still difficult?

What's "Integration"?

In short it's connecting two systems or data sets together.  This could be as mundane as ensuring that an app has access to your employee lists all the way to merging companies and ensuring that information is shared effectively across all of their systems.

For app/websites/platforms we're typically connecting data to the platform via APIs or data access and/or connecting multiple systems so the approach is seamless.

For data / BI / Big Data projects are largely about ensuring that the right data, at the right granularity and the right time period are viewable and usable.

The State of 2019

It's not that integration is getting harder; it's definitely improving over time.  But each year our clients increase the amounts of integrations they do, we expect less latency (i.e. the time between the data being in one system and being viewable/usable in another), for more and more data points, and we're increasingly using large applications from multiple vendors.

With all these things it's not just that the problem is getting harder (due to more data moved more quickly) but also that the amount of things we need to integrate are increasing.

So?  What's the problem?  Data continues to be in different shapes, with different formatting, to different standards, and, worst of all, some systems don't expose this data to allow integration.  

Just like jet packs we've been promised "drag and drop" integrations where we map data but these inevitably fail as soon as any changes are discovered or the data is outside of bounds/wrong.

As an example; we have a recent client who came to us needing their CRM data centralised and reportable.  Why's this a problem?  Because they have 4 different eCRM/marketing products (Marketo, Send In Blue, Urban Airship and Oracle's Responsys Interact) this means that consumer's contact preferences weren't easy to manage (so erring on the side of caution marketing was being curtailed) and different systems had different marketable users in them.  The end result is that the efficiency and efficacy of campaigning was impossible to judge and manage.

How then will we integrate products?

Luckily 2019 is the year that RPA (Robotic Process Automation) comes of age as well as having comprehensive support for integration (and handling faults) within AWS and Azure cloud services.

RPA allows us to integrate systems where APIs don't exist.  Using this technology allows us to automate the processes that people follow so that we can extract or enter data.

The clouds have excellent support for small elements of code that can be scheduled or triggered (for example they could be scheduled to run at a certain time or date, or they can run if a message appears or a document is added to a cloud fileshare).

This allows us to use a hybrid approach where RPA helps fill some of the gaps in various products APIs or services they provide.

We can also use AI to help automate some decisions and data sets - for example recognising faces in photos or examining the text automatically to understand the sentiment of the message or look for the key subjects and themes.

Will 2020 improve things?

Undoubtedly!  AI will continue to improve apace.  RPA automation is improving on a monthly basis and the clouds are investing heavily.

Will this provide some panacea of all integration?  I doubt it but it'll continue to make things simpler for integrators and allow clients to integrate more of their systems to improve efficiency and cost-effectiveness of their businesses.

So what's the good news? Well RPA and complex integration work today and can make significant efficiency and efficacy improvements in most businesses.  Contact us today if you want to know more.