The term “Integrations” is one of those catch-all words that have very different meanings based on who is referencing them. For example, an integration could be a simplistic click-to-deploy integration with Google or Apple to assist users in signing into a service more securely and conveniently. Or an integration could be as complex as a fully embedded technology where data, front-end programming libraries, and events are being shared in a well coordinated and highly fragile dance.
I think this reflects the greatest challenge when speaking about integrations, a challenge that hinges on who you talk to, what area of the business, or what endpoint, getting everyone on the same page.
As CTO here at Mercatus, I get a lot of questions around integrations from business leaders in the private markets. Here I try to provide guidance and clarity by sharing my answers to some of the most common ones.
What work goes into a typical integration?
When approaching integrations, the first challenge is always around definition. Typically there are many stakeholders and the requests come across “we must integrate X”. These requests come from people who assume integrations are like consumer apps in which you can simply “click to deploy.” What usually has not been accounted for is the significant amount of detailed definition that is always required in successfully setting up an integration.
Let’s move beyond scoping. Assume you have asked all of the hard questions like: How will data flow? What data? What events will trigger the exchange? How will we handle credentials? What security considerations and implications do we have to take into account? What interfaces will we use? Next we need to move to implementation, provisioning required infrastructure, building the integration, testing, and deployment of the solution.
However, getting an initial solution deployed is just the beginning. Much like markets, the integration requirements will continue to evolve, and the endpoints/infrastructure will also continuously change. This is where it becomes critical to have alignment and effective communication streams with the involved vendors, internal teams, and align mutual roadmap and product changes so that the integration can remain functional through its lifecycle. The last thing anyone wants, which happens more often than not, is to wake up to a broken integration just to learn that the vendor has deprecated a critical API endpoint, or changed security features and nothing is working.
Why can’t I just have someone on my engineering team do the integrations and maintain them?
Building a data integration or functional integration between two products takes not only skill, knowledge, and foresight to ensure a successful initial deployment; but it also requires commercial management and alignment to ensure that as the other products evolve the overall integrated solution also evolves along the way and is properly supported. This includes security, roadmap, legacy support, growth planning (ie. How many API calls do you need now? In 4 quarters?). I would say while a single engineer can build a high quality integration, it takes significant dedication across many roles (engineer, product owner, project manager, procurement, and information security) to ensure a successful integrated experience.
How are Mercatus integrations different from other solutions?
We learned quite early on that the mythical creature known as the “out of the box” integration was not a valid concept within the private markets investment landscape. We understood that standardization was not an option, therefore how can we rapidly address curating and managing bespoke integrations with as many different endpoints as possible in a way that can be managed, and can meet the unique needs of each of its customers’ businesses? This is where we leaned in and deployed an integration platform as a service (iPaaS) within our core platform to give unique advantages to our customers, our own service teams, and our engineering teams. By leveraging this approach we are able to create, deploy, and support complex data integrations across modern and legacy solutions in a matter of a week or two, versus months using the traditional route.
Much of this is due to the advantages of no-code assisted logic building. With iPaaS, our product and engineering teams can focus on building the most comprehensive Mercatus interfaces making as much of the platform extensible as possible, while the business teams can focus on building the integrations with an easy to understand visual logic builder right inside the application, and finally the iPaaS partner can focus on curating, building, and maintaining the endpoint ecosystem for acceleration and support.
Furthermore, by using this approach the difficult and typically overlooked aspects of error handling, throttling, performance, notification, authentication, version management, etc. are unified and available across all integrations in a standardized way. Business users may not appreciate these aspects, but ask a weary analyst or engineer at 3am when roused to fix a broken integration the night before the close of a quarterly reporting period, and you will understand why these are just as critical.
Once you’ve integrated with another system once, does it get easier?
Depends, but usually only slightly. Typically efficiency gains will come from familiarity with the endpoints and their current supported capabilities but that’s where the advantage falls off. Configurations and business uses typically change drastically from integration to integration, even for the same endpoints (systems). Additionally, with various versions floating around many times the actual endpoints capabilities vary, even within the same vendor offering. This means that it is usually very unlikely that you will see the same integration twice. The best you can hope for would be some reusability around shared aspects like authentication. Even these, however. can vary drastically from deployment to deployment.
What is the cost to integrate a system into Mercatus?
This varies wildly, similar to the scope of integrations. By leveraging our unique approach to integrations using our integration platform as a service (iPaaS), we are able to accelerate by orders of magnitude since we are not rebuilding core capabilities around authentication, endpoint management, error handling, reporting, notification, etc. Most of the cost in an integration comes down to license implications put on you by the vendors of the endpoints, volume of data exchanged and limits, and the big cost—labor. We decided to address the last, given it has the lion’s share of impact. This translates into an average reduction in annual cost of data uploads, integration builds and maintenance effort by 80% versus your traditional approach.
Why is there an ongoing cost for access to integrations after they’re set up?
Integrations are really just narrowly scoped products. Just like a product you have to continue to evolve it not only due to business requirements changing, but also you need to change with your endpoint vendors and their own product evolutions. Integrations are dynamic, and as such they will need dedicated support and adjustment continuously over their lives. We typically see changes on a continuous basis monthly if not weekly.
How long does it take to do an integration?
Just like cost, this will always vary greatly due to scope, complexity, and endpoint capabilities. However, building a moderately complex integration between two known and established endpoints in private markets the traditional way (ie. programmatically) will usually take between 2-3 months with an experienced team and supportive vendor team.
On the other hand, using Mercatus’ approach with iPaaS the same integration could be deployed and live in under two weeks.