Data orchestration is a growing priority for enterprise brands, as they seek to gain greater visibility and control of data across connected systems.
In a small, but growing number of cases, this takes place across fully-composable stacks, but only a minority of brands have been fortunate enough to future-proof their technology in this way. More often, companies need to ‘present-proof’ the stack, by optimizing the way that legacy systems and modern composable solutions interoperate.
Leading the charge in this space is Sana Remekie, co-founder of Conscia, a composable data orchestration solution which works for enterprise brands. Sana joined us on our latest podcast episode alongside Conscia’s implementation partner Orium, with CEO Jason Cottrell returning for his second episode of Martalks.
In this conversation we discussed:
- the specific challenges of API design in legacy systems and the very high costs of custom workarounds
- how a data orchestration layer improves on this, and how Conscia’s solution compares to competitors
- how system integrators (SIs) can support data orchestration, including through the development of accelerators
Listen to the full podcast here, or scroll down for a summary of the key points discussed.
How data orchestration compensates for legacy APIs
Where composable transformations have not been successful, in many cases, it’s been legacy approaches to data orchestration that have unsteadied the ship.
For this type of integration work, explains Sana, brands typically ended up…
“…using ETL scripts or tools like MuleSoft. It’s millions in effort and licences.
When you think about composable architecture, you add 5-10 more systems, you balloon that problem out of proportion, and you can’t use the traditional methods anymore.”
While legacy solution vendors may have anticipated the need of data orchestration, earlier versions of Salesforce assumed that orchestration work would take place in the presentation layer.
This becomes unmanageable in the context of multichannel commerce, because APIs in these legacy systems were “were never meant to be called directly from the front end.”
Sana gives the example of how category navigation structures may differ between the platform and the customer experience (CX).
“You have to make multiple calls to get the full tree back – because the one call only gives you the current category and its children. To put a full navigation on your website, you need the entire hierarchy. In that scenario, the API is really an issue…
If the front-end had to do that work in real-time, that’s a lot of APIs calls. You’re doing API chaining and stitching of data and caching… that’s going to take time, and the response times suffer.”
Clearly, simply adding composable services does not directly solve any challenges that the brand is facing relating to management of data. While this can, in theory, be worked around with hardcoded integration, you’d end up “simply building another monolith.”
Conscia’s solution is an abstraction layer; an “intermediary to improve access to the data model”. This preserves the hoped-for flexibility of the composable setup, while also greatly reducing time and cost of implementation.
For Conscia’s typical $1bn+ enterprise clients, Sana says, the time needed to solve the data orchestration problem, in a composable stack, falls from 9-12 months, to just 4-8 weeks.
System integrators, and the productization of composable connectors
As with most martech categories, options for data orchestration solutions have proliferated in recent years, leading to greater specialism and faster time to value.
Orium’s Jason Cottrell commented…
“You have far more productized options today compared to five, three, even one year ago. When you have a productized option you know the integration’s been done properly, and it’s probably been used by a couple of customers, so you’re not the first. So it reduces not only time and cost; it also reduces risk.”
As the category has matured, it has diversified into solutions which are highly optimized for certain use-cases.
Conscia’s point of differentiation is what Sana describes as a “superset of different capabilities”, highly optimized for multi-channel ecommerce.
The technology comprises two core components:
- the DX engine, which handles dataflows between frontend and backend systems
- the DX graph, essentially an API-first data hub which moves data horizontally between different connected systems, with a management interface where non-technical professionals can work with the data.
By contrast, Enterspeed and Octtoo, two competing vendors, operate more as ‘caching solutions’, which are highly useful for increasing flow rates but without the same data extension capabilities.
Jason elaborates that orchestration options are now available based on common data protocols in certain industries (i.e., commerce, fintech, finance) and use-cases (personalization, AI, auto-optimization).
These tailored solutions can save a huge amount of time and energy, including for less-technical client teams. Nonetheless, there remains a vital role for SIs to play in ensuring these solutions are chosen wisely and deployed effectively.
The value of collaborating with a systems integrator has been widely discussed in previous episodes of Martalks (see our recordings with Pivotree and Capgemini). In the field of data orchestration, Sana and Jason agree on the importance of collaborating with agencies which operate practices for both composable and legacy solutions.
As well as Orium, Sana lists EPAM, Royal Cyber and Valtech as partners which fit that profile, while Jason highlights the value of accelerators which the agency has purpose-built to knit together different platforms and solutions.
“An accelerator is not a SaaS product… it’s pre-written integrations, and any SI worth their salt in the composable space should be coming to the table with that… we’ve spent millions on pre-integration. You want one of a very defined subset who have truly productized a jumpstart.”
That implies integrations which have already been written and tested by the agency, which come at no direct cost to the customer, and which give the customer flexibility around different solutions.
For Orium, a partnership with a vendor such as Conscia greatly extends the scope and utility of this work. A one-time integration with a data layer, purposely designed for multi-channel commerce and multi-component commerce stacks, ultimately gets the customer live faster, and at lower risk and cost.
About The Rosenstein Group
The Martalks Podcast is published by The Rosenstein Group, the leader in martech executive search. For over 20 years, we’ve been recruiting heads of sales, channel sales leaders, and other members of the commercial team, across martech, supply chain, ecommerce, sales enablement and systems integration.
Visit RosensteinGroup.com to find out more.
Apple Podcasts
Spotify
Google Podcasts
