Much data management effort has gone into eliminating data silos and the Operational Data Store (ODS) is a solution commonly used by many businesses. However, the traditional ways of storing data, such as using an ODS, have certain limitations. New technologies demand rethinking the approach to meet a growing demand for speed, scalability and availability.
Solutions need to be found to step up data-management agility, support for data of all types and data harmonization. To handle an increased range of data and operations, institutions require a scalable data ecosystem. It should be possible to find the right data quickly and support many different use cases, from simple reporting to artificial intelligence and machine learning.
The limitations of traditional data ecosystems
The operational data store (ODS)
The ODS is a central database that integrates data from various sources. The disadvantage of a traditional operational data store is that it may support operational reporting but it is often based on a relational database which means that handling large amounts of data is a problem. New digital applications need low latency and this is not possible with a traditional ODS. If too many users access the data store concurrently, performance goes down. A periodic refresh rate is not acceptable for digital applications as they require real-time data.
The Data Warehouse (DW)
The data warehouse tackles integration by having a single destination for all the data across the organization. The data is moved from various sources through ETL (extract, transform, load) and to the data warehouse.
The resulting data warehouses make strategic decision-making possible with access to a single source of historical data. However, they provide a static source of data and have limited agility and adaptability. Various evolving trends, like big data and self-service analytics, have amplified the limitations.
The Data Lake
Data lake architecture emerged to deal with data warehouse limitations. It offered a new way of handling unstructured and differently structured data, thereby improving elasticity and scalability. The ability to store and process vast quantities of data and ingest it at high speed enabled advanced analytics.
However, various new challenges like governance and security became an issue. A data lake could become yet another data silo that existed alongside but disconnected from a data warehouse or an operational data store.
The evolution of the ODS and a paradigm shift
At first, an operational data store consisted of separate tactical and strategic systems that were populated by periodic batch updates. These evolved into real-time systems, with ODS data being updated when changes occurred on the source systems. However, there was still no single, consolidated view of the business. Much effort was invested in consolidating into a single ODS, which offered many advantages.
However, the traditional ODS is unable to keep up with the demands created by the introduction of new business models and services. For example, digital banks keep introducing more online services. Digital insurance companies can generate policies and pay claims within minutes.
Financial services companies often have many applications and each system of record has a different API. This makes it difficult to get a single view of data across all applications and platforms. Imagine a scenario where many digital applications are calling APIs directly to access multiple systems of record. Conventional architecture no longer suffices and the paradigm shift has resulted in the creation of a new generation of operational data stores.
What is a new generation ODS and what does it offer?
A new generation ODS can prevent back-end systems from being overwhelmed by excessive workloads and avoids complex integrations between back-end databases and front-end API services.
Fast performance: A new generation ODS has a distributed In-memory computing and storage engine with enough speed to power digital applications. The distributed in-memory core means that even with a high concurrency of users, performance is unaffected.
Autonomous scaling: A new generation ODS accommodates unplanned drops and peaks in volume. This makes it possible to maintain customer experience at the peaks and expensive over-provisioning of on-premise resources unnecessary.
High availability: A new generation ODS means decoupling the API layer from the system of record. This means that a business can keep functioning even if a system of record is down. For example, the credit status of a customer could be checked against the ODS if a credit-authorization system was down.
Accurate predictive modeling: It is possible to analyze real-time data as well as enrich it with historical data to enable robust predictive modeling.
AI-driven data tiers: It is possible to move data based on business rules to make sure that the most important data is readily available, while less important data resides on less expensive types of storage. The data moves automatically to balance cost and performance.
Hybrid deployment support: Many global organizations have data centers in remote areas for reasons such as regulation compliance. Others have data on-premise and in the cloud. A new generation ODS can support hybrid deployment and replicate data in real-time without impacting performance.
Choose from various options
Businesses often find that it makes more sense to augment their current architectures than discard them. This entails enhancing existing ODS deployment to boost performance and scale, making the introduction of new digital services possible. Adding a micro-services API layer for digital applications requires only a small investment.
An out-of-the-box solution usually comes with all the necessary components to enable fast data processing in real-time from systems of record. This is a great option for a small to medium-sized business as it provides a platform to launch a new project and a system that can scale up to support future growth.
Conclusion
Businesses invested in digital transformation need to modernize their architecture. They can do this by augmenting their current systems or replacing them. A next-generation ODS can aggregate multiple back-end systems and databases into a low-latency, shared data store. Businesses can experience the benefit of better performance, more speed, scalability and high availability, which gives them a strong, competitive advantage.
Be the first to comment