My previous blog highlighted how transforming into a data-driven organization requires access to high-quality information from across the enterprise. Data accessibility is a top challenge because data is often spread across multiple data stores that use different technologies.
Business users want one-stop shopping for data. A single enterprise data warehouse was the traditional strategy for providing data to the organization. However, enterprise data now lives across a variety of data stores including relational databases like SQL Server, a cloud data warehouse like Snowflake, data lakes like S3, services like Salesforce, and maybe even Excel files.
Our point of view is that many organizations will benefit from a data virtualization solution that provides a unified view of enterprise data assets because:
- It accommodates the reality that data exists across a variety of data stores and technologies
- It provides a centralized data security layer
- It offers a logical transformation capability that can dramatically improve time to market and lower long-term maintenance costs
Data virtualization is an excellent solution for data accessibility because it provides a single connection point for business users to access all enterprise data assets. Data assets of all types that use different technology platforms become accessible with SQL, even when it is not a relational database. It supports federated queries that makes it trivially easy to combine query results across different data stores, such as merging client data from the CRM system with investment data from the data warehouse. Rather than wasting time with data wrangling, the data virtualization layer can automatically query two or more systems and merge the results as if all the data is in one big SQL database. This saves a lot of time and effort.
The data virtualization layer can control who can see what because business users make an authenticated connection, and it offers role-based access controls. Sensitive data, like Personally Identifiable Information (PII), may be masked. The single data access point enables security to be more effectively monitored and controlled. We have seen how centralizing data access and permissions makes data governance easier.
Data virtualization offers a logical transformation capability that can dynamically integrate data, without replicating or moving the data physically, to create “virtual” data. Traditional physical transformation, which is often referred to as ETL, is comparatively more expensive to write and maintain and can create consistency issues. Logical transformations will likely not eliminate physical transformations, but they can reduce the need for physical transformations by 50%, reduce storage needs, and avoid consistency issues.
Virtual data views can combine the latest data across various source systems making it easier to deliver holistic views of information such as clients or investments. They can expedite the creation of Client 360 views. Firms without a Master Data Management (MDM) solution for investments (aka Security Master) may create a logical MDM that dynamically merges data from source systems.
For moderately complex data environments, data virtualization is worthy of consideration as a key component of your data platform. The convenience provided by a data virtualization solution helps enable self-service and empower business users. Leveraging it can free staff from data wrangling, allowing them to focus on higher-value tasks. We have seen it deliver significant ROI and agility benefits.