Can XBRL play a key role in modern data architectures?

Introduction

The digital age has ushered in an unprecedented era of data proliferation, with business information being generated and disseminated at an exponential rate. Traditional methods of business reporting and data analysis struggle with this surge in volume and complexity. Consequently, enterprises are turning to cloud-native platforms, data lakes, data lakehouses, and mesh networks for innovative solutions that can imbue business data with meaning, enabling automated processing and enhanced decision-making.

As organizations strive to unify diverse datasets in these tools and extract meaning at scale, the concept of a universal ‘semantic layer’ is gaining significant traction. These structures aim to provide a human understandable view of data stored by presenting information in meaningful business terms.

At the same time, the eXtensible Business Reporting Language (XBRL) has emerged as the pivotal standard in the realm of digital reporting and regulatory compliance, offering a structured and standardized approach to the electronic communication of business and financial data.

This article endeavours to explore the potential of leveraging XBRL as a semantic layer for business data, examining its capabilities, benefits, and limitations, and its position in relation to other semantic technologies. By delving into these aspects, this article seeks to provide a comprehensive understanding of how XBRL can contribute to a more semantically rich and intelligent ecosystem for business information.

Demystifying XBRL

XBRL, which stands for eXtensible Business Reporting Language, was designed for the exchange of business and financial data. Its core function is to automate the sharing of business information by attaching standardised tags in a report. These tags and their semantic relationships are defined in an independent XBRL taxonomy.

The taxonomy serves as a common dictionary and model (hierarchies, dimensions, etc.). Each tag in an XBRL report corresponds to a business concept provided by the taxonomy author or created as a new extension. Extensibility being a key feature of XBRL as it enables taxonomies to be generated from a set of standard taxonomies. Hence, a complex reporting model can be built up from simple building blocks.

For example, the European Single Electronic Format (ESEF) is based upon the IFRS reference taxonomy, plus technical taxonomies from XBRL International (XII). When reporting, individual companies can then add their own specific company extensions creating their own company specific ESEF extension taxonomy to file with their report. The report can then be automatically validated by XBRL processing software.

Beyond just definitions, an XBRL taxonomy includes components (linkbases) that specify various relationships, including how they should be presented (hierarchies), tables (grids), how calculation and consistency checks should be performed, associated labels for different languages, and references to authoritative literature. XBRL reports, technically called instances, also contain contexts that define the entity, period, unit, and precision of the data, e.g., millions of US dollars.

Inline XBRL (iXBRL), which embeds the XBRL data directly within an HTML document, makes these reports both human-readable and machine-readable in a single file. A key requirement for reports, which are published or printed but also where the data undergoes significant analysis, such as company annual financial and sustainability reports.

XBRL has recently further expanded its footprint with the Open Information Model (OIM) initiative, introducing XBRL-CSV and XBRL-JSON formats for more efficient and developer-friendly data representation and exchange, particularly for large datasets and web applications.

The next step in the OIM program is to simplify the definition of the XBRL taxonomy and the rules for checking its accuracy and consistency. This presents the XBRL community with the ideal opportunity to look further and to develop XBRL in the direction of being a key component in modern data stacks, i.e. part of a universal semantic layer.

What Is a Semantic Layer?

Today’s many stovepipe systems and applications using different labels and definitions for data make it extremely hard to create a unified organisational view and for transforming it into a common analysis system.

In the realm of data management and business intelligence, a semantic layer serves as an abstraction layer that translates technical data and complex layouts into meaningful business terms that are readily understandable by users. In addition, the semantic layer of a modern-day self-service business intelligence tool can provide the user with a consolidated view of data stored in multiple systems.

However, each BI tool has their own proprietary format and syntax, each data storage solution has its own access layers and data retrieval methods. While many data storage systems generate ‘data catalogs’ which are essential to understanding what data is available to be accessed and analysed, they do not contain the semantic information for the data they describe.

This is where the idea for a ‘universal’ semantic layer has developed from. It acts as a central foundation for where business concepts and related metadata are held, but also the definition of common calculations, and data metrics are standardised, ensuring that every department, from finance to marketing, works with data from a consistent view and terminology.

The emergence of tools incorporating these ideas is shown in the architecture diagram below from Dremio. It shows the vision of tools that are aimed at making these modern-day data architectures easier to manage and easier for business users to query.

A quick search reveals a number of vendors that provide standalone semantic layer platforms, i.e., that are independent of specific BI tools and enable ‘open’ access:

• AtScale: focus on providing consistent metrics, performance optimization, and streamlined governance. AtScale supports multiple modelling personas (code-first and no-code).
• Cube: offers features like data modelling, access control, caching, and APIs for integration with various endpoints. Cube also focuses on improving the accuracy of Generative AI by providing business context.
• dbt Semantic Layer (with MetricFlow): DBT is a popular data management and storage tool, its semantic layer extension allows for the central definition of key business terms and metrics using code.
• Dremio: connects to diverse data sources, providing a built-in catalog for organization, and supporting robust data governance. The Reflections feature optimizes the semantic layer for analytics and machine learning workloads by creating materialized views that accelerate query performance.
• Kyvos Insights: offers an AI-powered semantic layer designed to manage large and complex datasets with a focus on enhancing data, accelerating query performance, and providing rich data models with hierarchies and calculated measures.
• Progress Semaphore: focus on metadata management and semantic AI to create a semantic layer for a digital ecosystem.
• Veezoo: emphasizes a powerful and human-friendly Q&A experience with data. It uses a semantic modelling language (VKL) to allow business users to get instant insights through search and natural language queries.

Many traditional BI platforms, such as Microsoft Power BI, Tableau, SAP BusinessObjects, and Oracle Analytics Cloud (OAC) include semantic layer functionalities that could be developed further but today are tightly integrated with their own ecosystems. Google’s Looker also has a strong semantic modelling layer called LookML, which allows for the definition of data models and business logic which is the beginning of another option for a standalone semantic layer.

So, a typical semantic layer will include semantic model definitions, metadata management, a business logic layer for defining calculations and rules, a data access layer for managing user interaction and security, and often taxonomy and ontology management components. These components collectively enable the semantic layer to provide a simplified, consistent, and secure view of organizational data. How does XBRL compare against such a list?

Unlocking Semantic Meaning using XBRL

XBRL possesses noteworthy features to function as a semantic layer. As discussed, XBRL taxonomies establish a comprehensive semantic framework for business reporting, serving as standardized vocabularies, organising these elements in a hierarchical structure, facilitating logical grouping and navigation of information, reflecting the inherent relationships between different concepts and provision of rules to check for accuracy and consistency. This structured model forms a semantic foundation upon which financial reporting frameworks can be built.

The potential of XBRL as the basis for compliance systems and reporting frameworks is already being realised.

• National initiatives like the Standard Business Reporting (SBR) program in the Netherlands coordinate numerous government financial reporting requirements using XBRL: annual company returns, tax, statistics collection, housing corporation budgets, education sector, and commercial loan monitoring.
• The European Insurance and Occupational Pensions Authority (EIOPA) and the European Banking Authority (EBA) use the XBRL data format to define common reporting models for 27 European countries to collect a range of reports from thousands of banks and insurance companies. (Unfortunately, the EBA has started straying from the semantic model path by using abstract encoded labels from their internal database to identifying facts, negating the benefits in understanding and access. Other articles cover this issue, but hopefully they can return to the right road in the future.)
• The ESEF taxonomy, published by the European Securities Management Authority (ESMA) is a good example of combining multiple taxonomy building blocks into a single model. It is also translated into all European languages to enable European listed firms to report in their local language.

However, despite the numerous advantages of using XBRL, there are several concerns and limitations that warrant consideration:

• The complexity associated with XBRL is often cited by critics as a barrier. The taxonomies can be vast and intricate, encompassing thousands of elements and complex relationships. However, the models are complex because the nature of the domains they describe appear simple to humans but are complex at the data layer.
• Another adoption barrier often cited is the initial costs associated with implementing XBRL, including the acquisition of specialized software, providing adequate training to staff, and potentially engaging external consultants. However, seen as part of the wider cost of standardisation and digital reporting, XBRL tagging costs are relatively small and partially a result of a nascent software industry.
• Understanding and implementing the correct taxonomy for a specific industry or jurisdiction requires specialized knowledge and expertise. The skills issue is real but again can be seen as part of the growing pains of a fast-growing technology.
• Resistance to change within organizations can also impede the adoption process, as it may require significant adjustments to existing reporting workflows and processes. All part of the issues of moving to a digital world and one driven by increasing use of AI that will be faced by everybody.
• Finally, the extensibility of XBRL, while offering flexibility, can also pose challenges. The ability for companies to create custom extensions can lead to difficulties in structured analysis systems and ensuring comparability across different filers. Striking a balance between allowing necessary flexibility and maintaining a level of standardization that enables effective comparison is an ongoing challenge in the XBRL community — a ‘marmite’ issue, if you will.

My view is that these are typical technology issues. However, while XBRL significantly improves the automated validation and data analysis capabilities in the above examples and many more, it is limited today to storage and analysis mechanisms that are specifically able to treat its unique structure and syntax. This one factor is holding the adoption of XBRL back.

XII has shown that by converting XBRL reports to XBRL-JSON and linking them to tools like Tableau and LLMs, the XBRL semantic definitions can be productively used. Plus, there exists today commercial XBRL databases, such as UBPartner’ s XT Database, which can help transform volumes of XBRL reports into usable data. However, it does not appear to be enough to move the dial on the wider use of XBRL.

So how can OIM be delivered in such a way that it moves XBRL in the direction of a universal semantic layer?

XBRL in the Context of the wider Semantic Web

To fully appreciate the role of XBRL as a semantic layer, it is beneficial to compare it with other established semantic technologies, particularly those associated with the Semantic Web, such as RDF (Resource Description Framework) and OWL (Web Ontology Language). The Semantic Web is an extension of the World Wide Web that aims to make internet data machine-readable, enabling computers to understand and process information more effectively.

• RDF provides a standard model for data interchange on the Web. It allows for the creation of a network of interconnected data where the relationships between different pieces of information are explicitly defined. For example, you could use RDF to state that “Company A” “hasRevenue” “1000000 USD”.
• OWL builds upon RDF and offers a more sophisticated language for defining ontologies, which are formal representations of knowledge, outlining concepts, their properties, and the relationships between them. OWL allows for the creation of richer and more complex semantic models.

The primary use of RDF and OWL is semantic searching of large and mostly unstructured data. SPARQL Query Language is typically used for RDF data stores and allows users to specify graph patterns to match against the RDF data, enabling complex searches based on relationships and properties, enabling users to find information based on meaning rather than just keywords.

Knowledge graphs built with RDF and OWL can be navigated and searched to discover connections and insights that might not be apparent in traditional data stores. Graph traversal and querying techniques can be used to explore the relationships between entities, significantly enhancing data search. While tools like Ontopic Studio allow for the design of semantic layers based on W3C standards, providing a more knowledge graph-centric approach.

OWL supports more sophisticated logical reasoning and inference than standard XBRL. OWL’s model-theoretic semantics allows for automated consistency checking, classification, and other forms of reasoning over the data. However, in terms of adoption and ecosystem, XBRL has achieved widespread adoption within the financial reporting domain, with a mature ecosystem of regulatory mandates, software tools, and service providers.

The adoption of RDF and OWL is growing across various sectors, but the ecosystem for business is less developed. It appears that they also suffer from many of the issues identified above for XBRL. In summary, they differ as follows:

There are ongoing initiatives and approaches aimed at combining the strengths of XBRL and Semantic Web technologies. Charlie Hoffman (often cited as ‘the father’ of XBRL) has often mused over such links in his blog on Digital Reporting. These projects and ongoing initiatives underscore the growing recognition and utilisation of XBRL’s potential as a semantic layer, at least for financial data.

The question remains whether XBRL and semantic web tools can be integrated to define a universal semantic model and whether they can be used together to define Taxonomies for the new OIM specifications. Closer ties between the XBRL Standards Board (XSB) and the OWL community would appear to be beneficial.

The Future of Semantic Data with XBRL

The analysis above underscores the significant potential of XBRL to serve as a semantic layer for business data. Its inherent structure, standardized tagging system, and comprehensive taxonomies provide a robust foundation for imbuing information with meaning that transcends mere syntax.

The ongoing evolution of XBRL with new formats and the simplification of taxonomies suggests that XBRL’s use for regulatory compliance will expand and deliver even greater benefits. As AI continues to advance, we should expect growing demand for XBRL tagging, where contrary to many uninformed statements, XBRL provides the ‘syntactical’ and ‘semantic’ structure for AI to better interpret financial reports.

For the XBRL community, the OIM Taxonomy specifications will be a major break with the past, which gives the XBRL community the opportunity to address a common concern that XBRL is not yet ‘visibly’ making tasks easier, simpler, or faster. The first step, will be ensuring that OIM is a high-quality platform to support the current ambitions of the community, including more sophisticated data quality checking rules and the improved processing speed.

While the journey towards widespread and effective utilization of XBRL as a semantic layer is promising, it will not be without its challenges. It will require strong governance and continued community collaboration to realise its full potential.

Despite the above warning, I would still recommend that the XBRL community should look further ahead:

• Reviewing how XBRL could integrate into modern data stacks.
• Look to provide a truly universal sematic layer for these deployments in a more consistent and standardised way.
• Ensure that XBRL’s future includes semantic governance across taxonomies, as the example reporting frameworks mentioned above have been developed as point solutions which means the same concept is described differently by different taxonomies. New initiatives such as the ECB’s BIRD aim to build dictionaries in a more consistent way which XBRL needs to support.

The XSB should provide a long-term vision as part of the direction they take on OIM, to encourage software vendors to invest in new tools and visualisation approaches which help users build and review taxonomy models.

By addressing these challenges early in the OIM definition process and continuing to innovate, XBRL can solidify its role as a cornerstone of a more semantically intelligent future for financial reporting.

The author is Martin DeVille of AM2 Limited