1️⃣Overview The Data Dictionary serves as the backbone for structured data definitions across financials, operations, projects, compliance, and workforce domains. It ensures standardised terminology, clear metadata definitions, and structured governance for all key data entities. Key Functions: ✅ Standardised Definitions – Ensures clarity and uniformity across datasets.✅ Metadata-Driven Organisation – Supports schema validation and query … Read more
Proposal for a white paper on Common Data Model (Datamarts) Proposed Structure for the White Paper 1. Executive Summary 2. Introduction 3. Objectives 4. Proposed Standardised Format 5. Naming Conventions 6. Standardised Structure for Views 7. Metadata Format 8. Implementation Roadmap 9. Benefits 10. Conclusion Naming Suggestions for Views Here are some options for naming … Read more
In its most basic definition, sensitive data is a specific set of “special categories”: Article 9 of the GDPR prohibits special category data from being processed unless you can rely on an exemption.
Core Views and their Standard Structures Each DataMart will be built on six primary views: 1. Details (or Core) View Purpose: Schema: Column Name Data Type Description company VARCHAR Primary entity identifier object_id VARCHAR Unique object identifier (project_id) seq_json JSON Structured metadata reference for linking name VARCHAR Name of the project/entity description TEXT Brief descriptive … Read more
This draft framework provides a robust SLA for DataMart availability, balancing business needs, technical feasibility, and stakeholder communication. 1. Availability Target Set a clear uptime percentage that aligns with the criticality of the DataMart to your business processes. 2. Operational Hours Define the expected operational hours of the DataMart. Specify if it needs to be … Read more
MVC, which stands for Model-View-Controller, is a design pattern used in software engineering, particularly in web and application development. It divides an application into three interconnected components, which separates internal data logic from the user interface, allowing for more manageable and scalable code. 1. Model: 2. View: 3. Controller: Workflow Example: Benefits of MVC:
Overview: The Platinum Layer is the most advanced tier in the data warehouse, dedicated to performing complex business transformations and machine learning (ML) exercises. This layer leverages advanced analytics to generate insights that support budget, forecasting and predictive analytics. The resulting transformed data and predictive models are then returned to the Gold Layer for delivery. … Read more
Overview: The Gold Layer serves as the delivery centre of the data warehouse, providing secure and efficient access to the finalised data. This layer is responsible for delivering data to end-users and applications, ensuring that it is ready for analysis, reporting, and other business intelligence activities. Key Characteristics: Workflow Summary: Importance: RAID Risks: Issues: Dependencies: … Read more
Overview: The Silver Layer, also referred to as the ‘Staging’ SQL database, is where data undergoes further refinement and transformation. This layer focuses on structuring the data into well-defined snapshots, adhering to a common data model. The goal is to create standardized and consistent data sets that are ready for analytical purposes and easy integration … Read more
Overview: The Bronze Layer is the initial raw data storage tier in the data warehouse, representing the true start of the data lake. It is designed to store data in its raw, untransformed state, yet in an organised and accessible manner. This layer ensures that the data is preserved in its original form, allowing for … Read more