Objective: The Snapshot pipeline aims to streamline data analysis and management by creating optimised views from raw data files. Process Overview: Loading parquet files and Delta lake table Creation: View Creation API Access and loading into the data warehouse Conclusion: The Snapshot pipeline offers an efficient solution for transforming raw data into actionable insights. By … Read more
Purpose The Common View for Resource Details, excluding dates or measures, acts as a unified and reliable source for data on project resources. It is accessible to different departments and systems across the organisation. This view supports integration with fact tables for reporting and analysis and provides a limited set of dimension attributes. Use Case … Read more
Purpose The Supplier Invoice (value) fact table is designed to display quantitative data related to invoices generated. This table integrates with dimension tables, such as supplier_details, to provide detailed insights into the financial aspects of projects, including invoicing, revenue, and billing trends. The primary purpose is to support financial analysis and reporting across the organisation. Use … Read more
Efficient data storage and processing are crucial for businesses and organizations dealing with large datasets. Apache Parquet is a popular columnar storage format offering fast query performance and data compression, while CSV is a row-based format that may not be suitable for large-scale processing. This blog post covers how to convert CSV files to Parquet … Read more
The merge() function in Pandas is a powerful tool for combining two or more dataframes based on one or more keys. It is analogous to the JOIN operation in SQL databases and offers various options to customize the merge behavior. Here’s the basic syntax of the merge() function: pandas.merge(left, right, how=’inner’, on=None, left_on=None, right_on=None, left_index=False, … Read more
Purpose Customer Invoice, provides high-level invoice details primarily focused on essential customer and financial information. It is structured to include invoice metadata, billing details, amounts, and various identifiers. The view originates from multiple snapshots and iterations, with each change tracked meticulously to support functionality and flexibility for different reporting and integration requirements. Use Case Owner: Soon … Read more
Purpose The Customer Order Line view provides essential details for individual order lines within customer orders. It serves as a granular component, detailing each line item’s specifics, such as quantity, pricing, revenue, dates, and relevant codes. Designed to align closely with the Customer Order view, this view enables precise tracking of each line item in … Read more
Copy plugins/my/ includes/classes/fields_types
Purpose for Live order intake and Pipeline Review Business Opportunity Line Details Use Case Owner: Soon TanTypical Use Case: Live order intake and Pipeline ReviewStyle: Tailored Fact Based on mvw.busopp_details_plus DataMarts CompanyCustomerBusiness Opportunity Source From Where Sample Exec Response SQL
Purpose The Project Forecast fact table is designed to display projected data related to future project performance, including anticipated costs, revenues, timelines, and resource requirements. This table integrates with dimension tables, such as Project_Details, to provide detailed insights into future planning and strategic decision-making. The primary purpose is to support forecasting, budgeting, and proactive project … Read more