Mastering Slowly Changing Dimensions (SCD) "Type 2" with Azure Data Factory: A Step-by-Step Guide
Implement SCD "Type 2" with Azure Data Factory
The Data Junction
Latest articles
Mastering Slowly Changing Dimensions (SCD) Type 1 with Azure Data Factory: A Step-by-Step Guide
Mastering DataFlow Techniques in Azure Data Factory with a Data Transformation example:
Simplifying Data Integration // Data Transformations with ADF: Merge Sources and Export to Parquet.
Azure Data Factory: "Join" 2 or more CSV Files and Convert to JSON Format
Optimizing "Data Transfer" and "Data Transformation" in ADF: Filtering Even Customer IDs from CSV to SQL
Streamline Data Processing in ADF: Transfer Even Customer IDs from CSV to SQL
Copy data from the Data Lake to the SQL Database, deleting any existing data each time before loading the new data:
Step 1: Check the existing data in the employee table: Step 2: Change the data manually in the csv file in data lake: we need to check whether this change will be visible in the table or not (definitely at the end). Step 3: Settings in “Sink” opti...
Data Lake to SQL DB Data Movement ( or CSV to SQL Table data movement)
(We will ensure that table is automatically created!)
Important Business Use Case - Solved by ADF (Azure Data Factory)
“Use Case”: Whenever we have more than 100 records in the customer table, we copy the customer data to another table customer_copy within the SQL DB. However, whenever we do this copy, we first truncate the table 'customer_copy' and then copy the dat...
How to Transfer All SQL Database Tables to Azure Data Lake in One Go?
Step 1: Create an instance of Azure Data Factory: Step 2: Set up Linked Services: Step 3: Create a dataset for both (source-SQL Database and destination-Azure Data Lake): Step 4: Build the Data Pipeline with the help of datasets: First I will us...
How to Copy SQL Database Tables Post-Join in Azure Data Lake via ADF!!
Steps to Copy Joined SQL Tables into Azure Data Lake Using ADF