The Data Junction

Write

Mastering Slowly Changing Dimensions (SCD) Type 1 with Azure Data Factory: A Step-by-Step Guide

Dec 2, 20241 min read19

Mastering DataFlow Techniques in Azure Data Factory with a Data Transformation example:

Dec 2, 20241 min read13

Mastering Slowly Changing Dimensions (SCD) "Type 2" with Azure Data Factory: A Step-by-Step Guide

Implement SCD "Type 2" with Azure Data Factory

Dec 2, 20243 min read659

Simplifying Data Integration // Data Transformations with ADF: Merge Sources and Export to Parquet.

Dec 2, 20241 min read15

Azure Data Factory: "Join" 2 or more CSV Files and Convert to JSON Format

Dec 2, 20241 min read18

Optimizing "Data Transfer" and "Data Transformation" in ADF: Filtering Even Customer IDs from CSV to SQL

Streamline Data Processing in ADF: Transfer Even Customer IDs from CSV to SQL

Dec 2, 20241 min read8

Copy data from the Data Lake to the SQL Database, deleting any existing data each time before loading the new data:

Step 1: Check the existing data in the employee table: Step 2: Change the data manually in the csv file in data lake: we need to check whether this change will be visible in the table or not (definitely at the end). Step 3: Settings in “Sink” opti...

Nov 12, 20241 min read11

Data Lake to SQL DB Data Movement ( or CSV to SQL Table data movement)

(We will ensure that table is automatically created!)

Nov 12, 20241 min read12

Important Business Use Case - Solved by ADF (Azure Data Factory)

“Use Case”: Whenever we have more than 100 records in the customer table, we copy the customer data to another table customer_copy within the SQL DB. However, whenever we do this copy, we first truncate the table 'customer_copy' and then copy the dat...

Oct 24, 20241 min read13

How to Transfer All SQL Database Tables to Azure Data Lake in One Go?

Step 1: Create an instance of Azure Data Factory: Step 2: Set up Linked Services: Step 3: Create a dataset for both (source-SQL Database and destination-Azure Data Lake): Step 4: Build the Data Pipeline with the help of datasets: First I will us...

Oct 24, 20241 min read26

How to Copy SQL Database Tables Post-Join in Azure Data Lake via ADF!!

Steps to Copy Joined SQL Tables into Azure Data Lake Using ADF

Oct 23, 20241 min read5

Command Palette

Latest articles