The Data Junction
Tag

data-engineering

#data-engineering

More content

Read more stories on Hashnode

Articles with this tag

Mastering Slowly Changing Dimensions (SCD) "Type 2" with Azure Data Factory: A Step-by-Step Guide

Dec 2, 20243 min read7 views

Implement SCD "Type 2" with Azure Data Factory · Introduction to Slowly Changing Dimensions (SCD) Type 2 Slowly Changing Dimensions (SCD) Type 2 is a...

Mastering Slowly Changing Dimensions (SCD) "Type 2" with Azure Data Factory: A Step-by-Step Guide

Mastering Slowly Changing Dimensions (SCD) Type 1 with Azure Data Factory: A Step-by-Step Guide

Dec 2, 20241 min read6 views

(SCD Type 1 implementation via ADF) Step 1: Setting Up Your Azure SQL Database for SCD Type 1. Create the emp_scdtype1 table in Azure SQL...

Mastering Slowly Changing Dimensions (SCD) Type 1 with Azure Data Factory: A Step-by-Step Guide

Simplifying Data Integration // Data Transformations with ADF: Merge Sources and Export to Parquet.

Dec 2, 20241 min read5 views

Step 1: Inspecting the CSV File in Data Lake and SQL Table present in Azure SQL DB Step 2: Overview of the Dataflow for the task and then we will dig...

Simplifying Data Integration // Data Transformations with ADF: Merge Sources and Export to Parquet.

Azure Data Factory: "Join" 2 or more CSV Files and Convert to JSON Format

Dec 2, 20241 min read7 views

Step 1: Inspecting the CSV Files in Data Lake: Your First Step to Data Optimization Step 2: Configuring the Data Flow Sources: Pointing to the...

Azure Data Factory: "Join" 2 or more CSV Files and Convert to JSON Format

Optimizing "Data Transfer" and "Data Transformation" in ADF: Filtering Even Customer IDs from CSV to SQL

Dec 2, 20241 min read8 views

Streamline Data Processing in ADF: Transfer Even Customer IDs from CSV to SQL · Step 1: Inspecting the CSV File in Data Lake: Your First Step to Data...

Optimizing "Data Transfer" and "Data Transformation" in ADF: Filtering Even Customer IDs from CSV to SQL

Copy data from the Data Lake to the SQL Database, deleting any existing data each time before loading the new data:

Nov 12, 20241 min read6 views

Step 1: Check the existing data in the employee table: Step 2: Change the data manually in the csv file in data lake: we need to check whether this...

Copy data from the Data Lake to the SQL Database, deleting any existing data each time before loading the new data: