Author: Alexandre Cloud Silva
-
Data Factory Databricks Job activity parameterization
—
by
With Databricks Jobs being possible to be called from Azure Data Factory (not available in Azure Synapse Analytics as time of writing: November 2025), and being already Generally Available, one concern which might arise is working with JobIDs parameterization when promoting code to higher environments. Purely, as an observation, when using this activity, the Job…
-
Fabric Airflow Job Sizing and Pricing: Understanding Pools and SKUs
—
by
The goal of this article is not to explore (in depth) what Airflow is. Instead, is to provide insights on a relationship between capacity SKU and Airflow pool types. Capacity planning and forecast is a very important so that you don’t risk throttling it. In the case of Airflow jobs (pricing), you pay for the…
-
Fabric Mirroring: Data Retention
—
by
Until recently, Fabric Mirrored Databases data retention (OneLake) was of 7 days (default) and it was not possible to change this property. Fabric May 2025 Feature Summary | Microsoft Fabric Blog | Microsoft Fabric As of today, the default retention period for such artifacts created after June 2025 is of 1 day. And you can…
-
SHIR release notes notification automation
—
by
The goal of this article is to: For Azure Data Factory (and Synapse Analytics) users, keeping up with SHIR (Self-Hosted Integration Runtime) notes might be a cumbersome task. It involves downloading a .doc file, costing a couple of clicks, through the following url: https://www.microsoft.com/en-us/download/details.aspx?id=39717 Having this in mind, I have created an automation to send…
-
Trigger VS Debug
—
by
Azure Data Factory (ADF) offers 2 ways of executing a pipeline: Selecting Debug actually runs the pipeline. For example, if the pipeline contains copy activity, the test run copies data from source to destination. As a result, we recommend that you use test folders in your copy activities and other activities when debugging. After you’ve debugged the…