The goal of this article is to:
- Showcase Microsoft Fabric potential
- Propose a SHIR release notes automation mechanism for notifications
For Azure Data Factory (and Synapse Analytics) users, keeping up with SHIR (Self-Hosted Integration Runtime) notes might be a cumbersome task. It involves downloading a .doc file, costing a couple of clicks, through the following url: https://www.microsoft.com/en-us/download/details.aspx?id=39717
Having this in mind, I have created an automation to send me an email, every time (dependent on the schedule of the pipeline) a new version is released, so that I keep track of changes and also to be informed about new versions.
To do this, I developed a FDF (Fabric Data Factory) Data pipeline which:
through a notebook, downloads the .doc file to a lakehouse, decides whether this is a new file or not based on its hash, and in the case that it is a new file, the file is parsed and the content is then distributed via email.
The content distribution can be achieved through any (or both) of these activities:
- https://learn.microsoft.com/en-us/fabric/data-factory/outlook-activity
- https://learn.microsoft.com/en-us/fabric/data-factory/teams-activity
I share this solution on my github page and numerous changes could be done and proposed, be it naming of the activities, or the way in which some pyspark (python) has been written to store metadata, to name an example.
I cannot take full credit for the code, as I relied on copilot to help me.
The solution
You may find the steps and files to import this template into your workspace in this repository: acsilva96/shir-releases
I hope you find use to this template, any feedback is more than welcome!