databricks Archives - Saman Ahmadi

Azure Key Vault Secret Updates using Logic App

Posted on 13.09.2021

Azure Key Vault Secret updates using Logic App are challenging as there are no connectors, but using the API is a secure and managed way to do it. Here is some background information on the use case. During my recent data and integration project, the team used Azure Logic apps to batch update and processed data to Salesforce. The Salesforce API requires a pre-defined schema for batch updating its objects. The ETL pipe using Azure Databricks makes the data ready…

DataBricks Automation with Azure DevOps

Posted on 01.06.2020

Databricks on Azure is essential in data, AI and IoT solutions, but the env. automation can be challenging. Azure DevOps is a great tool for automation. Using Pipelines and product CLI integrations can minimise or even remove these challenges. My team is currently working on a cutting edge IoT platform where data flows from edge devices to Azure. We are dealing with data which is sensitive, and under GDPR so no one should have direct access to the data platform…