https://pprakash.me/tags/virtual-network/ Recent content in Virtual Network on Technology Enlightened Hugo en &copy; <a href="https://github.com/p-prakash/">Prakash P</a> 2015 - 2023 Mon, 11 Dec 2023 18:00:00 +0100 https://pprakash.me/tech/2023/12/11/scale-azure-databricks-secure-network-access-to-azure-data-lake/ Mon, 11 Dec 2023 18:00:00 +0100 https://pprakash.me/tech/2023/12/11/scale-azure-databricks-secure-network-access-to-azure-data-lake/ <p>Many organisations store their data in a centralised data lake on Azure Storage accounts. In this post we will look at a solution to scale the secure network access between the centralised data lake storage account and Azure Databricks workspaces VNets.</p> <p><img src="https://pprakash.me/images/2023/12/Databricks_Network_Design.png" alt="Architecture diagram of having storage account with Private Endpoint connected to Azure Databricks workspaces VNets"></p> <p>The recommended approach to achieve a scalable and secure network access to the data lake storage is to use Private Endpoints for the storage accounts.</p>