> For the complete documentation index, see [llms.txt](https://azure.kasinathanramesh.vip/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://azure.kasinathanramesh.vip/azure-databricks/deployment-steps.md).

# Deployment Steps

## Deploy Azure Databricks in your Azure virtual network (VNet injection) <a href="#deploy-azure-databricks-in-your-azure-virtual-network-vnet-injection" id="deploy-azure-databricks-in-your-azure-virtual-network-vnet-injection"></a>

### Sign in to the Azure portal - <https://portal.azure.com/> <a href="#sign-in-to-the-azure-portal" id="sign-in-to-the-azure-portal"></a>

### Create a virtual network <a href="#create-a-virtual-network" id="create-a-virtual-network"></a>

Steps 1: From the Azure portal menu, select **Create a resource**. Then select **Networking > Virtual network**.

<figure><img src="/files/zspiaFISumqp4On8aanQ" alt=""><figcaption></figcaption></figure>

Steps 2: Under **Create virtual network**, apply the following settings:

| Setting        | Suggested value                                    | Description                                                                                                          |
| -------------- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Subscription   | \<Your subscription>                               | Select the Azure subscription that you want to use.                                                                  |
| Resource group | databricks-project                                 | Select **Create New** and enter a new resource group name for your account.                                          |
| Name           | databricks-vnet                                    | Select a name for your virtual network.                                                                              |
| Region         | \<Select the region that is closest to your users> | Select a geographic location where you can host your virtual network. Use the location that's closest to your users. |

<figure><img src="/files/ys55yu8SuRtU08YyVVSs" alt=""><figcaption></figcaption></figure>

Select **Next: IP Addresses >** and apply the following settings. Then select **Review + create**.

| Setting              | Suggested value | Description                                                                                                                                                                   |
| -------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| IPv4 address space   | 10.26.1.0/24    | The virtual network's address range in CIDR notation. The CIDR range must be between /16 and /24                                                                              |
| Subnet name          | default         | Select a name for the default subnet in your virtual network.                                                                                                                 |
| Subnet Address range | 10.26.1.0/24    | The subnet's address range in CIDR notation. It must be contained by the address space of the virtual network. The address range of a subnet which is in use can't be edited. |

<br>

<figure><img src="/files/bV1CGPiRndE714aA5jx0" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/IilJ4T4hXtWjuXbwgyhs" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/FtKNI6200KaZTVjqdokG" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/Wz5ahhGpjPjWrz4EIKRc" alt=""><figcaption></figcaption></figure>

### Create an Azure Databricks workspace <a href="#create-an-azure-databricks-workspace" id="create-an-azure-databricks-workspace"></a>

From the Azure portal menu, select **Create a resource**. Then select **Analytics > Databricks**.

<figure><img src="/files/OzMgS1sD17FgLQr3M3mW" alt=""><figcaption></figcaption></figure>

Under **Azure Databricks Service**, apply the following settings:

| Setting        | Suggested value                                    | Description                                                                                                                        |
| -------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| Workspace name | databricks-workspace                               | Select a name for your Azure Databricks workspace.                                                                                 |
| Subscription   | \<Your subscription>                               | Select the Azure subscription that you want to use.                                                                                |
| Resource group | databricks-project                                 | Select the same resource group you used for the virtual network.                                                                   |
| Location       | \<Select the region that is closest to your users> | Choose the same location as your virtual network.                                                                                  |
| Pricing Tier   | Choose between Standard or Premium.                | For more information on pricing tiers, see the [Databricks pricing page](https://azure.microsoft.com/pricing/details/databricks/). |

<figure><img src="/files/8zVU8srbnpe8iui07SfY" alt=""><figcaption></figcaption></figure>

Once you've finished entering settings on the **Basics** page, select **Next: Networking >** and apply the following settings:

| Setting                                                          | Suggested value                                         | Description                                                                              |
| ---------------------------------------------------------------- | ------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Deploy Azure Databricks workspace in your Virtual Network (VNet) | Yes                                                     | This setting allows you to deploy an Azure Databricks workspace in your virtual network. |
| Virtual Network                                                  | databricks-quickstart                                   | Select the virtual network you created in the previous section.                          |
| Public Subnet Name                                               | [databricks-public-subnet](https://portal.azure.com/#)  | Use the default public subnet name.                                                      |
| Public Subnet CIDR Range                                         | 10.26.1.0/25                                            | Use a CIDR range up to and including /25.                                                |
| Private Subnet Name                                              | [databricks-private-subnet](https://portal.azure.com/#) | Use the default private subnet name.                                                     |
| Private Subnet CIDR Range                                        | 10.26.1.128/25                                          | Use a CIDR range up to and including /25.                                                |

<figure><img src="/files/LbXMjEDaF6wP8PHCIkWd" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/MX7F4DAyJvVvn233ImAp" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/tgLCr0LuTjgZ64vAB2g2" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/1CMSZqOBNdkZKokqoOMb" alt=""><figcaption></figcaption></figure>

Once the deployment is complete, navigate to the Azure Databricks resource. Notice that virtual network peering is disabled. Also notice the resource group and managed resource group in the overview page.

<figure><img src="/files/2ENnJ9OWpaOcFHbIh6Jw" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/Dp7VUnBJoZ32bXQvCge0" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/iHnxpsLbz78d75PHCGXd" alt=""><figcaption></figcaption></figure>

The managed resource group is not modifiable, and it is not used to create virtual machines. You can only create virtual machines in the resource group you manage.

<figure><img src="/files/xXn3VSMijqjLyr2zuc77" alt=""><figcaption></figcaption></figure>

When a workspace deployment fails, the workspace is still created in a failed state. Delete the failed workspace and create a new workspace that resolves the deployment errors. When you delete the failed workspace, the managed resource group and any successfully deployed resources are also deleted.

<figure><img src="/files/GWUTzBVqXCMFRLOuiDRc" alt=""><figcaption></figcaption></figure>

### Create a cluster <a href="#create-a-cluster" id="create-a-cluster"></a>

1. Return to your Azure Databricks service and select **Launch Workspace** on the **Overview** page.

Select **Clusters** > **+ Create Cluster**. Then create a cluster name, like *databricks-project-cluster*, and accept the remaining default settings. Select **Create Cluster**.

<figure><img src="/files/6WhkHOjWCaL7zSchBSeE" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/pycuCM2FbTcYDytjF5hk" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/eJ2OnGxDa5vaCoyqqQPq" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/ecpeGdoQiBAweHbAc3nb" alt=""><figcaption></figcaption></figure>

Once the cluster is running, return to the managed resource group in the Azure portal. Notice the new virtual machines, disks, IP Address, and network interfaces. A network interface is created in each of the public and private subnets with IP addresses.

<figure><img src="/files/pJth7qj3t9nLQcHZrKRh" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/6mNRPV6fEXHIZy1INUxs" alt=""><figcaption></figcaption></figure>

Return to your Azure Databricks workspace and select the cluster you created. Then navigate to the **Executors** tab on the **Spark UI** page. Notice that the addresses for the driver and the executors are in the private subnet range. In this example, the driver is 10.26.1.132 and executors are 10.26.1.133 and 10.26.1.134 and Your IP addresses could be different.

<figure><img src="/files/StY92PtChQGmmA0dc3lV" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/gz3DxtGVfoUojReUolKD" alt=""><figcaption></figcaption></figure>

### Clean up resources <a href="#clean-up-resources" id="clean-up-resources"></a>

After you have finished the article, you can terminate the cluster. To do so, from the Azure Databricks workspace, from the left pane, select **Clusters**. For the cluster you want to terminate, move the cursor over the ellipsis under **Actions** column, and select the **Terminate** icon. This stops the cluster.

If you do not manually terminate the cluster it will automatically stop, provided you selected the **Terminate after \_\_ minutes of inactivity** checkbox while creating the cluster. In such a case, the cluster automatically stops, if it has been inactive for the specified time.

If you do not wish to reuse the cluster, you can delete the resource group you created in the Azure portal.

{% embed url="<https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/vnet-inject>" %}

{% embed url="<https://www.databricks.com/blog/2020/02/28/securely-accessing-azure-data-sources-from-azure-databricks.html>" %}

{% embed url="<https://medium.com/@antaliagacortes/restricting-traffic-from-azure-databricks-055e062dc2f5>" %}

{% embed url="<https://www.databricks.com/blog/data-exfiltration-protection-with-azure-databricks>" %}

<mark style="color:red;">**`Creating a private link for DBFS root storage`**</mark>

{% embed url="<https://community.databricks.com/t5/administration-architecture/creating-a-private-link-for-dbfs-root-storage/td-p/50099>" %} <mark style="color:purple;">**`The term DBFS comes from Databricks File System`**</mark>
{% endembed %}

{% embed url="<https://learn.microsoft.com/en-us/azure/databricks/resources/supported-regions#control-plane-ip-addresses>" %}

{% embed url="<https://learn.microsoft.com/en-us/azure/databricks/dbfs/dbfs-root>" %}

{% embed url="<https://learn.microsoft.com/en-us/azure/databricks/admin/workspace-settings/notebook-results>" %}

## Data Exfiltration Protection with Azure Databricks

{% embed url="<https://www.databricks.com/blog/data-exfiltration-protection-with-azure-databricks>" %}

<figure><img src="/files/62QPnbmSjNe5W1ss12kX" alt=""><figcaption></figcaption></figure>