feature: databricks sync

This commit is contained in:
Scott Wilson
2025-02-10 18:38:27 -08:00
parent 4e48ab1eeb
commit 4de8888843
105 changed files with 1548 additions and 173 deletions

View File

@@ -0,0 +1,4 @@
---
title: "Available"
openapi: "GET /api/v1/app-connections/databricks/available"
---

View File

@@ -0,0 +1,4 @@
---
title: "Create"
openapi: "POST /api/v1/app-connections/databricks"
---

View File

@@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/app-connections/databricks/{connectionId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/app-connections/databricks/{connectionId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/app-connections/databricks/connection-name/{connectionName}"
---

View File

@@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/app-connections/databricks"
---

View File

@@ -0,0 +1,4 @@
---
title: "Update"
openapi: "PATCH /api/v1/app-connections/databricks/{connectionId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Create"
openapi: "POST /api/v1/secret-syncs/databricks"
---

View File

@@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/secret-syncs/databricks/{syncId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/secret-syncs/databricks/{syncId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/secret-syncs/databricks/sync-name/{syncName}"
---

View File

@@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/secret-syncs/databricks"
---

View File

@@ -0,0 +1,4 @@
---
title: "Remove Secrets"
openapi: "POST /api/v1/secret-syncs/databricks/{syncId}/remove-secrets"
---

View File

@@ -0,0 +1,4 @@
---
title: "Sync Secrets"
openapi: "POST /api/v1/secret-syncs/databricks/{syncId}/sync-secrets"
---

View File

@@ -0,0 +1,4 @@
---
title: "Update"
openapi: "PATCH /api/v1/secret-syncs/databricks/{syncId}"
---

Binary file not shown.

After

Width:  |  Height:  |  Size: 392 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 774 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 446 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 743 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 512 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 410 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 363 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 808 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 792 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 853 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 827 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 778 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 804 KiB

View File

@@ -62,7 +62,7 @@ Infisical currently only supports one method for connecting to Azure, which is O
## Setup Azure Connection in Infisical
<Steps>
<Step title="Navigate to the App Connections">
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings** page. ![App Connections
Tab](/images/app-connections/general/add-connection.png)
</Step>

View File

@@ -61,7 +61,7 @@ Infisical currently only supports one method for connecting to Azure, which is O
## Setup Azure Connection in Infisical
<Steps>
<Step title="Navigate to the App Connections">
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings** page. ![App Connections
Tab](/images/app-connections/general/add-connection.png)
</Step>

View File

@@ -0,0 +1,64 @@
---
title: "Databricks Connection"
description: "Learn how to configure a Databricks Connection for Infisical."
---
Infisical supports the use of [service principals](https://docs.databricks.com/en/admin/users-groups/service-principals.html) to connect with your Databricks workspaces.
## Configure a Service Principal for Infisical
<Steps>
<Step title="Databricks Workspace Settings">
Navigate to your Databricks Workspace **Settings** via the dropdown in the top right.
![Workspace Settings Page](/images/app-connections/databricks/workspace-settings.png)
</Step>
<Step title="Manage Service Principals">
Under the **Identity & Access** tab, click the **Manage** button in the **Service Principals** section.
![Manage Service Principals](/images/app-connections/databricks/manage-service-principals.png)
</Step>
<Step title="Service Principal Management">
Click the **Add Service Principal** button.
![Add Service Principal](/images/app-connections/databricks/add-service-principal.png)
</Step>
<Step title="Add Service Principal">
Select the **Add New** option and create a service principal for Infisical.
![Create Service Principal](/images/app-connections/databricks/create-service-principal.png)
</Step>
<Step title="Generate Service Principal Secret">
Click on your new service principal, select the **Secrets** tab and click the **Generate Secret** button.
![Generate Secret](/images/app-connections/databricks/service-principal-secrets.png)
</Step>
<Step title="Service Principal Secret">
Copy your service principal **Secret** and **Client ID** for use in the following steps.
![Generate Secret](/images/app-connections/databricks/service-principal-ids.png)
</Step>
</Steps>
## Setup Databricks Connection in Infisical
<Steps>
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings**
page. ![App Connections
Tab](/images/app-connections/general/add-connection.png)
</Step>
<Step title="Add Connection">
Select the **Databricks Connection** option from the connection options modal.
![Select Databricks
Connection](/images/app-connections/databricks/select-databricks-connection.png)
</Step>
<Step title="Authorize Connection">
Select the **Service Principal** method, add your **workspace URL** and **service principal credentials**, then click **Connect to
Databricks**. ![Connect via Databricks
service principal](/images/app-connections/databricks/create-databricks-service-principal-method.png)
</Step>
<Step title="Connection Created">
Your **Databricks Connection** is now available for use. ![Databricks Service Principal
Connection](/images/app-connections/databricks/databricks-service-principal-connection.png)
</Step>
</Steps>

View File

@@ -81,7 +81,7 @@ Infisical supports [service account impersonation](https://cloud.google.com/iam/
## Setup GCP Connection in Infisical
<Steps>
<Step title="Navigate to the App Connections">
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings**
page. ![App Connections
Tab](/images/app-connections/general/add-connection.png)

View File

@@ -69,7 +69,7 @@ Infisical supports two methods for connecting to GitHub.
## Setup GitHub Connection in Infisical
<Steps>
<Step title="Navigate to the App Connections">
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings** page.
![App Connections Tab](/images/app-connections/general/add-connection.png)
</Step>
@@ -135,7 +135,7 @@ Infisical supports two methods for connecting to GitHub.
## Setup GitHub Connection in Infisical
<Steps>
<Step title="Navigate to the App Connections">
<Step title="Navigate to App Connections">
Navigate to the **App Connections** tab on the **Organization Settings** page.
![App Connections Tab](/images/app-connections/general/add-connection.png)
</Step>

View File

@@ -0,0 +1,140 @@
---
title: "Databricks Sync"
description: "Learn how to configure a Databricks Sync for Infisical."
---
**Prerequisites:**
- Set up and add secrets to [Infisical Cloud](https://app.infisical.com)
- Create a [Databricks Connection](/integrations/app-connections/databricks)
<Tabs>
<Tab title="Infisical UI">
1. Navigate to **Project** > **Integrations** and select the **Secret Syncs** tab. Click on the **Add Sync** button.
![Secret Syncs Tab](/images/secret-syncs/general/secret-sync-tab.png)
2. Select the **Databricks** option.
![Select Databricks](/images/secret-syncs/databricks/select-databricks-option.png)
3. Configure the **Source** from where secrets should be retrieved, then click **Next**.
![Configure Source](/images/secret-syncs/databricks/databricks-source.png)
- **Environment**: The project environment to retrieve secrets from.
- **Secret Path**: The folder path to retrieve secrets from.
<Tip>
If you need to sync secrets from multiple folder locations, check out [secret imports](/documentation/platform/secret-reference#secret-imports).
</Tip>
4. Configure the **Destination** to where secrets should be deployed, then click **Next**.
![Configure Destination](/images/secret-syncs/databricks/databricks-destination.png)
- **Databricks Connection**: The Databricks Connection to authenticate with.
- **Scope**: The Databricks secret scope to sync secrets to.
<Note>
You must create a secret scope in your Databricks workspace prior to configuration. Ensure your service principal has [Write permissions](https://docs.databricks.com/en/security/auth/access-control/index.html#secret-acls) for the specified secret scope.
</Note>
5. Configure the **Sync Options** to specify how secrets should be synced, then click **Next**.
![Configure Options](/images/secret-syncs/databricks/databricks-options.png)
- **Initial Sync Behavior**: Determines how Infisical should resolve the initial sync.
- **Overwrite Destination Secrets**: Removes any secrets at the destination endpoint not present in Infisical.
<Note>
Databricks does not support importing secrets.
</Note>
- **Auto-Sync Enabled**: If enabled, secrets will automatically be synced from the source location when changes occur. Disable to enforce manual syncing only.
6. Configure the **Details** of your Databricks Sync, then click **Next**.
![Configure Details](/images/secret-syncs/databricks/databricks-details.png)
- **Name**: The name of your sync. Must be slug-friendly.
- **Description**: An optional description for your sync.
7. Review your Databricks Sync configuration, then click **Create Sync**.
![Confirm Configuration](/images/secret-syncs/databricks/databricks-review.png)
8. If enabled, your Databricks Sync will begin syncing your secrets to the destination endpoint.
![Sync Secrets](/images/secret-syncs/databricks/databricks-created.png)
</Tab>
<Tab title="API">
To create an **Databricks Sync**, make an API request to the [Create Databricks Sync](/api-reference/endpoints/secret-syncs/databricks/create) API endpoint.
### Sample request
```bash Request
curl --request POST \
--url https://app.infisical.com/api/v1/secret-syncs/databricks \
--header 'Content-Type: application/json' \
--data '{
"name": "my-databricks-sync",
"projectId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"description": "an example sync",
"connectionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"environment": "dev",
"secretPath": "/my-secrets",
"isEnabled": true,
"syncOptions": {
"initialSyncBehavior": "overwrite-destination"
},
"destinationConfig": {
"scope": "my-scope"
}
}'
```
### Sample response
```bash Response
{
"secretSync": {
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"name": "my-databricks-sync",
"description": "an example sync",
"isEnabled": true,
"version": 1,
"folderId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"connectionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"createdAt": "2023-11-07T05:31:56Z",
"updatedAt": "2023-11-07T05:31:56Z",
"syncStatus": "succeeded",
"lastSyncJobId": "123",
"lastSyncMessage": null,
"lastSyncedAt": "2023-11-07T05:31:56Z",
"importStatus": null,
"lastImportJobId": null,
"lastImportMessage": null,
"lastImportedAt": null,
"removeStatus": null,
"lastRemoveJobId": null,
"lastRemoveMessage": null,
"lastRemovedAt": null,
"syncOptions": {
"initialSyncBehavior": "overwrite-destination"
},
"projectId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"connection": {
"app": "databricks",
"name": "my-databricks-connection",
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
},
"environment": {
"slug": "dev",
"name": "Development",
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
},
"folder": {
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"path": "/my-secrets"
},
"destination": "databricks",
"destinationConfig": {
"scope": "my-scope"
}
}
}
```
</Tab>
</Tabs>

View File

@@ -392,10 +392,11 @@
"group": "Connections",
"pages": [
"integrations/app-connections/aws",
"integrations/app-connections/github",
"integrations/app-connections/gcp",
"integrations/app-connections/azure-app-configuration",
"integrations/app-connections/azure-key-vault",
"integrations/app-connections/azure-app-configuration"
"integrations/app-connections/databricks",
"integrations/app-connections/gcp",
"integrations/app-connections/github"
]
}
]
@@ -409,10 +410,11 @@
"pages": [
"integrations/secret-syncs/aws-parameter-store",
"integrations/secret-syncs/aws-secrets-manager",
"integrations/secret-syncs/github",
"integrations/secret-syncs/gcp-secret-manager",
"integrations/secret-syncs/azure-app-configuration",
"integrations/secret-syncs/azure-key-vault",
"integrations/secret-syncs/azure-app-configuration"
"integrations/secret-syncs/databricks",
"integrations/secret-syncs/gcp-secret-manager",
"integrations/secret-syncs/github"
]
}
]
@@ -825,27 +827,15 @@
]
},
{
"group": "GitHub",
"group": "Azure App Configuration",
"pages": [
"api-reference/endpoints/app-connections/github/list",
"api-reference/endpoints/app-connections/github/available",
"api-reference/endpoints/app-connections/github/get-by-id",
"api-reference/endpoints/app-connections/github/get-by-name",
"api-reference/endpoints/app-connections/github/create",
"api-reference/endpoints/app-connections/github/update",
"api-reference/endpoints/app-connections/github/delete"
]
},
{
"group": "GCP",
"pages": [
"api-reference/endpoints/app-connections/gcp/list",
"api-reference/endpoints/app-connections/gcp/available",
"api-reference/endpoints/app-connections/gcp/get-by-id",
"api-reference/endpoints/app-connections/gcp/get-by-name",
"api-reference/endpoints/app-connections/gcp/create",
"api-reference/endpoints/app-connections/gcp/update",
"api-reference/endpoints/app-connections/gcp/delete"
"api-reference/endpoints/app-connections/azure-app-configuration/list",
"api-reference/endpoints/app-connections/azure-app-configuration/available",
"api-reference/endpoints/app-connections/azure-app-configuration/get-by-id",
"api-reference/endpoints/app-connections/azure-app-configuration/get-by-name",
"api-reference/endpoints/app-connections/azure-app-configuration/create",
"api-reference/endpoints/app-connections/azure-app-configuration/update",
"api-reference/endpoints/app-connections/azure-app-configuration/delete"
]
},
{
@@ -861,15 +851,39 @@
]
},
{
"group": "Azure App Configuration",
"group": "Databricks",
"pages": [
"api-reference/endpoints/app-connections/azure-app-configuration/list",
"api-reference/endpoints/app-connections/azure-app-configuration/available",
"api-reference/endpoints/app-connections/azure-app-configuration/get-by-id",
"api-reference/endpoints/app-connections/azure-app-configuration/get-by-name",
"api-reference/endpoints/app-connections/azure-app-configuration/create",
"api-reference/endpoints/app-connections/azure-app-configuration/update",
"api-reference/endpoints/app-connections/azure-app-configuration/delete"
"api-reference/endpoints/app-connections/databricks/list",
"api-reference/endpoints/app-connections/databricks/available",
"api-reference/endpoints/app-connections/databricks/get-by-id",
"api-reference/endpoints/app-connections/databricks/get-by-name",
"api-reference/endpoints/app-connections/databricks/create",
"api-reference/endpoints/app-connections/databricks/update",
"api-reference/endpoints/app-connections/databricks/delete"
]
},
{
"group": "GCP",
"pages": [
"api-reference/endpoints/app-connections/gcp/list",
"api-reference/endpoints/app-connections/gcp/available",
"api-reference/endpoints/app-connections/gcp/get-by-id",
"api-reference/endpoints/app-connections/gcp/get-by-name",
"api-reference/endpoints/app-connections/gcp/create",
"api-reference/endpoints/app-connections/gcp/update",
"api-reference/endpoints/app-connections/gcp/delete"
]
},
{
"group": "GitHub",
"pages": [
"api-reference/endpoints/app-connections/github/list",
"api-reference/endpoints/app-connections/github/available",
"api-reference/endpoints/app-connections/github/get-by-id",
"api-reference/endpoints/app-connections/github/get-by-name",
"api-reference/endpoints/app-connections/github/create",
"api-reference/endpoints/app-connections/github/update",
"api-reference/endpoints/app-connections/github/delete"
]
}
]
@@ -908,30 +922,17 @@
]
},
{
"group": "GitHub",
"group": "Azure App Configuration",
"pages": [
"api-reference/endpoints/secret-syncs/github/list",
"api-reference/endpoints/secret-syncs/github/get-by-id",
"api-reference/endpoints/secret-syncs/github/get-by-name",
"api-reference/endpoints/secret-syncs/github/create",
"api-reference/endpoints/secret-syncs/github/update",
"api-reference/endpoints/secret-syncs/github/delete",
"api-reference/endpoints/secret-syncs/github/sync-secrets",
"api-reference/endpoints/secret-syncs/github/remove-secrets"
]
},
{
"group": "GCP Secret Manager",
"pages": [
"api-reference/endpoints/secret-syncs/gcp-secret-manager/list",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/get-by-id",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/get-by-name",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/create",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/update",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/delete",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/sync-secrets",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/import-secrets",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/remove-secrets"
"api-reference/endpoints/secret-syncs/azure-app-configuration/list",
"api-reference/endpoints/secret-syncs/azure-app-configuration/get-by-id",
"api-reference/endpoints/secret-syncs/azure-app-configuration/get-by-name",
"api-reference/endpoints/secret-syncs/azure-app-configuration/create",
"api-reference/endpoints/secret-syncs/azure-app-configuration/update",
"api-reference/endpoints/secret-syncs/azure-app-configuration/delete",
"api-reference/endpoints/secret-syncs/azure-app-configuration/sync-secrets",
"api-reference/endpoints/secret-syncs/azure-app-configuration/import-secrets",
"api-reference/endpoints/secret-syncs/azure-app-configuration/remove-secrets"
]
},
{
@@ -949,20 +950,45 @@
]
},
{
"group": "Azure App Configuration",
"group": "Databricks",
"pages": [
"api-reference/endpoints/secret-syncs/azure-app-configuration/list",
"api-reference/endpoints/secret-syncs/azure-app-configuration/get-by-id",
"api-reference/endpoints/secret-syncs/azure-app-configuration/get-by-name",
"api-reference/endpoints/secret-syncs/azure-app-configuration/create",
"api-reference/endpoints/secret-syncs/azure-app-configuration/update",
"api-reference/endpoints/secret-syncs/azure-app-configuration/delete",
"api-reference/endpoints/secret-syncs/azure-app-configuration/sync-secrets",
"api-reference/endpoints/secret-syncs/azure-app-configuration/import-secrets",
"api-reference/endpoints/secret-syncs/azure-app-configuration/remove-secrets"
"api-reference/endpoints/secret-syncs/databricks/list",
"api-reference/endpoints/secret-syncs/databricks/get-by-id",
"api-reference/endpoints/secret-syncs/databricks/get-by-name",
"api-reference/endpoints/secret-syncs/databricks/create",
"api-reference/endpoints/secret-syncs/databricks/update",
"api-reference/endpoints/secret-syncs/databricks/delete",
"api-reference/endpoints/secret-syncs/databricks/sync-secrets",
"api-reference/endpoints/secret-syncs/databricks/remove-secrets"
]
},
{
"group": "GCP Secret Manager",
"pages": [
"api-reference/endpoints/secret-syncs/gcp-secret-manager/list",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/get-by-id",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/get-by-name",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/create",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/update",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/delete",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/sync-secrets",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/import-secrets",
"api-reference/endpoints/secret-syncs/gcp-secret-manager/remove-secrets"
]
},
{
"group": "GitHub",
"pages": [
"api-reference/endpoints/secret-syncs/github/list",
"api-reference/endpoints/secret-syncs/github/get-by-id",
"api-reference/endpoints/secret-syncs/github/get-by-name",
"api-reference/endpoints/secret-syncs/github/create",
"api-reference/endpoints/secret-syncs/github/update",
"api-reference/endpoints/secret-syncs/github/delete",
"api-reference/endpoints/secret-syncs/github/sync-secrets",
"api-reference/endpoints/secret-syncs/github/remove-secrets"
]
}
]
},
{