Integrate with HashiCorp Vault

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

Applies from version 2025.1.1

Overview

Alation supports integration with HashiCorp Vault for data source authentication using an OCF connector. Using HashiCorp Vault allows you to consolidate your credentials in a single, secure location, preventing “credential sprawl” and enabling your organization to comply with IT security policies. You can store secrets such as database passwords and usernames, Kerberos authentication information, JDBC URI keys, and more.

When the HashiCorp Vault integration is set up, Alation will read credentials from HashiCorp Vault when performing metadata extraction (MDE), query log ingestion (QLI), sampling, and profiling.

Alation supports integration with HashiCorp Vault through the Alation Agent, Alation Cloud Service, and customer-managed instances of Alation.

If you integrate with HashiCorp Vault through an Alation Agent, the authentication flow goes like this:

  1. Alation stores the encrypted vault credentials in its database. When you make a request to start MDE, for example, your Alation instance sends the request, along with the vault credentials, through its encrypted TLS v1.3 connection to the Alation Agent.

  2. The Alation Agent uses the vault credentials to authenticate with HashiCorp Vault and obtain credentials for your data source.

  3. The Alation Agent uses the retrieved data source credentials to authenticate against the data source and complete the request. Your data source credentials never have to leave your VPC, and they are not stored in Alation or the Alation Agent.

Supported HashiCorp Vault Features

Alation supports the following HashiCorp Vault features.

Deployment Types

Alation supports the following HashiCorp Vault deployment types:

  • HashiCorp Cloud Platform (HCP)

  • Self-managed Vault deployments

Authentication Methods

HashiCorp Vault supports many types of authentication methods. Alation supports the following authentication methods for HashiCorp Vault:

Secrets Engines

In HashiCorp Vault, a secrets engine is a component that stores, generates, or encrypts data. Secrets engines can be used to store arbitrary secrets, generate dynamic secrets, or encrypt and decrypt data. HashiCorp Vault supports multiple types of secrets engines, each with its own purpose and functionality.

Alation supports storage and retrieval of secrets from the following HashiCorp Vault secrets engines:

  • kv (key-value) secrets engine v1

  • kv (key-value) secrets engine v2

  • PKI (public key infrastructure) secrets engine

The kv secrets engines are used to store arbitrary secrets in string format. kv v2 supports versioning of secrets, while kv v1 does not. The PKI secrets engine is used to generate and manage X.509 certificates. For more details, see:

Prerequisites

  • You must be on Alation version 2025.1.1 or later.

  • If you’re connecting to your data source through an Alation Agent, you must be using Alation Agent version 1.8.10.5154 or later.

  • If you’re connecting to your data source through an Alation Agent, you must install Authentication Service Add-on version 5.14.0.2303 or later on the Alation Agent.

  • You must have the Server Admin role to set up the integration in Alation.

Step 1: Configure HashiCorp Vault

This step is performed in HashiCorp Vault

  • Create a kv v1 or v2 secrets engine or PKI secrets engine in HashiCorp Vault, or use one you already have.

  • In the secrets engine, create a test secret that you can use to test the connection from Alation. You can also use an existing secret if you prefer.

    • For basic authentication, create a secret with username and password fields.

    • For key-pair authentication, create a secret with username, passphrase, and certificate fields.

      Note

      The value for the certificate field must be base64-encoded.

Step 2: Install the Authentication Service Add-on on the Alation Agent

This step is performed in Alation or on the Alation Agent machine

If you’re connecting to your data source through an Alation Agent, you must install the Authentication Service add-on on the Agent. If not, you can skip this step.

See Install the Authentication Service Add-on for information on installing the Authentication Service add-on.

Step 3: Create an Authentication Profile

This step is performed in Alation

To create an authentication profile for the Secrets Manager integration:

  1. Log in to your Alation instance as a Server Admin.

  2. Click the Admin Settings gear icon on top right to open the Admin Settings page.

  3. Click Authentication to open the Authentication tab. Locate the section Authentication Configuration Methods for External Systems.

  4. If you’re on a customer-managed (on-prem) instance of Alation, skip to the next step.

    If you’re on an Alation Cloud Service instance, click the drop-down menu next to See configurations for to choose the location of the configuration:

    • Choose Alation Cloud Service to create the authentication configurations for Alation Cloud Service itself.

    • Choose the name of an Alation Agent to create the configuration for an Agent. An Agent name will only be listed if you’ve installed the Authentication Service Add-on on that Agent.

  5. Click Add Configuration, and then select HashiCorp Vault as the method type. The Authentication Configuration Method page will open in a new browser tab.

  6. In Config Name, enter a unique name for the configuration. Save it for future reference when configuring the data source.

  7. In Vault URL, enter the URL for your HashiCorp Vault.

  8. In Secret Path for test connection, enter the path to the test secret from Step 1. The path must be in the following format:

    • For kv v1: <mount_path>/<secret_path>

    • For kv v2: <mount_path>/data/<secret_path>

      Note

      kv v2 secrets can have versions. In HashiCorp Vault, the path is displayed with the secret version prepended. You should not include the version in the Secret Path for test connection field.

      For example, if the secret path in HashiCorp Vault is /v1/mykv/data/testSecret, you should enter mykv/data/testSecret in the Secret Path for test connection field.

    • For PKI: pki/cert/<certificateName>

  9. Under Authentication Type, select either LDAP or App Role. Additional fields will appear based on your selection.

    • If you selected LDAP, enter the following information:

      • Username—The username for your LDAP account.

      • Password—The password for your LDAP account.

    • If you selected App Role, enter the following information:

      • Role ID—The role ID for your AppRole.

      • Secret ID—The secret ID for your AppRole.

  10. Click Save. Alation attempts to create a connection, and if the connection is successful, the configuration is saved.

Now, you can use your integration with an OCF connector. See next: Configure Authentication with AWS Secrets Manager for a Data Source.

Step 4: Configure Authentication with HashiCorp Vault for a Data Source

This step is performed in Alation

Use the steps in this section to configure your data source to read the data source credentials from HashiCorp Vault. You need the Data Source Admin level of the data source access or the Server Admin role to configure settings of a data source.

To configure authentication with HashiCorp Vault on a data source:

  1. Log in to Alation and go to the settings page of the OCF data source for which you’re setting up authentication with HashiCorp Vault.

  2. Open the General Settings tab of the settings page and click the vault option for each setting you want to configure using HashiCorp Vault.

  3. Click Select a configuration and select the appropriate HashiCorp Vault authentication profile.

  4. Enter the key for the secret you want to use. Keys must follow a specific format as follows:

    • For kv v1: kv1:<mount_path>/<secret_path>:<keyPath>

    • For kv v2: kv2:<mount_path>/data/<secret_path>:<keyPath>

    • For PKI: pki:pki/cert/<certificateName>

  5. Repeat as needed for additional settings.

  6. Click Save. The configurations and keys you entered will be used to look up the actual username and password stored in the HashiCorp Vault.

Now, when a Data Source Admin performs MDE, QLI, sampling, and profiling, Alation will read the appropriate credentials from HashiCorp Vault.

Testing the Configuration

  1. Click the Test Connection button in the Alation Authentication configuration.

  2. Verify that the secret path is correctly derived for your KV engine version.

  3. If the test fails, check:

    • The status of the Vault server.

    • Correctness of credentials and secret paths.

    • Appropriate configuration of the chosen authentication method.

    • Proper Vault access policies.

Troubleshooting

Logs

On customer-managed instances of Alation, you can review the following files if issues arise:

  • Alation Logs:

    • /opt/alation/site/logs/alation-debug.log

    • /opt/alation/site/logs/alation-error.log

    • /opt/alation/site/logs/alation-info.log

  • Authentication Service Logs:

    • /opt/alation/site/logs/authserver_err.log

    • /opt/alation/site/logs/authserver.log

    • /opt/alation/site/logs/authserver_out.log

  • Authentication Service Configuration Files:

    • /opt/alation/site/config/authserver/

Error Messages

  • “Failed to authenticate with Vault” error:

    • Check the Vault server status.

    • Confirm that the credentials and secret paths are correct.

    • Ensure the correct authentication method (LDAP or AppRole) is configured.

  • “Permission denied” error:

    • Verify the secret path format.

    • Ensure that the Vault policies allow access to the specified secret.