Amazon Web Services PrivateLink

Amazon Web Services’ PrivateLink is an AWS service that provides private connectivity between VPCs without exposing traffic to the public Internet. Keeping traffic in the Amazon network reduces the data security risk associated with exposing your Warehouse traffic to the Internet.

You can configure AWS PrivateLink for Databricks, RDS Postgres, Redshift, and Snowflake. Only warehouses located in regions us-east-1, us-west-2, or eu-west-1 are eligible.

Usage limits for each customer during the AWS PrivateLink Private Beta include the following:

• Up to 2 AWS PrivateLink VPC endpoints.
• A monthly data transfer limit of 300GB total for all PrivateLink VPC endpoints connected to Segment.

Databricks

The following Databricks integrations support PrivateLink:

• Databricks storage destination
• Databricks Reverse ETL source
• Databricks Profiles Sync
• Databricks Data Graph

Prerequisites

Before you can implement AWS PrivateLink for Databricks, complete the following prerequisites in your Databricks workspace:

• Databricks account must be on the Enterprise pricing tier and use the E2 version of the platform.
• Databricks workspace must use a Customer-managed VPC and Secure cluster connectivity.
  • Configure your VPC with DNS hostnames and DNS resolution
  • Configure a security group with bidirectional access to 0.0.0.0/0 and ports 443, 3306, 6666, 2443, and 8443-8451.

Implement PrivateLink for Databricks

To implement Segment’s PrivateLink integration for Databricks:

1. Follow the instructions in Databricks’ Enable private connectivity using AWS PrivateLink documentation. You must create a back-end connection to integrate with Segment’s front-end connection.
2. After you’ve configured a back-end connection for Databricks, let your Customer Success Manager (CSM) know that you’re interested in PrivateLink.
3. Segment’s engineering team creates a custom VPC endpoint on your behalf. Segment then provides you with the VPC endpoint’s ID.
4. Register the VPC endpoint in your Databricks account and create or update your Private Access Setting to include the VPC endpoint. For more information, see Databricks’ Register PrivateLink objects documentation.
5. Configure your Databricks workspace to use the Private Access Setting object from the previous step.
6. Reach back out to your CSM and provide them with your Databricks Workspace URL. Segment configures their internal DNS to reroute Segment traffic for your Databricks workspace to your VPC endpoint.
7. Your CSM notifies you that Segment’s PrivateLink integration is complete. If you have any existing Segment Databricks integrations that use your Databricks workspace URL, they now automatically use PrivateLink. Any new Databricks integrations created in the Segment app using your Databricks workspace URL will also automatically use PrivateLink.

RDS Postgres

The following RDS Postgres integrations support PrivateLink:

• RDS Postgres storage destination
• RDS Postgres Reverse ETL source
• RDS Postgres Profiles Sync

Prerequisites

Before you can implement AWS PrivateLink for RDS Postgres, complete the following prerequisites:

• Set up a Network Load Balancer (NLB) to route traffic to your Postgres database: Segment recommends creating a NLB that has target group IP address synchronization, using a solution like AWS Lambda. If any updates are made to the Availability Zones (AZs) enabled for your NLB, please let your CSM know so that Segment can update the AZs of your VPC endpoint.
• Configure your NLB with one of the following settings:
  • Disable the Enforce inbound rules on PrivateLink traffic setting
  • If you must enforce inbound rules on PrivateLink traffic, add an inbound rule that allows traffic belonging to Segment’s PrivateLink/Edge CIDR: 10.0.0.0/8

Implement PrivateLink for RDS Postgres

To implement Segment’s PrivateLink integration for RDS Postgres:

1. Create a Network Load Balancer VPC endpoint service using the instructions in the Create a service powered by AWS PrivateLink documentation.
2. Let your Customer Success Manager (CSM) know that you’re interested in PrivateLink. They will share information with you about Segment’s AWS principal.
3. Add the Segment AWS principal as an “Allowed Principal” to consume the Network Load Balancer VPC endpoint service you created in step 1.
4. Reach out to your CSM and provide them with the Service Name for the service that you created above. Segment’s engineering team provisions a VPC endpoint for the service in the Segment Edge VPC.
5. Segment provides you with the VPC endpoint’s private DNS name. Use the DNS name as the Host setting to update or create new Postgres integrations in the Segment app.

Redshift

The following Redshift integrations support PrivateLink:

• Redshift storage destination
• Redshift Reverse ETL source
• Redshift Profiles Sync
• Redshift Data Graph

Prerequisites

Before you can implement AWS PrivateLink for Redshift, complete the following prerequisites:

• You’re using the RA3 node type: To access Segment’s PrivateLink integration, use an RA3 instance.
• You’ve enabled cluster relocation: Cluster relocation migrates your cluster behind a proxy and keeps the cluster endpoint unchanged, even if your cluster needs to be migrated to a new Availability Zone. A consistent cluster endpoint makes it possible for Segment’s Edge account and VPC to remain connected to your cluster. To enable cluster relocation, follow the instructions in the AWS Relocating your cluster documentation.
• Your cluster is using a port within the ranges 5431-5455 or 8191-8215: Clusters with cluster relocation enabled might encounter an error if updated to include a port outside of this range.

Implement PrivateLink for Redshift

To implement Segment’s PrivateLink integration for Redshift:

1. Let your Customer Success Manager (CSM) know that you’re interested in PrivateLink. They will share information with you about Segment’s Edge account and VPC.
2. After you receive the Edge account ID and VPC ID, grant cluster access to Segment’s Edge account and VPC.
3. Reach back out to your CSM and provide them with the Cluster Identifier for your cluster and your AWS account ID.
4. Segment’s engineering team creates a Redshift managed VPC endpoint within the Segment Redshift subnet on your behalf, which creates a PrivateLink Endpoint URL. Segment then provides you with the internal PrivateLink Endpoint URL.
5. Use the provided PrivateLink Endpoint URL as the Hostname setting to update or create new Redshift integrations in the Segment app.

Snowflake

The following Snowflake integrations support PrivateLink:

• Snowflake storage destination
• Snowflake Reverse ETL source
• Snowflake Profiles Sync
• Snowflake Data Graph

Prerequisites

Before you can implement AWS PrivateLink for Snowflake, complete the following prerequisites:

• Your Snowflake account is on the Business Critical Edition or higher.
• Your Snowflake account is hosted on the AWS cloud platform.

Implement PrivateLink for Snowflake

To implement Segment’s PrivateLink integration for Snowflake:

1. Follow Snowflake’s PrivateLink documentation to enable AWS PrivateLink for your Snowflake account.
2. Let your Customer Success Manager (CSM) know that you’re interested in PrivateLink. They will provide you with Segment’s AWS Edge account ID.
3. Create a Snowflake Support Case to authorize PrivateLink connections from Segment’s AWS account ID as a third party vendor to your Snowflake account.
4. After Snowflake support authorizes Segment, call the SYSTEM$GET_PRIVATELINK_CONFIG function while using the Snowflake ACCOUNTADMIN role. Reach back out to your Segment CSM and provide them with the privatelink-vpce-id and privatelink-account-url values from the function output. Note down for yourself the privatelink-account-name value.
5. Segment’s engineering team creates a custom VPC endpoint on your behalf. Segment also creates a CNAME record to reroute Segment traffic to use your VPC endpoint. This ensures that Segment connections to your privatelink-account-name are made over PrivateLink.
6. Your CSM notifies you that the setup on Segment’s side is complete. Use your privatelink-account-name as the Account setting to update or create new Snowflake integrations in the Segment app.