io.github.randyridgley.cdk.datalake.constructs.package-info Maven / Gradle / Ivy

Go to download
Show more of this group Show more artifacts with this name
Show all versions of cdk-datalake-constructs Show documentation
AWS CDK Constructs that can be used to create datalakes/meshes and more
There is a newer version: 0.0.64
Show newest version
/**
 * cdk-datalake-constructs  
 * 
 * Very experimental until version 1.0.
 * This is my attempt at simplifying deploying various datalake strategies in AWS with the CDK.
 * 

 * 
 * 
 * 
 *  
 *    
 * 

 * Table of Contents
 * 

 * 

 * Features
 * Installation
 * Usage
 * 
 * 

 * Basic
 * Data Mesh
 * 
 * Documentation
 * 
 * 

 * Construct API Reference
 * 
 * Supporting this project
 * License
 * 
 * 
 * 
Features
 * 
 * 

 * Easy to Start - Create a Datalake in a few lines.
 * Easy to Expand - Expand into multiple accounts and into a data mesh.
 * Easy to Admin - Initial governance created on deploy.
 * 
 * 
 * 
Installation
 * 
 * TypeScript/JavaScript
 * 

 * 
 * $ npm install @randyridgley/cdk-datalake-constructs
 * 
 * 
 * Python
 * 

 * 
 * $ pip install cdk-datalake-constructs
 * 
 * 
 * .Net
 * 

 * 
 * $ nuget install CDK.Datalake.Constructs
 * 
 * # See more: https://www.nuget.org/packages/CDK.Datalake.Constructs/
 * 
 * 
 * 
Usage
 * 
 * 
Basic
 * 
 * 
 * // Example automatically generated. See https://github.com/aws/jsii/issues/826
 * import randyridgley.cdk.datalake.constructs.DataLake;
 * 
 * Array<dl.Pipeline>[] taxiPipes = List.of(pipelines.YellowPipeline(), pipelines.GreenPipeline());
 * 
 * Array<dl.DataProduct>[] dataProducts = List.of(Map.of(
 *         "pipelines", taxiPipes,
 *         "accountId", lakeAccountId,
 *         "dataCatalogAccountId", "123456789012",
 *         "databaseName", "taxi-product"));
 * 
 * // deploy to local account
 * // deploy to local account
 * DataLake.Builder.create(this, "LocalDataLake")
 *         .name("data-lake,")
 *         .accountId(centralAccountId)
 *         .region("us-east-1")
 *         .policyTags(Map.of(
 *                 "classification", "public,confidential,highlyconfidential,restricted,critical",
 *                 "owner", "product,central,consumer"))
 *         .stageName(Stage.getPROD())
 *         .dataProducts(dataProducts)
 *         .createDefaultDatabase(false)
 *         .build();
 * 
 * 
 * 
Data Mesh
 * 
 * You can setup cross account access and pre-created policy tags for TBAC access in Lake Formation
 * 

 * 
 * // Example automatically generated. See https://github.com/aws/jsii/issues/826
 * Object lakeAccountId = app.node.tryGetContext("lakeAccountId");
 * Object centralAccountId = app.node.tryGetContext("centralAccountId");
 * Object consumerAccountId = app.node.tryGetContext("consumerAccountId");
 * 
 * Array<dl.Pipeline>[] taxiPipes = List.of(pipelines.YellowPipeline(), pipelines.GreenPipeline());
 * 
 * Array<dl.DataProduct>[] dataProducts = List.of(Map.of(
 *         "pipelines", taxiPipes,
 *         "accountId", lakeAccountId,
 *         "dataCatalogAccountId", centralAccountId,
 *         "databaseName", "taxi-product"));
 * 
 * // deploy to the central account
 * // deploy to the central account
 * DataLake.Builder.create(this, "CentralDataLake")
 *         .name("central-lake,")
 *         .accountId(centralAccountId)
 *         .region("us-east-1")
 *         .policyTags(Map.of(
 *                 "classification", "public,confidential,highlyconfidential,restricted,critical",
 *                 "owner", "product,central,consumer"))
 *         .stageName(Stage.getPROD())
 *         .crossAccount(Map.of(
 *                 "consumerAccountIds", List.of(consumerAccountId, lakeAccountId),
 *                 "dataCatalogOwnerAccountId", centralAccountId,
 *                 "region", "us-east-1"))
 *         .dataProducts(dataProducts)
 *         .createDefaultDatabase(true)
 *         .build();
 * 
 * // deploy to the data product account
 * Object datalake = DataLake.Builder.create(this, "LocalDataLake")
 *         .name("local-lake")
 *         .accountId(lakeAccountId)
 *         .region("us-east-1")
 *         .stageName(Stage.getPROD())
 *         .dataProducts(dataProducts)
 *         .createDefaultDatabase(true)
 *         .build();
 * 
 * // Optionally add custom resource to download public data set products
 * datalake.createDownloaderCustomResource(accountId, region, props.getStageName());
 * 
 * // deploy to consumer account
 * Object datalake = DataLake.Builder.create(this, "ConsumerDataLake")
 *         .name("consumer-lake")
 *         .accountId(consumerAccountId)
 *         .region("us-east-1")
 *         .stageName(Stage.getPROD())
 *         .policyTags(Map.of(
 *                 "access", "analyst,engineer,marketing"))
 *         .createDefaultDatabase(true)
 *         .build();
 * 
 * 
 * 
Documentation
 * 
 * 
Construct API Reference
 * 
 * See API.md.
 * 

 * 
Supporting this project
 * 
 * I'm working on this project in my free time, if you like my project, or found it helpful and would like to support me any contributions are much appreciated! ❤️
 * 

 * 
License
 * 
 * This project is distributed under the MIT.
 */
@software.amazon.jsii.Stability(software.amazon.jsii.Stability.Level.Experimental)
package io.github.randyridgley.cdk.datalake.constructs;