Skip to content

cli-release.md

Latest commit

 

History

History
168 lines (124 loc) · 8.48 KB

cli-release.md

File metadata and controls

168 lines (124 loc) · 8.48 KB

CLI Release Process

The release process is handled by .github/workflows.

GoReleaser Automation

MSI Installer

  • Exe files for Windows distributed through Winget and Chocolatey have to be bundled in installers
  • MSBuild with WiX Toolset is used
  • The MSI file is created in the release-msi-windows step of the GitHub Workflow
  • The MSI file is uploaded to S3 in the same step

WinGet

  • WinGet is a package manager for Windows by Microsoft
  • The manifest is created using wingetcreate tool and pushed to the repository using update-repositories-windows step of the GitHub Workflow
  • The manifest is published to the official repository
  • Initial PR
  • Our release workflow sends a new PR to microsoft/winget-pkgs
  • We need to wait for one of the maintainers to merge the PR, this can take a few days
  • To install Keboola CLI with winget, use winget install --id=Keboola.KeboolaCLI -e

macOS

  • macOS binaries are signed with an Apple Developer code signing certificate
  • The certificate is issued by Tomas Netrval's Apple Developer account for the time being
  • macOS binaries are distributed using Homebrew distribution

Homebrew

Chocolatey

  • Chocolatey is a package manager for Windows
  • The package is created and pushed to the community repository using update-repositories-windows step of the GitHub Workflow
  • The package is published to the community repository: https://community.chocolatey.org/packages/keboola-cli
  • Updates to the package are authenticated by API key stored in CHOCOLATEY_KEY secret
  • Our release workflow sends the new version to Chocolatey which then runs 3 steps called "Validation", "Verification" and "Scan"
  • Check the current status on top of the page for the new version which is linked from the version history
  • Sometimes Chocolatey checks may decide that a manual review is necessary which can take longer

Scoop

Linux Repositories

S3 Distribution

  • The S3 bucket is publicly available on url https://cli-dist.keboola.com
  • The bucket is provisioned in dedicated AWS account using Terraform.

CLI Distribution Terraform Setup

Terraform backend init

Testing:

export AWS_PROFILE="Test-Keboola-As-Code-Assets"
export AWS_DEFAULT_REGION="eu-central-1"
export TERRAFORM_BACKEND_STACK_PREFIX="keboola-ci-kac-assets"
./provisioning/cli-dist/scripts/create-backend.sh

Production:

export AWS_PROFILE="Prod-Keboola-As-Code-Assets"
export AWS_DEFAULT_REGION="eu-central-1"
export TERRAFORM_BACKEND_STACK_PREFIX="keboola-prod-kac-assets"
./provisioning/cli-dist/scripts/create-backend.sh

OIDC authorization for GitHub Actions

See the documentation for the OIDC background between AWS and GitHub Actions.

1. Create a GitHub OIDC Provider

  • got to the IAM console -> Identity providers
  • click Add new provider
  • select OpenID Connect
  • fill provider url: https://token.actions.githubusercontent.com (Don't forget to click Get Thumbprint)
  • fill audience: sts.amazonaws.com
  • click add provider

2. Create AWS role for GitHub Actions to testing environment

Fill ARN from the previous step in env GITHUB_OIDC_PROVIDER_ARN

export AWS_PROFILE="Test-Keboola-As-Code-Assets"
export AWS_DEFAULT_REGION="eu-central-1"
export GITHUB_ORGANIZATION="keboola"
export GITHUB_REPOSITORY_NAME="keboola-as-code"
export GITHUB_OIDC_PROVIDER_ARN=arn:aws:iam::813746015128:oidc-provider/token.actions.githubusercontent.com
./provisioning/aws/scripts/create-github-testing-role.sh

The script will return the ARN full admin access role you will use in aws-actions/configure-aws-credential as a parameter role-to-assume to testing workflow.

3. Create AWS roles for GitHub Actions to production environment

  • fill ARN from the step one in env GITHUB_OIDC_PROVIDER_ARN
  • fill terraform backend prefix CF stack in env TERRAFORM_BACKEND_STACK_PREFIX
export AWS_PROFILE="Prod-Keboola-As-Code-Assets"
export AWS_DEFAULT_REGION="eu-central-1"
export GITHUB_ORGANIZATION="keboola"
export GITHUB_REPOSITORY_NAME="keboola-as-code"
export GITHUB_OIDC_PROVIDER_ARN=arn:aws:iam::455460941449:oidc-provider/token.actions.githubusercontent.com
export TERRAFORM_BACKEND_STACK_PREFIX=keboola-prod-kac-assets
./provisioning/aws/scripts/create-github-production-role.sh

The script will return the ARN roles:

  • full admin access role that can be called in GitHub Actions only over the main branch
  • read only role for the whole account and attached policy which allows you to run terraform provisioning plan, you can use this role over any branch

AWS ACM Certificate configuration

ACM Certificate for Cloudfront distribution is prepared and validated manually:

Test-Keboola-As-Code-Assets

  1. Login into Test-Keboola-As-Code-Assets AWS account as Administrator
  2. Go to AWS Certificate manager in us-east-1 region
  3. Request Public certificate
    • Fully qualified domain name: *.keboola.dev
    • Validation methond - DNS validation
  4. Copy the CNAME name and CNAME value of requested certificate
  5. Switch to Prod-KBC-multi-tenant-legacy and create CNAME DNS record from previous step in Route 53 keboola.dev Hosted Zone
  6. Switch back

Prod-Keboola-As-Code-Assets

  1. Login into Prod-Keboola-As-Code-Assets AWS account as Administrator
  2. Go to AWS Certificate manager in us-east-1 region
  3. Request Public certificate
    • Fully qualified domain name: *.keboola.com
    • Validation methond - DNS validation
  4. Copy the CNAME name and CNAME value of requested certificate
  5. Switch to Prod-KBC-multi-tenant-legacy and create CNAME DNS record from previous step in Route 53 keboola.com Hosted Zone
  6. Switch back to Prod-Keboola-As-Code-Assets AWS Account and wait until the certificate is validated