staging-branches

staging-branches is a repo hosting the branches configuration for commcare-hq's and formplayer's staging environments.

Workflow

The general workflow is virtually the same across repositories. The convention is for a script named rebuildstaging to live in the project's scripts directory (e.g., commcare-hq/scripts/rebuildstaging) which is responsible for fetching the appropriate file from this repository, and building a branch to deploy to the staging environment.

• Add the branch you'd like to deploy to the appropriate staging configuration file in this repository. The easiest approach is to use the GitHub UI for editing the file directly, but you can choose to edit locally if so desired.
• Commit your changes directly to main. Don't worry about a detailed commit message unless you're doing something unusual.
• From the root of the repository you want to deploy (e.g., commcare-hq, formplayer, etc), run ./scripts/rebuildstaging. This will build a branch called autostaging that contains all branches specified in this file.
  • To check if your newly added branch causes any conflicts before pushing to autostaging, run:

    $ scripts/rebuildstaging --no-push
    

• After rebuilding the autostaging branch, you need to deploy the new branch to staging.

  $ commcare-cloud --control staging deploy
  ##### OR #####
  $ scripts/rebuildstaging --deploy (NOTE: `commcare-cloud` must be available in your shell)
  

Automated Staging Rebuild

The autostaging branch is rebuilt automatically in two ways:

1. Config changes: When commcare-hq-staging.yml is pushed to main in this repo, a workflow here triggers rebuild-staging in dimagi/commcare-hq via workflow_dispatch.
2. Branch pushes: The rebuild-staging workflow in dimagi/commcare-hq also runs on push to any branch. It checks this config file and skips the rebuild if the pushed branch isn't listed. This means pushes to master or to any branch already in the config will trigger a rebuild automatically.

Auto-removal of merged branches

When a branch in the staging config is merged to master in commcare-hq, the rebuild-staging workflow detects this and triggers the remove-branch workflow in this repo instead of rebuilding. The branch is automatically removed from the config file, and the resulting push to main triggers a clean rebuild via trigger 1 above.

This prevents rebuild failures caused by branches that have been deleted after merge. Note that +-joined conflict resolution branches (e.g. foo+bar) are not automatically removed — see Resolving Branch Conflicts for how to clean those up manually.

The only case where autostaging can go stale is when new commits are pushed to a branch declared in a submodule's staging config.

You can monitor triggered runs in the commcare-hq Actions tab.

For implementation details, maintenance, and how to extend this to other repos, see docs/automated-rebuild-trigger.md.

Resolving Branch Conflicts

First, determine where the conflict lies. All of these steps should be taken from the root of the repository you're working on (eg, commcare-hq)

Branch foo conflicts with master

$ git checkout -b foo origin/foo
$ git pull origin master

try to resolve conflict

$ git push origin foo

Branch foo conflicts with branch bar

You can't just merge foo into bar or vice versa, otherwise the PR for foo will contain commits from bar. Instead make a third, conflict-resolution branch:

$ git checkout -b foo+bar --no-track origin/foo
$ git pull origin bar

try to resolve conflict

$ git push origin foo+bar

add the branch foo+bar to staging.yml and move branches foo and bar to right below it:

- foo+bar
- foo
- bar

Later on branch bar gets merged into master and removed from staging.yml. Perhaps the person who removes it also notices the foo+bar and does the following. Otherwise anyone who comes along and sees foo+bar but not both branches can feel free to assume the following needs to be done:

• Merge foo+bar into foo. Since bar is now merged and deleted, you want to merge the resolution into foo, otherwise foo will conflict with master.
• Remove foo+bar from staging.yml. It's no longer necessary since it's now a subset of foo.

If you are unsure of how to resolve a conflict, notify the branch owner.