[infra] Patch release scripting (#3072)

This PR adds tooling to enable a PR-based workflow for publishing
'patch' releases.

### How releases currently work

Quick recap of how the 'major' and 'minor' releases work:

- You trigger them manually in the github actions UI
- It only works on the `main` branch.
- You select a mode: `'major'`, `'minor'`, or `'override'` with a
specific version. The override option is mainly for transitioning in and
out of prerelease mode, but potentially also skipping unlucky numbers
like 13 if you're feeling superstitious 🧙🏼
- It bumps the version numbers in the `package.json` and `version.ts`
files.
- It compiles a changelog based on descriptions/titles from all the PRs
that have gone in to `main`.
- It tags the commit with the version number e.g. `v2.0.0` and pushes
all the changes made to `main` (i.e. changelogs, version bumps and the
tag)
- It creates a github release, e.g.
https://github.com/tldraw/tldraw/releases/tag/v2.0.0
- It deploys the packages to npm
- It tells huppy bot about the release (for now-defunct purposes, we can
remove that code later)
- It triggers the template repo update workflow

### Introducing: Release Branches

This PR adds one step into the above process: creating a 'release'
branch. e.g. if it publishes a new version tagged `v2.1.0` it will also
create a branch named `v2.1.x`.

These branches are protected in the following ways:
- Only huppy bot can create or delete them (ad-hoc admin overrides are,
of course, still doable should the need arise)
- Like `main` they can only be updated via pull request.

The process to create a patch release becomes simple:

1. Checkout the `v<major>.<minor>.x` branch you want to create a patch
release for. e.g.

       git fetch && git checkout v2.1.x

4. Branch off, e.g.

       git checkout -b david/my-patch-release

6. Cherry-pick any commits you need from `main` into your branch,
resolving any conflicts if they arise. **important**: don't do new work
here because it won't be merged back into `main` automatically. Fix the
thing in `main` first and then cherry-pick, unless you're in a big rush
or whatever. e.g.

       git cherry-pick abdeaf234 cde234d09 ab23af287
       
7. Push your new branch to github as normal and make a PR targeting the
`v<major>.<minor>.x` branch.

8. Merge it.

Congratulations, you just triggered a patch release build.

### What happens (differently) during a patch release build.

⚡ A key thing to understand here is that **this script allows us to
deploy patch versions of _older_ major/minor releases**. This will
happen when we have customers pinned to older versions and they need a
quick bugfix but don't have time to upgrade to the latest due to some
breaking change. This will also happen if we ever adopt a kind of 'LTS'
release model.

With that said, here's how things go down differently:

- Firstly, the build happens automatically after the PR is merged, and
you don't select 'major' or 'minor' or anything, it just does its thing.
- It bumps the version numbers in the `package.json` files and the
`version.ts` files but these changes stay within the release branch,
they don't get propagated to `main` (nor should they).
- It compiles a changelog entry featuring just your one PR's
description/title, and also pushes this to the release branch (but not
`main`).
- It still tags the commit and creates a github release as normal.
- It still deploys the packages to npm (obvs). HOWEVER it only uses the
`latest` tag if this will indeed be the latest version of the public
packages. Otherwise, if we're patching an older release, it uses the
`revision` tag. Unfortunately it doesn't seem to be an option to deploy
with _no_ tag, but using `revision` still allows version strings like
`~2.0.0` to capture subsequent patch releases like `2.0.3`.
- Similarly it _only_ notifies huppy bot and _only_ triggers the
template repo update if the version being deployed is actually the
latest version.

I'm going to merge this now to test it out but I'd still appreciate
reviews.

.github/workflows/publish-manual.yml

@@ -2,6 +2,10 @@ name: (Re)Publish public packages manually
 # This only attempts to publish the public packages to npm.
 # It does not bump the version, it does not update the changelogs, it does not create a github release.
 
+# Prevent more than one non-canary npm publishing job from running at the same time
+concurrency:
+  group: npm-publish
+
 # Package publishing is manually triggered on github actions dashboard
 on: workflow_dispatch
 
@@ -12,6 +16,9 @@ jobs:
     timeout-minutes: 15
     runs-on: ubuntu-latest-16-cores-open
 
+    outputs:
+      is_latest_version: ${{ steps.publish_step.outputs.is_latest_version }}
+
     steps:
       - name: Check out code
         uses: actions/checkout@v3
@@ -20,6 +27,7 @@ jobs:
         uses: ./.github/actions/setup
 
       - name: Publish
+        id: publish_step
         run: yarn tsx ./scripts/publish-manual.ts
         env:
           GH_TOKEN: ${{ github.token }}
@@ -28,9 +36,8 @@ jobs:
   publish_templates:
     name: Publishes code templates to separate repositories
     uses: tldraw/tldraw/.github/workflows/publish-templates.yml@main
+    if: ${{ needs.deploy.outputs.is_latest_version == 'true' }}
     secrets:
       VITE_TEMPLATE_REPO_SSH_DEPLOY_KEY: ${{ secrets.VITE_TEMPLATE_REPO_SSH_DEPLOY_KEY }}
       NEXTJS_TEMPLATE_REPO_SSH_DEPLOY_KEY: ${{ secrets.NEXTJS_TEMPLATE_REPO_SSH_DEPLOY_KEY }}
-    needs: [
+    needs: deploy
-      deploy
-    ]

.github/workflows/publish-new.yml

@@ -1,6 +1,10 @@
 name: Publish new packages from main
 # This bumps the version, updates the changelogs, publishes a GitHub release, and publishes the packages to npm.
 
+# Prevent more than one non-canary npm publishing job from running at the same time
+concurrency:
+  group: npm-publish
+
 # Package publishing is manually triggered on github actions dashboard
 on:
   workflow_dispatch:

.github/workflows/publish-patch.yml

@@ -0,0 +1,61 @@
+name: Publish patch release
+# This bumps the patch version, updates the changelogs in the release branch only, publishes a GitHub release, and publishes the packages to npm.
+
+# Prevent more than one non-canary npm publishing job from running at the same time
+concurrency:
+  group: npm-publish
+
+# Package publishing is manually triggered on github actions dashboard
+on:
+  push:
+    branches:
+      - 'v[0-9]*.[0.9]*.x'
+
+jobs:
+  deploy:
+    name: Publish patch release
+    environment: npm deploy
+    timeout-minutes: 15
+    runs-on: ubuntu-latest-16-cores-open
+    outputs:
+      is_latest_version: ${{ steps.publish_step.outputs.is_latest_version }}
+
+    steps:
+      - name: Generate GH token
+        id: generate_token
+        uses: tibdex/github-app-token@b62528385c34dbc9f38e5f4225ac829252d1ea92
+        with:
+          app_id: ${{ secrets.HUPPY_APP_ID }}
+          private_key: ${{ secrets.HUPPY_APP_PRIVATE_KEY }}
+
+      - name: Check out code
+        uses: actions/checkout@v3
+        with:
+          token: ${{ steps.generate_token.outputs.token }}
+
+      - name: Prepare repository
+        # Fetch full git history and tags for auto
+        run: git fetch --unshallow --tags
+
+      - name: Run our setup
+        uses: ./.github/actions/setup
+
+      - name: Publish
+        id: publish_step
+        run: |
+          git config --global user.name 'huppy-bot[bot]'
+          git config --global user.email '128400622+huppy-bot[bot]@users.noreply.github.com'
+          yarn tsx ./scripts/publish-patch.ts
+        env:
+          GH_TOKEN: ${{ steps.generate_token.outputs.token }}
+          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
+          HUPPY_TOKEN: ${{ secrets.HUPPY_TOKEN }}
+
+  publish_templates:
+    name: Publishes code templates to separate repositories
+    uses: tldraw/tldraw/.github/workflows/publish-templates.yml@main
+    if: ${{ needs.deploy.outputs.is_latest_version == 'true' }}
+    secrets:
+      VITE_TEMPLATE_REPO_SSH_DEPLOY_KEY: ${{ secrets.VITE_TEMPLATE_REPO_SSH_DEPLOY_KEY }}
+      NEXTJS_TEMPLATE_REPO_SSH_DEPLOY_KEY: ${{ secrets.NEXTJS_TEMPLATE_REPO_SSH_DEPLOY_KEY }}
+    needs: deploy

scripts/publish-manual.ts

@@ -1,8 +1,22 @@
-import { publish } from './lib/publishing'
+import { appendFileSync } from 'fs'
+import { exec } from './lib/exec'
+import { getLatestVersion, publish } from './lib/publishing'
 
 // This expects the package.json files to be in the correct state.
 // You might want to run this locally after a failed publish attempt on CI.
 // Or if you need to hotfix a package it might be desirable to run this.
 
 // Generate a npm automation token and run this with the NPM_TOKEN env var set.
+async function main() {
+	const latestVersionInBranch = await getLatestVersion()
+	const latestVersionOnNpm = (await exec('npm', ['show', 'tldraw', 'version'])).trim()
+
+	const isLatestVersion = latestVersionInBranch.format() === latestVersionOnNpm
+	if (process.env.GITHUB_OUTPUT) {
+		appendFileSync(process.env.GITHUB_OUTPUT, `is_latest_version=${isLatestVersion}\n`)
+	}
+
 	publish()
+}
+
+main()

scripts/publish-new.ts

@@ -70,6 +70,8 @@ async function main() {
 	const releaseType = getReleaseType()
 	const nextVersion = await getNextVersion(releaseType)
 
+	const isPrerelease = parse(nextVersion)!.prerelease.length > 0
+
 	console.log('Releasing version', nextVersion)
 
 	await setAllVersions(nextVersion)
@@ -111,10 +113,18 @@ async function main() {
 		title: `v${nextVersion}`,
 	})
 
+	const gitTag = `v${nextVersion}`
+
 	// create and push a new tag
-	await exec('git', ['tag', '-f', `v${nextVersion}`])
+	await exec('git', ['tag', '-f', gitTag])
 	await exec('git', ['push', '--follow-tags'])
 
+	// create new 'release' branch called e.g. v2.0.x or v4.3.x, for making patch releases
+	if (!isPrerelease) {
+		const { major, minor } = parse(nextVersion)!
+		await exec('git', ['push', 'origin', `${gitTag}:refs/heads/v${major}.${minor}.x`])
+	}
+
 	// create a release on github
 	await auto.runRelease({ useVersion: nextVersion })
 

scripts/publish-patch.ts

@@ -0,0 +1,111 @@
+import { Auto } from '@auto-it/core'
+import fetch from 'cross-fetch'
+import glob from 'glob'
+import { assert } from 'node:console'
+import { appendFileSync } from 'node:fs'
+import { exec } from './lib/exec'
+import { nicelog } from './lib/nicelog'
+import { getLatestVersion, publish, setAllVersions } from './lib/publishing'
+import { getAllWorkspacePackages } from './lib/workspace'
+
+async function main() {
+	const huppyToken = process.env.HUPPY_TOKEN
+	assert(huppyToken && typeof huppyToken === 'string', 'HUPPY_ACCESS_KEY env var must be set')
+
+	const latestVersionInBranch = await getLatestVersion()
+	const latestVersionOnNpm = (await exec('npm', ['show', 'tldraw', 'version'])).trim()
+
+	const isLatestVersion = latestVersionInBranch.format() === latestVersionOnNpm
+	if (process.env.GITHUB_OUTPUT) {
+		appendFileSync(process.env.GITHUB_OUTPUT, `is_latest_version=${isLatestVersion}\n`)
+	}
+
+	const nextVersion = latestVersionInBranch.inc('patch').format()
+	// check we're on the main branch on HEAD
+	const currentBranch = (await exec('git', ['rev-parse', '--abbrev-ref', 'HEAD'])).toString().trim()
+	if (currentBranch !== `v${latestVersionInBranch.major}.${latestVersionInBranch.minor}.x`) {
+		throw new Error('Branch name does not match expected format: v{major}.{minor}.x')
+	}
+
+	// we could probably do this a lot earlier in the yml file but 🤷‍♂️
+	const numberOfCommitsSinceBranch = Number(
+		(await exec('git', ['rev-list', '--count', `HEAD`, '^main'])).toString().trim()
+	)
+
+	if (numberOfCommitsSinceBranch === 0) {
+		// Skip release if there are no commits since this branch was created during the initial release
+		// for this <major>.<minor> version.
+		nicelog('Initial push, skipping release')
+		return
+	}
+
+	nicelog('Releasing version', nextVersion)
+
+	await setAllVersions(nextVersion)
+
+	// stage the changes
+	const packageJsonFilesToAdd = []
+	for (const workspace of await getAllWorkspacePackages()) {
+		if (workspace.relativePath.startsWith('packages/')) {
+			packageJsonFilesToAdd.push(`${workspace.relativePath}/package.json`)
+		}
+	}
+	const versionFilesToAdd = glob.sync('**/*/version.ts', {
+		ignore: ['node_modules/**'],
+		follow: false,
+	})
+	console.log('versionFilesToAdd', versionFilesToAdd)
+	await exec('git', [
+		'add',
+		'--update',
+		'lerna.json',
+		...packageJsonFilesToAdd,
+		...versionFilesToAdd,
+	])
+
+	const auto = new Auto({
+		plugins: ['npm'],
+		baseBranch: currentBranch,
+		owner: 'tldraw',
+		repo: 'tldraw',
+		verbose: true,
+		disableTsNode: true,
+	})
+
+	await auto.loadConfig()
+
+	// this creates a new commit
+	await auto.changelog({
+		useVersion: nextVersion,
+		title: `v${nextVersion}`,
+	})
+
+	// create and push a new tag
+	await exec('git', ['tag', '-f', `v${nextVersion}`])
+	await exec('git', ['push', '--follow-tags'])
+
+	// create a release on github
+	await auto.runRelease({ useVersion: nextVersion })
+
+	// if we're on the latest version, publish to npm under 'latest' tag.
+	// otherwise we don't want to overwrite the latest tag, so we publish under 'revision'.
+	// semver rules will still be respected because there's no prerelease tag in the version,
+	// so clients will get the updated version if they have a range like ^1.0.0
+	await publish(isLatestVersion ? 'latest' : 'revision')
+
+	if (isLatestVersion) {
+		nicelog('Notifying huppy of release...')
+		const huppyResponse = await fetch('https://tldraw-repo-sync.fly.dev/api/on-release', {
+			method: 'POST',
+			headers: {
+				'Content-Type': 'application/json',
+			},
+			body: JSON.stringify({ apiKey: huppyToken, tagToRelease: `v${nextVersion}`, canary: false }),
+		})
+		nicelog(
+			`huppy: [${huppyResponse.status} ${huppyResponse.statusText}] ${await huppyResponse.text()}`
+		)
+	}
+}
+
+main()