Ian J MacIntosh.com

Using Lighthouse CI with GitHub Actions

This article explains the steps to integrate Lighthouse into your pull request process using GitHub Actions. Lighthouse is a web page auditing tool, and GitHub Actions provides GitHub users a way to perform continuous integration (“CI”) tasks right within GitHub itself.

By combining them, you can automatically generate Lighthouse audit reports showing how changes proposed in pull requests will affect your app's most important pages.

Steps

  1. Pick what pages to audit
  2. Run Lighthouse from Google Chrome
  3. Run Lighthouse from the CLI
  4. Run Lighthouse from GitHub Actions
  5. Add the Lighthouse CI GitHub App
  6. Merge Your Changes

Step 1: Pick What Pages to Audit 📋

Before I start auditing pages, I need to figure out what pages to audit. This should be informed by organization goals for how visitors should use the site.

I’d take a few minutes to consider:

If my site has different types of visitors with different goals, I’ll make a list for each of them. If working with teammates (especially ones who are more explicitly tasked with advocating for different types of visitors) this is a good time to get their input.

Once I’ve got a list of my important use cases and associated pages, I’ll store my notes somewhere convenient for reviewing later, since even outside the context of this exercise, it provides useful insight on how the org thinks about users. For my immediate purposes, I’ll make a plaintext list of all the pages identified, write them as relative URLs, and remove any duplicates.

By the end of this step, you’ll have a list of URLs for your site’s most important pages.

Step 2: Run Lighthouse from Google Chrome 🔬

At this point, I use Chrome DevTools Lighthouse tab to manually audit those pages on a local server, opening each page one by one, opening Chrome DevTools, switching to the Lighthouse tab, and running audits for performance, accessibility, best practices, and SEO using the “mobile device” setting.

I could skip this step, but I feel better when I run my audits manually and see the results with a nice in-browser UI.

Tips

By the end of this step, you’ll have some Lighthouse audit scores for your most important pages.

Step 3: Run Lighthouse CI from the CLI 🖥

After learning how to get my server prepared for audits and seeing how my pages score, I’ll get Lighthouse locally from the CLI the same way I’ll be doing with GitHub Actions.

Step 3.1: Install LHCI's CLI tool

npm install -g @lhci/[email protected]

You don’t necessarily have to use sudo to install a package globally. Sindre Sorhus explains how in this guide.

Step 3.2: Confirm it's installed globally

lhci --version

This should return something like:

0.7.2

Step 3.3: Write a configuration file

LHCI will look for a file named lighthouserc.js, which should tell LHCI to:

Example lighthouserc.js:

module.exports = {
  ci: {
    upload: {
      target: 'temporary-public-storage',
    },
    collect: {
      staticDistDir: './dist',
      url: [
        'http://localhost/',
        'http://localhost/articles/',
        'http://localhost/code-projects/',
      ],
    },
  },
}
Test Faster! ⚡️

Even though I showed my configuration file testing three different pages, I recommend reducing your cycle time by auditing only one or two pages while setting this up. You can add the rest once everything is working.

Step 3.4: Build the app

For my example, I'm working with a static site generator that I can build by running npm run generate which puts all the built assets into a dist/ directory. This will vary from project to project! If I’m testing an app that isn’t running on a static site generator, I’ll use the startServerCommand to tell Lighthouse CI how to start my server.

Step 3.5: Run Lighthouse CI and confirm it worked

lhci autorun

Confirm it worked right:

✅  Configuration file found

Running Lighthouse 3 time(s) on http://localhost:56251/
Run #1...done.
Run #2...done.
Run #3...done.
Done running Lighthouse!

Uploading median LHR of http://localhost:56251/...success!
Open the report at https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1626375889671-9480.report.html

By the end of this step, you’ll have the ability to run LHCI with your project’s specific settings from the command line.

Step 4: Run Lighthouse CI from GitHub Actions 🏗

Now it’s time to get all this automated. If I’m working on a new project that isn’t already using GitHub Actions, I’ll make a new directory: .github/actions/ and put a file ci.yml in it. If the project already uses GitHub Actions, I’ll need to decide between updating an existing workflow with a new Lighthouse job or updating an existing job with a new Lighthouse step. Whoever’s responsible for guiding the project’s testing process should be able to inform this decision.

Since this Lighthouse audit is an end-to-end test, it should theoretically be run after ensuring your code passes lighter/quicker tests. If I’ve got linting and unit tests and integrated tests already running in a single job, I prefer to put the Lighthouse step at the end.

Example ci.yml:

name: End-to-End Tests
on: [push]
jobs:
  lhci:
    name: Lighthouse
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v2
      - name: Set up Node.js 12.x
        uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - name: Install dependencies and build
        run: |
          npm install
          npm run build
      - name: Test site with Lighthouse CI
        run: |
          npm run generate # Or whatever will build the project
          npm install -g @lhci/[email protected]
          lhci autorun

I’ll commit this file to my repo on a new branch and open a pull request. In that pull request’s “Checks” tab, after a moment I should see a log showing GitHub Actions running my Lighthouse audits. If I need to change anything in the workflow, GitHub will use workflow files on my branch itself to guide its PR’s checks. For a faster review cycle when troubleshooting, I’ll use the act library to run GitHub Actions locally.

By the end of this step, you’ll have an open pull request with some automated checks showing an output log pretty close to what you saw when you ran the tests locally.

Step 5: Add the Lighthouse CI GitHub App ✨

Finally, to view my Lighthouse scores as checks without needing to go to the “Checks” tab, I can install the Lighthouse CI GitHub App.

  1. Install the Lighthouse CI GitHub App and grant it permission to access my app’s repository
  2. After authorizing Lighthouse CI, I’ll see a confirmation screen with an access token string that will look like 82cgwlT9UrQSWfV:18297505:r5SnfRqsN. If I lose it, I’ll have to get a new one by uninstalling and reinstalling the Lighthouse CI GitHub App.
  3. Add my access token to my GitHub repo’s secrets
    1. In the GitHub UI, I’ll go to my repository and click the “Settings” tab
    2. Choose “Secrets”
    3. Choose “New Repository Secret”
    4. Name the secret LHCI_GITHUB_APP_TOKEN and set its value using the token from that authorization page.
  4. Pass a reference to that secret in my GitHub Actions workflow so the step to run Lighthouse has access to it:

Example ci.yml:

...
      - name: Run Lighthouse
        run: |
          npm run generate
          npm install -g @lhci/[email protected]
          lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
...

I’m showing my run commands to give some context for where exactly I put the reference in my own workflow, but your run commands will likely be a little different. The important part you’ll be copying and pasting is that last env part where I set LHCI_GITHUB_APP_TOKEN using GitHub’s syntax for referencing secrets.

If you see a message in your logs that says it successfully updated the status check for GitHub but you’re not seeing a new status check on your pull request, confirm your workflow’s checkout step checks out the same hash your branch is pointed at. For troubleshooting, the official GitHub checkout action offers “Checkout out the ref” output in its log:

Checking out the ref
/usr/bin/git log -1 --format='%H'
'5af80f0a7fda021196b7adedc63bb06ce865e27e'

Compare that third line with your most recent commit hash and ensure it's not different. I don’t understand why this sometimes happens, but it’s a known issue. If the hash listed doesn’t match the commit your branch is pointed at, use this fix:

Example ci.yml:

...
      - name: Checkout 🛎
        uses: actions/checkout@master
        with:
          ref: ${{ github.event.pull_request.head.sha }}
...

By the end of this step, you’ll see your site’s Lighthouse scores from your GitHub Pull Request’s “Conversation” tab under where it shows “Checks.”

Step 6: Merge Your Changes 🚢

If you took my advice earlier about only auditing one or two pages while getting this set up, now's a good time to add the rest of the pages. Assuming everything still works, celebrate and merge your changes! 🍾