Tag: github

GitHub Actions Braindump
The following are from a braindump I did for my teamn (everything here is public knowledge):

Getting Setup to Building and Developing with the Workflows

This section outlines setting up your development environment for working with workflows:
1. Download the Visual Code. This tool is best to sit outside of your environment.
2. Click Extensions > Search for PowerShell and install the PowerShell. This feature will also install PowerShell local to your system. PowerShell is used in the Windows workflow.
3. Install ShellCheck. This feature is used to check your code and make sure you are following best practices when generating the shell scripts.
4. Add an alias to your terminal settings:
alias code=’/Applications/Visual\ Studio\ Code.app/Contents/Resources/app/bin/code’

For me, this is in the .zshrc file.

You should also have Docker installed (or aliased nerdctl aliased to docker). You should also have IBM Java 11 installed (v11.0.14.1).

You’ll also need access to:
- Bash shell
- git tools
- gpg
Debugging GitHub Actions

If the GitHub Action fails, you should check the following:
1. Review the PR Workflow
1. Click on Details
  1. Select the Job that is failing
- Click on the Settings > View raw logs
- Navigate down to the end of the file to the BUILD FAILURE
  - Scroll up from the BUILD FAILURE point (at least in Maven)
  - Look for the failing tests
- If there are not enough details, go back to Step (b)
  - Click on Summary
- Scroll down to Artifacts
  - Download the Artifacts related to the build. These files are kept for 90 days or until we hit a storage limit. Note, we purposefully use the upload-artifacts step.
1. Extract the files and you should have the Docker Console log file, test results any other workflow logging.
2. IBM/FHIR Repository Actions – You can filter through list of Actions.
  1. Navigate to https://github.com/IBM/FHIR/actions?query=is%3Afailure
  1. You can filter on the failing workflows and get a good big picture.
3. GitHub Status – You should subscribe to the site’s updates (via email). This is going to be very helpful to figure out what’s going on with GitHub. You should also check this when the transient errors appear systemic or non-deterministic – e.g. not failing in the same spot. At least one person on the team should sign up for the GitHub Status.
4. GitHub Community: Actions – I rarely go here, but sometimes I find that someone has posted with the same question, it may have an answer. Very rarely, I post directly there.
Debugging

If you encounter anything that looks transient – e.g., network download (Wagon Failure), disk space, filesystem failure – you can choose to rerun the failing workflow.
1. Navigate to the failing Job-Workflow
2. Click Re-run all jobs
3. Repeat for all Workflows that failed
If you see a failure in a particular GitHub Step, such as actions/checkout or actions/setup-java, you should go search that actions issues:
1. If actions/setup-java is failing, navigate to https://github.com/actions/setup-java
2. Check the Issues (Search for the issue)
3. Search for settings that may help
Note, sometimes they intentionally fail a GitHub Action workflow to signal that you should upgrade or change.

How to do Local Development with GitHub Actions
1. Navigate to https://github.com/ibm/fhir
2. Click Fork
  1. If a Fork already exists, be sure to Fetch Upstream > Fetch and Merge
- Click Pull Requests and Create the ci-skip label
  - Click Labels
  - Click New Label
  - Click Create label
- Clone the fork – git clone https://github.com/prb112/FHIR.git
  - I typically create a local folder called prb112 then clone into it.
- Once the main branch is active, git checkout -b new-ci
- Open your code using your alias: code ~/dev/fhir
- You’ll see:
- Edit your automation files in .github and build
- Click Terminal > New Terminal
1. Update the .github/workflows/<file> you are changing so the job.<workflow_job>.if condition is invalid:
jobs:

e2e-audit:

runs-on: ubuntu-latest

if: “!contains(github.event.pull_request.labels.*.name, ‘ci-skip’)”

I change ci-skip to ci-skip-ignore so that I can run just that one targeted workflow.
1. Test the steps locally by executing the steps in the workflow line-by-line in the terminal session.
2. Once you are comfortable with the changes:
  1. git add <files>
  1. git commit -s -m “My edits for issue #2102010”
3. Push your new branch – git push -u origin new-ci
4. Create your new Pull Request targeting the IBM:fhir main branch and add ci-skip.
The targeted workflow you are building with is the only one that runs. Note, you have a LIMITED number of execution minutes for GitHub Workflows.

Finding GitHub Actions to use/tips in Use

There are many folks using GitHub Actions, and many have figured out better patterns (or evolved to have better patterns).
1. Go here – https://github.com/actions/
2. Search: <my query> file:yml site:github.com
Workflow Details

Each workflow runs in a hosted-runner (or virtual-environment). These hosted-runners are containers specified declaratively in the workflow:

Flavor Virtual-Environment
windows windows-2019
All Other Ubuntu2004

These hosted-runners have a number of pre-installed libraries and tools – Docker, podman, java-11, jq, perl, python, yarn et cetra.

These workflows (except the site, javadocs and release) follow a similar pattern:
1. Setup Prerequisites
2. Run the Pre-integration Steps
3. Execute the Tests
4. Run the Post Integration Steps
5. Archive the Results
This pattern evolved from build.yml and integration.yml as the starting point all the way to the most recent migration.yaml. Migration.yml is the most sophisticated workflow-jobs that are created.
2022-03-01
Gatsby & Carbon: Build with Github Action
As some of you know, I work on the IBM FHIR Server and with my colleagues, I have started automating some of the actions we take – Build, Test, Deploy, Deploy our website.

More specific to the “Deploy our website” automation, our website uses technologies, such as Gatsby, Carbon, Gatsby Carbon Theme. Fundamentally, a static site generation technology, like Jekyll, Gatsby uses Node, Yarn and some nice React code.

To build our site with GitHub actions, I built out a site workflow. The key elements to this workflow are:
- Triggers
- Node.js and Ubuntu Images
- Build
- Add, Commit and Push to GH Pages
- Debugging and Replicating Locally
Triggers

For the Triggers, I recommend limiting the site generation to master branches. The master branch filter and on push, limits the re-deployment, also keep your site building on on docs/** changes.

on:

push:

paths:

– “docs/**”

branches:

– master

There is a subtlety the websites are cached for 10 minutes, confirmed on the site – Caching assets in website served from GitHub pages

Node.js and Ubuntu Images

I opted to use Ubuntu with Node.js

jobs:

build:

runs-on: ubuntu-latest

strategy:

matrix:

node-version: [12.x]

The important thing is ubuntu-latest which has some incompatibility with Gatsby Carbon’s build.

Build

I build the system as follows:

Checkout the repo to a folder

steps:

-name: Grab the Master Branch

uses: actions/checkout@v1

with:

working-directory: fhir

ref: refs/heads/master

fetch-depth: 1

path: fhir

Activate Node

– name: Use Node.js ${{ matrix.node-version }}

uses: actions/setup-node@v1

with:

node-version: ${{ matrix.node-version }}

Setup the build

echo “Check on Path”

pwd

cd docs/

npm install -g gatsby-cli

gatsby telemetry –disable

Install the packages, note, fsevents is not used on linux images, so use–no-optional (these plugins are suspect).

npm install –no-optional –save react react-copy-to-clipboard react-dom react-ga classnames carbon @carbon/addons-website carbon-components carbon-components-react carbon-addons-cloud carbon-icons gatsby gatsby-theme-carbon-starter markdown-it gatsby-plugin-manifest gatsby-plugin-slug gatsby-plugin-sitemap gatsby-plugin-sharp

With ubuntu, you can’t use gatsby build directly per https://github.com/gatsbyjs/gatsby/issues/17557,

so I use the suggestion as a workaround due to path/issues in the gatsby component dependency of fsevents.

npm –prefix-paths run build

cp -R public/ ../../public/

Grab the GH-Pages branch

– name: Grab the GH Pages Branch

uses: actions/checkout@v1

with:

working-directory: gh-pages

ref: refs/heads/gh-pages

fetch-depth: 1

path: docs

token: ${{ secrets.GITHUB_TOKEN }}

Per Bypassing Jekyll on GitHub Pages, be sure to add the .nojekll to the root of the gh-pages. I added a guard in the shell script to check if the file is there, and create the file if it does not exist.

If you need Environment variables, you should add the environment variables to the step.

Add, Commit and Push to GH Pages

I add the gitignore and nojekll files while removing any cached files, before moving in the new files.

I also like to make sure when this runs there is a build.txt file to trace when the site is built. (This file contains the build time Thu Nov 21 19:39:49 UTC 2019

I then use the github environment variables passed in to push the contents to the repo the branch is from.
```
-name: Commit and Add GH Pages

run: |

echo "cleaning up the prior files on the branch"

if [ ! -f .nojekyll ]

then

touch .nojekyll

rm -f _config.yml

fi

rm -f *.js webpack.stats.json styles-*.js styles-*.js.map webpack-runtime-*.js.map webpack-runtime-*.js manifest.webmanifest component---*.js* app-*.js*

rm -rf docs/node_modules docs/public docs/.cache

echo "Moving the files around for gh-pages"

cp -Rf ../public/* ./

find .

date > build.txt

git config --global user.email "${{ secrets.GITHUB_ACTOR }}@users.noreply.github.com"

git config --global user.name "Git Hub Site Automation"

git add .

git commit -m "Update to GH-Pages"

- name: Push changes to GH Pages

run: |

echo "Push Changes"

git branch

remote_repo="https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"

git push "${remote_repo}" HEAD:gh-pages

env:

GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GITHUB_REPOSITORY: ${{ secrets.GITHUB_REPOSITORY }}

GITHUB_ACTOR: ${{ secrets.GITHUB_ACTOR }}

CI: true
```
Debugging and Replicating Locally

If you are troubleshooting, you can use a couple of approaches:

1 – Create a Docker Image

Create the Image

docker run -itd –name gatsby-node -v docs:/data node:latest

Copy the Files

docker cp ~/FHIR/docs 6d810efb3b586739932166d424641003ee9b238de506543fcdd47eb7e7d41699:/data

Launch the shell and try the build

npm install –no-optional –save react react-copy-to-clipboard react-dom react-ga classnames carbon @carbon/addons-website carbon-components carbon-components-react carbon-addons-cloud carbon-icons gatsby gatsby-theme-carbon-starter markdown-it gatsby-plugin-manifest gatsby-plugin-slug gatsby-plugin-sitemap gatsby-plugin-sharp

Run the gatsby build

npm –prefix-paths run build

2. If you want complicated navigation, refer to https://github.com/gatsbyjs/gatsby/blob/master/www/src/data/sidebars/doc-links.yaml however… gatsby-carbon-theme’s sidebar links only uses the to value not the href value.

3. If you have an issue with your deployment check a couple of things:

Check your Deployed Environment. You should see a deployment in the last few seconds.

Check your Settings You should see no issues, else investigate the site locally on the gh-pages branch, and check Troubleshooting Jekyll build errors for GitHub Pages sites

Best of luck with your build!
2019-11-27