feat(build): parallelize go-build-release and github actions with matrix#200
Conversation
f0fda34 to
b4c726e
Compare
|
😍 looks amazing! Are you good to go on this or were you adding anything else. Just checking as its a draft. |
|
Left it in draft mode because I want to figure out why the docker-build-release rule is still failing CI. |
|
I think its because qemu and buildx might not be setup for multi platform builds like we do in the cicd pipeline. But for tests I dont think we need to build docker on each platform - maybe we just have a separate docker build test command to build on the native amd64 for the test? |
|
So, I came to the realization that using The bottleneck for build time now is the Docker image build, which now runs in a minute, so that's much better than the 4-7 minutes it was taking before. |
oschwartz10612
left a comment
There was a problem hiding this comment.
LGTM only think we should still build latest tag for docker.
| echo "Error: tag is required. Usage: make docker-build-release tag=<tag>"; \ | ||
| exit 1; \ | ||
| fi | ||
| docker buildx build --platform linux/arm/v7,linux/arm64,linux/amd64 -t fosrl/newt:latest -f Dockerfile --push . |
There was a problem hiding this comment.
I think we still want to tag latest when we run the docker build release but IDK if we are even still using this in the cicd but could be good to have for other use.
There was a problem hiding this comment.
Hmm, I was a bit confused by this at first. I thought it was just a redundant run of buildx using a different parameter.
Was it actually necessary to both run this using a tag of latest and to require the tag var as well?
There was a problem hiding this comment.
To me those make sense to run as two separate rules in the matrix if that is the case.
There was a problem hiding this comment.
Actuall I think we can just double tag in the same command like:
docker buildx build --platform linux/arm/v7,linux/arm64,linux/amd64 -t fosrl/newt:latest -t fosrl/newt/$(tag) -f Dockerfile --push .
There was a problem hiding this comment.
Ah very cool, so the tag var should ideally still be required then for this rule?
b6ccad7 to
1dcb68d
Compare
|
LGTM! THANKS! |
Community Contribution License Agreement
By creating this pull request, I grant the project maintainers an unlimited,
perpetual license to use, modify, and redistribute these contributions under any terms they
choose, including both the AGPLv3 and the Fossorial Commercial license terms. I
represent that I have the right to grant this license for all contributed content.
Description
This PR splits the
go-build-releaseMakefile rule into individual targets based on arch, and composes them such that they can be parallelized usingmake -j.It also applies this parallelization to the pull request CI workflow using a build matrix with each Makefile rule being a separate entry in said matrix, which cuts CI times massively.
How to test?
go-build-releaseshould work the same as before, and CI times should be cut by a non-trivial number.