Hacker News new | past | comments | ask | show | jobs | submit login

With actions you can run multiple tasks in parallel, restart failed portions without retrying the whole CI, run certain sections with conditions like which branch you are on (different tasks when commit to master vs feature branch)



>With actions you can run multiple tasks in parallel,

If you mean "I want to run one build with the foo feature and one build with the bar feature. Actions lets me run those in parallel", then that is the "strategy" part of the workflow, not the "tasks" part. My comment was about the latter. ("and make your GH Actions run `make` in a script task.")

If you mean "I want to run two steps of a job in parallel and then run the rest of the job after they're complete", then shell is a much simpler DSL for that. Running things in parallel is literally a single `&` character.

>restart failed portions without retrying the whole CI, run certain sections with conditions like which branch you are on (different tasks when commit to master vs feature branch)

So can a shell script.


> Running things in parallel is literally a single `&` character.

Yes now try waiting for all parallel tasks, and error out if at least one errors. And try separating their output so that they don't get interleaved into a big mess. And try by default hiding the outputs of commands that succeeded, only showing those that failed, except when the user explicitly asks for it.

Your "simple" shell script now suddenly isn't so simple anymore.


Using GNU make you can get very nice output sync:

    all: variant-1 variant-2
        @echo done with all
    
    variant-%:
        @echo starting $@
        @sleep 1
        @echo done $@
Use `make -j2 --output-sync=target`.


    brew install make
    gmake -j2 --output-sync=target
on macOS.


>Yes now try waiting for all parallel tasks,

`wait`

>and error out if at least one errors.

`wait` propagates the exit status of the waited task. `set -e` triggers the script to exit on any command failure.

>And try separating their output so that they don't get interleaved into a big mess.

`| while read -r line; do echo "task X: $line"; done`

Learn the tools instead of being so confident that it can't be done or that it's complicated.


You remind me of that guy who said there's no point in Dropbox because one can "just simply setup an sftp server".


That’s what GNU Parallel is for. Or Pueue, which gives you a GH-Actions-level feature set but it’s less likely to be installed on any particular machine. Pretty sure you could fetch a pueue binary at the start of an actions script and do everything that way.

(These can’t project their tasks over multiple GH Actions runners, eg for multiple OSes. For that you will need to use the YAML. Good compilers will already do work in parallel and max out however many cores they are given, so multi machine is the main use case. Unfortunate.)


What I mean is if you have 50,000 unit tests, writing an actual CI config will let you split that up in to 20 jobs that all run at once and if a test fails, you can retry that 1/20th of the tests instead of the whole thing.

The CI runners aren’t multi core VMs I believe so you can’t just use standard shell utilities, you have to indicate to the CI system you want to run multiple tasks.


Yes, that's the "strategy" part.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: