Having gone down the worktree rabbit hole for a month or so, I am giving up in favor of multiple checkouts to enable many agents to work across many repos.
Worktrees worked great for me when my agents' work was mostly contained to a single repo (they got me to finally hitting rate limits, not that that was a goal). As my homelab scales, agents increasingly work across multiple repos, and that's where (my approach to) worktrees broke down; agents were spawned in a repo worktree and so avoided stepping on the toes of other agents in the same repo without any special instruction, but as soon as they needed to touch another repo, they would default to working in that repo directly, without a worktree, and thus collide with other agents, and muddy a merge process that expected the main repo clone to be clean (which turns out to have been an unnecessary design quirk, but resolving it would still not stop agents from stepping on each other's toes in secondary repos).
After a detour through ZFS dataset clones and some mounting magic that made /srv/src/ appear to be a distinct hierarchy for each agent process, I am simplifying even further and giving each agent a bare directory (sth like /srv/dev/<slug>/) where they can check out any repos they want. The git remote (/srv/git/) becomes the only integration point.
Maybe I should just bite the bullet and move into containers, but performant as they are the ergonomics still bum me out.
Edit: I may actually hold on to ZFS datasets with mounting magic. It does add a bit of complexity to dev tools, but it also reduces path-based trust bloat in harness configs, provides some interesting zfs features (zfs diff, etc)... And it's done.
Worktrees worked great for me when my agents' work was mostly contained to a single repo (they got me to finally hitting rate limits, not that that was a goal). As my homelab scales, agents increasingly work across multiple repos, and that's where (my approach to) worktrees broke down; agents were spawned in a repo worktree and so avoided stepping on the toes of other agents in the same repo without any special instruction, but as soon as they needed to touch another repo, they would default to working in that repo directly, without a worktree, and thus collide with other agents, and muddy a merge process that expected the main repo clone to be clean (which turns out to have been an unnecessary design quirk, but resolving it would still not stop agents from stepping on each other's toes in secondary repos).
After a detour through ZFS dataset clones and some mounting magic that made /srv/src/ appear to be a distinct hierarchy for each agent process, I am simplifying even further and giving each agent a bare directory (sth like /srv/dev/<slug>/) where they can check out any repos they want. The git remote (/srv/git/) becomes the only integration point.
Maybe I should just bite the bullet and move into containers, but performant as they are the ergonomics still bum me out.
Edit: I may actually hold on to ZFS datasets with mounting magic. It does add a bit of complexity to dev tools, but it also reduces path-based trust bloat in harness configs, provides some interesting zfs features (zfs diff, etc)... And it's done.