proposal: move webhook to spindle

Context:

#320: proposal: trigger pipeline from spindle (not from knot)

currently, sh.tangled.pipeline event is emitted from tangled when git ref update triggered the pipeline. Conceptually this is unnecessary logic for knot. We can make spindle to do this instead.

1. spindle listen for sh.tangled.git.refUpdate from knot event stream
2. when ref update happens, fetch repo from knot, parse .tangled/workflows/*.yml and trigger the pipeline.

#526: proposal: make the trigger event payload accessible to pipeline workflow runs

For comparison, GitHub Actions trigger events are unified with their webhooks, so the full webhook payload is available in triggered workflow (example).

Could Tangled do something similar? https://docs.tangled.org/webhooks#webhook-payload

and maybe #368: proposal: spindle rewrite (sh.tangled.ci.* and "spindle adapters")

Spindle is the core of automation in Tangled. Its entire existence is to run CI. And webhook is also heavily related to the automation, usually serving as a trigger for third-party CI services. If triggering pipeline event from spindle makes sense, considering pipeline & webhook trigger are pretty identical, I think it makes sense to just move entire webhook logic to the spindle.

Users will still be able to use current appview webhook, but internal implementation will be changed to spindle1.

This will also solve the centralization problem around tangled webhooks. If someone cannot trust Tangled, they can just use other spindle service or run their own.

Technical problems

Spindle should listen to any events referencing the repository.

• git event (refUpdate)
• new issue/PR
• new fork
• new comment on issue PR
• issue/PR state change

git event is ok, spindle is already listening to knotstream for that.
Others, basically atproto events are the problem. Ideally we want spindle to only subscribe for events related to its repos. To do that we can either:

1. Just listen to entire firehose. Spindle requires quite high spec machine anyways.
2. Introduce tangled-specific, jetstream-like service that understands the schema and stream records only related to the git repos.

I personally think second option sounds more ideal. We will want that kind of service for various reasons.

tl;dr

• webhooks and pipelines are similar in both concept and implementation
• implement webhook from spindle and not appview. current appview webhooks will just use spindle1 under the hood.
• technical difficulties would be self-hosted spindles consuming repo-referencing records from the atmosphere.