Git forge, own VCS engine

Cairn

A self-hosted Git forge built on a version control engine written from scratch instead of shelling out to git. It reads and writes real packfiles, resolves deltas, walks commit graphs and speaks the smart HTTP protocol, so an unmodified client can clone and push against it. The web layer sits directly on that engine, which means blame, diff and history are computed in process rather than parsed out of subprocess output.

Printed plate for Cairn: a stack of engraved stones with a green node graph branching away from the topmost one.

The engine

Loose objects, packfiles, the index, refs and reflogs, all implemented directly. Delta resolution is the interesting part: a packfile stores most objects as a chain of deltas against a base, and resolving them efficiently means processing in an order that keeps the working set small rather than in the order they appear.

Commit graph traversal is where a forge lives or dies. Generation numbers are computed and cached so that reachability questions, which are the ones a forge asks constantly, do not degrade into a full walk of history.

Speaking the protocol

Smart HTTP means implementing reference advertisement, want and have negotiation, and packfile generation. A real git client is a demanding peer: it will use every capability it advertises, and it will fail in unhelpful ways if the negotiation is subtly wrong.

The test for this is not a unit test. It is a matrix of real git versions cloning, fetching, shallow fetching, pushing, force pushing and pushing with a stale ref, against a server that starts from an empty directory each time.

Why not shell out

Forges that call the git binary pay for a process, a fork, and a parse on every page view, and they are limited to what the porcelain chooses to print. Working against the object store directly means blame can share a traversal with history, and a diff can be computed once and rendered three ways.

It also means every bug is mine, which is the trade being made.