PineForge runs your Pine Script backtests in CI and fails the build when trades drift.
Why parity in CI
Strategies drift like any code. A parameter rename, a refactor of a helper function, a Pine v6 API change you didn’t notice: any of these can silently shift signals without breaking any existing test. Without a parity gate in CI, a commit you made two months ago can change last week’s trade list and you’ll only find out when the live P&L looks wrong.
The discipline of committing your reference trade list alongside your strategy source is the same discipline that makes software reliable: you describe the expected output, and your build fails if the actual output diverges. For a backtest, the expected output is the trade list (entry bar, exit bar, direction, size, fill price) against a pinned historical dataset.
When that comparison runs on every commit, you get a ratchet: your strategy’s historical behavior can only improve, never accidentally regress. You know exactly which commit changed the output, because the build failed on that commit. You have a git blame for your equity curve.
This matters more than most quants realize at first. The gap between “I tested this strategy” and “I have a reproducible, version-controlled record of every historical signal this strategy has ever produced” is enormous. The first is a screenshot. The second is an audit trail. CI is how you maintain the second without manual effort on every push.
PineForge fits here because the runtime is a Docker image: it runs wherever Docker runs, gives the same trades for the same inputs, and writes a JSON report you can parse with jq. There is no browser, no login and no rate limit on the backtest itself.
GitHub Actions example
The full workflow: check out your strategy, transpile it to C++ with the pineforge-release image, run the backtest against your pinned OHLCV CSV, parse the JSON report with jq, compare it with your committed baseline, and fail the build if a trade changes side or bar or the net PnL delta exceeds your threshold.
The runtime is a container image, so there’s no API key to manage. Just docker run it in the job. Create .github/workflows/backtest.yml:
name: Backtest parity
on:
push:
branches: [main, "feat/**"]
pull_request:
jobs:
backtest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
# Pin the image version so results only change when you change it.
- name: Transpile strategy to C++
run: |
docker run --rm --network=none \
-e PINEFORGE_TRANSPILE_ONLY=1 \
-v "$PWD/strategy.pine":/in/strategy.pine:ro \
ghcr.io/pineforge-4pass/pineforge-release:1.4.0 > strategy.cpp
- name: Run backtest
run: |
docker run --rm --network=none \
-v "$PWD/strategy.cpp":/in/strategy.cpp:ro \
-v "$PWD/data/ohlcv.csv":/in/ohlcv.csv:ro \
ghcr.io/pineforge-4pass/pineforge-release:1.4.0 \
> report.json
- name: Assert parity
run: |
# Every trade keeps its side and its entry and exit bars,
# and net P&L moves by at most THRESHOLD.
BARS='[.trades[] | [.side, .entry_bar_index, .exit_bar_index]]'
ACTUAL=$(jq '.summary.net_pnl' report.json)
BASELINE=$(jq '.summary.net_pnl' baseline/report.json)
DELTA=$(echo "$ACTUAL $BASELINE" | awk '{d=$1-$2; print (d<0)?-d:d}')
THRESHOLD="0.01"
if [ "$(jq -c "$BARS" report.json)" != "$(jq -c "$BARS" baseline/report.json)" ]; then
echo "Parity drift: a trade changed side, entry bar or exit bar"
elif awk "BEGIN{exit !($DELTA > $THRESHOLD)}"; then
echo "Parity drift: net_pnl delta $DELTA exceeds threshold $THRESHOLD"
else
echo "Parity OK - same sides and bars, net_pnl delta $DELTA (threshold $THRESHOLD)"
exit 0
fi
# One trade per line, so diff's line numbers are trade numbers.
TRADE='.trades[] | {side, entry_bar_index, exit_bar_index, entry_price, exit_price, qty, pnl}'
diff <(jq -c "$TRADE" baseline/report.json) <(jq -c "$TRADE" report.json)
exit 1Store your baseline report at baseline/report.json in the repository, committed alongside the strategy source. When a PR changes strategy behavior intentionally, the author updates the baseline as part of the PR. That diff becomes a permanent record of what changed and why.
The same workflow runs identically on GitLab CI, Bitbucket Pipelines, or any runner that supports Docker. Swap the YAML syntax, keep the shell commands.
Exit codes and asserting parity
The PineForge runtime exits with code 0 on a clean run and a non-zero code on failure: 2 for a missing input file, 3 for a compile failure, 4 for a backtest failure and 5 for a transpile failure (unsupported Pine or a syntax error). CI picks these up with no special handling.
Checking the result is up to your workflow. The report looks like this (trimmed):
{
"engine": "pineforge",
...
"summary": {
"total_trades": 3483,
"wins": 1288,
"losses": 2194,
"win_rate_pct": 36.9796,
"net_pnl": 2518.39,
"max_drawdown": -1382.76,
...
},
"trades": [
{
"n": 1,
"side": "long",
"entry_time": 1729402200000,
"exit_time": 1729422000000,
"entry_price": 2640.96,
"exit_price": 2645.15,
"qty": 1.0,
"pnl": 4.19,
"entry_bar_index": 34,
"exit_bar_index": 56,
...
}
],
"metrics": { "all": { "profit_factor": 1.06, ... }, "equity": { "sharpe_tv": ..., ... } },
"equity_curve": [ ... ],
"fingerprint": { "digest": "sha256:...", ... }
}For a simple numeric parity check, compare summary.net_pnl and summary.total_trades against the baseline. The workflow above also reads the trades array: each trade must keep its side and its entry and exit bars. When a check fails, the workflow diffs the two trade lists one trade per line, so diff’s line numbers are trade numbers and each line shows a trade’s side, bars, prices, quantity and P&L, before and after.
A practical threshold for numeric fields: accept floating-point deltas up to 0.01 (one cent of net P&L) and fail on anything larger. Trades get no tolerance: the workflow above fails on any change of side, entry_bar_index or exit_bar_index, whatever the P&L. A trade landing on a different bar means your signal logic changed.
A practical pattern: update the baseline in a separate workflow step gated on a PR label, so an intentional change shows up in code review next to the Pine change that caused it.