Channel testing

# Channel testing

## Goal

Design a fair test of acquisition channels so the company can keep the few that produce beachhead conversations and commitments at a cost it can bear, and kill the rest.

This is experiment design. Do not buy ads, send outreach, or declare a winner from imagined results. A testing plan is not a growth engine.

Complete the analysis autonomously. Do not stop to ask clarifying questions. When evidence is missing, make the narrowest assumption that permits a test or `HOLD`, label it, and state what changes if it is false.

Do not modify application code.

## Prerequisites

Read the latest ICP, market, positioning, pricing, campaign, and nurture files; any historical channel notes, spend, or lists in the repository; founder time and cash constraints; and any existing `docs/gtm/channels.md`, `docs/gtm/channels.yaml`, and `docs/gtm/channels-changelog.md`.

Use the market audit’s reachability work. Do not test a channel with no beachhead concentration just because it is fashionable.

If no campaign plan exists, this prompt may propose tests, but each test must still name the story, offer, and commitment it is testing.

## Evidence standard

Use the shared GTM labels.

A channel is not winning because it is cheap to start, easy to post on, or praised by other founders. It is winning when it produces qualified, on-ICP commitments at a fully loaded cost the offer can survive.

Separate:

* cost to try the channel
* cost per conversation
* cost per qualified account
* cost per commitment
* cost per activated customer

Do not collapse these into one CAC guess.

## Step 1: Name the candidates honestly

List channels already in the campaign plan plus any historically used. For each:

* beachhead concentration
* access method
* time and money to a first fair test
* expected false-positive type
* whether past evidence exists, including failures

Reject vanity channels with no path to the economic buyer or champion.

## Step 2: Pre-register the tests

For each channel you will actually test in the next 30–60 days, write:

* hypothesis
* audience definition
* the exact offer and commitment
* treatment: what will be said and shown
* sample or time box
* primary metric and secondary metrics
* confirmation threshold
* falsification threshold
* kill / iterate / scale rule
* fully loaded cost model, including founder hours

Run as few tests as capacity allows. Two honest tests beat six theatrical ones.

Do not A/B a headline when the channel, audience, or offer is still untested. Test one variable per test unless the repository already has a stable baseline.

## Step 3: Define what may not be learned yet

State the questions this round cannot answer: long-term CAC, brand, partner quality, or retention by channel. Do not let a two-week test claim those.

## Step 4: Comparison rules

Channels may be compared only when they ask for the same commitment from the same beachhead definition. Do not crown content “better than outbound” if one asked for a click and the other asked for a paid pilot.

If historical numbers exist, keep them, label their definitions, and do not mix incompatible denominators.

## Step 5: Desk-test

1. Each test can fail.
2. Each test names a commitment, not an impression.
3. Cost includes founder time.
4. The audience is the beachhead.
5. Claims in creative stay under the evidence ceiling.
6. Capacity can run the tests and the resulting conversations.

## Decision

* `TEST`: named tests may be prepared
* `TEST NARROWLY`: one channel only
* `HOLD`
* `KEEP`: only if observed repository evidence already meets confirmation thresholds; then say what is already proven
* `KILL`: a channel already failed on observed evidence and should not be retested without a new hypothesis

Lead with:

“As of [date], channel testing for [beachhead] will [test / hold / keep / kill] [channels], measuring [commitment] at fully loaded cost, with decision [TEST / TEST NARROWLY / HOLD / KEEP / KILL].”

## Deliverables

### 1. `docs/gtm/channels.md`

Candidate review, pre-registered tests, incomparable metrics called out, desk tests, open questions.

### 2. `docs/gtm/channels.yaml`

`version`, `status`, `decision`, `candidates`, `tests`, `comparison_rules`, `cost_model`, `confirmation_conditions`, `falsification_conditions`, `assumptions`, `unknowns`, `sources`.

`status: planned` unless appending later observed results. Never invent outcomes.

### 3. `docs/gtm/channels-changelog.md`

Append only.

## Boundaries

* Do not spend money or contact people.
* Do not forecast conversion rates as facts.
* Do not declare product-market fit from a channel test.
* Do not retest a killed channel with the same hypothesis.
* Do not optimize for lowest CPL if the leads are off-ICP.
* Preserve uncertainty.

## Done when

* Tests are pre-registered and capacity-constrained.
* Metrics distinguish attention from commitment.
* Fully loaded cost is defined.
* Desk tests passed or forced `HOLD`.
* The three files agree and contain no fabricated results.