Boston • New York • Detroit

What Should a Recommendation Marketing Pilot Be Willing to Disprove?

Learn what an honest Recommendation Marketing pilot should be willing to disprove and the six commitments a buyer should require before saying yes.

If a pilot is designed only to confirm the sale, it is not designed to teach you anything. Most pilot conversations begin with the same question: What will success look like?

Fair question. It is not the first one I would ask.

Before a business puts time, money, and reputation behind a test, I want to know what would make us stop. What would tell the provider, the client, and everyone who wants the idea to work that it did not work here?

If nobody can answer that before the work begins, the pilot is not designed to learn. It is designed to produce a favorable story.

That risk is not theoretical. In a 2024 RAND study based on interviews with 65 experienced data scientists and engineers, 84% of interviewees cited leadership-driven causes as a primary reason AI projects fail. Those causes included choosing the wrong business problem and setting expectations the work could not responsibly meet. [1]

That is not a failure rate for Recommendation Marketing pilots. It is a warning that the most consequential pilot mistakes may happen before anyone measures a result.

A Recommendation Marketing pilot should be willing to disprove its own premise. That is what makes the result worth believing, whichever way it points.

TL;DR

  • A pilot is a verification cycle with a business decision attached.
  • Its product is not a favorable result. Its product is a credible decision: continue, adjust, or stop.
  • Before anything is measured, both parties should finish the sentence: “We will conclude this did not work here if…”
  • The questions, baseline, decision rule, and obligations must be agreed before anyone knows which outcome will look best.
  • The standard applies to any provider, including Right Angle.

A Pilot Buys a Decision, Not a Result

Last week I wrote about Recommendation Marketing’s first verification cycle: what it should measure and what it has no right to claim.

A pilot uses the same discipline. The buyer questions are fixed before the work begins. The baseline records the starting position and normal movement. The comparison returns to the same questions under conditions that are as consistent as we can responsibly make them.

The difference is not the machinery. The difference is the consequence.

McKinsey’s 2025 workplace AI research makes the distinction useful. Among 238 C-level executives surveyed, 39% described their organizations as emerging, where generative AI pilots were beginning to show value. Only 1% described their rollouts as mature and driving substantial business outcomes. [2] The respondents were leaders of large organizations, so I would not apply those percentages directly to a growth-stage business. I would carry forward the distinction: a pilot beginning to show value is not the same thing as mature business impact.

When the pilot ends, somebody has to decide something. Continue. Adjust. Stop. Spend more. Spend differently. Walk away.

That changes the room.

The provider wants the work to continue. The client wants the choice to look wise. The internal champion wants to defend the time spent getting everyone there. None of that makes anyone dishonest. It does make everyone invested in the answer.

Recommendation Measurement gives the room a standard it cannot quietly rewrite after seeing the result. The pilot buys a decision, not a result. If the decision was already made, the measurement is decoration.

What Is the Pilot Actually Testing?

Recommendation Marketing is the discipline of building a business that buyers and AI systems can understand, verify, consider, and recommend.

A pilot inside that discipline tests a narrower premise: that defined improvement work, applied to this business, can move its standing on a governed set of buyer questions beyond the normal variation recorded before the work began.

That sentence matters because it can be wrong.

Before the pilot begins, both sides should be able to complete this sentence:

We will conclude this did not work here if…

A blank field inside a fixed frame, with a small seal-like geometric mark to represent an agreed condition set in advance

The ending has to be observable. It cannot be “if we do not feel enough momentum” or “if the results are disappointing.” Those phrases leave the real decision for later, when everyone has a reason to reinterpret what happened.

The most trustworthy sentence in a pilot proposal is the one that tells you when the provider would recommend that you stop paying.

Five Commitments I Would Require Before the Pilot Begins

1. Freeze the buyer questions first

Decide which buyer questions matter before anyone knows which answers will look favorable. That is the Frozen Question Set. Stakes are not a reason to loosen the questions. They are the reason to lock them.

2. Record the baseline before improvement work

I want the starting position and the normal movement recorded before anyone begins changing the content, evidence, website, or market presence. Without that baseline, the “before” picture becomes a memory—and memory is remarkably cooperative when a relationship is on the line.

3. Write the decision rule in advance

What will both sides do if the questions move beyond the normal range? What happens if they remain flat? What happens if nearly everything moves at once and the pattern looks more like system drift than a change in the business’s standing?

The details will vary by engagement. The branches should not.

4. Put both sides’ obligations on the page

A provider has work to do. So does the client. Access, approvals, evidence, internal decisions, and permission to make agreed changes all affect whether the pilot has a fair chance to teach us anything.

If the client’s obligations are vague, a flat result arrives with a dispute already attached. The provider blames delay. The client blames the work. The measurement cannot settle a responsibility nobody wrote down.

5. Let the pilot end

A pilot needs a defined conclusion and a recorded decision. No quiet rollover. No endless extension because the result was almost good enough. No waiting for one more favorable observation to rescue the story.

The ability to honor a stop decision is what gives a later yes its value.

What a Successful Pilot Still Does Not Prove

A pilot can succeed on its own terms and still have limits. Money and attention do not make two points into a trend, movement into causation, or an early result into a forecast.

It does not prove Recommendation Marketing

One pilot at one business is one governed observation. A good outcome does not prove the discipline. A disappointing outcome does not disprove it. We do not get to claim the favorable result as universal proof and explain away the unfavorable one as an exception.

It does not price the ongoing relationship

The order still matters: measurement first, accountability second, pricing third.

The pressure to move faster is real. In The CMO Survey’s 2025 study of 281 U.S. marketing leaders, 99% at the vice-president level or higher, 63% reported increased pressure from CFOs to prove marketing value. Sixty-one percent reported greater scrutiny from CEOs, and 50% from board members. [3]

That pressure does not make an early result more complete. It makes a pre-agreed decision rule more important.

A pilot may begin earning confidence in the measurement. It does not give either party permission to build a long-term commercial model around one favorable interval.

It is not automatically a case study

A private decision becomes public evidence only with permission, with the conditions attached, and with the limitations as visible as the result. Until then, it is a private decision.

The Equation Keeps Us Honest

Visibility + Engagement = Opportunity.

A Recommendation Marketing pilot will usually observe the visibility side first: how the business is found, represented, cited, compared, and recommended across the governed buyer questions.

A three-part sequence with only the first segment lit, showing that visibility is observed while engagement and opportunity remain unproven

That matters. It is not the whole equation.

The pilot does not automatically tell us that a buyer cared, explored further, started a conversation, or became an opportunity. Those are later questions, with their own evidence.

The equation is a sequence, not a shortcut. A favorable visibility result does not earn the right to jump over engagement and claim opportunity.

The Buyer’s Pilot Checklist

If a pilot proposal lands on your desk, from an agency, a platform, an internal team, or Right Angle, you should be able to find six things before you say yes:

  1. The premise: what the pilot expects to happen, written so that it could be wrong.
  2. The failure sentence, agreed by both sides before observation begins.
  3. The Frozen Question Set and baseline, recorded before improvement work.
  4. The decision rule for movement, no movement, and likely drift.
  5. The obligations on both sides, including what the client must permit and by when.
  6. A plain statement of what the pilot will not claim, whatever the outcome.

Six identical checkpoints in a single row, representing the six items a pilot proposal should already answer

If the proposal describes success in detail but failure only in generalities, you already know what the pilot was built to produce.

Where Sightline Fits

Sightline helps Right Angle observe the visibility side of Recommendation Marketing consistently enough to compare. It examines how a business appears across governed buyer questions, which sources support the answer, how competitors are represented, and where the business may be absent or misunderstood.

That capability can support a responsible pilot design. It does not remove the need for a Frozen Question Set, a baseline, a decision rule, or restraint. And it does not turn visibility movement into proof of engagement or opportunity.

That is the standard I would require from any pilot, including one proposed by us.

The Question for the Comments

Think about a pilot you have run, funded, sold, or joined in any discipline.

Did anyone write down what failure would look like before it began? And did that make the final decision easier or harder?

Coming next: Who recommended you this year, and did you earn it?

Frequently Asked Questions

What should a Recommendation Marketing pilot be willing to disprove?

Its own premise: that defined improvement work can move a business’s standing on governed buyer questions beyond the normal variation recorded before the work began. The failure condition and the decision it triggers should be agreed in advance.

How is a pilot different from a first verification cycle?

The measurement discipline is similar: frozen questions, a baseline, and a scheduled comparison under consistent conditions. A pilot attaches a business decision to the outcome, which increases the pressure and therefore the need for pre-agreed rules.

What makes a pilot proposal credible?

A premise that could be wrong, a failure sentence, a decision rule written before the results, obligations on both sides, a defined ending, and visible limitations. A proposal that can only produce “success” is not designed to test anything.

Should a successful pilot determine long-term pricing?

Not by itself. Measurement first, accountability second, pricing third. One pilot can begin building confidence in the measurement; it cannot establish a durable commercial model or forecast future performance.

What should happen when a pilot ends?

The decision it was designed to produce: continue, adjust, or stop, based on the rule both parties agreed before seeing the outcome. The result and its limitations should be recorded whichever way the decision points.

Sources

  1. RAND Corporation. The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed. 2024. Interviews with 65 experienced data scientists and engineers; the 84% finding refers to interviewees citing one or more leadership-driven causes as a primary reason AI projects fail.
    https://www.rand.org/content/dam/rand/pubs/research_reports/RRA2600/RRA2680-1/RAND_RRA2680-1.pdf

  2. McKinsey & Company. Superagency in the Workplace: Empowering People to Unlock AI’s Full Potential. 2025. Survey of 3,613 employees and 238 C-level executives conducted in October and November 2024. The maturity-stage figures cited in the article come from C-level respondents; all U.S. C-suite respondents worked at organizations with at least $1 billion in annual revenue.
    https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work

  3. The CMO Survey. Marketers Claim a Broader Role and Increased Influence Amid Pressures. April 1, 2025. Findings are based on 281 marketing leaders at for-profit U.S. companies; 99% held vice-president-level or higher roles.
    https://cmosurvey.org/marketers-claim-a-broader-role-and-increased-influence-amid-pressures/

We're Here to Help.

More Posts

What Is Marketing Actually For?

Marketing closes the gap between what buyers know and what businesses can prove. See the foundation beneath Recommendation Marketing.

Visibility + Engagement = Opportunity.

We build engagement engines that turn visibility into connection, and connection into opportunity.

Get In Touch

Boston • New York • Detroit