Reviews
Part of When to trust B2B demand generation reviews and when to verify claims
Scoring a B2B demand generation review methodology without vendor spin
How to score a B2B demand generation review methodology without vendor spin, separating independent evidence from claim, with a worked example and compliance checks.
What to take away
- A Manchester agency shortlist landed on a desk this week with three proposals, three case studies and almost no way to tell which numbers were verified, so the scoring method matters more than the pitch deck.
- Score reviews on what was assessed, who assessed it and what evidence sits behind each claim, not on how confident the wording sounds.
- Keep vendor claims, independent evidence and editorial judgement in separate columns so a strong claim cannot quietly outrank a weak proof point.
- Treat compliance as a scored criterion, not a footnote, because enforcement records are public and dated.
- No hands-on testing is claimed here; this is a method for reading and comparing published material.
Separate the three kinds of statement
Every line in a demand generation review falls into one of three buckets.
Three Kinds of Statement
Vendor claim
- Source
- Provider itself
- Example
- Pipeline influenced
- Checkable
- No
- Risk if mixed
- Claim reads as proof
Independent evidence
- Source
- Dated public record
- Example
- Filing or regulator notice
- Checkable
- Yes
- Risk if mixed
- Buried under assertion
Editorial judgement
- Source
- Reviewer's reasoning
- Example
- Reasoned view
- Checkable
- Labelled as such
- Risk if mixed
- Borrows false authority
Vendor claims are assertions the provider makes about itself, such as pipeline influenced or cost per qualified lead. Independent evidence is a dated, checkable source: a filing, a regulator notice, a published standard, a named client reference. Editorial judgement is the reviewer's reasoned view and should be labelled as such.
When the buckets are mixed, a confident claim reads like proof. When they are separated, a reader can see how much of the score rests on assertion alone. Our wider guide to B2B demand generation: reviews and comparison methods sets out how those buckets feed a comparison table.
Score the process, not the polish
A defensible method asks four questions of each review. What was assessed? Who did the assessing, and what access did they have? What would change the score? When was it last checked? A review that cannot answer all four is a summary, not an assessment.
Set weights before you read the material, not after. For example, a buying team might weight evidence quality at 40 per cent, compliance record at 25 per cent, commercial terms at 20 per cent and reference depth at 15 per cent. Fixing weights early stops a persuasive final meeting from shifting the ranking.
Put compliance in the score
The Information Commissioner's Office publishes enforcement action against organisations that breach marketing rules, and those notices are dated and public. A supplier with an unresolved notice should score lower on compliance, and the reviewer should say so plainly.
The Advertising Standards Authority also names non-compliant online advertisers, which is useful when a provider's own advertising is part of the pitch. Checking both registers takes minutes and gives the review a factual spine.
Worked example with labelled figures
Suppose a Leeds software firm scores three agencies on a 100-point method. Evidence quality carries 40 points, compliance 25, commercial terms 20 and references 15.
Agency A scores 30, 25, 15 and 10, giving 80, while Agency B scores 34, 15, 17 and 12, giving 78. Agency C scores 22, 25, 18 and 9, giving 74.
The illustrative fee gap is £3,000 a month, or £36,000 a year, between the cheapest and dearest proposal. Agency B looks strongest on paper, but its lower compliance score reflects a live advertising ruling. The buying team wants a written remediation plan before proceeding.
That is the point of a method: it surfaces the question rather than hiding it behind an average.
What was not assessed
No hands-on testing was carried out for this method, and no provider systems were accessed. Platform capability, team seniority and delivery quality were not independently verified. Vendor claims about results remain claims unless a dated source supports them.
The method does not score cultural fit or chemistry, which many buyers weigh heavily. Put those in a separate, clearly labelled section so they do not borrow authority from scored criteria.
Where AI features are central to a pitch, reviewers should check whether staff hold relevant training, such as the AI marketing courses from the Chartered Institute of Marketing.
Common questions
Can a review score a provider it has never worked with?
Yes, if the review is explicit that it assessed published material only. Scoring evidence quality, compliance records and commercial terms does not require a working relationship, but the limits must be stated.
How should a reviewer handle a vendor's own case study?
Treat it as a claim unless the client is named and the figures are dated. If the client cannot be contacted, the case study can support a narrative but should not carry the evidence score. Our England-focused piece on B2B demand generation agency reviews in England shows how that distinction changes a shortlist.
How often should a review methodology be revisited?
At least annually, and whenever a regulator publishes new enforcement or the provider changes ownership. A score built on evidence older than twelve months should be marked as stale.
What single change improves most review methods?
Separating claims from evidence. It is the cheapest fix and it removes the most common source of inflated scores.
