Why we publish the scoring rubric where competitors can read it

The usual objection arrives within about ten seconds of anybody hearing that we publish the full scoring rubric on a public page: if the companies being scored can read the standard, cannot they just game it?

Yes. That is the point.

What “gaming” the rubric actually means

Read the tier requirements for the Conversion lens. To move from tier 3 to tier 4, a company has to offer more than one path matched to buyer readiness, with trust evidence at the point of decision and a stated response commitment.

A competitor who reads that and thinks “we should add a booking link alongside the form, put a response commitment next to it, and stop burying our proof in the footer” has not gamed anything. They have improved their conversion path. Their score goes up because the thing being measured got better.

That is what happens when a rubric measures the actual behaviour rather than a proxy for it. Gaming and improving become the same activity.

The rubrics you cannot publish are the ones measuring proxies. If your score depends on a keyword density or a page count or a follower number, then yes, publishing it invites people to manufacture the proxy without doing the work. The right response to that is not secrecy. It is to stop measuring proxies.

The real reason to publish

The reason is not generosity, and it is not confidence. It is that a hidden standard cannot do the job we need it to do.

A client is going to make real financial decisions from these scores. Budget reallocations, hiring, sometimes a repositioning. They are entitled to check the reasoning, and “trust our proprietary methodology” is not checking, it is deference.

Publishing changes the relationship in a specific way. It means:

A client can dispute a score on the merits. Not by questioning our judgement in the abstract, which goes nowhere, but by pointing at a requirement and saying we do meet that, here is the evidence you missed. That is a conversation that improves the report. It is only possible if they can read the bar.

We cannot quietly move the bar. If the standard is published and dated, moving it to produce a more convenient result is visible. That constraint is on us, and it is the most valuable thing publication buys.

The score means the same thing across clients. A tier 4 in one engagement and a tier 4 in another are the same claim, because they are awarded against the same published text. Without publication there is nothing keeping them aligned except memory and good intentions.

Absolute, which is what makes publication possible

None of this works with a curve.

If a score is relative to the cohort, there is no fixed requirement to publish. The requirement is “be better than these five other companies,” which changes every time the roster changes and cannot be written down in advance.

Our scale is absolute. A tier is earned by meeting that tier’s published requirements, full stop. Being better than the others is not a criterion. If an entire market lands at par, the grid is flat, and the flatness is the finding rather than a failure of the method.

That property is what makes the rubric publishable at all, and it is also what makes it useful. A ranked list tells you that you came third. An absolute standard tells you exactly what tier 4 requires and precisely what you are missing.

The evidence rule, which is the load-bearing part

Publishing tier descriptions alone would not be enough, because a description can be interpreted generously. The rule that stops that is about evidence rather than about judgement:

Three independent, dated, verbatim pieces of evidence are required to award tier 4 or tier 5.

This has a consequence that is not obvious until it bites you. A company that is thinly researched is structurally capped. If you have only gathered six pieces of evidence on a competitor, they cannot reach a top tier regardless of how good they actually are, because the proof required to award it does not exist in your file.

Which means a rubric with an evidence floor silently converts uneven research into a scoring advantage for whoever you researched hardest. Usually that is the client paying for the report.

We know this because it happened to us, and the fix became the parity gate: evidence depth across the cohort is measured automatically, and a report cannot publish as final until the field has been researched to comparable depth.

The published rubric and the parity gate are the same commitment approached from two sides. One says here is the bar. The other says here is the proof that we applied it evenly.

What publication costs

It costs the ability to claim a proprietary methodology as a moat, which is a real commercial thing to give up. Plenty of firms sell exactly that.

It is worth giving up, because a proprietary methodology is not actually a moat. The moat is the evidence pipeline: the grounding gate that verifies quotes byte for byte, the negative controls, the adversarial second pass, the parity measurement. A competitor can read our rubric in ten minutes. Building something that can populate it honestly, at depth, across a cohort, without inventing anything, is the hard part and it is not on a page.

Anybody can publish a rubric. Almost nobody can fill one in without flattering somebody.

The line we hold

There is a version of transparency that is theatre: publishing something vague enough to be unfalsifiable and calling it openness.

The test is whether a published standard can be used against you. Ours can. A client who reads a tier requirement and thinks we scored them too harshly has a concrete, specific thing to argue with, and if they are right the score changes.

If a standard cannot be used against the person who published it, it was marketing, not a standard.

The full rubric, every lens and every tier requirement, is on the methodology page. So are the metric definitions and the known limits. If any of it does not hold up, we would genuinely rather hear it.