Methodology
v1 · effective July 2026Every score on this site is a number we can show our work for. Every workspace gets its own rating methodology, developed by the engine from that workspace's scoring configuration — the weights, normalization strategies, and data-quality policies below are read live from Orbital Rockets, and this page lets you reproduce any published score from the same data and math the ranking job uses. Worked examples use AI Model Rankings, the platform's reference workspace. See it applied: open the Orbital Rockets comparison.
- · Every metric value carries a source, a confidence level, and a verified date.
- · Scores are cohort-relative — normalized against the rockets published today, not a fixed scale.
- · The breakdown under every score is recomputed live from the same math on this page — never a cached explanation.
- · Changing the methodology gets a new version number and an effective date, not a silent edit.
Normalization
Each metric's raw value maps to a 0–100 score against the min and max of published rockets in the same cohort — not a hardcoded target. The mapping depends on the metric's normalization strategy:
- Linear
- Straight min–max scaling. A value at the cohort minimum scores 0, the maximum scores 100. Flipped when lower is better.
- Log-scaled
- Same min–max scaling, applied after a log(1+x) curve. Used for metrics that span orders of magnitude (context window, max output tokens) so a jump from 4K→128K tokens doesn't dwarf 128K→256K.
- Inverse
- A 1/x curve, min–max scaled. Built for price metrics, where it rewards cheap values. A metric flagged "higher is better" gets the same curve flipped, so the flag and the curve can't disagree.
- Pass-through
- The raw value is clamped into 0–1 as-is, for metrics already expressed on that scale. Boolean metrics (does it support tool calling?) skip normalization entirely: true/false maps straight to 100/0, flipped when lower is better.
Weights
Each metric's normalized score is multiplied by a weight and summed. This table is read live from the workspace's scoring config — if a weight changes, this table changes with it, on the next request. In AI Model Rankings, benchmark scores and pricing carry the most weight because they are what the ranking claims to be about; capability booleans carry the least, because they gate what a model can do rather than how well it does it.
| Metric | Weight | Direction | Normalization |
|---|---|---|---|
| LEO Capacityleo_capacity_kg | 25% | Higher is better | Log-scaled |
| Successful Launchessuccessful_launches | 25% | Higher is better | Log-scaled |
| Launch Costlaunch_cost_usd | 20% | Lower is better | Inverse (rewards low values) |
| Failed Launchesfailed_launches | 10% | Lower is better | Linear |
| Launch Masslaunch_mass_t | 10% | Higher is better | Log-scaled |
| Lengthlength_m | 5% | Higher is better | Linear |
| Diameterdiameter_m | 5% | Higher is better | Linear |
| Reusablereusable | 0% | Higher is better | Linear |
Weights sum to 100%. This is the exact config compiled into the site right now — not a snapshot.
Missing data
Not every rocket has a verified value for every metric. One of three policies decides what happens — this vertical currently runs exclude:
- penalize
- Missing metrics count as 0 against the full weight total. There's no way to improve a score by leaving a weak metric undocumented — an unverified value costs more than a disclosed bad one.
- exclude
- Missing metrics are dropped and the remaining weights are renormalized to 100%, as long as at least 50% of the total weight has a value. Below that the score is scaled down in proportion, so a sparse item can't outrank a fully documented one, and its page shows the coverage it was scored on.
- insufficient-data
- Behaves like
exclude, but returns no score at all when less than half of the total weight has data — better unranked than misleading.
Confidence adjustment
Every metric value is tagged with a confidence level when it's recorded. The base score is multiplied by the average of the confidence factors across every metric that had data:
| Confidence | Multiplier | Meaning |
|---|---|---|
| high | ×1.00 | Verified directly from the primary source. |
| medium | ×0.97 | Verified, minor uncertainty. |
| low | ×0.90 | Best available, not independently confirmed. |
| stale | ×0.85 | Was verified, but is past its freshness window. |
| conflicting | ×0.85 | Sources disagree. |
| unknown | ×0.80 | No confidence recorded. |
Enabled for this vertical: true.
Freshness adjustment
Data goes stale. Once the oldest verified value among the metrics that count toward the score passes 365 days, the score decays linearly — down to a 15% penalty once that data reaches 730 days old, where it bottoms out. One old scored metric drags down the whole item's score even if everything else was just verified — that's deliberate: it's the stalest data point that determines whether the score can be trusted. A value for a metric with no weight in the score is ignored, and the breakdown names the metric and date behind any penalty.
In AI Model Rankings, a scheduled agent cross-checks prices against three independent sources (models.dev, LiteLLM, OpenRouter) — don't take that sentence's word for it: every source card shows its own last verified timestamp, which is the actual evidence of when checking last succeeded.
Enabled for this vertical: true.
Score bands and thresholds
Every cut-off the engine applies is listed here, read from the same constants the scoring code uses. Because scores are normalized against the current cohort, one new rocket can nudge every other score. The hold margin and the rank-change rule below exist so a nudge is not reported as news. They reduce that noise; they do not remove its cause, which needs fixed per-metric anchors.
Score bands
- Strong
- 70
- A score of 70 or more.
- Fair
- 40
- A score of 40 or more, and under 70.
- Weak
- under 40
- A score under 40.
- Band hold margin
- 2 points
- When rankings are recalculated, an item keeps the band it held until its score falls 2 points below that band's minimum: strong holds down to 68, fair down to 38. Entering a band always takes the full minimum.
Rankings and decisions
- Rank change worth announcing
- 1 point
- Ranks are always published exactly. A move of a single place is left out of the changelog and rank alerts when the item's own score moved less than 1 point and its band did not change.
- Close call
- 5 points
- Two items within 5 points of each other are a close call. An alternative is only offered in place of the top pick when it is a close call or still scores at least 40.
- Minimum data coverage
- 50%
- Where missing metrics are excluded, an item needs published values for at least 50% of the score's weight to be scored at full strength. Below that its score is scaled down in proportion. A workspace may set its own minimum.
Limitations — what this score is, and is not
The score is a published policy applied to documented evidence — a weighted average with disclosed, editable parameters — not an objective measurement of which rocket is best. That distinction is standard in decision science, and it has concrete consequences we'd rather you read here than discover:
- — Strengths can buy back weaknesses. A weighted sum is compensatory: a terrible value on one metric can be offset by excellence elsewhere. If something is a hard requirement for you, don't read the ranking — use the filters or the decide tool, which fails an item outright on an unmet or unknown requirement.
- — Scores are relative to today's cohort. Every metric is scaled to the current min and max, so adding or removing one rocket can shift — and occasionally reorder — the scores of rockets whose data never changed. A single extreme value compresses everyone else on that metric (one free entry, for instance, flattens price differences among paid ones). For the same reason, scores aren't comparable across time or across workspaces: a 78 today and a 78 last month were measured against different cohorts. Which items we include is an editorial decision, and it moves scores.
- — Default weights are a starting point, not the truth. They originate from AI analysis and editorial review — not from elicited preferences of any user, and not validated by independent domain experts. A weight also says nothing about spread: a heavily weighted metric on which every rocket is nearly identical contributes almost nothing in practice. The published ranking is one defensible weighting among many — the what-if weights panel exists so you can substitute yours, and the winner-flip analysis shows how close the call is.
- — A low score can mean bad data, not a bad rocket. Confidence and freshness multiply the score down, so poorly documented merit and well-documented mediocrity can land on similar numbers. The per-item breakdown shows which is which. The confidence multipliers themselves (1.00 down to 0.80) express an editorial ordering, not measured error rates; they average across metrics regardless of weight; and a value labeled stale can be discounted twice — once by its label, once by the freshness decay. The 15% freshness penalty and its 2× window are policy constants, not estimated decay rates.
- — Missing data is handled by policy, and each policy has a cost. Under penalize, the score measures documented merit — an under-documented rocket ranks low even if it's excellent (deliberate, to reward disclosure). Under exclude, items are effectively scored on different metric subsets, so two scores may summarize different questions. Under insufficient-data, coverage below half the total weight means unranked — a hard line, not a gradient.
- — Bands are display buckets. Strong / fair / weak cut the cohort-relative score at fixed thresholds (70 / 40). "Strong" means strong within this cohort under this methodology — not against any external standard, and a rocket can change band because a rival entered the cohort.
- — The math makes simplifying assumptions. Adding normalized metrics assumes they contribute independently to overall value (interactions between metrics aren't modeled) and that the normalization curves — linear, log, inverse — are reasonable shapes, which is a modeling judgment, not an elicited fact. The winner-flip sensitivity is computed under a simplified linear model and locates flips to the nearest 5 percentage points, so it can differ slightly from the published configuration.
Why publish a single number at all? Because a transparent, reproducible, versioned policy you can inspect, contest, and re-weight beats an opaque one — not because any single number can settle what "best" means. Everything above is why the reproduce widget, the per-value provenance, and the editable weights exist.
What we don't do
- — Affiliate and sponsored relationships never influence a score. The scoring pipeline above has no input for "paid," "sponsored," or "featured." This vertical doesn't run affiliate links at all.
- — No manual override on a published score. The only way to change a ranking is to change the underlying metric data, or the methodology itself — which gets a new version number.
- — Agent- and import-suggested data never publishes directly. It queues for human review first. Every public fact requires a cited source.
Reproduce this score
Pick any published rocket below. The breakdown is recomputed right now, from the live database, using the exact pipeline documented on this page — the same one the item page's score breakdown and the ranking job both use.
Soyuz URussian Federal Space Agency (ROSCOSMOS)
View full page →OptiSift Score breakdown
How this score was calculated — weighted metrics, normalized against the full dataset.
Data coverage 80% of the score's weight has a published value.
Every comparison, its own methodology
The engine develops a rating methodology with each workspace — same math, that workspace's own metrics, weights, and policies. Open any of them:
| Workspace | Items | |
|---|---|---|
| AI Model Rankings | 17 | Methodology·Comparison |
| Agent Skill Rankings | 0 | Methodology·Comparison |
| Golf Cart Comparisons | 65 | Methodology·Comparison |
| Crypto Markets | 20 | Methodology·Comparison |
| GPU Cloud Comparison | 82 | Methodology·Comparison |
| Largest US Banks | 1 | Methodology·Comparison |
| Orbital Rockets | 19 | Methodology·Comparison |
| Top Federal Contractors | 18 | Methodology·Comparison |
| World Economies | 20 | Methodology·Comparison |
See the full rankings, or read more about data provenance and human review on the transparency page.