Methodology White Paper

From Survey Response to Ranking

How CONSUMATICS converts individual survey answers into transparent, statistically-grounded brand rankings — the complete calculation engine, step by step.

Overview: the seven-stage pipeline

Every published ranking is the output of a fixed, auditable pipeline. Raw answers enter at Stage 1; a rank position leaves at Stage 7. Each stage is deterministic and reproducible — given the same response data and parameter set, the engine always returns the same ranking.

#	Stage	Input → Output
1	Response capture	Questionnaire answers → validated response rows
2	Quality filtering	Response rows → clean rows (bots/straight-liners removed)
3	Attribute scoring	2–3 items per attribute → one score per attribute per respondent
4	Key-driver weighting	Attribute scores → attribute weights (per sub-category)
5	Weighted brand score	Weights × brand attribute means → raw brand score
6	Bayesian shrinkage	Raw score + sample size → stabilised score
7	Ranking & aggregation	Stabilised scores → sub-category ranks → category ranks

Response capture

Surveys are designed per sub-category — for example, a dedicated questionnaire exists for Inverter Air Conditioners, not for the broad Appliances category. Each questionnaire contains 8–15 questions grouped so that 2–3 questions measure each of the five core attributes. Respondents authenticate via Google sign-in, guaranteeing one response per person per survey.

Attribute	Example items	What it captures
Quality	Q1, Q2, Q3	Build quality, materials, performance
Value	Q4, Q5	Price-to-worth, value for money
Design	Q6, Q7	Aesthetics, usability, form factor
Service	Q8, Q9	After-sales, support, warranty handling
Reliability	Q10, Q11	Durability, consistency, failure rate

All rating items use a common 1–5 scale (Likert or star), so they are directly comparable and averageable. Non-rating items (multiple choice, free text) are captured for analysis but do not enter the score.

Quality filtering

Before any scoring, low-quality responses are removed so they cannot distort results:

Attention checks — each survey embeds one instructed-response item (e.g. "select 'Agree' for this question"). Failing it voids the response.
Straight-lining — respondents who give the identical answer to every item (zero variance) are flagged and dropped.
Speeding — completions faster than one-third of the median duration are removed as likely non-genuine.
Duplicate identity — enforced at capture by Google authentication.

Attribute scoring

For each respondent, the score for an attribute is simply the arithmetic mean of that attribute's items. If a respondent rated Quality on Q1=5, Q2=4, Q3=4, their Quality score is 4.33.

attribute_score(r, a) = mean( items of attribute a answered by respondent r )

This collapses 8–15 raw answers into exactly five attribute scores per respondent: Quality, Value, Design, Service, Reliability. To reduce the influence of extreme responses, each attribute's distribution is winsorised at the 1st and 99th percentiles — outliers are capped to those bounds rather than deleted, preserving sample size.

Key-driver weighting (per sub-category)

Not every attribute matters equally to consumers, and the balance differs by product type — design drives smartphone choice far more than it drives refrigerator choice. So the engine learns the weights from the data using a key-driver regression, run independently for each sub-category.

Each respondent's overall satisfaction rating is regressed on their five attribute scores:

overall = β₀ + β₁·Quality + β₂·Value + β₃·Design + β₄·Service + β₅·Reliability + ε

The standardized coefficients (β) show how strongly each attribute drives overall satisfaction. They are converted into weights as follows:

Fit with OLS; if predictors are collinear (VIF > 5), switch to ridge or PLS regression for stability.
Take standardized betas so attributes on the same scale are comparable.
Floor negatives at 0 — a negative driver is treated as "no positive contribution".
Rescale so the five weights sum to 1.

Provisional vs. Confirmed. A sub-category's learned weights are only trusted once the model explains enough variance (R² ≥ 0.5) and has ≥ 200 responses. Until then the engine falls back to equal weights (0.20 each) and the sub-category is flagged Provisional.

Weighted brand score

Within a sub-category, each brand's five attribute means (averaged across its respondents) are combined using the learned weights into a single raw score:

raw_score(brand) = Σₐ ( weight_a × brand_mean_a )

Example — an Inverter AC brand with weights Quality 0.27, Value 0.26, Design 0.10, Service 0.18, Reliability 0.19 and matching brand means of 4.5, 4.4, 4.2, 4.3, 4.6 yields a raw score of 4.42.

Bayesian shrinkage

A brand with only 12 responses should not leapfrog a brand with 900 just because a handful of enthusiasts rated it highly. Bayesian shrinkage pulls small-sample scores toward the category average until enough evidence accumulates:

score = ( n / (n + C) ) × brand_mean + ( C / (n + C) ) × category_mean

Here n is the brand's response count and C is the shrinkage constant (default C = 40). When n is small, the category mean dominates; as n grows, the brand's own mean takes over. The result is a stabilised score that is fair across very different sample sizes.

Brand responses (n)	Weight on brand's own mean	Weight on category mean
10	20%	80%
40	50%	50%
120	75%	25%
400	91%	9%

Ranking & category aggregation

Brands are ranked within each sub-category by their stabilised score. Two further rules govern eligibility and ties:

Eligibility thresholds

Fewer than 30 responses → brand does not appear.
Between 30 and 100 responses → brand appears but is flagged "Emerging — limited data".

Tie-breaker cascade (applied in order)

Larger sample size (n)
Narrower confidence interval
More recent responses
Alphabetical (final fallback)

Category aggregation. A category ranking (e.g. Appliances) is never surveyed directly — it is built up from its sub-categories. Category attribute weights are the response-weighted average of the sub-category weights:

category_weight_a = Σ_sub ( subcategory_weight_a × n_sub ) / Σ_sub n_sub

Because the aggregate is weighted by response count, sub-categories with more data have proportionally more influence on the category-level picture — while every sub-category still keeps its own independent, directly-surveyed ranking.

Update cadence & auditability

Two clocks run in parallel:

Running scores update on every new validated response, so internal dashboards always reflect the latest data.
Published rankings refresh on a fixed nightly rebuild (and on-demand when an admin triggers the engine), preventing the public table from flickering with every single submission.

At the close of each quarter the engine writes an immutable snapshot — the full ranking, the response count, and the exact weight-model version used. These snapshots power the rise-and-fall trend charts and make every historical ranking fully reproducible and auditable.

Two data sources, one engine. The database ingests data from two separate streams — (1) live visitor survey responses collected on the website, and (2) supplementary datasets uploaded by administrators as CSV/Excel. Both are stored separately in the cloud, tagged by source, then read by the same calculation engine so the pipeline above applies identically regardless of origin.

This document describes the CONSUMATICS calculation engine as configured in the admin panel. Parameters shown (C = 40, min sample = 30, emerging threshold = 100, R² gate = 0.5) are the current defaults and are adjustable by authorised Data Analysts and Super Admins.