Mixed Methods & Evidence-Backed Segmentation

Why Mixed Methods Usually Break in Practice

The phrase sounds sophisticated. Mixed methods. In reality the failure mode is painfully ordinary. The analytics team runs a funnel and produces a cohort split. The research team runs interviews and produces an empathy-heavy readout. Each team is internally rigorous. The organization still ends up confused because the two outputs cannot dispute, confirm, or refine each other at row level. They only meet as presentations.

That is not a methods problem. It is a storage problem. If quant and qual do not land in one evidence system, the reconciliation step becomes theater. Somebody says, "The survey proves users love self-serve." Somebody else says, "The interviews say they hate it." Nobody can see whether those statements are about the same cohort, the same time window, the same definition of success, or even the same service moment.

ASD's answer is blunt: one register, three evidence kinds, one artifact pipeline. Survey toplines arrive as telemetry because they are measured distributions. Survey instruments, questionnaires, store procedures, and consent forms arrive as document. Human words arrive as quote. The boundary is not academic. It is what lets me fuse a wait-time extract, a field note, and a verbatim complaint into one evidence-backed node without inventing a fourth class called "insight" halfway through.

Vesper Pharmacy makes the point cleanly. The chain's dashboards said click-and-collect should be the low-effort channel. The survey said customer effort was worse than walk-in. Interviews said the ready-notification could not be trusted. Store operations said the problem was customer unreliability. All of those claims were individually plausible. None were useful until the evidence entered one register and could be cited together inside the same artifact rows.

Telemetry, Survey Data, and Documents Under One Roof

The second-edition discipline matters here because the platform never added a special survey evidence kind. That omission is wise. A survey produces several kinds of evidence. The raw instrument is a document. The distribution of answers is telemetry. Free-text comments are quotes. Treating the whole survey as one mystical object is exactly how teams hide category errors inside a respectable method name.

In Vesper, for example, a ready-notification timing extract is telemetry. A store SOP that explains when staff are supposed to move orders onto the shelf is a document. A shopper saying, "The app said ready. It was not ready. So now I phone," is a quote. The mixed-methods job is to place those three rows next to each other and let them argue honestly. The analytics row tells me how often the failure happens. The document row tells me what the organization thinks should happen. The quote tells me what the failure costs in trust.

Evidence-strength tagging is the underrated part of this pattern. In Vesper's triangulation matrix, a vivid interview line never outranked a weak distribution on charm alone. A theme reached strong only when the quant row had adequate sample and effect, the qual row had at least three independent sources, and operations corroborated the same mechanism. Moderate meant two streams. Weak meant one. That sounds procedural, but it changes the politics of synthesis. The loud anecdote loses its ability to hijack the room. The rare but severe event can still stay visible, yet it stays labelled as rare until more evidence upgrades it.

Conflict handling follows the same logic. If telemetry says wait time is short and interviews say waiting feels endless, I do not average them into mush. I ask what they are each measuring. Usually one is measuring duration and the other legibility. At Vesper that distinction mattered because queue opacity, not raw minutes, explained the low CSAT in the fast elderly cohort. The conflict was the insight.

asd evidence-add --projectId vesper-pharmacy-1c7293aa   --source pickup-events-2026q2   --kind telemetry   --body "Prepared orders no-show rate 18 percent; first-time click-and-collect no-show rate 31 percent; premature-ready cohort 2.3x higher than properly shelved orders."

asd evidence-add --projectId vesper-pharmacy-1c7293aa   --source sop-ready-shelf-v4   --kind document   --body "Orders should be scanned onto the ready shelf before the customer-facing ready notification is sent."

asd evidence-add --projectId vesper-pharmacy-1c7293aa   --source int-31   --kind quote   --body "The app said ready. It was not ready. So now I phone, which defeats the point."

Notice what this does to interpretation. I no longer have to choose between numbers and stories. I can write an insight node that cites all three. When the Critic later reviews the artifact, it can ask a real question: does the proposed claim follow from the telemetry, the document, and the quote together, or is the team leaning too hard on one stream?

Kestrel Energy followed the same rule at much larger scale. Daily aggregate usage distributions, portal-usage shares, switching rates, and contact volumes became telemetry. The metering mandate and the data-protection agreement became documents. Interviews and diary excerpts stayed as quotes. That separation is what let the team combine 2.4 million household records, 1.9 million contact logs, 400 interviews, 60 diary studies, and 28,400 survey verbatims without turning the segmentation into a bag of incompatible formats.

From Coded Evidence to Segments That Can Survive Audit

Segmentation gets sloppy when people jump straight from raw data to labels. I do not. I want an intermediate layer of coded evidence and observable behavioral dimensions. Otherwise the cluster is just a prettier demographic bucket. The whole point of mixed methods is to carry the behavioral signal from the raw evidence into the segment definition.

At Vesper, that meant joining quant and qual around explicit dimensions: trust in status messages, channel preference under uncertainty, readiness to wait, sensitivity to queue opacity, and tolerance for self-serve. Some of those dimensions came from telemetry first. Some emerged from interviews first. What mattered was that each dimension had to be backed by evidence rows rather than by organizational lore. For Kestrel, the dimensions expanded into payment posture, channel posture, contact posture, price posture, and consumption posture, but the logic stayed the same. Build the variables from what users do and say. Then cluster.

This is where behavioural-personas earns its keep. The method does not ask the Synthesizer to write colorful cards out of thin air. It tells the Synthesizer to cluster participants by observed behavior and goal, never by demographic attribute alone. It tells the next step to draft one persona per behavioral cluster, citing the evidence for the defining goal and the main frustration. It tells the Critic to reject any persona whose defining goal is demographic rather than behavioral.

asd method-run --methodId behavioural-personas   --projectId kestrel-energy-f4180ce3

{
  "method": {
    "id": "behavioural-personas",
    "name": "Behavioural Persona Construction",
    "phase": "synthesis",
    "reviewGate": "critic"
  },
  "acceptance": [
    {
      "rule": "min-nodes",
      "min": 2
    },
    {
      "rule": "min-evidence-per-node",
      "min": 2
    },
    {
      "rule": "field-non-empty",
      "field": "goal"
    },
    {
      "rule": "forbid-field-values",
      "field": "description",
      "deny": [
        "demographic"
      ]
    }
  ],
  "outputKind": "persona-set"
}

That final denial rule is the quiet assassin of lazy segmentation. It does not ban age from appearing anywhere in a dataset. It bans demographic identity from masquerading as the behavioral explanation. "Over 65" is not a goal. "Needs visible certainty before leaving home" might be. That is why the elderly repeat-patient cohort at Vesper could become a meaningful design segment instead of a stereotype. Their defining issue was not age. It was the need for acknowledged, trustworthy progress through the pickup experience.

The Behavioural-Personas Artifact Is a Boundary Object, Not a Poster

Once the mixed-methods evidence and the clustering logic are in place, the output has to stay useful outside the analytics room. That is the job of the persona-set artifact. In ASD a persona is not a slide with a stock photo. It is a typed node inside a registered artifact kind, with fields that downstream methods can read and the Critic can inspect. The artifact becomes a boundary object between research, mapping, ideation, measurement, and delivery.

For that reason I care about the fields. The node needs a goal. It needs evidence. It needs a frustration. It needs a label the rest of the team can use without translating it every time. The acceptance floor does not guarantee a beautiful persona. It does guarantee that the persona has at least the minimum ingredients to be defensible.

[
  {
    "id": "c2-paper-trail-keeper",
    "label": "Paper trail keeper",
    "fields": {
      "goal": "Get certainty that the smart-meter visit will happen exactly as promised.",
      "description": "Phone-first, documentation-seeking, reassurance-driven household.",
      "frustration": "Mandatory-looking changes arrive without clear next steps or trusted confirmation."
    },
    "evidence": [
      {
        "evidenceId": "e1c27ce41870f130e3a2f7e827d4d4fe5df12d3d9f35ea71ca98fcf74db2c8bc"
      },
      {
        "evidenceId": "8d8d59d615c1a9ddff0d3ca1d6482da2e6ab224f69d42f0f0852958f46dbd2e3"
      }
    ]
  },
  {
    "id": "c3-tariff-hawk",
    "label": "Tariff hawk",
    "fields": {
      "goal": "Check whether every tariff change is economically rational for this household.",
      "description": "Calculation-driven, channel-flexible, quick-to-switch if arithmetic is hidden.",
      "frustration": "Opaque pricing messages make comparison work necessary and churn more likely."
    },
    "evidence": [
      {
        "evidenceId": "443f5dca7c070d3ea499c98e88be5aa0a654bf82f943465ce26cb79e2c994d3d"
      },
      {
        "evidenceId": "79b710828f711f5a2e4b00f0b89da2d0c840466f5c22e2d26730d88d977f6225"
      }
    ]
  }
]

Because this is a typed artifact, downstream work sharpens. A journey map can be regenerated through the lens of c2-paper-trail-keeper. A prototype plan can state which persona it is trying to help. A KPI tree can measure whether the service improved the right behavior. I am no longer waving at a persona in a workshop. I am passing a registered object through the system.

Case Spine: Vesper Pharmacy and Kestrel Energy

Vesper Pharmacy shows the mixed-methods loop at medium scale and high operational consequence. The research team carried 1.8 million pickup events, a 12,038-response survey, 35 shopper interviews, 20 pharmacist shadow days, and 15 mystery shops through one evidence discipline. The breakthrough was not merely discovering that no-shows were high. It was explaining why. The telemetry showed the premature-ready cohort. The document row showed the intended shelf-scan logic. The quote row exposed the trust wound. Once those three lived together, the fix became obvious: ready means ready, and queue visibility matters more than shaving another minute off a wait for the cohort already being served quickly.

Kestrel Energy scales the same discipline into segmentation proper. The utility inherited four demographic buckets from marketing and a mountain of contradictory data. The team rebuilt the evidence base, engineered behavioral features, ran clustering, built a separate qualitative typology, then fused the two into a segmentation that could survive an adversarial check. The most important decision was not computational. It was conceptual. The team refused to let vulnerability become a persona type and refused to let age become the story. That is why the final personas changed operations instead of merely decorating them.

The resulting interventions were precise. Kestrel's paper-trail keepers received assisted-digital booking routes, visible phone paths, and clearer access-preparation guidance. Tariff hawks got transparent arithmetic instead of generic reassurance. Vesper's first-time click-and-collect users got a guided script and a truthful status line. Elderly repeat patients got visible queue progress and acknowledgement rather than faster-but-silent processing. None of those interventions would have emerged cleanly from quant alone or qual alone.

Validation matters here too. Kestrel did not stop at plausible clusters. The household model had to clear a mean silhouette of 0.35, preserve usable cluster share, and survive an independent qualitative derivation that matched most of the resulting types. That is a much better discipline than asking whether the persona names feel right in a workshop. A segment survives because it separates behavior, predicts something the service can act on, and does not collapse when another model family attacks it.

Vesper's equivalent validation lived in the pilot. The mixed-methods logic predicted that status truthfulness and queue visibility would matter more than generic speed optimization for the elderly repeat-patient cohort. The pilot then checked the bet with telemetry, follow-up interviews, and mystery shops. In other words, the segmentation claim stayed reversible until the service proved it in the world.

That is why I treat evidence-backed segmentation as a service-design capability rather than a marketing deliverable. The end product is not a set of audience cards. It is a reliable way to decide which route, script, message, queue, or support pattern fits which behavioral situation. When the artifact is typed, reviewed, and replayable, the segmentation becomes part of the operating system. When it is only persuasive, it dies the first time a dashboard or a stakeholder mood shifts.

One more practical rule: do not let survey percentages float free of cohort definition. If a survey result cannot be tied back to a named slice of the service and a named segment hypothesis, it becomes decoration. Mixed methods earns its keep when every number knows which behavior it is trying to explain.

The practical lesson is simple. Put every method under one register. Store survey findings, documents, telemetry, and quotes in their right evidence forms. Build the behavioral variables explicitly. Demand persona nodes that can survive the Critic. Then let the service change. That is mixed methods with teeth.

That is the hidden advantage of ASD here: the platform gives segmentation a memory. Months later I can still inspect the rows, rerun the check, and see why a service choice belonged to this segment and not another.

One caution from practice: mixed-method projects drift when teams treat the quantitative arm as an audit of the qualitative arm instead of as a second lens on the same behavior. Telemetry that contradicts an interview theme is not an embarrassment to reconcile away. It is usually the strongest finding in the project, because it marks exactly where stated intent and lived behavior diverge. I put those contradictions into their own evidence rows and make the personas carry them explicitly. A segment description that admits where its two evidence forms disagree is worth more than a tidy one that quietly picked a side.