Research Methodology

Our query generation process begins with seed queries derived from three sources: real search query datasets (anonymized), industry keyword databases, and manually curated queries designed to test specific citation behaviors. Each seed query is then expanded into multiple variants to account for phrasing differences, specificity levels, and intent nuances.

Queries are classified into four intent categories following a modified version of the Broder taxonomy:

Each query is submitted individually via API with a clean conversation context — no prior messages, no system prompts, and no user profile data. This ensures we measure the model's baseline citation behavior rather than personalized responses. Temperature is set to 0 for deterministic output, and each query is run three times to verify consistency.

As of Q1 2026, we test four major AI models that represent the majority of consumer-facing AI search interactions:

Each model has different grounding capabilities and knowledge cutoffs, which directly affect citation behavior. Perplexity and ChatGPT with browsing tend to cite more recent sources, while models without web access rely more heavily on training data. We report both aggregate and model-specific metrics to account for these differences.

Not all citations are equal. A direct recommendation (“I recommend using Notion for this”) carries different weight than a passing mention (“tools like Notion, Coda, and others”). Our scoring framework classifies each brand mention into one of four tiers:

Citation tier classification is performed using a combination of rule-based NLP patterns and a fine-tuned classifier trained on 8,000+ manually labeled examples. Inter-annotator agreement for the labeling task was 91.3% (Cohen's kappa = 0.87), indicating strong reliability.

All reported citation rates include 95% confidence intervals calculated using the Wilson score interval method, which provides better coverage properties than the standard Wald interval for proportions near 0 or 1.

Quarter-over-quarter comparisons use a two-proportion z-test with Bonferroni correction to account for multiple comparisons across industries. We only report a change as statistically significant if p < 0.005 after correction. Effect sizes are reported using Cohen's h for proportion comparisons.

For brand-level rankings, we use a Bayesian hierarchical model that accounts for query difficulty and industry-level variance. This prevents brands in low-volume categories from being disproportionately affected by sampling noise.

We believe transparency about limitations is essential to maintaining research credibility. The following are known limitations of our current methodology: