# Choice-Level Analysis

## Why Choice-Level Analysis?.

Choice-level analysis is simpler, easier, and more powerful than profile-level analysis.

Conjoint designs originated in market research and psychology where each respondent is asked to rate each of two different profiles (e.g., two products). Each of the two ratings provided separate information, and the two are analyzed as separate observations. Researchers with this profile-level design find it convenient to arrange their data with one profile per row, and thus twice as many rows as respondents.

Unfortunately, when social scientists adopted the conjoint survey design, they kept the same profile-level design but changed the outcome measure from separate ratings to a single choice between the two profiles (e.g., to reflect a voter choice between two candidates). In this situation, respondents asked to make one choice between the two profiles that are exactly dependent, as choosing one necessarily meant not choosing the other (e.g., in a two-candidate partisan election, one observation would be “Democrat” and the other would be “not the Republican”). Using this profile-level design with 2*n rows but only n independent observations requires the introduction of complicated statistical procedures to correct for the dependence induced solely by the researcher’s decision to organize the data in this complicated way.

We recommend the much simpler and more powerful choice-level design. The idea is to arrange data at the level of the respondent’s choice, so that each row in the data matrix includes information about one choice (and both profiles together, with n observations and n rows). Our AJPS article clarifies this point, shows how this choice-level analysis vastly simplifies the notation, statistical analysis procedures, and intuition, and greatly expands the substantive questions conjoint analysis be used to answer.

Key Issues:

- Profile-level quantities like AMCEs, which require the assumption of independently generated profiles and disregard the context of comparison, prevent researchers from investigating many questions that (implicitly or explicitly) assume dependence between profiles.

- Examples of choice-level research questions

  - Do voters choose a white candidate over a non-white candidate?
    - The levels (white vs. Asian, Black, Hispanic) always differ between profiles.
  - Do Asian Democrat respondents prefer an Asian Republican over a white Democrat?
    - The two profiles are specifically designed based on multiple attributes.
  - Do voters care about candidate electability?
    - The sum of the two percentages should be 100.
  - Do voters prefer a status quo option over a policy proposal?
    - One profile is always fixed, while another varies across tasks.
  - How much do voters prefer extreme left-leaning or extreme right-leaning policies?
    - The left-leaning (right-leaning) candidate’s policies (attributes) should always be positioned consistently on the left (right).

- When individuals compare two profiles side-by-side (as in most conjoint tasks), their evaluations are often psychologically influenced by the alternative (See Horiuchi and Johnson 2025).

## Why Move to Choice-Level Analysis?

- Choice-level analysis models the decision between two profiles, not the evaluation of a single profile.
- This approach more closely mirrors:
  - How people make real-world tradeoffs (e.g., choosing between candidates, products, policies)
  - How comparative contexts shape judgments (e.g., assimilation, contrast effects)
- Rather than estimating the probability of selecting a standalone profile, choice-level analysis estimates the probability of choosing one profile over another, conditional on both the attributes of interest and other attributes.

✅ Better matches real-world behavior
✅ Explicitly captures comparative decision-making
✅ Reveals true tradeoffs and feature prioritization

## Summary

Profile-Level Analysis | Choice-Level Analysis
--- | ---
Treats profiles independently | Models the decision between profiles
Ignores comparative psychology | Captures influence of side-by-side comparisons
May blur or bias tradeoffs | Highlights real tradeoffs
Can be misleading | Much more informative
Requires complicated statistics | Allows simple methods

## Key Takeaway

> 🔎 If your conjoint design presents two profiles for comparison, choice-level analysis is essential for valid and insightful inference.

> 📈 It provides deeper insights, more accurate estimates, and a closer reflection of actual decision-making.

## 📚 References

- Clayton, Horiuchi, Kaufman, King, Komisarchik (Forthcoming).
  “Correcting Measurement Error Bias in Conjoint Survey Experiments.”
  Forthcoming, American Journal of Political Science.
  Pre-Print Available
- Horiuchi and Johnson (2025).
  “Advancing Conjoint Analysis: Delving Further into Profile Comparisons.”
  Work-in-progress.