Shlomo Kalish (Leon Recanati Graduate School of Business Administration, Tel Aviv University, Tel Aviv, Israel 69978) and Paul Nelson (William E. Simon Graduate School of Business Administration, University of Rochester, Rochester, New York 14627)
Marketing Letters 2:4, (1991): 327-335
Objectives
- Compare RP to traditional MA from predictive validity perspective
- Empirically compares traditional ranking and rating preference in CA to reservation prices (direct monetary measure of product value)
Key Methods/Approach
- used consumer survey results to compare predictive ability and goodness of fit of RB model using directly measured reservation prices with those of MA model using common rank and rating preference measures
- full profile representations of 12 hypothetical products
- full profile representations of 12 hypothetical products
- each respondent state preferences using ranking, rating or reservation price
- product: round-trip airline tickets from New York City to Hawaii
- Attributes and Levels
* service Level: minimum, regular or premium
* seating room: regular or spacious
* non-stop or not
- included price when rating and ranking, but no price with reservation price
- price determined by part worth costing of features where cost of each feature reflected approximate market price at time
- $100 was added or subtracted from full cost to avoid correlation problems?
- estimation done on individual level
- ranks and ratings fit as a linear function
- stated reservation prices fit as a linear function of products' attribute levels only
respondents
- administer to undergraduate and first year graduate students in 6 different business classes
- no students had previous class exposure to conjoint analysis
- 225 useable questionnaires with 1/3 using each measurement method
* rank were measured from most preferred to least preferred
* rating measured on scale of 0 to 100 points in 5-point intervals
* reservation price respondent was asked to indicate what sum of money would make them indifferent between product and money
- preference order predicted for both original twelve and four holdout products
- estimation was done using OLS regression analysis, justified for RP and only partly justiied for ratings and less so for ranks
- fit measured using average across individuals of the Spearman rank correlation between ranks of actual measurements and fitted values (traditional measures (R^2) cannot be used since dependent variable are measured on different scales
Results and Conclusions
- reservation prices do very well in terms of fit, but are inferior in terms of predicting choice on a holdout sample
- little difference is found in performance of ranks and ratings
- Goodness of Fit
* Fit is high for all cases (rank: .91, rate: .88, reservation price: .96), but highest with reservation price perhaps because it allows a much larger range of values for dependent variable than ratings or rankings
* ranking and rating may have some favorable bias because there are 5 rather than 4 parameters, but mitigated since RP incorporates price before preferences thus accounting for price effects
* ratings scored lowest possibly because of response style bias (Kalwani and Silk, 1982)
- Predictive Validity
* rank and rate measurement consistently outperform reservation prices (Percent #1 correct- rank and rating: 62%, reservation: 46%), median (rank: .80, rate: .80 and RP: .60) and average also consistently larger (rank: .57, rate: .49 and RP: .43)
* reported correlations are averages and medians of rank correlations between choice ranking of four holdout products (stated choice) and predicted ranks (estimated choice)
- hypothesis that average correlations are equal for ranking and rating or ratings and reservation prices cannot be rejected
- hypothesis of no difference between 3 measurement scales cannot be rejected
- regarding prediction of first choices
* no difference between 3 scales can be rejected at 5% level
* ratings outperform RP at 3% level
* rankings outperform RP at 5% level
* no statistically significant difference between rankings and ratings
- study was broadened to other products
* rankings averaged 56% first choice predictions and RP 40% (difference larger for low ticket items and high involvement items)
* when respondents are well informed all three measure perform roughly equal
* RP not as robust to respondent involvement as ranks and ratings
- future improvements in RP measurement may make it acceptable alternative to ranks and ratings in conjoint analysis
Errors in experimental design, statistical analyses or analytical approaches
- Sample size was not exactly split 3 ways (rank: 68, rate: 97, reservation price: 89)
Opinion
This study has been cited 62 times according to Google Scholar. The authors do point out that the result refer to a single study of a single product class and may or may not be generalizable. The paper was straight-forward and concisely written meeting its state objective of comparing rank, rating and reservation price via the single study. They also briefly discusses other examples which helped support their conclusions.
This study has been cited 62 times according to Google Scholar. The authors do point out that the result refer to a single study of a single product class and may or may not be generalizable. The paper was straight-forward and concisely written meeting its state objective of comparing rank, rating and reservation price via the single study. They also briefly discusses other examples which helped support their conclusions.
This information will be helpful when setting up the conjoint experiment for restaurant chefs. Given this information we would not use reservation price. According to PingSun rating is easier than ranking for participants. I think that rating could be used to sum up part-worths. Choosing ranking or rating may not be necessary if choice experiment is used, however we used questions in the value chain survey that could be useful as part-worth (Please rank the below factors according to importance when purchasing fish and seafood). Ranking or rating could be used to sum part-worths, but I think rating would be more telling of the participant’s attitude.
Useful Information
- Advantages of Reservation Price (RP) over Multiattribute (MA) models:
* directly connected with traditional economic demand theory (function plots number of consumers with reservation price greater than a said price vs. price)
* provides simplified algorithmic framework to evaluate price and positioning alternatives through choice simulation
- Reservation Price
* hypothesizes each customer has a maximum price they are willing to pay for a given product which equals the product's value to the consumer
* compares reservation price for each product with purchase price and chooses based on utility maximization (consumer surplus or net value according to Hauser and Urban (1986))
* because consumer input and response difficulty are larger for reservation prices than rank or rating, it is predicted to have lower fit and predictive ability
- Possible advantages of RP over MA:
* product description do not always need to include price, reducing attributes by one Icould be significant in some cases)
* in some applications a monetary valued utility function is easier and more intuitive
* directly measured RP is ration scaled allowing for use in regression and other metric estimation techniques
* scale is comparable across individuals eliminating difficulties of interpersonal utility that exist with rating and ranking
- RP vs. MA
* RP: monetary (ratio) scaled measure, does not take price into account, chooses product with highest utility
* MA: treats price as an additional attribute taking into account
* equivalent if prices enters MA model linearly (Ratchford, 1979 and Srinivasan, 1982), rescaled by dividing price coefficient is measured in monetary units
- 3 ways to compare measurement scales
* evaluate consistency of preferences across 3 scales (not used here because respondent workload is high and learning effects significant)
* fit estimated model to original preferences
* predictive validity: after doing tasks respondent asked to rank choice among 4 hypothetical products
- 2 types of models: 1) main effects part worths model vs 2) linear model (used in this paper), could affect fit or predictive validity
- little difference between OLS and monotonic regression results (Green and Srinivasan, 1978)
Useful References
Cameron, T.A., and M.D. James. (1987). "Estimating Willingness to Pay from Survey Data: An Alternative Pre-Test-Market Evaluation Procedure," Journal of Marketing Research 24, 389-395.
Ratchford, B.T. (1979). "Operationalizing Economic Models of Demand for Product Characteristics," Journal of Consumer Research 6, 76-85.
Green, P.E., and A.M. Krieger (1985). "Models and Heuristics for Product Line Selection," Marketing Science 4, 1-19."
Cattin, P. and D.R. Wittink (1982). "Commercial Use of Conjoint Analysis: A Survey," Journal of Marketin 46, 44-53.
Kalwani, M. and A.J. Silk. (1982). "On the Reliability and Predictive Validity or Purchase Intention Measures," Marketing Science 1, 243-286.
No comments:
Post a Comment