CASE STUDY
7 Minute Read
Consideration Set Formation
Investigating how consumers shortlist complex financial products
CASE STUDY
7 Minute Read
Consideration Set Formation
Investigating how consumers shortlist complex financial products
CASE STUDY
7 Minute Read
Consideration Set Formation
Investigating how consumers shortlist complex financial products
Summary
About
Objective
Learn about the information and strategies used by consumers when selecting a credit card, and provide recommendations to inform the design, content, and features of our comparison platform.
Outcome
Insights into target consumer behavior and recommended feature improvements were shared with the organization.
Role
I contributed to this project as the sole UX researcher embedded in a product team. The work presented represents my individual contributions.
Skills Applied
Survey, content analysis
Summary
About
Objective
Learn about the information and strategies used by consumers when selecting a credit card, and provide recommendations to inform the design, content, and features of our comparison platform.
Outcome
Insights into target consumer behavior and recommended feature improvements were shared with the organization.
Role
I contributed to this project as the sole UX researcher embedded in a product team. The work presented represents my individual contributions.
Skills Applied
Survey, content analysis
Reliance on Incentives
MoneySmart's product selection was extensive, but also largely non-proprietary—the same products were also offered on competing aggregators' sites. As such, MoneySmart relied heavily on costly incentive-driven strategies to drive transactions on its platform.
This project was driven by the desire to reduce our reliance on incentives by offering a superior comparison experience as a point of differentiation.
As other aggregators increased the value of their incentives to compete, this resulted in a "race to the bottom" which was seen as unsustainable in the long term.
Reliance on Incentives
MoneySmart's product selection was extensive, but also largely non-proprietary—the same products were also offered on competing aggregators' sites. As such, MoneySmart relied heavily on costly incentive-driven strategies to drive transactions on its platform.
This project was driven by the desire to reduce our reliance on incentives by offering a superior comparison experience as a point of differentiation.
As other aggregators increased the value of their incentives to compete, this resulted in a "race to the bottom" which was seen as unsustainable in the long term.
Reliance on Incentives
MoneySmart's product selection was extensive, but also largely non-proprietary—the same products were also offered on competing aggregators' sites. As such, MoneySmart relied heavily on costly incentive-driven strategies to drive transactions on its platform.
This project was driven by the desire to reduce our reliance on incentives by offering a superior comparison experience as a point of differentiation.
As other aggregators increased the value of their incentives to compete, this resulted in a "race to the bottom" which was seen as unsustainable in the long term.
Facilitating Easier Comparisons
Spoilt for Choice
MoneySmart's product listings presented users with a large assortment of complex and feature-rich products to compare. For example, there were more than 60 credit cards to choose from. The sheer variety made it potentially difficult for (especially inexperienced) consumers to select the right product for their needs.
Credit cards (and other financial products) are often differentiated by perks and features that are not easily or directly comparable, such as reward mechanics or merchant-specific tie-ups.
Facilitating Easier Comparisons
Spoilt for Choice
MoneySmart's product listings presented users with a large assortment of complex and feature-rich products to compare. For example, there were more than 60 credit cards to choose from. The sheer variety made it potentially difficult for (especially inexperienced) consumers to select the right product for their needs.
Credit cards (and other financial products) are often differentiated by perks and features that are not easily or directly comparable, such as reward mechanics or merchant-specific tie-ups.
Facilitating Easier Comparisons
Spoilt for Choice
MoneySmart's product listings presented users with a large assortment of complex and feature-rich products to compare. For example, there were more than 60 credit cards to choose from. The sheer variety made it potentially difficult for (especially inexperienced) consumers to select the right product for their needs.
Credit cards (and other financial products) are often differentiated by perks and features that are not easily or directly comparable, such as reward mechanics or merchant-specific tie-ups.
A Top-Down Approach
Unfortunately, our interface didn't always help. Decisions on what specific information to highlight as "key features" were made through an informal combination of expert discretion and reference to the providers' own marketing. There was no evidence to suggest that these features were in fact the ones that mattered most to potential customers.
One implication of this unstructured approach was that key features were often not directly comparable between different products.
A Top-Down Approach
Unfortunately, our interface didn't always help. Decisions on what specific information to highlight as "key features" were made through an informal combination of expert discretion and reference to the providers' own marketing. There was no evidence to suggest that these features were in fact the ones that mattered most to potential customers.
One implication of this unstructured approach was that key features were often not directly comparable between different products.
A Top-Down Approach
Unfortunately, our interface didn't always help. Decisions on what specific information to highlight as "key features" were made through an informal combination of expert discretion and reference to the providers' own marketing. There was no evidence to suggest that these features were in fact the ones that mattered most to potential customers.
One implication of this unstructured approach was that key features were often not directly comparable between different products.
Different key features between products
Different key features between products
Different key features between products
A More User-Centered Approach
I set out to design a study to identify the product attributes that mattered most to consumers when shortlisting credit cards. With this knowledge, we could potentially craft a comparison experience that highlighted the most relevant information, minimizing cognitive and search costs.
While this study focused on credit cards, I also intended the study to be replicable across different products.
A More User-Centered Approach
I set out to design a study to identify the product attributes that mattered most to consumers when shortlisting credit cards. With this knowledge, we could potentially craft a comparison experience that highlighted the most relevant information, minimizing cognitive and search costs.
While this study focused on credit cards, I also intended the study to be replicable across different products.
A More User-Centered Approach
I set out to design a study to identify the product attributes that mattered most to consumers when shortlisting credit cards. With this knowledge, we could potentially craft a comparison experience that highlighted the most relevant information, minimizing cognitive and search costs.
While this study focused on credit cards, I also intended the study to be replicable across different products.
Theoretical Foundations
Decision-making research (e.g. Hauser, 2014) suggested that consumers utilize a two-step process when selecting from a wide array of possible options. First, a smaller, more manageable group of options (the consideration set) are shortlisted. Then, the remaining options are evaluated more thoroughly to arrive at the final choice. This implies that products that do not make it past the first step are eliminated from consideration.
This pattern was supported by analytics data that showed engagement and conversions dropping off sharply after the first few cards—suggesting that consumers are not performing exhaustive comparisons on the entire product selection.
Theoretical Foundations
Decision-making research (e.g. Hauser, 2014) suggested that consumers utilize a two-step process when selecting from a wide array of possible options. First, a smaller, more manageable group of options (the consideration set) are shortlisted. Then, the remaining options are evaluated more thoroughly to arrive at the final choice. This implies that products that do not make it past the first step are eliminated from consideration.
This pattern was supported by analytics data that showed engagement and conversions dropping off sharply after the first few cards—suggesting that consumers are not performing exhaustive comparisons on the entire product selection.
Theoretical Foundations
Decision-making research (e.g. Hauser, 2014) suggested that consumers utilize a two-step process when selecting from a wide array of possible options. First, a smaller, more manageable group of options (the consideration set) are shortlisted. Then, the remaining options are evaluated more thoroughly to arrive at the final choice. This implies that products that do not make it past the first step are eliminated from consideration.
This pattern was supported by analytics data that showed engagement and conversions dropping off sharply after the first few cards—suggesting that consumers are not performing exhaustive comparisons on the entire product selection.
Illustration of the two-step process
Illustration of the two-step process
Illustration of the two-step process
Research Design
Method
I decided on the survey method after reviewing the literature on the evaluation of relative preferences, a common objective in marketing research. The survey design was inspired by the unstructured direct elicitation (UDE) method described by Ding and colleagues (2011).
I considered more complex approaches (e.g. conjoint analysis, MaxDiff analysis), but decided against them due to lack of expertise and access to software.
Research Design
Method
I decided on the survey method after reviewing the literature on the evaluation of relative preferences, a common objective in marketing research. The survey design was inspired by the unstructured direct elicitation (UDE) method described by Ding and colleagues (2011).
I considered more complex approaches (e.g. conjoint analysis, MaxDiff analysis), but decided against them due to lack of expertise and access to software.
Research Design
Method
I decided on the survey method after reviewing the literature on the evaluation of relative preferences, a common objective in marketing research. The survey design was inspired by the unstructured direct elicitation (UDE) method described by Ding and colleagues (2011).
I considered more complex approaches (e.g. conjoint analysis, MaxDiff analysis), but decided against them due to lack of expertise and access to software.
Items
The main item involved asking participants to write a hypothetical email to "a friend" with instructions to help them shortlist a few credit cards on their behalf.
Items
The main item involved asking participants to write a hypothetical email to "a friend" with instructions to help them shortlist a few credit cards on their behalf.
Items
The main item involved asking participants to write a hypothetical email to "a friend" with instructions to help them shortlist a few credit cards on their behalf.
Prompt for the writing task
Before being asked to write the "email", participants were showed a sample prompt and response to familiarize them with the requirements of the task.
Prompt for the writing task
Before being asked to write the "email", participants were showed a sample prompt and response to familiarize them with the requirements of the task.
Prompt for the writing task
Before being asked to write the "email", participants were showed a sample prompt and response to familiarize them with the requirements of the task.
In addition to the writing task, I also included measurements for the following possible moderating factors:
Perceived product complexity
Prior product experience
Interest in personal finance
Inclination to perform pre-purchase research
Comfort with online shopping
Devices used for online shopping
Understanding of the writing task (manipulation check)
Perceived difficulty of the writing task (manipulation check)
The survey was created in Typeform, and pilot tested with MoneySmart employees to refine the phrasing of the items. Data was collected over a 2 week period.
Respondents were recruited through a post on MoneySmart's Facebook account. A total of 182 valid responses were received. A small incentive was offered to encourage participation.
In addition to the writing task, I also included measurements for the following possible moderating factors:
Perceived product complexity
Prior product experience
Interest in personal finance
Inclination to perform pre-purchase research
Comfort with online shopping
Devices used for online shopping
Understanding of the writing task (manipulation check)
Perceived difficulty of the writing task (manipulation check)
The survey was created in Typeform, and pilot tested with MoneySmart employees to refine the phrasing of the items. Data was collected over a 2 week period.
Respondents were recruited through a post on MoneySmart's Facebook account. A total of 182 valid responses were received. A small incentive was offered to encourage participation.
In addition to the writing task, I also included measurements for the following possible moderating factors:
Perceived product complexity
Prior product experience
Interest in personal finance
Inclination to perform pre-purchase research
Comfort with online shopping
Devices used for online shopping
Understanding of the writing task (manipulation check)
Perceived difficulty of the writing task (manipulation check)
The survey was created in Typeform, and pilot tested with MoneySmart employees to refine the phrasing of the items. Data was collected over a 2 week period.
Respondents were recruited through a post on MoneySmart's Facebook account. A total of 182 valid responses were received. A small incentive was offered to encourage participation.
Full demo of the survey
Content Analysis
After cleaning the data to remove invalid responses, I analyzed the responses in ATLAS.ti.
First, I read through and coded 20 responses to discover initial themes that would inform a preliminary framework for analyzing the rest of the data.
Next, I read and coded the entire dataset with the focus on the attributes that respondents included in their instructions. New themes were added and initial themes were amended or combined as patterns emerged from the data.
Then, I tabulated the frequency of identified themes and calculated the average number of options requested by respondents.
Content Analysis
After cleaning the data to remove invalid responses, I analyzed the responses in ATLAS.ti.
First, I read through and coded 20 responses to discover initial themes that would inform a preliminary framework for analyzing the rest of the data.
Next, I read and coded the entire dataset with the focus on the attributes that respondents included in their instructions. New themes were added and initial themes were amended or combined as patterns emerged from the data.
Then, I tabulated the frequency of identified themes and calculated the average number of options requested by respondents.
Content Analysis
After cleaning the data to remove invalid responses, I analyzed the responses in ATLAS.ti.
First, I read through and coded 20 responses to discover initial themes that would inform a preliminary framework for analyzing the rest of the data.
Next, I read and coded the entire dataset with the focus on the attributes that respondents included in their instructions. New themes were added and initial themes were amended or combined as patterns emerged from the data.
Then, I tabulated the frequency of identified themes and calculated the average number of options requested by respondents.
Coding the responses
Coding the responses
Coding the responses
Themes
A total of 70 codes were generated. Three overarching themes emerged, suggesting that consumers considered attributes of (1) the product, (2) their own personal circumstances and habits, as well as (3) the provider when shortlisting credit cards.
Themes
A total of 70 codes were generated. Three overarching themes emerged, suggesting that consumers considered attributes of (1) the product, (2) their own personal circumstances and habits, as well as (3) the provider when shortlisting credit cards.
Themes
A total of 70 codes were generated. Three overarching themes emerged, suggesting that consumers considered attributes of (1) the product, (2) their own personal circumstances and habits, as well as (3) the provider when shortlisting credit cards.
All themes and subthemes
All themes and subthemes
All themes and subthemes
Frequency of themes
I then tabulated the frequency of themes and subthemes, which might reflect the relative salience of attributes to consumers.
Frequency of themes
I then tabulated the frequency of themes and subthemes, which might reflect the relative salience of attributes to consumers.
Frequency of themes
I then tabulated the frequency of themes and subthemes, which might reflect the relative salience of attributes to consumers.
Relative frequency of themes and subthemes
Relative frequency of themes and subthemes
Relative frequency of themes and subthemes
Selected insights
Credit Cards as Lifestyle Products
Respondents frequently mentioned deals and discounts at their favorite merchants in their instructions. In contrast, relatively fewer respondents mentioned interest rates, suggesting that consumers in our sample saw credit cards as lifestyle products rather than a source of financing.
In contrast, our product listings at the time tended to emphasize information related to card specifications (e.g. rates), which were one step removed from tangible lifestyle benefits.
Selected insights
Credit Cards as Lifestyle Products
Respondents frequently mentioned deals and discounts at their favorite merchants in their instructions. In contrast, relatively fewer respondents mentioned interest rates, suggesting that consumers in our sample saw credit cards as lifestyle products rather than a source of financing.
In contrast, our product listings at the time tended to emphasize information related to card specifications (e.g. rates), which were one step removed from tangible lifestyle benefits.
Selected insights
Credit Cards as Lifestyle Products
Respondents frequently mentioned deals and discounts at their favorite merchants in their instructions. In contrast, relatively fewer respondents mentioned interest rates, suggesting that consumers in our sample saw credit cards as lifestyle products rather than a source of financing.
In contrast, our product listings at the time tended to emphasize information related to card specifications (e.g. rates), which were one step removed from tangible lifestyle benefits.
Concerns about Eligibility and Costs
Many respondents mentioned cut-off points for monthly spending and annual income requirements, as these factors would impact their eligibility for certain cards or benefits. Preferences for easy annual fee waivers and no minimum spend criteria were also frequently mentioned.
In contrast, our listings at the time tended to de-emphasize the conditions and fees associated with the cards.
Concerns about Eligibility and Costs
Many respondents mentioned cut-off points for monthly spending and annual income requirements, as these factors would impact their eligibility for certain cards or benefits. Preferences for easy annual fee waivers and no minimum spend criteria were also frequently mentioned.
In contrast, our listings at the time tended to de-emphasize the conditions and fees associated with the cards.
Concerns about Eligibility and Costs
Many respondents mentioned cut-off points for monthly spending and annual income requirements, as these factors would impact their eligibility for certain cards or benefits. Preferences for easy annual fee waivers and no minimum spend criteria were also frequently mentioned.
In contrast, our listings at the time tended to de-emphasize the conditions and fees associated with the cards.
Selected Recommendations
The thrust of my recommendations were to move beyond simply listing products towards matching relevant product attributes to users' circumstances and preferences (personal attributes). In particular, I advocated for:
Selected Recommendations
The thrust of my recommendations were to move beyond simply listing products towards matching relevant product attributes to users' circumstances and preferences (personal attributes). In particular, I advocated for:
Selected Recommendations
The thrust of my recommendations were to move beyond simply listing products towards matching relevant product attributes to users' circumstances and preferences (personal attributes). In particular, I advocated for:
Extending Sort and Filter Functionality
The existing system enabled users to filter cards by ongoing promotions, card association, and provider. Only a default sort by "recommended" was available.
To facilitate less effortful shortlisting of cards, I recommended enabling users to sort and filter products by frequently mentioned attributes, particularly those concerning costs or conditions.
For example, being able to filter cards based on annual income might help users quickly eliminate from consideration cards they do not qualify for. Similarly, users could be offered the functionality to sort cards based on annual fees, or minimum monthly spend, etc.
Extending Sort and Filter Functionality
The existing system enabled users to filter cards by ongoing promotions, card association, and provider. Only a default sort by "recommended" was available.
To facilitate less effortful shortlisting of cards, I recommended enabling users to sort and filter products by frequently mentioned attributes, particularly those concerning costs or conditions.
For example, being able to filter cards based on annual income might help users quickly eliminate from consideration cards they do not qualify for. Similarly, users could be offered the functionality to sort cards based on annual fees, or minimum monthly spend, etc.
Extending Sort and Filter Functionality
The existing system enabled users to filter cards by ongoing promotions, card association, and provider. Only a default sort by "recommended" was available.
To facilitate less effortful shortlisting of cards, I recommended enabling users to sort and filter products by frequently mentioned attributes, particularly those concerning costs or conditions.
For example, being able to filter cards based on annual income might help users quickly eliminate from consideration cards they do not qualify for. Similarly, users could be offered the functionality to sort cards based on annual fees, or minimum monthly spend, etc.
Shifting Emphasis from Product Specifications to Lifestyle Benefits
Respondents frequently included information about how they intended to use the credit card in their instructions, expressed either as spending categories (e.g. groceries) or specific merchants (e.g. Shopee). This suggested a preference for cards that offered benefits in line with their established spending habits.
I recommended exploring the possibility of shifting the emphasis of our product information from card specifications to lifestyle benefits. This approach (exemplified by WhatCard.sg) might enable users to more easily identify cards that provided the most attractive benefits relevant to their intended usage.
Shifting Emphasis from Product Specifications to Lifestyle Benefits
Respondents frequently included information about how they intended to use the credit card in their instructions, expressed either as spending categories (e.g. groceries) or specific merchants (e.g. Shopee). This suggested a preference for cards that offered benefits in line with their established spending habits.
I recommended exploring the possibility of shifting the emphasis of our product information from card specifications to lifestyle benefits. This approach (exemplified by WhatCard.sg) might enable users to more easily identify cards that provided the most attractive benefits relevant to their intended usage.
Shifting Emphasis from Product Specifications to Lifestyle Benefits
Respondents frequently included information about how they intended to use the credit card in their instructions, expressed either as spending categories (e.g. groceries) or specific merchants (e.g. Shopee). This suggested a preference for cards that offered benefits in line with their established spending habits.
I recommended exploring the possibility of shifting the emphasis of our product information from card specifications to lifestyle benefits. This approach (exemplified by WhatCard.sg) might enable users to more easily identify cards that provided the most attractive benefits relevant to their intended usage.
A table of prioritized feature improvement ideas
I communicated the insights from the study through a report which was shared with the department. I also compiled specific recommendations and feature ideas in a spreadsheet to facilitate discussion and prioritization within the team.
A table of prioritized feature improvement ideas
I communicated the insights from the study through a report which was shared with the department. I also compiled specific recommendations and feature ideas in a spreadsheet to facilitate discussion and prioritization within the team.
A table of prioritized feature improvement ideas
I communicated the insights from the study through a report which was shared with the department. I also compiled specific recommendations and feature ideas in a spreadsheet to facilitate discussion and prioritization within the team.
My takeaways
This project gave me the opportunity to apply a relatively novel survey design that balanced the advantages of both qualitative and quantitative approaches.
It also highlighted to me the importance of designing instruments that are engaging for respondents. I was frankly quite surprised at the number of high quality responses we received given the effort required by the writing task, the modest incentive offered and the short timeframe for data collection!
My takeaways
This project gave me the opportunity to apply a relatively novel survey design that balanced the advantages of both qualitative and quantitative approaches.
It also highlighted to me the importance of designing instruments that are engaging for respondents. I was frankly quite surprised at the number of high quality responses we received given the effort required by the writing task, the modest incentive offered and the short timeframe for data collection!
My takeaways
This project gave me the opportunity to apply a relatively novel survey design that balanced the advantages of both qualitative and quantitative approaches.
It also highlighted to me the importance of designing instruments that are engaging for respondents. I was frankly quite surprised at the number of high quality responses we received given the effort required by the writing task, the modest incentive offered and the short timeframe for data collection!