Overview

I successfully led the end-to-end design of a new in-app engagement surface to enhance suite awareness and engagement across Microsoft 365, a cloud-based service.

Through 3-4 rounds of design, research studies, and presentations to leadership and stakeholders, we secured funding for this project.

The MVP version launched to 100% in 2023 in Word, Excel, PowerPoint, Outlook, and the Microsoft 365 app, resulting in a +5.5% uplift in overall Net Promoter Score (NPS).

The scope of my work

Design from scratch

We started this project as a design-driven concept. We defined the goal and scope, developed scenarios and use cases, and went through multiple ideations and design iterations. After conducting several user tests, we successfully shipped the design for the first launch.

Highly collaborative

Collaborated with user researchers to conduct user testing and worked with the accessibility team to ensure the design was accessible to everyone. I also coordinated with designers across app feature teams to produce the final design assets and worked closely with engineers for implementation.

Launched MVP

We first released it for internal users, then to en-US consumer subscribers in Microsoft Insider ring. We continued fixing bugs with engineers and launched the MVP for the production A/B experiment, tracking data and metrics throughout. Then rolled out to 100%.

Understanding

Solving problem for 67 million consumers

Microsoft 365 Premium has 67 million subscribers, but many users are unaware of or don't fully utilize their subscription benefits. Some don't even consider their subscription as premium. Additionally, there's a lack of cross-app awareness and engagement. We need to address this gap to drive meaningful cross-app engagement and help users maximize the value of their Microsoft 365 subscription.

Data indicated this is a real problem

User research and NPS results show that overall suite value awareness is low.

Data reveals that less than 40% of users know about our benefits, about 38% use our features, and only 20% of family plan subscribers share their plan with others.

Making a huge impact on the business

With 60 million consumers paying $9.99 monthly or $99 annually, high user engagement significantly boosts retention and loyalty, ultimately making a substantial positive impact on the business.

We aim to create an in-app communication surface that empowers users discover the value of their subscription, learn, get inspired, and deeply engage with Microsoft 365 features and benefits.

Introducing “My Benefits”

”My Benefits" is a cross-suite, cross-platform, programmable in-app engagement surfac that highlights the value of Microsoft 365 subscriptions. It helps users set up their subscriptions and introduces them to premium features.

It updates content periodically and has consistent suite-wide placement with the diamond icon the header, “My Benefits” is initially available to paid and trial consumers on Win32 Word, Excel, PowerPoint, and Outlook in en-us markets. After successful A/B testing, it expanded to Web, Mobile, and other M365 applications in 2023.

Persistent Entry Point

We don’t want this message shows up too frequently, and respect user’s limited resources of attention and help them to focus on what’s important to them. This is only show once per content refresh, which is around once per quarter.

My Benefit Home

Welcome user to Microsoft 365 subscription. Inform and educate user about their subscription benefits and encourage user to take advantage of it.

In this MVP version, instead of taking user to a new page, we wanted to keep user in the context while they are working on their documents. Make it easy to access, and also make it easy for user to return. Post MVP, we want to make it a scaleable surface that can be in-app dialog, side panel or on web.

The hero card is app-centric according to the app users are in.

Word

Excel

PowerPoint

Outlook

Navigate to other pages

Apps and Benefits Tab

Inspire user to use cross-suite benefits.

Allow user to take inline action, such as download mobile app, back up files and share family plan.

Premium features Tab

Inspire user to explore premium features across Microsoft 365 products

Allow user to try out the premium features immediately

Process

We have gone through a lot of process and design iterations

The design process includes creating information architecture, user stories, several rounds of user testing and research, and tons of design iterations.

Building UX architecture

In order to have a holistic perspective when dive into design, we build an architecture system and mapped out scenarios across the product lifecycle and uncover key triggers and moments from user perspective. Key moments include:

Suite welcome and onboarding
Premium feature engagement (paid user)
New app launch
Premium feature upsell (free user) [Out of scope for MVP]
Subscription expiration

Map Out Information architecture

As cross-suite value communication is the ultimate goal. Our strategy was to engage user from a single feature level as a start point, and then drive user to explore a broader value.

The exploration - Defining interactive structure

Mapping user stories

To weigh out what's the catch there between informative design and intrusive design, we built narrative user stories along with initial design mockups, it helped us to explore possibilities and to make decisions, it also helped stakeholders to better understand the design rationales.

Design iteration 1: Initial design mockup

PMs requested a navigation mock-up based on Microsoft 365’s brand story guide. We suggested that the content might be too abstract and marketing-focused, so the team decided to create and test the mockup.

User testing round 1 findings

Our main focus was to determine if the content and navigation worked, if it fulfilled an unmet need, provided value to users, fit into their workflows and mental models.

😃 People like the idea of discovering / exploring cross-suite value within the module

🤔 People did not initially realize the module's purpose, and they were confused by its content, structure and navigation.

🙁 Participants initially thought the notifications and module were an upsell

Design iteration 2: Content and Navigation update

A new homepage with programmable content

Reinforced the idea that this is the place to discover subscription benefits and features
More Explicit on what is available with subscription
More grounded and tangible on the structure and content

We came up a new navigation system for the sub-cards, and want to test if it works.

User testing round 2 findings

🤔 The home screen navigation was clear. However, users only explored sections if they were interested in the featured content.

😃 Participants understood the subscription benefits section and liked the top options for easy navigation.

👎 Many struggled with the sub-card navigation and didn't notice the sidebar navigation.

🤔 We also realize our consumer subscribers fall along these spectrums. How can we facilitate both of them?

Design iteration 3: Simple structure

After went through tons of iterations, design reviews and critiques, we landed on the MVP version for the first launch.

During development, we also participated in the Microsoft 365 Ground Truth study series. Users rated the "My Benefits" experience as a 10/10, highlighting its importance and value.

Key Milestones

01/2021 Launched MVP to Dogfood

Launched MVP to Dogfood

04/2021 Launched MVP to Insider audiences

Launched MVP to Insider audiences. In the meantime, kept working with engineer to fix bugs.

Results from Insider audiences:

Audience Size	51K
CTR on diamond	10.99% (with TeachingUI)
Return Rate (Users with 2+ Unity sessions)	28.33%
Overall Card CTR	20.0%
Most Actioned Card	Money in Excel

05/2021 Launch a 50/50 experiment for 6 weeks

Experiment: 6-week 50/50 trial with paid and trial consumers on Win32.
Positive Results: Improved suite-engagement metrics and NPS; Diamond CTR increased from 4.8% to 5.3%.
Negative Results: Low entry point CTR; increased OCV 'frowns' and decreased 'smiles'; dislike for new header icon.
Key Metrics:

Interaction Rate	31.7%
Card Interaction Rate	7.1%
Return Rate (2+ sessions)	18.3%
Return Rate (3+ sessions)	6.5%

Insights: Unity's success relies on discoverability and perceived value. Increased interaction with Unity correlates with higher NPS scores for M365. Unity 2.0 aims to enhance engagement and retention.

12/2021- 08/2023 Full roll out to various products

We launched to 100% for all Win32 paid and trial consumers in Word, Excel, PowerPoint, Outlook, and on microsoft365.com

Since May 2021, we've improved the Teaching popover for the entry point, added experimentation in Unity, renamed it to 'My Benefits', and conducted more user research. These changes led to a +5.5% uplift in overall NPS. Then we gradually rolled it out to 100% users.

Next steps: My Benefits V2.0

As our team expanded, I transitioned to a new priority project. A new designer joined and continued iterating on this project throughout 2023. Here is a list of post-MVP explores:

User intent capture and content personalization
Bookmark function and Search function
Information retrieval for older issues
Expandability and versatility as an in-app and cross platform communication framework
Serve as an upsell surface for free users

Now My Benefits 2.0, which provides a scalable framework, library with components, triggered across multiple platforms, for free, paid and small business users, that delivers diverse content based on users' apps and interests.