Project overview

Introduction

Microsoft Research wanted to develop an AI tool to help companies untap the power of AI to transform their businesses in ways that weren't possible before. My two primary tasks were working with drug developer Novartis to help develop AI tools to speed up drug development and product design for the overall platform. * To honor my NDA, I'll only be showing images of the final product that have been officially released by Microsoft. I also am not at liberty to discuss in detail any work that was done on behalf of Novartis.

https://www.microsoft.com/en-us/research/project/project-s/

Project overview

My Role

I joined the team at Microsoft Research as a Senior Product Designer and as a contractor. Our design team consisted of 6 designers, we worked very closely with our UX research team, platform developers, AI modelers, C-level execs, and our client team at Novartis. Specifically, I owned the design work for Novartis, the command line feature, and the data visualization component. I also worked collaboratively with my design team to conceptualize and problem solve higher-level work on the overall platform.

Project overview

Product Overview

AI & Machine Learning has the power to transform how entire companies do business, but accessing these technologies often requires an effort and cost so great that it becomes a risk to even pursue it. The goal of Microsoft's AI platform was to democratize access to the power of AI and machine learning. Currently, most AI/ML solutions are custom built for each individual purpose, however, there is great power in having a modular, plug and play platform that allows businesses and individuals alike the ability to use more generalized AI models for whatever needs that see fit. Combine that with Microsoft's already established ability to securely manage big data and you have a stable environment for hosting and processing big data ways that one-off AI models just cannot.

Project overview

Key Challenges

1. Designing one experience for both technical and non-technical users

   Simply based on the products official mission statement of democratizing AI, we know that one of our major design goals and challenges is to create a product that's simple enough for non-technical users to understand and use, yet still contains all of the powerful features that a highly technical user would expect when using AI or ML models.
   
   

2. Creating a product where users future objectives are unknown

   An interesting aspect of this project was that we were creating a sandbox-like product. Giving users the ability to mix and match any number or AI/ML models meant that the product could potentially be used for anything from industrial mining applications to social media campaigns. How do we design the product if we don't know exactly how it will be used?
   
   

3. Helping Novartis create solutions for a product that's still under development

   Another unique part of this project was that we were working with the products clients while we were simultaneously building the product. While this was a god send from a use case and product development perspective, it meant that I was heavily involved in the development of AI models specific to the problems that Novartis was trying to solve.

Project overview

The Design Process

Our design team structure was that each designer was fully responsible for the features that we had been assigned. Another important element to keep in mind here is that each team at Microsoft Research is run like a little startup. So we all wear many hats and do whatever is needed to get the product to launch.

Before we were assigned our own features, the design team worked collaboratively to create an overall framework for the product. We were then assigned our features, created a list of questions that we wanted our UX Research team to dig into, and begin conducting our own design research. We did internal and external design reviews, worked with and created specs for devs, teamed with c-level product owners, met with external clients, and honestly, so much more.

Research

Understanding AI/ML Tools and how they're used

Our design team was full of inquisitive, self motivated designers, so design research and UX research ran in parallel at the start. Essentially, both teams were trying to answer different aspects of the same questions.

Who are we designing for?
What do they want do with the product?
How do they want to use the product?

The UX Research team gave us an amazingly detailed view of the tools that AI and ML developers use, what they prefer, what they dislike, and an analysis of tools that were in the same wheelhouse as ours. This research helped to build product requirements.

Design Exploration

New Technology / Familiar Framework

Now that we understood the requirements for our primary user types, we started to explore the initial structure for the application based on those requirements. But there were other factors that we needed to consider as well. While we did technically have carte blanche to design the product in any way we saw fit, we ultimately chose to air on the side of using design patterns that the user might be more familiar with. Many of our users might be using AI or ML for the first time and we didn't want to encumber the user with the additional task of having to learn wildly new design patterns on top of having to learn about AI itself. With all of the excitement over building a new product this was no easy task. We did explore some more radical and innovate design patterns, but ultimately cooler heads prevailed and I believe that we made the best decision for our users.

I created lots of example flows based on existing clients and potential future clients. Those flows were built around each clients end goals for an AI/ML application. Then, I used those example flows to test how different framework designs would perform in a variety of different use cases. This helped me to understand the strengths, weaknesses, flat out failings of certain structures, design patterns, and configurations of information architecture.

Working through different high-level frameworks and flows

Design Exploration

Powerful enough for technical users, easy enough for non-technical users

One of the product's overarching principles is to make AI/ML more accessible than it ever has been before, allowing people from outside the tech industry to utilize the power of this great technology. As lofty as that sentence is, it means that in order for us to truly deliver the product that we promise, we have to create a singular product that both a manufacturing company and a data scientist can use.

So while the platform used lots of familiar design patterns to help non-technical users, we also created features to help empower more technical users who wanted to perform advanced functions. One such feature is the formula bar. Conceptually similar to the formula bar in Excel, this feature allows technical users to configure AI models, link data sets, and perform advanced functions. This was one of the features that I owned.

I adapted one of my favorite design patterns from IDEs (the type of program that developers write code in). Essentially, as you type your formula, a dropdown will visually show you all of the possible options for each section of the formula. It also filters this list as you type. The pattern is good for technical users who are mostly likely already familiar with this style of pattern and its easy to understand for users who aren't familiar with it. This feature also involved doing information architecture work to ensure that the structure of the formulas were bulletproof and couldn't easily return errors.

Design Exploration

A Modular Data Visualization System

At the end of the day, understanding the data that comes out of an AI or ML model is the only thing that matters. I built a system of data visualization modules that work independently, but also can be combined for maximum impact. Because this platform could contain any type of AI/ML model I focused on building a design system for visualizations but also for AI specific visualizations such as image recognition, OCR, and data overlayed on maps.

Other Concerns

Negotiating with Dev Teams

While we as designers would love to just throw our craziest ideas the developers, this is almost never how it actually works. I worked with the dev teams for each of my features starting from the concepting phase. This allowed them to scope the different directions that I was considering and to get ahead of any technical issues that each design direction presented. In the end, we negotiated about the function of certain design features so both teams could achieve their goals.

The Outcome

Microsoft Science Engine

After I left the project it was rebranded as Science Engine. Though the core product remained the same, its being marketed more as a research and development tool for science related industries. The end result of our work was an AI/ML platform that's approachable, flexible, doesn't require coding (from the user), and easily allows data to be understood through the use of data visualizations. * To honor my NDA, I'll only be showing images of the final product that have been officially released by Microsoft.

https://www.microsoft.com/en-us/research/project/project-s/

https://www.microsoft.com/en-us/research/project/project-s/