Making Formulas Easy and Building Smartsheet’s First Generative AI Feature
Multi-phase improvements to remediate customer’s number one support topic and take the code requirement out of formula creation
2023 – 2024
Delivered to Market
Formulas are Smartsheet's ‘secret ingredient’
They’re highly flexible, and Smartsheet offers a wider range of utility than many of its primary Project Management focused competitors like Monday and Asana.
They're also over half of support questions.
Formulas are responsible for most of Smartsheet’s community forum posts. They require lots of customer education, troubleshooting, and support.
Key Outcomes
Shipped AI-powered natural language formula creation experience
This delivery brought Smartsheet’s first Generative AI feature to market and earned customer compliments for how much easier it made formula creation.
It provided a valuable playbook for future Generative AI feature launches, helping craft a go to market approach (strategy, collateral, messaging) for other AI features.
Redesigned formula syntax editing UX for Smartsheet's New Views
As Smartsheet worked to rebuild its core asset – the sheet – and associated controls, I led design work to improve formula syntax editing to be a best-in-class code editor UX.
Borrowing from familiar IDE conventions and integrating smart accelerators, this shipped a more flexible form factor and intuitive editor for syntax power users
Merged AI power with core experience
Our initial delivery of natural language powered formula editing came as a minimally viable add-on feature, separate from the core Formula editor UX
With New Views going to market, we had an opportunity to integrate this feature more tightly, and crafted an intuitive ‘translation’ modality for formula editing
Delivered strategic direction for future
A key to the success of this work was setting a vision early, and delivering components of it incrementally.
I helped develop a roadmap for several quarters of further enhancements – like bringing formulas more ‘natively’ to the sheet as pre-built column types, providing built in templates for formula creation, and enhancing in-product help.
Key Metrics
over
30%
decrease
In Enterprise formula support cases following AI implementation
10%
fewer
community forum posts about formula difficulties
over
80%
accuracy
Delivering valid formula outputs via AI powered creation
Very Positive
Customer sentiment around editing improvements
Design Process
Stop the bleeding: AI formula creation MVP
Syntax use is a full-stop, do-not-pass-go moment for many customers – stopping them from realizing value.
With GenAI tools rapidly improving at code completion, we saw an opportunity to quickly ship an MVP of NL formula creation to test the waters.
Core formula editor experience redesign
Making syntax optional was the key to expanding the total user base and engagement, but improving syntax editing was key for power users.
We decided that after unlocking basic NL creation, improving syntax editing was key.
+
Unified formula editing and AI powered creation
Our GenAI tool needed to become tightly embedded in the core formulas experience to make discovering and using the tool intuitive.
And, it needed to migrate from our Legacy sheet UX to our New Views experience.
Minimum Viable AI Formulas UX
Conceptualized how GenAI might power up Smartsheet’s core tool user experiences – from filters, to formulas, to activity log, and generating data and structure.
Proved technical feasibility of passing column names and necessary sheet structure to an LLM and returning a valid Smartsheet formula.
The hackathon project resonated with leadership – and won a first place prize. The 'Homepage Hero' is a project that would be worthy of home page emphasis.
We experimented and tested full conversational, as well as simple input/output modalities – and found customers preferred the simple form.
They knew what to ask for, and what to expect back. Added bonus: this made it much easier to ensure predictability of responses, lowered costs, and negated context window issues.
At the time, a fairly new pattern that later was incorporated into most of our AI experiences. This provided explanatory context for where an answer came from – or didn’t
Built in a check using formula lexer to validate that returned formulas were valid Smartsheet syntax.
Tied-in rating modality provided a way for us to gather examples and refine model prompting. And, a pulse on customer sentiment and specific issues.
AI whitepaper developed in tandem with product marketing to provide clarity on data egress, trust. Also built in feature flags to turn off entire feature for some customers.
New Views Syntax Editing
Research Discoveries
Discoverability barriers
Customers had to know to type ‘=’ in cell, and the function names. The toolbar 'quick access' to the SUM function was often truncated.
Column formula creation – the most powerful way to use formulas – was hidden in a menu on cells with existing formulas.
Moving throughout long formulas closed the editor, resulting in unintended formula saves.
Moving throughout the sheet while editing a formula functionally impossible, since it opened inside a cell editor.
The formula editor often covered cells that customer wanted to reference, requiring the cell name to be manually entered using proper syntax.
Syntax display had no clear delineation between elements, and no bracket matching or formula hierarchy representation.
Design Solutions
Clear CTA in toolbar with explanation of formula value to customer.
Provides a stable surface with abundant room for edits.
Includes a ‘staged’ apply button, so inadvertent changes are less severe.
Highlighting syntax elements follows common IDE conventions and makes long formulas more readable.
Smart bracket matching makes it clear where edits are happening. Bracket auto-completion reduces syntax errors.
Ability to add a column reference simply by typing name. Autocompletes and adds dynamic ‘@row’ reference.
A key for scalable formula creation. Enabled via split button – add to current column, or add as new column.
Unified AI and Syntax Experience
Research Discoveries
Over half of Smartsheet's support forum posts are about formula syntax, which requires learning a whole new language to craft logic.
Users who had a grasp of syntax were still using the MVP AI formula panel – for natural language troubleshooting, putting a formula they'd written in syntax in the panel so they could 'read' the logic.
Two places to go for slightly different experiences is innately confusing – and was never a long term ambition.
Design Solutions
Confusing syntax completely bypassed with tight NL ‘translation’ integration within Table View Formula panel.
The ability to quickly ‘swap’ between syntax and NL editing allows customers the best of both worlds – tight control of syntax, more fuzzy intent interpretation with NL, and easy explanations as writing and editing.
Putting these modalities together enhances learning – like having a WYSIWYG editor alongside the code it outputs.
Additionally, one experience to maintain with clear product team mapping simplified future maintenance.
Lessons Learned
Seizing the Moment
The project began as a hackathon idea, proving feasibility and desirability – on a large stage with great visibility across company leadership. That helped build a lot of momentum to continue investment.
This project was well positioned for a moment when AI investments were salient and hyped – and still solved a core customer problem. That pairing of meeting a market narrative and a customer need provided wind in our sails as we built this into a full fledged product.
Improving Incrementally
Shipping an MVP with limited functionality enabled rapid user feedback, which guided subsequent improvements. Metrics (e.g., 30% fewer enterprise support cases and 80% formula accuracy) highlighted the impact of changes and guided iterative improvements. Gathering user ratings and feedback helped refine the AI model and provided insights into real-world issues.
Cross-functional Collaboration
Tight collaboration with leadership, marketing, and technical teams ensured a successful go-to-market strategy for Smartsheet’s first generative AI feature. Breaking down those boundaries as we collaborated let everyone feel invested – and made the project more fun to work on.