AI-powered assistant converting your speech to forms using Google Cloud Run & Functions
Figma
Maze
3 Weeks
.svg)
Developers
.svg)
Designers
Our objective was to build a creative, enjoyable, and valuable solution for optimizing a resource-intensive business function using Cloud Run, Cloud Functions, and Workflows.
We built a serverless web application, reimagining the workflow of filling out multi-input forms through AI-powered speech recognition.
Teaming up with 3 developers (Santiago Uriarte, Brandon Huynh, Daniel Vu) and a designer (Tiffany Trinh), we opted to join the Google Cloud Easy as Pie Serverless Hackathon, aiming to foster skill growth and collaborative innovation together.
Objective: Build and deploy serverless apps with Cloud Run, Cloud Functions, and Workflows.
Prompt: "What’s your most sticky, time-consuming, budget-draining business function? Imagine that hot mess transformed into a sleek, simple process."
Our team brainstormed a few issues together to decide which business problem we wanted to tackle together for this project. Ultimately, we decided to address the pain points of forms using speech-to-text to:
Help increase workflow efficiency
Address the documentation needs of customers
Create a scalable SaaS product applicable to several industries
Discovery SessionMy team's brainstorming notes on FigJam during discovery.
We surveyed 30 participants about their recent experiences with forms and speech-to-text so that we could pinpoint any user points, determine the value of speech-to-text, and identify potential markets for scalability.
What is the current experience like with forms?
How often do users use speech-to-text?
Where do users typically use forms?
say filling out forms are tedious
rarely or never use speech recognition
say speech recognition is convenient
Repetitive
Writing Burdens
Time-Consuming
Government
Healthcare
Education
Although 67% of participants rarely or never use speech-to-text, 53% found it convenient. This reveals an opportunity to expand speech recognition usage, especially within healthcare, government, education, or food industries.
Due to time constraints, we needed to focus on high-priority features for the purposes of a minimum viable product (MVP).
We crossed out lower priority features, making our main features of the app to be:
Logging in as an existing user
Creating a SMART form
Accessing the transcribed form
Creating a template
Editing a form
Downloading a completed form
User FlowAfter logging in, the user can select a template to create a SMART form. The app then uses AI to transcribe the users' speech in real-time. Once the form has processed, users can view, edit, and download the transcribed form.
Using Figma, Tiffany and I created low-fidelity wireframes to depict potential designs for the app. We aimed the designs to be:
Familiar and learnable since this was not an ordinary transcription app.
Optimized for mobile device to allow for flexible usage wherever, whenever.
Simple for ease of use and navigation when performing the MVP tasks.
Wireframes 1.0The initial set of wireframes based on the app’s MVP features.
Wireframes 2.0A more detailed set of wireframes based on the MVP features.
Using Maze, I conducted 4 remote, moderated and 10 remote, unmoderated usability tests. My goals were to assess user navigation and ease by asking participants to complete the following tasks below.
Select a Template
Create a SMART Form
View SMART Form
Success Rate
Misclick Rate
Duration
"Nothing really confused me, it was pretty straightforward."
"I thought it was a list, but I wasn’t sure if I had to click anywhere else before...it seemed like an unclickable list."
"Anything where you purchase at a counter...or anything if they want to take note what the customer or buyer needs, this would be useful."
"...The way it was laid out and when it gave me the time accessed and when the file was opened, it kind of confused me because I wasn’t sure if this was a template or a file that I made."
Based on feedback, we realized that users needed more context and clarification. Refinements done to address these main concerns are as shown:
Templates ScreenSince users confused the templates for previously created forms, the list items were redesigned as cards with a preview image to better distinguish templates from forms.
Forms ScreenSince users expected to create a new form from the "Forms" screen, a button was added at the top to provide users quicker access to this capability. The list items were also further spaced out to increase visibility as clickable items.
Intended to be a scalable SaaS product, we decided that the branding of this app should be modern, clean, and professional.
Through this project, I learned how to rapidly collaborate with developers and designers in an agile environment. I really enjoyed collaborating with this team and successfully creating a demo product that could potentially open up more opportunities for speech recognition.
Here are a few key outcomes and reflections:
After implementing the iterations, the usability improved by nearly 10% based on metrics such as mission success & duration, test exits, and misclicks.
If I had more time:
I would have explored an onboarding flow to guide users on what the app is about and its main features.
I would have further explored motion design and animation for the audio waves to increase accessibility.
What I learned:
Attending weekly stand-ups and reviews helped us identify roadblocks, expectations, and new learnings from each other.
Being able to bounce off ideas with one another during brainstorming sessions helped spur creative solutions.
