LoudMouth Engagement Tool

Imagine being a busy single parent that is doing their best to just get by and put food on the table. Now imagine your child/children are going to a fast-paced virtual online school where being behind a week means most likely not catching back up. It might seem impossible to stay on top of your kids and making sure they’re logging on, what their grades are, who their teachers are and how to contact them. This is where Loud Mouth comes in!

Naming of the product aside, Loud Mouth is an intervention & engagement tool for schools. It delivers actionable information and insights to parents and students early and often, so that they can stay up to date and informed about what’s going on. The solution is currently completely interface-less and is purely text-message based. My main role on this project was research, UX copy, and at one point even Product Owner. 

Loud Mouth seeks to engage the disengaged, reduce clerical work, and humanize the teacher.

Okay, but what IS it?

Let’s get a little bit into the specifics of what the product is and what it has to offer. Loud Mouth is a text message facilitator that talks to various internal API’s to get student data and deliver them at appropriate times via Twilio. So far, over the course of its almost two year life cycle, Loud Mouth has sent over 2,000,000 messages to over 50,000 students, parents, and guardians, and that numbers grows every day.

That’s over two million data points that users would otherwise have had to seek out themselves. We have also received thousands of responses from users that have helped give understanding to teachers and administrators as well as help facilitate conversations that otherwise would not have happened. And with a 98% opt-in rate we know we have a solution that that most users seems to appreciate and find value in. We’ve also seen value demonstrated to clients with reports that course completion rates and seat time increased over their usual historic numbers with no other obvious influencing factors besides Loud Mouth, and that was just in the first few months.

Clients reported an increase in course completion rates by 5% and seat time by 12% within 4 months of starting to use Loud Mouth

Loud Mouth is a fairly simple offering. It pushes academic data to certain users at times and intervals we have found provide the best impact. Here’s how it breaks down:

1. User receives the “System Welcome” message as soon as they are enrolled in their courses and term has started, which includes opt-out information and expectations.
2. On the first Monday of courses the users receive a “Class Welcome” message, which includes teacher name, class name, teacher contact info
3. Every Wednesday morning the users receive a “Grade Message” which includes course grade information in the format the school wants users to see.
4. An activity message is sent out at whatever interval the customer has set to reach students before they fall too far behind.

Above is a little sample of the messages we have developed for students specifically, there’s another set that is tailored for parents. This does, however beg the question, how do we know that our messages are driving behavior we want to see?

We know because with each iteration (messaging is past iteration 15 at this point) of messaging, we ran studies and surveys to see if there was any way to improve and iterate on how we presented the information and how we presented it. We combined this with a new way that we found we could rapidly gain insights into our copy, Amazon MTurk (Mechanical Turk).

Utilizing Amazon MTurk

This was probably one of the cooler ways I’ve seen to gain and crowd-source insights. A question arose around verbiage behind one of our message offerings, with users reporting being confused by the combination of “current” and “overall” in context of grade.

“Your student’s current overall grade is 86% in Algebra 2A as of 1/4/19 with 22 days left. If you are confused about your grade or have questions about the course, your teacher Jane Smith can help you out at (555) 555-5555.”

This wasn’t the first time this question had arisen so my team set out to squash this confusion once and for all. Amazon MTurk allowed my team and I to send out small samples of messaging and get great quantitative and qualitative feedback in a very quick amount of time and be confident in our results. Through MTurk we were able to come to quick decisions on verbiage and the overall structure of our messages that made sense to the users, not just ourselves.

In the form of a quick survey, we laid out context, different descriptions about the particular term that the user were seeing (none of these users have context of our systems or the jargon associated with it) and multiple options to choose from that conveyed the message clearly. Below is the message sent to MTurk workers for feedback.

Context

“Your student’s current overall grade is 86% in Algebra 2A as of 1/4/19 with 22 days left. If you are confused about your grade or have questions about the course, your teacher Jane Smith can help you out at (555) 555-5555.”

Questions

• 1. Which statement best describes this message?
  • a. Your student’s current grade in Algebra 2A is 86%.
  • b. Your student has completed 86% of Algebra 2A.
  • c. Your student’s final grade in Algebra 2A is 86%.
  • d. Your student’s grade for an assignment in Algebra 2A is 86%.
  • e. Your student’s grade for an exam in Algebra 2A is 86%.
  • f. None of the above.
• 2. If none of the above options are accurate, how would you describe this message?

Results

• 229 Participants @ 50¢ a participant
  • a. 141
  • b. 22
  • c. 55
  • d. 5
  • e. 6
  • f. 10

Overall we were able to gain quantitative data rapidly and solidify an actionable plan to move forward with the first option that removes the “overall” terminology. Through removing this one word we were able to reduce support calls concerning this specific issue by over 70%.

Reflection

This was a really fun and different way to gain quick feedback about design decisions that I could see being utilized in other areas of UX copy and metaphor validation. And while this was a great use of an out-of-the box strategy to gain user feedback, there are some questions I can think someone would have about it.

One that comes to mind is how can we be sure of efficacy of the results that we’re receiving, or more simply put, how can we trust these responses. Bots have been made to automate MTurk responses and make workers money without actually doing things, and some people will just speed through trying to finish as many tasks as possible. I believe that’s where the quantitative aspect comes in. If you have more responses than you can be a little more assured that the ones you’re receiving are going to be actual people.

---

Along with these messages we also offer a few other message types based off of customer needs as well as a PowerBI dashboard which utilizes the database that we store our messages and responses in to help provide valuable data to the school.