Lead UI Designer & UX Researcher
The project's main goal was to help children with autism better understand how to follow ordered steps. Our secondary goal was to do a good enough job that it would create buy-in from the Idaho school board or the Idaho chapter of GEAR Up. Our project (along with a couple of others) was successful in this. The GIMM Works organization that I was a part of, employed by Boise State, successfully partnered with both.
The majority of this project was spent attempting to understand our end user. We struggled to create a user persona since each child with autism could have a widely varying level of access to our app. The best we could do was perform extensive generative research on the community and conduct interviews with our advisor who had a deep understanding of autism. In the end, we decided we would design for a child with access to sight and who had the mobility to hold the iPad camera steady or who could be with a teaching aid that would hold the iPad camera steady.
Our first idea was to have a set of lego instructions that were overlayed on the screen as the student pointed the iPad camera at the legos.
Our biggest concern here was how the progression between steps should be handled and how we could give positive feedback to the student that they had completed the correct step.
After testing out the first prototype, we realized we wanted everything to be docked on the edges and corners of the screens so that the middle with the camera wasn't overwhelming.
Additionally, we liked having the 'next step' available at the bottom corner of the screen for transparency of the steps.
The project started in January 2020 with a deadline of April 2020. With our second idea in February, we were feeling pretty good about our progress and were looking forward to having actual students try it out.
Our third prototype, further refined what we had ideated in the second prototype. This idea got approved by our supervisor and we began development.
Since the beginning of the project in January, we had been learning Swift and ARkit in order to use the Apple iPads that would be available in the classrooms that we were delivering the product to. After getting approval for our 3rd design, we had become very well versed in the Apple software and tools that we were using and knew how we would go about developing our prototype.
Then, quite suddenly, the pandemic and resulting quarantine caused us to lose access to the university-supplied "Apple lab" which we had been developing in. We expected that in a couple of weeks, things would have run their course and we would be back in the lab.
Paper Prototype (Kind of?)
After the second week of quarantine, when things started feeling like they weren't going back to normal anytime soon, we accepted that we would not be able to use ARKit and Swift. We started learning Vuforia within Unity and the idea we kept coming back to was 'what if we replaced the legos with a puzzle?'
Without access to a printer and desperate to grasp onto some new idea, I cut up an Eggo box and we spent two hours on Zoom as a team wondering how to turn this into something viable in the next two weeks.
For full transparency, morale was low. Like, really, really low. Every conversation turned back to COVID. We questioned the point of the project if students weren't going to be in school. We did not foster a very collaborative environment during our shared feelings of hopelessness. We procrastinated and held onto the puzzle idea, while learning new technology, until 5 days before the deadline.
While attempting to learn Vuforia, we discovered the overlay of assets on top of 2d images. We had a discussion about whether we wanted to overlay the steps using this method or use the method to do something else entirely. We went with the latter. We realized if the whole point was teaching students to follow directions, what if we just made following directions fun? Our final idea was born, with mere days to spare.
In the final app, a student uses a mobile device to add context to a list of instructions that are written on paper. The added context comes in the form of 3d animations overlayed above the instructions. In our MVP, the instructions are for washing your hands. As the student’s mobile device registered which instruction was being viewed, it overlayed an animation of that step. The hope was that after many repetitions with the cards and the app, the student may then take the cards home with them and attach them to a mirror and retain an understanding of what the ordered list represented. Additional haptics, graphics, and sounds were added to make the completion of a step very viscerally satisfying. The attached video titled Puzzle Masters has a walkthrough of the MVP app.
With such a broad scope, our team struggled greatly in picking a good idea to move forward with. Of course, much of this indecision came from a place of us wanting to deliver the best possible product and being hesitant in declaring what exactly that product was. A global pandemic didn't help. We did our best to leverage our humility and desire to do well by rapidly prototyping and testing. While we weren't given many constraints at the beginning, the more we prototyped and tried things, the more we were able to write off certain ideas as unfeasible and thus limit our scope and make it easier to come to decisions.
What I learned
A session that stands out to me is when I was in a meeting with stakeholders and there was a ton of confusion going back and forth attempting to clarify what versions of difficulty there would be on the games. As the PM speaking on behalf of my team, I was making the mistake of talking too close to the content as I believed that was where the miscommunication lied. We went back and forth about which pairs of flashcards were on which screen's difficulty. Our stakeholders kept stating the difficulty levels were mixed up and I was struggling to understand. We confused one another for a while longer until we tabled it for the next meeting when I promised to show up with a visual.
The next requirements gathering I was feeling quite defeated because my first few weeks as a PM were a nonstop confusion of emails and miscommunications. However, when I showed my visual representation of the flashcard types, the stakeholders immeditately identified the problem. We were missing the flashcards that contained phonetics. In the folder that the stakeholder had shared with us to complete the project, there was a heap of content to go through. Some of it was direct curriculum and some of it was supplementary to the curriculum. When I saw the folder full of phonetic cards, I had assumed that the Center on Literacy and Deafness had included them as supplemental material for the developers and not something they used with their students. I did not know that they had a curriculum for students 'with access to sound' and students 'without access to sound' as they later informed me. I had no previous exposure to the community of people we were working with and as a result, made an erroneous assumption on the content. The issue was quickly resolved and the project marched forward, but the experience stays with me.
When I think about my takeaway here, I don't think too much about the epiphany 'aha' moment that the stakeholders and I shared a laugh about. I instead think about just a couple of hours earlier when I was looking at the list of flashcards that I did have and that I was so confident was inclusive of all the flashcards they had given us. Today, anytime I feel that research is conclusive or that I am confident about an assumption, I always hesitate and look for information that I may have missed. I am constantly seeking out information that I don't yet know... that I don't know.
Additionally, I learned that a visual is the best communication tool. Whenever it's possible during communication with someone, I try to have something visual that can be referred to during our conversation.