When we released the Learning Insights Dashboard (LID), we anticipated it would empower tutors to make better decisions and support students effectively. However, actual usage fell short of expectations. What went wrong? This challenge pushed us to dive deeper into understanding tutor needs, identify the gap, and redesign the experience.
PROJECT OVERVIEW
"I didn't know LID can be used to monitor progress, I have never really seen the small module title under name change"
"I always try to give at least every student like 5 min of my time"
"What does “working” mean? Students who don’t have any problem? How does the algorithm determine the “May need help”?"
In this instructional and UX design project, I aimed to redesign the Learning Insights Dashboard (LID) and onboarding training to make it more accessible, intuitive, and actionable for tutors. This journey was guided by an iterative process, including usability testing, need-finding research, alignment with stakeholders, and multiple redesigns—all framed within the ADDIE model for instructional design.
Analyze
Identifying Key Challenges
Problem Identification: After the LID Dashboard was released, it became clear that tutors were not engaging with it as expected. To uncover why, we conducted usability testing with novice tutors and need-finding interviews with previous users of the dashboard. We identified key pain points: tutors found it difficult to understand how to use the dashboard effectively, and the student cards lacked the detailed information necessary for decision-making.
Stakeholder Communication: We used clear documentation and workshops to communicate findings asynchronously with busy stakeholders and subject matter experts, including researchers and tutor supervisors, ensuring their time was respected. The most urgent problem identified was that the current dashboard and training, were not providing tutors, who were not professional educators, with sufficient, relevant information to make informed decisions.
Design
Aligning with Stakeholders and Creating Solutions
Stakeholder Alignment: After identifying the core issues, we reconvened with stakeholders to present possible solutions. During this process, stakeholders emphasized the need for prescriptive guidance—tutors needed clear instructions on interpreting student statuses and actionable steps to take. We also confirmed with SMEs that the "round robin" method of checking in with students, as reported by tutors during interviews, was not the most efficient use of their time and did not align with the service's goals.
Design Considerations: Based on this alignment, we focused on adding prescriptive elements to the experience. This included clear instructions for tutors on how to act based on changes in student learning status and what specific language to use when engaging with students.
Development
Iterative Design of Student Cards and Training
Student Card Redesign: We iterated on the student card design to ensure tutors had clear information about each student’s learning status. We replaced ambiguous icons and colors with semantic labels such as "Needs Motivation" and "Needs Content Help." Each label was accompanied by additional descriptions, providing clarity on the type of support required. This change improved the accessibility and intuitiveness of the dashboard.
Developing Prescriptive Training: In addition to redesigning the dashboard, we created a focused tutorial to onboard tutors. This tutorial, implemented on Notion, included use cases on the best way to engage with students on Zoom and how to use LID to prioritize helping students who needed immediate attention. It provided step-by-step instructions on how to interpret learning statuses and what to say to support students effectively.
Implementation
Training Delivery and Dashboard Release
Initial Training Implementation: Initially, all training materials were included in a single tutor handbook on Notion. However, we noticed that this created challenges for tutors trying to locate specific information. The articles were difficult to navigate, and tutors had to search deeply to find the resources they needed.
Help Center Creation: To address this, we built a dedicated Help Center for the company using Intercom. This new Help Center provided organized, easy-to-find access to key resources, including the LID onboarding tutorial. In parallel, the redesigned student card features were also released to tutors. These efforts collectively contributed to a 193.3% increase in user engagement with the LID dashboard.
Evaluation
Assessing Effectiveness and Planning Improvements
Ongoing Evaluation: To evaluate the effectiveness of the redesigned dashboard and training, we are currently conducting both quantitative and qualitative assessments. Using Microsoft Clarity, we are analyzing tutor interactions with the dashboard to understand where they are clicking, when, and how they use the information to support students.
Tutor Recordings Analysis: We are also analyzing recordings of real tutoring sessions to observe how tutors used LID during live interactions. This will give us critical insights into the practical application of our tools and guided our next steps for improvement.
Next Steps: Given our previous round of need finding, our evaluation will likely reveal opportunities to redesign the "deep dive" feature, which some tutors found cumbersome. We hope to identify and confirm this need, and provide better use cases within the tutorial to help tutors understand when and how to utilize deep dives effectively. These improvements are part of our plan to further streamline tutor workflows and enhance decision-making capabilities.
More projects
STAR Framework for Behavioral Interview
Assessment Design, Learner Testing, Andragogy
Storymate: Personalized writing chatbot
Interactive Course Design, Gen AI
Will Tara Pass the Class?
Scenario-Based Learning, Faculty Training, DEI
Building Healthy Peer Relationships
Curriculum Design, Backward Design, UDL