ChatGPT in Education
Executive Summary
Introduction
This study aimed to analyze how ChatGPT 3.5’s interface and features impact users' ability to complete educational tasks. The primary objective was to identify usability issues and pain points that users encountered while using the platform in an educational context, and to provide insights for improving the user experience.
Methodology
A digital survey was used to gather insights from University of Washington students and recruit participants. These insights guided our testing, which involved three tasks: reading an academic article, summarizing it, and creating study materials. Sessions were conducted on May 14th and 16th with one participant and two researchers per session, starting with interviews and ending with a debrief.
Key Findings
Information Validity: Participants struggled to assess the accuracy of ChatGPT 3.5’s responses.
Prompting Issues: Users were uncertain about starting chats and using prompts effectively.
Regenerated Responses: Participants found regenerated responses unsatisfactory and had difficulty understanding the tool’s capabilities.
Interface Navigation: While users appreciated the ease of use, some found long chat logs challenging to navigate and the regenerate button hard to find.
Recommendations
Enhance Information Transparency: Introduce indicators or explanations to help users better assess the validity of ChatGPT’s responses.
Improve Prompt Guidance: Provide guidance or examples for effective prompting to reduce user confusion and improve task completion.
Refine Regenerate Feature: Make the regenerate button more accessible and provide clearer feedback when responses are regenerated.
Improve Navigation of Chat History: Redesign chat log navigation to help users quickly locate specific parts of the conversation and improve overall usability.
Conclusion
By addressing these key findings, the user experience of ChatGPT 3.5 in educational contexts can be significantly improved. This will help students use the tool more effectively, leading to a better understanding of its capabilities and fostering greater confidence in its use for academic tasks.