01
02
03
04
05
06The group will observe a student's interaction with ChatGPT, a conversational artificial intelligence (AI) assistant developed by OpenAI that allows users to communicate through natural-language prompts and receive generated responses (OpenAI, 2022). The communication situation will consist of a student creating two separate reviewers. The first will involve the student seeking assistance in creating a reviewer regarding environmental ethics, while the second will involve composing an email addressed to a teacher concerning a project's guidelines. The biology reviewer will serve as preparation for the upcoming long exam in Biology 4. ChatGPT will assist throughout the entire process, beginning with the student's first prompt, and ending when the reviewer is completed. The main purpose of the exchange is for the student to seek assistance in creating and refining the reviewer.
The student and ChatGPT will alternate as sender and receiver, with the ChatGPT website serving as the channel through which the communication takes place. The messages include the student's questions, requests, instructions, and corrections. The interaction takes place online. Feedback will occur through the responses and follow-up prompts exchanged throughout the conversation. The observation will focus on the student's emotional semantics and developing reliance on AI.
To support the analysis of emotional semantics, the group used three sentiment-analysis tools: Daniel Soper's Sentiment Analyzer (Soper, 2019), Valence Aware Dictionary and sEntiment Reasoner or VADER (Hutto & Gilbert, 2014), and CardiffNLP Twitter-RoBERTa sentiment model (Barbieri et al., 2020). The tools were used to provide multiple computational perspectives on the student's messages rather than relying on a single sentiment score. Their outputs were compared with the actual context and progression of each interaction to identify changes in the student's expressed sentiment and to examine differences among the tools. VADER is a rule-based sentiment model designed for text commonly found in social media and microblogging contexts, while the CardiffNLP model is a transformer-based RoBERTa model fine-tuned for English sentiment classification. Possible communication barriers include poorly worded or ambiguous prompts alongside inaccurate or misunderstood responses.
The model of communication involved will be interactive, as the student and ChatGPT—as aforementioned—continuously alternate as sender and receiver, where each response influences the succeeding messages. This model is particularly applicable because the communication involves active participation from both parties. The study itself focuses on the communication process only, including feedback and psychological context, rather than whether an individual message is transmitted and received.
The group aims to study how communication between a student and a generative AI develops throughout the creation of a Biology 4 reviewer on environmental ethics and the composition of an email to a teacher regarding project guidelines.
Project detailsExamine the emotional semantics of the student's language towards ChatGPT — gratitude, politeness, frustration, impatience, hostility, and other attitudes — supported by automated sentiment analysis.
Determine how the student's emotional language changes throughout the exchange.
Examine how reliance on AI develops throughout the communication process, particularly through repeated requests, acceptance or rejection of AI-generated information, and incorporation of AI suggestions.
With the participant's consent, the group will use screen recording, photographic documentation, field observation, and note-taking to document the communication process between the student and ChatGPT. The screen recording will preserve the sequence of prompts and responses throughout the task, while photographs will provide evidence of the observation process.
Field observation and note-taking will be used to record relevant contextual and behavioral observations, such as hesitation, confusion, laughter, frustration, or changes in the student's interaction with ChatGPT. A short follow-up interview may also be conducted, with the participant's consent, to clarify observations that cannot be determined from the recorded interaction alone.
The screen-recorded exchanges between the participant and ChatGPT will be organized chronologically to map the progression of the conversation. The group will encode the student's messages and relevant ChatGPT responses into a structured table. Each student message will then be examined according to its purpose and emotional semantics, including requests, clarifications, corrections, gratitude, frustration, impatience, and criticism.
Three sentiment-analysis tools will be used as supplementary computational tools: Daniel Soper's Sentiment Analyzer, VADER, and the CardiffNLP Twitter-RoBERTa sentiment model. Soper's Sentiment Analyzer produces scores from -100 to +100, where lower values indicate more negative sentiment and higher values indicate more positive sentiment (Soper, 2019). VADER and for this study, the CardiffNLP Twitter-RoBERTa sentiment model produces a sentiment score from -1 to +1, with scores closer to -1 indicating more negative sentiment and scores closer to +1 indicating more positive sentiment.
The scores will be recorded and compared across different stages of the interaction to observe changes in the student’s expressed sentiment. The group will also manually examine the messages and surrounding conversation to account for contextual meanings that automated sentiment analysis may overlook, such as sarcasm, humor, or indirect expressions of emotion. The group will simultaneously identify indicators of AI reliance, including repeated consultation, acceptance or rejection of ChatGPT's suggestions, requests for ChatGPT to make decisions, and incorporation of AI-generated material. The two observations will then be compared to identify patterns between sentiment, reliance, feedback, and communication barriers.
Based on the synthesized observations, the group will develop an interactive model of communication representing the observed interaction between the student and ChatGPT. The model will illustrate how the student and ChatGPT alternate roles as sender and receiver, with each response functioning as feedback that influences the succeeding message.
It will also incorporate the psychological context of the student, particularly the emotional semantics identified throughout the exchange, as well as the communication barriers that may affect the quality and continuation of the interaction.
The study has limitations that should be considered when interpreting the findings. First, the study will only involve observing the same student twice, so the findings may not represent how other students communicate with or rely on ChatGPT. The student's familiarity with ChatGPT, prior knowledge, and personal communication style may also influence the results.
Next, the group will not assume that a single message or sentiment-analysis result represents a definitive emotional state. The sentiment-analysis tools will be used to support the analysis of emotional semantics, while the surrounding conversation and context will also be considered when interpreting the student's language.
Lastly, ChatGPT's responses may vary depending on the exact wording of the prompts, conversation history, model version, and platform settings. Therefore, the observations will be treated as focused cases of human-AI interaction rather than universal representation of how students communicate with AI.