Monday, November 27, 2023

I let students use ChatGPT freely

This semester I asked students to use ChatGPT freely for homework submissions, but they had to tell me when they used it. No other guidelines: they could use it in full or in part, to reword something, or for anything. For high stakes examinations, students were allowed to earn up to 5% points using ChatGPT if the told me they used it.

This is what I found:
  • Around 2 students in a group of 12 said they used it in some assignment, and 2 students in a group of 18 said they used it. The others said they didn't use it, and I agreed they didn't use it since their results sounded authentic.
  • The four students who used ChatGPT used it on a regular basis. There is no "occasional users". It's an all or nothing tool.
  • Among the four students who used it, three were whites. Only one indigenous reported using ChatGPT. No Hispanics reported using it.
  • Among the four students who used it, three were really good students who didn't need to use it to obtain a good grade. The other student started doing better when she stopped using it.
  • In one discussion the students had to identify if their classmates used it. Only those who used it on a regular basis were able to identify other people using it. They identified them because they got familiar with the linguistic style of the ChatGPT. As preservice teachers, non-users will be at a disadvantage since they are unaware of the style.
  • ChatGPT makes good students better and bad students worse.
  • I believe the students used it more often last semester when I was unaware that they were using it. Giving them freedom to use it really discouraged usage.
  • Long papers to read yielded high usage. One student asked ChatGPT to summarize the paper, and he didn't capture the core of the ideas. The student would have been earned a low grade if not protected by the open-ended policy.
  • Short written assignments yielded less usage. The students were aware that it would take longer to pull it down from ChatGPT than producing a writing.
Implications into my teaching:

Students need detailed guidelines on how to use ChatGPT on each assignment. An assignment where ChatGPT was absolutely prohibited should also be noted clearly at the top of such assignment. For example, I realized ChatGPT was very good to provide translation in context to produce their flashcards in Spanish/English. I encouraged students using it for this assignment, and it yielded higher usage. In one written and proctored exam, one student used it to respond to only three questions in the exam as advised at the top of the instructions. I believe such report was accurate due to this student's performance in the exam. If she used it longer than that, it didn't work.

Students who are not using it may be at disadvantage when the professor is not aware of such potential usage. For example, in such Flashcard assignments, students who didn't use it produced poorer translations, but more authentic. In another assignment, students used it to analyze a song. The students who used ChatGPT produced the most accurate responses to what I was expecting. Saying students that I wouldn't grade grammar, spelling or accuracy helped students earn authenticity. I learned to see values in choppy sentences, spelling mistakes and inaccurate thinking.