Interactive Learning Environments

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/nile20

The effectiveness of Gen AI in assisting students’

knowledge construction in humanities and social
sciences courses: learning behaviour analysis

Shuai He & Yu Lu

To cite this article: Shuai He & Yu Lu (22 Oct 2024): The effectiveness of Gen AI in assisting
students’ knowledge construction in humanities and social sciences courses: learning
behaviour analysis, Interactive Learning Environments, DOI: 10.1080/10494820.2024.2415444

To link to this article: https://doi.org/10.1080/10494820.2024.2415444

View supplementary material

Published online: 22 Oct 2024.

Submit your article to this journal

Article views: 178

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=nile20
INTERACTIVE LEARNING ENVIRONMENTS
https://doi.org/10.1080/10494820.2024.2415444

The effectiveness of Gen AI in assisting students’ knowledge

construction in humanities and social sciences courses: learning
behaviour analysis
a b
Shuai He and Yu Lu
a
School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing, People’s
Republic of China; bSchool of Humanities, Beijing University of Posts and Telecommunications, Beijing, People’s
Republic of China

ABSTRACT ARTICLE HISTORY

Currently, generative AI has undergone rapid development. Numerous Received 24 April 2024
studies have attested to the benefits of Gen AI in programming, Accepted 4 October 2024
mathematics and other disciplines. However, since Gen AI mostly uses
KEYWORDS
English as the intrinsic training parameter, it is more effective in Generative AI; ChatGPT;
facilitating the teaching of courses that use international common learning behaviour;
notation, but few scholars have researched the fitness of Gen AI- knowledge construction;
assisted teaching of humanities courses in Chinese-language higher education
environments. To address these gaps, this study examined the learning
behaviours of 30 students using Gen AI to help them answer questions
on economic law tests using the Lag Sequential Analysis. The results
show that the following: (1) The use of Gen AI to aid learning in an
economic law course did not significantly improve the cognitive level of
academics from the perspective of knowledge construction. (2)
According to the characteristics of students’ behavioural paths via Gen
AI-assisted learning, their behavioural patterns can be classified into
autonomous and innovative, moderate, and lacking innovation. (3)
Different learning modes when Gen AI-assisted teaching was used
affected the final results, which were as follows: High-performing
students favoured the autonomous and innovative pattern, medium-
performing students favoured the moderate pattern, and low-
performing students favoured the lacking innovation pattern.

1. Introduction
The advent of modern information technology has instigated significant transformations within the
education sector (Karabacak et al., 2016; Mugruza-Vassallo & Suárez, 2016). Following the wide­
spread adoption of generative artificial intelligence (Gen AI)—, spearheaded by ChatGPT at the
end of 2022, with its highly efficient text analysis and data processing capabilities—educators
have become increasingly interested in integrating this technology into their teaching methods.
This integration aims to foster an intelligent and personalised educational paradigm (Bozkurt,
2023; Lee et al., 2024); Dwivedi et al., 2023). In the Horizon Report 2023: Teaching and Learning
Edition, numerous higher education experts pinpoint Gen AI as one of the most transformative tech­
nologies of our era and a pivotal trend in the future evolution of higher education. Within this report,
Pelletier (2024) underscores that Gen AI offers unique benefits such as enhancing productivity, pro­
moting pedagogical innovation, aiding in research depth, and bolstering student engagement.

CONTACT Yu Lu [email protected]
Supplemental data for this article can be accessed online at https://doi.org/10.1080/10494820.2024.2415444.
© 2024 Informa UK Limited, trading as Taylor & Francis Group
2 S. HE AND Y. LU

Leveraging large language models (LLMs) (Mahowald et al., 2024)) in Gen AI can facilitate learning
and teaching through input, interaction, and the autonomous generation of simple text commands
(Baidoo-Anu & Owusu Ansah, 2023; Bahrini et al., 2023; Steele, 2023). Notably, owing to its emphasis
on language usage and content creation, Gen AI aligns intrinsically with the pedagogical require­
ments of humanities disciplines. For example, in law programmes, Gen AI, exemplified by
ChatGPT, have been shown to possess capabilities such as admittance to out-of-country legal qua­
lifying exams, the ability to draft legal documents, the ability to pass law schools’ final exams, and
the ability to aid in reducing the teaching workload, among other demanding professional compe­
tencies (Perlman, 2022).
Currently, ChatGPT’s training parameters predominantly consist of English corpora, which results
in superior teaching assistance outcomes for courses involving international, universal disciplines
such as foreign languages, mathematics, and programming (Hettiarachchilagea, 2023; Kuhail
et al., 2024). In contrast, there is a relative lack of pertinent measurements in the field of education
in the Chinese context. For example, using the Chinese search engine Zhi.com, 774 search results
were found when using Gen AI and education as keywords, but no results can be found when
using Gen AI and Chinese context as keywords. Most relevant studies use students’ performance
as the only criterion for judging whether Gen AI-assisted teaching is effective, which produces a
lower depth of research. Focusing on law courses, since the Chinese corpus is more confusing
than the English corpus and the narratives of legal cases are more subjective and complex, it is
easier for Gen AI to produce wrong outputs to mislead students, and the specific effect of assisted
teaching is unknown. Second, there are relatively few studies on students’ learning behaviours when
Gen AI is used to aid learning. For instance, 879 search results were found when using students’ learn­
ing behaviours as keywords, but only 1 result was found when using Gen AI and students’ learning
behaviours. Moreover, it is impossible to accurately identify students’ specific behavioural patterns
when Gen AI is used to assist learning; consequently, it is difficult to make reasonable explanations
for other problems (with the exception of contextual changes) and to judge whether students’ own
cognitive level has improved after Gen AI is employed to assist learning. To address this gap, our
study focused on college students’ economic law courses and examined both learning performance
and behavioural data. We explored the efficacy of Gen AI-assisted teaching in the Chinese context,
delved into students’ behavioural patterns, and identified the reasons behind Gen AI’s impact on
learning outcomes from a behavioural standpoint. In terms of the future, we aim to effectively
guide the design and execution of subsequent teaching and learning endeavours.

2. Literature review
This section provides a detailed review of the literature concerning our two focuses: Gen AI-assisted
education and students’ behavioural pathways.

2.1. Gen AI-assisted education

With the great advancement of educational technologies, computer-supported collaborative learn­
ing has gained increasing attention in higher education. For example, in the context of the public use
of Gen AI, especially ChatGPT, many education researchers have conducted studies related to
ChatGPT-assisted education. Yilmaz (2023b) investigated how ChatGPT affects college students’ pro­
gramming thinking and programming self-efficacy; the results revealed that the use of ChatGPT-
assisted teaching in programming learning can significantly improve students’ computational think­
ing ability and learning motivation. Yılmaz and Yılmaz (2022) and other scholars (Zekaj, 2023)
asserted that by using AI tools or creating a supportive environment when teaching programming,
students can ask questions and receive feedback in real time, thus creating personalised education
for each student. Focusing on the teaching of humanities courses, Agustin (2023) explored the effect
of ChatGPT on learning English through questionnaires and interviews, and found that timely
 INTERACTIVE LEARNING ENVIRONMENTS 3

feedback from ChatGPT can effectively enhance students’ self-reflection and self-evaluation, thus
improving their academic level of English. Perlman (2022), in the context of teaching law courses
at a law school, utilised ChatGPT to administer test items, legal qualification exams, and final
exams. Students who were aided in learning by ChatGPT showed better comprehension of legal
language, French, and legal intent recognition; Perlman suggested that the reasonable use of
ChatGPT-type technology can effectively reduce the teaching burden on law professors. Fathi
et al. (2024) explored the effectiveness of using Gen AI in speaking instruction through test scores
and semi-structured interviews; the use of AI significantly improved students’ English-speaking
skills, and most students had positive attitudes towards AI intervention in speaking instruction.
Drawing on empirical studies of ChatGPT-assisted education outside China, many wonder if similar
results can be obtained in China. To this end, many scholars have conducted relevant research in
various educational fields. In computer education, Jiang and Duan (2023) investigated the bright
future of ChatGPT-assisted computer education, which can yield better results in terms of students’
code writing, understanding of complex concepts, and pressure on teachers. Zhou (2023) verified
that the use of ChatGPT in English-language teaching significantly improved students’ vocabulary
learning and fulfilled their individual learning needs through controlled experimental methods.
The effects of ChatGPT-assisted teaching in subjects such as computer science and English—both of
which use international corpora—are consistent with findings from foreign studies. However, as for the
teaching of humanities in Chinese contexts such as law and history, where professional knowledge is
relatively in depth and topic inquiries are fairly obscure, Chinese scholars have been unable to replicate
findings from abroad. For example, Liping et al. (2024) horizontally compared the answer data of
several Gen AI models (including ChatGPT, Wenxin Yiyin, and WeLM) when examining objective
and subjective questions in secondary school history taught domestically; the results revealed that
although the current Gen AI is able to produce a complete and fluent expression of its own output
answers, it is still far below the average level of secondary school-age children, as shown by a quiz
of secondary school history knowledge. Moreover, in terms of knowledge reserve, logical reasoning
still has considerable room for improvement. As for legal instruction, scholars in China have not per­
formed many empirical studies. Wang (2023) suggested that the core of the legal industry is the under­
standing and use of legal language and the generation of legal knowledge based on legal language.
From the perspective of technological logic, the legal industry and ChatGPT have an intrinsic logical fit,
but the current version of ChatGPT in terms of the Chinese language corpus is relatively weak in its
ability to understand legal language; furthermore, the use of ChatGPT-type technologies in legal edu­
cation scenarios still belongs to the transition from lower-order to higher-order technology.

2.2. Analysis of students’ behaviour

In recent years, the emphasis in education has shifted from lower-order thinking to higher-order
thinking. The evaluation of learning has also shifted from a single performance-based decision to
a multi-level, multi-method, comprehensive evaluation of the entire process. The emergence of
learning analytics has provided educators with a viable solution to study the previously unobserva­
ble relationship between students and academic outcomes, and has provided theoretical guidance
for all-encompassing evaluation and the improvement of instructional programmes. For the fourth
consecutive year, the Horizon Report has identified learning analytics as one of the six key technol­
ogies that will influence the future of education (Xu Dan, 2024). Learning analytics refers to “under­
standing and optimising learning and learning environments by measuring, collecting, analysing,
and reporting data about learners and their environments”. Learning behaviour analysis is an impor­
tant part of learning analytics, which aims to purposefully examine behavioural data recorded during
the learning process. The purpose of learning behaviour analysis is to explore the valuable infor­
mation hidden behind behavioural data (such as behavioural patterns and habits), and ultimately
to promote the understanding and optimisation of the learning process, outcomes, and environ­
ment (Siemens & Long, 2011).
4 S. HE AND Y. LU

Learning behaviour analysis provides ideological and theoretical support for investigating the
intrinsic elements of students’ learning effectiveness, whereas lag sequential analysis1 (Sackett,
1978) and GSEQ2 (Akeman & Quera, 1995) software provide researchers with software functional
support. We chose LSA because many scholars have already carried out studies on students’ behav­
iour using LSA in secondary schools and universities (Eryilmaz et al., 2013; Hou, 2012; Lai & Hwang,
2015); the results show that LSA has better application prospects in the field of learning behaviour
analysis, which can help researchers and teachers accurately grasp the potential behavioural pat­
terns of learners, explain why technology enhances learning effects from a behavioural point of
view, and at the same time effectively guide the design and implementation of subsequent teaching
and learning activities. Focusing on higher education technology, the use of LSA to examine stu­
dents’ learning behaviours has gradually become professional and popular. For example, Sun Dan
et al. (2024) recognised the use of Gen AI in programming instruction as an effective way to
reduce pressure on teachers and to increase students’ motivation to learn; Sun Dan analysed stu­
dents’ behavioural paths when they used ChatGPT to aid their learning via lagged sequential analy­
sis (LSA) and explained why ChatGPT can effectively empower programming education from
students’ perspectives. Yang et al. (2015), in order to determine whether a two-tier testing strategy
had a significant effect on students’ cognitive level, employed LSA to explore all students’ learning
behaviours in the experimental process, and determined that the two-tier test helped students
improve their programming skills in terms of behavioural patterns.

2.3. Research questions

The literature suggests that ChatGPT cannot achieve similar results to those collected outside of
China in assisted instruction in the humanities. Apart from the objective variable of switching
from an English corpus to a Chinese corpus, other factors may contribute to changes in the effective­
ness of ChatGPT-assisted teaching. Second, the current stage of research on ChatGPT-assisted edu­
cation involves evaluating its effectiveness based on performance levels. This approach overlooks
the learning process itself, which includes how students build knowledge through recognising,
understanding (Sung & Hwang, 2018), and solving problems (Van Aalst, 2009). Weinberger and
Fischer (2006) argued that studying the process of knowledge construction can effectively reveal
the relationship between students’ learning effectiveness and their external learning environment.
Hence, we took an undergraduate economic law course as an example from the perspective of stu­
dents’ process of using ChatGPT to assist teaching. We applied learning behaviour analysis technol­
ogy to collect, analyse, and mine the granularity of learners’ processes when using ChatGPT. The
following questions are answered through a statistical comparison of students’ learning patterns
and learning performance:
(1) Does the integration of Gen AI in economic law coursework significantly influence students’ aca­
demic performance, particularly in terms of cognitive level?
(2) What distinct characteristics do students’ learning patterns reveal when Gen AI is utilised to
facilitate answering questions during an economic law course?
(3) In the context of using Gen AI to facilitate answering questions in an economic law course, what
is the relationship between various learning patterns and the efficacy (i.e. the knowledge con­
structed) of employing Gen AI to enhance instructional strategies?

3. Methods
3.1. Participants
The participants were 30 sophomore A students majoring in business administration who were
taking an economic law course at a certain postal university in the fall of 2023, whereas a
 INTERACTIVE LEARNING ENVIRONMENTS 5

Table 1. Experimental introduction.

Experimental Group A1 A2 A3 B1 B2 B3
Use of Gen AI in courses √ √ √ × × ×
Use of Gen AI in testing sessions × √ × × × ×
Pre-test √ × × √ × ×

homogeneous Group of 30 sophomore B students majoring in business administration were

selected as the experimental control Group (the two classes were randomly Grouped into indepen­
dent classes, and both classes were taught by the experimenter, which ensured that the experiment
was manageable and avoided large sample gaps). The course consisted of 16 class periods, with
ChatGPT-assisted instruction provided in the first 15 class periods for Class A (including traditional
economic law and the use of ChatGPT) and only traditional economic jurisprudence in Class
B. The course was taught in the first fifteen class periods. A quiz on economic law knowledge
without ChatGPT assistance was administered to students in Groups A and B in the thirteenth
class, with scores recorded as A1 and B1 as a pre-test for the overall experiment. In the last class,
the students in Group A were asked to use ChatGPT for assisted open-book answer scores (recorded
as A2), whereas the students in class B were asked to use open-book answer scores (recorded as B2)
with the teacher’s materials only. The significant difference between the A2 and B2 Groups was
determined based on the level of students’ scores in the final submission of their answers.
Second, to establish the level of influence of ChatGPT on the same body, after the end of the
exam, the students in Group A were arranged in the absence of ChatGPT, and the open-book exam­
ination scores were recorded as A3 to maximise the experimental control variables in the same con­
ditions of the quiz regarding the Group B scores (recorded as B3; the experiments are outlined in
Table 1). All three Groups of students were told that their scores would not affect their final
grades in order to avoid differences in subjective motivation caused by the pressure of credits,
thus affecting the objectivity of the experiment. To study the behavioural patterns of Group A stu­
dents during the question-answering process via ChatGPT, we used the online question-answering
proctoring function of SuperStudyPass to record the students’ question-answering behaviour for
subsequent research. The Gen AI model used in this study was unified with the ChatGPT 3.5-
turbo model developed by Open AI, Inc. and it was made available to students in the form of a
fully functional website.

3.2. Instruments and procedure

The experimental idea and tools proposed are shown in Figure 1. To study students’ cognitive con­
struction behaviour when using ChatGPT to assist in answering questions, the first step involved

Figure 1. Experimental methods.

6 S. HE AND Y. LU

analysing students’ behavioural patterns. However, the cognitive process (as a psychological
process) cannot be observed directly with the naked eye, so we viewed the students’ test manipu­
lation behaviours as an external representation of their cognitive activities (Jamaludin et al., 2009),
and we employed LSA to examine the behaviours recorded via SuperStarLearn.
First, we used the final course exam from the previous class as the primary method of documen­
tation. The students’ question-answering behaviour was meticulously captured throughout the
examination via SuperStarLearn’s proctoring function, providing a comprehensive record. The
choice of the final exam for documentation is due primarily to two reasons. First, after fifteen ses­
sions, students become more familiar with the key points in the economic law course. This familiarity
mitigates any potential impact on deeper inquiries into the disparities in learning patterns among
differently performing Groups. These differences are often attributed to students’ generally low per­
formance due to their lack of knowledge regarding test questions. Second, following the fifteen
ChatGPT-assisted sessions, the students developed greater familiarity with using ChatGPT. This fam­
iliarity has helped to prevent any negative effects on experimental outcomes caused by excitement,
curiosity, and other subjective emotions. A thorough literature review on data coding aspects indi­
cated that numerous researchers have previously coded students’ classroom learning behaviours
and developed more mature behavioural coding scales. Consequently, we reference the findings
of Blikstein et al. (2014), adapting their behavioural analysis of the coding scale for the programming
process to one suitable for economic law students (Table 1). Two coding members coded students’
behaviour in terms of the modified coding sheet and cross-verified each other’s findings. The
reliability coefficient between the two coding members was 0.83, implying a high level of consist­
ency in the adapted coding sheet for measuring students’ behaviour. This finding provides a
robust foundation for subsequent experiments.
Second, we analysed the codes for student behaviour gathered by coders using the GSEQ5 soft­
ware. This facilitated the creation of tables and adjusted residual tables to measure the frequency of
behavioural transition. These tables enabled the coders to construct a network diagram illustrating
the learning behaviour patterns of all students. The behavioural network diagram offers a compre­
hensive view of overall student behaviour in the experimental Group, highlighting its unique fea­
tures. We subsequently evaluated data from Group A2 (categorised as low performance, middle
performance, and high performance) and mapped the learning patterns of students with varying
levels of performance. We did so to investigate how the learning patterns of students with
different levels of performance are characterised by their use of ChatGPT. Through this research,
we explored the internal relationship between the impact of students’ use of ChatGPT to answer
questions and their learning mode.

4. Results and analysis

4.1. Analysis of students’ achievement data
The three Groups of quizzes in this experiment are the results of the dual evaluation by the Super­
StarLearn platform and the classroom teachers (the evaluation score criteria include the score rate,
the completeness of the knowledge points, the error rate of the legal texts, the logical reasonable­
ness, and the natural fluency of the language). Table 2 displays the statistics of the three Groups’ quiz
scores in which both Groups A and B consisted of 30 data samples; the findings from the mean value
indicate that Group A2 had the best performance among Group A data. Group A2 also demonstrated
the highest performance among Groups A and B. In addition, Group A2, regarding the number of
high-performing people and scores, shows an excellent level. The performance of Group B1 is the
best; from the perspective of the mean values of A1, A2, B1, B2, and B3 are similar. To further
explore the level of significant difference between the three Groups, we carried out normality
tests and independent samples t-tests on the data of the three Groups; the results are shown in
Figure 2 (the normality test graph), and Tables 3 and 4 (the independent samples t-test).
 INTERACTIVE LEARNING ENVIRONMENTS 7

Table 2. Coding form for students’ behaviour.

Encoding Name of activity Categorisation Behavioural description
UT Understanding the task Commencement Students view topic details through the task window
RAM Referring to additional Students refer to teacher-provided resources to answer
materials questions
CA Coding the answers Compiling answers Students compile answers in the SuperStarLearn answer
box
UTA Understanding the Students check their answers by swiping left and right
answers across the answers with the mouse.
CTS Checking the score Students observe the fractions given in SuperStarLearn
RCM Reading the console The student returns to the answer box to check for errors
message based on the score given by SuperStarLearn
ANQ Asking new questions ChatGPT-assisted answering Students independently ask new questions on the
of questions ChatGPT platform
ARQ Asking relevant Students search the ChatGPT platform for questions
questions related to the question stem
COQ Copy the original Students paste the original question on the ChatGPT
question platform
RF Reading feedback Students read feedback in the ChatGPT platform
CPA Copy and paste the Students paste the answers given by ChatGPT into the
answers answer box
SO Stop the operation Close No manipulative behaviour by the student to end
answering the question

Figure 2. Plot of the normality test results.

As illustrated in Figure 1, the three datasets illustrate the outcomes of the normality test con­
ducted on the variable achievement data. The normality plot predominantly exhibits a bell-
shaped curve (with peaks in the middle and troughs at the extremities), suggesting that while the
data are not strictly normal, they can generally be deemed normally distributed. Hence, these con­
ditions meet the prerequisites for conducting subsequent independent samples t-tests.
Table 3 shows the level of statistical significance of Groups A2 and B2. The mean value of A2 and
B2, in terms of achievement, is 82.97/74.67. Due to the satisfaction of the chi-square test, the p-value
8 S. HE AND Y. LU

Table 3. Statistical table of student performance.

Average Standard Number of high Number of middle Number of low
Groups Aggregate value deviation Median performers/mean performers/mean performers/mean
Group 30 76.6 7.08 76.5 6/86.5 12/78.75 12/69.5
A1
Group 30 82.97 6.63 83.5 9/90.89 14/82.71 7/73.29
A2
Group 30 77.37 6.72 78 6/86.83 13/79.38 11/69.82
A3
Group 30 76.73 6.82 76.5 5/86.8 12/79.67 13/70.15
B1
Group 30 74.67 6.29 76 4/85.5 15/76.73 11/67.90
B2
Group 30 77.3 7.17 77 7/87.57 12/78.08 11/69.90
B3
Note: Performance grades are categorised as follows: high-performance grades are defined as X> = 85, middle-performance
grades are denoted as 75< = X<85, and low-performance grades are represented by X<75.

Table 4. Independent samples t-test for Groups A and B.

Variable Sample Average Standard Welch’s t- Cohen’s d- Mean
Variant value size value deviation t-test value value difference
Experiment A2 30 82.97 6.63 T = 4.975 P T = 4.975 P 1.285 8.3
1 B2 30 74.67 6.29 = 0.000*** = 0.000***
(Grand) 60 78.82 7.65 Note: ***, **, * represent the 1%, 5%, and 10%
total significance levels, respectively.

of the significance result on the independent samples t-test is 0. 000***, which indicates a significant
difference between A and B in terms of achievement. The magnitude of the difference, Cohen’s d
value, is 1.285. The magnitude of the difference is very high. Does this verify that the use of
ChatGPT in teaching can significantly improve students’ achievement level? The answer is no.
ChatGPT is similar to search engine technology to a certain extent and can provide users with
more relevant information in a shorter period of time. Although the accuracy of the information is
unsatisfactory, it can effectively reduce the time pressure on students to answer the questions,
thus affecting the test results compared with the open-book test in Group B. To determine
whether ChatGPT can improve students’ learning from the perspective of knowledge cognition,
we conducted an experiment with Group A3 (an open-book test with ChatGPT removed). The follow­
ing table shows the results of independent samples t-tests between Groups A1, A2, A3, B1, and B3.
The results are shown in the table above. Experiment 2 indicates that the mean values of A1 and
B1, in terms of achievement, are as follows: 76.6/75.53. Due to satisfying the variance of the chi-
square, the p-value of the significance result for the independent samples t-test is 0.561, so the stat­
istical outcome is not significant; this means that there is no significant difference between A1 and B1
in terms of achievement. Experiment 3 shows that the mean values of A2 and A3, in terms of achieve­
ment, are 82.967 and 77.367, respectively, and the p-value of the significance result for the indepen­
dent samples t-test is 0. 002*** due to the fulfilment of the chi-square requirement. Thus, the
statistical outcome is significant, which implies a significant difference between A2 and A3 in
terms of achievement. The magnitude of the difference, Cohen’s d, is 0.839, which is very large.
Experiment 4 indicates that the mean value of Groups A3 and B3, in terms of achievement, is
77.367/77.3. Due to the fulfilment of variance of the chi-square requirement, the p-value of the sig­
nificance result on the independent samples t-test is 0.971. Thus, the statistical outcome is not sig­
nificant, indicating no significant difference between Groups A3 and B3 in terms of achievement. The
magnitude of the difference, Cohen’s d, is 0.414, which is a very small magnitude of the difference.
Experiment 2 illustrates that the students in Group A, although they received 13 lessons of
ChatGPT-assisted experience, still did not differ significantly from the traditionally taught Group B
 INTERACTIVE LEARNING ENVIRONMENTS 9

students on the pre-test without the ChatGPT-assisted quiz. Experiment 1 revealed that students
achieved superior Group A2 scores on the ChatGPT-assisted quiz, which produced significant differ­
ences from Group B2 participants who did not use ChatGPT. However, this finding does not serve as
direct evidence that instruction via ChatGPT led to increased cognitive levels of student learning.
Control Experiments 3 and 4 showed that when ChatGPT was absent, the overall test level of
Group A3 declined significantly compared with that of Group A2 and did not differ significantly
from that of Group B3. This suggests that the cognitive level of the students in Group A3 was not
significantly different from that of the students in B3. In Control Experiment 5, the mean values of
A1 and A3, in terms of performance, were 76.6/77.37. Due to the satisfaction of the chi-square
test, the p-value of the significance result on the independent samples t-test was 0.0.669, so the stat­
istical result was not significant. This indicates no significant difference between the performance of
A1 and A3, i.e. in the conditions of withdrawing the ChatGPT-assisted test, the performance of Group
A returned to the original level of the pre-test, and its intra-brain cognition did not significantly differ
from that of Group B3 students. Moreover, intracerebral cognition did not significantly improve after
the experiment in Group A2. Our results thus converge with those of other scholars in China.
What factors contribute to the cognitive impacts of using ChatGPT in assisting student learning in
China? We undertook a subsequent investigation, focusing on students’ behavioural patterns.

4.2. Analysis of learning behaviour

4.2.1. Descriptive statistical analysis of learning behaviour
We generated a total of 1,140 behavioural transition sequences for the 30 experimental participants
in Group A. Table 5 presents the frequency and distribution of each behaviour. Statistical analysis
revealed that learners’ behaviours exhibited the following characteristics: (1) Without the quiz,
ChatGPT-assisted question-answering was the most frequent activity, followed by compiling
answers. The least frequently observed behaviours were the commencement and close categories,
which were aligned with the course content attributes. (2) Within the compiling answers category,
“Coding the Answers” (CA) was the most common behaviour, followed by checking the score (CTS),
understanding the answers (UTA), and finally reading the console message (RCM). This pattern
suggests that learners focused primarily on compiling answers and obtaining scores during
testing, with minimal time dedicated to understanding the answers. (3) In ChatGPT-assisted ques­
tion-answering behaviours, in addition to reading feedback (RF), the most prevalent activities
were asking new questions (ANQs) and asking relevant questions (ARQs). This finding indicates
that learners predominantly utilised ChatGPT as a questioning tool, incorporating their own insights

Table 5. Independent samples t-tests for five Groups.

Variable Sample Average Standard Welch’s t- Cohen’s d- Mean
Variant value size value deviation t-test value value difference
Experiment A1 30 76.6 7.08 T = 0.584 T = 0.584 0.151 1.067
2 B1 30 75.53 7.06 P = 0.561 P = 0.561
total 60 76.07 7.03 Note: ***, **, * represent the 1%, 5%, and 10%
significance levels, respectively.
Experiment A2 30 82.97 6.63 T = 3.248 P T = 3.248 P 0.839 5.6
3 A3 30 77.37 6.73 = 0.002*** = 0.002***
total 60 78.82 7.65 Note: ***, **, * represent the 1%, 5%, and 10%
significance levels, respectively.
Experiment B3 30 77.3 7.17 T = 0.037P T = 0.037P 0.01 0.067
4 A3 30 77.37 6.73 = 0.971 = 0.971
total 60 76.02 6.60 Note: ***, **, * represent the 1%, 5%, and 10%
significance levels, respectively.
Experiment A1 30 76.6 7.08 T = −0.43 T = −0.43 0.111 0.767
5 A3 30 77.37 6.73 P = 0.669 P = 0.669
Total 60 76.98 6.86 Note: ***, **, * represent the 1%, 5%, and 10%
significance levels, respectively.
10 S. HE AND Y. LU

when interacting with it. Only a limited number of students consistently exhibited behaviours such
as copy the original question (COQ) and copy and paste answers (CPA), treating ChatGPT merely as
an answer source.

4.2.2. The sequential transformation of learning behaviour and behavioural network

analysis
We examined students’ behavioural patterns during the use of ChatGPT for assisted question-
answering. We analysed the coding results via latent semantic analysis (LSA). Initially, the coded
data were processed with GSEQ5 software to generate a table for the frequency of behavioural trans­
formation and a table for the adjusted behavioural transformation residuals (Table 6). We labelled
behavioural sequences where the residual value (Z score) exceeded 1.96 as significant, indicating
a behavioural path (Bakeman, 1997; Kucuk & Sisman, 2017).
To observe the sequence of student behaviours more intuitively, we employed data visualisation
techniques to represent the results of latent semantic analysis (LSA) in the form of a linear network. In
this network, circles symbolise behaviour, numbers denote the frequency of occurrence, and
numbers on connecting lines represent the residual value of the transitional relationship. The
larger the value is, the stronger its significance is. The direction of the connecting line indicates
the direction of the behavioural transition. Figure 2 illustrates the transformation network of
Group A’s learning behaviours.
As depicted in Figure 2, the behavioural transitions of learners exhibited several distinct
characteristics.

(1) Independent innovation: The sequence of actions is as follows: ANQ → RF → CA. The ratio of
ANQ to RF is 8.21, whereas the ratio of RF to CA is 17.78, indicating that learners frequently
pose innovative questions on the ChatGPT platform. They subsequently generate unique
answers by interpreting the feedback provided by ChatGPT after processing it through their cog­
nitive abilities. This suggests that, to a certain extent, learners depend on ChatGPT for taking
tests, primarily by utilising it as an information source rather than an answer source. They
invest significant time in extracting information from ChatGPT that they deem useful, continu­
ously writing answers to validate their own ideas, ultimately formulating their responses.
Throughout this process, they identify problems and solve them via ChatGPT, thereby reinfor­
cing the knowledge points they recognise.
(2) General mode: The sequence of actions is as follows: ARQ → RF → CA. The ratio of ARQ to RF is
10.4, whereas the ratio of RF to CA is 17.78. This suggests that learners’ behavioural patterns pre­
dominantly involve a significant amount of time spent asking ChatGPT-specific questions about
topics that are relatively narrow in scope. Through multiple rounds of question‒answer
exchanges on these topics, students may obtain more comprehensive answers. However,
when confronted with ChatGPT’s inability to provide accurate responses or misinformation, stu­
dents struggle to make accurate judgements. Instead, they tend to simply copy the information
provided by ChatGPT and output the results after processing them through their existing cog­
nitive processes.
(3) Lacking innovation: The sequence of actions is as follows: COQ → CPA. The ratio of COQ to CPA is
5.6. The study reveals a pattern of innovation deficiency characterised by learners’ tendency
towards CPA after obtaining output feedback from ChatGPT. Learners’ actions indicate a prefer­
ence for copying the original question rather than engaging in the cognitive process of inter­
preting the feedback. Consequently, they tend to paste the answer directly into the answer
box without integrating the tool’s responses with their existing cognitive knowledge. This over­
reliance on ChatGPT as a sole solution may lead to increased errors or even more complex pro­
blems as it does not leverage the student’s existing knowledge points.
Table 6. The frequency statistics of learner behaviour.
Categorisation Question type Frequency Percentage Categorisation Question type Frequency Percentage
Commencement UT 39 3.4% 8.9% ChatGPT-assisted Question and answer ANQ 119 10.4% 46.4%
RAM 63 5.5% ARQ 122 10.7%
Compiling answers CA 207 18.2% 42.1% COQ 52 4.6%
UTA 77 6.8% RF 183 16.1%
CTS 117 10.3% CPA 54 4.7%
RCM 77 6.8% Close SO 30 2.6% 2.6%
INTERACTIVE LEARNING ENVIRONMENTS
11
12 S. HE AND Y. LU

4.2.3. Analysis of learning behaviour in differently performing groups

To elucidate the intrinsic relationship between students’ utilisation of ChatGPT for question-answer­
ing assistance and their behavioural patterns, we derived descriptive statistics on learning beha­
viours (Table 7) and visualised the network of behavioural patterns (Figure 3) for Group A, which
had been previously categorised based on performance (Table 8).
Based on the table showing descriptive statistics—which presents the learning behaviours of stu­
dents across varying levels of performance—several significant features emerge.
The mean frequency of behavioural transitions diminishes progressively from the high-perform­
ing cohort to the low-performing cohort, with the former Group exhibiting approximately 12 more
conversions per individual than the latter. These findings suggest that the frequency of interaction
with ChatGPT throughout the instructional process plays a pivotal role in determining the ultimate
learning outcomes (Figure 4).
The distribution disparities of three inquiry modes (ANQ, ARQ, and COQ) are notably pronounced
among Groups with different levels of performance when ChatGPT is utilised for task completion.
Specifically, the percentages of ANQ occurrence in the high, medium, and low Groups are 16%,
10%, and 2%, respectively. For ARQ, these percentages stand at 11%, 12%, and 9%, respectively.
In the case of COQ, the respective frequencies are 1%, 4%, and 11%, respectively. This implies
that high-performing Groups exhibit a greater propensity to pose questions to ChatGPT about
their desired information, integrating it with their internal knowledge to formulate answers.
Medium-performing Groups tend to query ChatGPT about a topic, merging this with their personal
knowledge to generate final responses. Low-performing Groups predominantly copy ChatGPT’s
original queries to receive feedback from ChatGPT, bypassing the crucial step of amalgamating it
with their cognitive knowledge and directly copying the response. The frequency percentages of
CPA also indirectly corroborate these conclusions. The occurrence of CPA is observed at frequencies
of 2%, 5%, and 9% in the high, medium, and low Groups, respectively.
When we examine the network diagram depicting students’ learning behaviours across varying
levels of performance, several distinct features emerge.

Figure 3. Network diagram of the learning behaviours of Group A.

Table 7. Residuals for behavioural data in A.
Given. UT RAM CA UTA CTS RCM ANQ ARQ COQ RF CPA SO
UT −0.61 12.54*** −0.96 −1.73 −1.65 −1.73 1.49 −0.65 0.91 −2.82 −1.44 −1.06
RAM −0.78 −2 1.4 1.35 −0.27 −2.23 1.36 2.56* 3.1* −2.93 −1.24 −1.36
CA 3.36* 1.73 −7.48 8.35** 10.3*** −1.34 1.17 −1.64 −3.18 −3.99 −3.26 1.61
UTA −0.87 −1.21 0.18 −2.48 7.65 0.31 −0.48 −0.15 −0.34 −2.77 −0.96 −0.06
CTS −1.09 −2.38 −3.73 −1.97 −3.92 14.58*** 1.1 2.28* −1.15 −4.02 −2.58 8.35**
RCM 0.38 −0.19 −3.15 −1.09 −2.74 −2.48 6.02** 2.89* 4.14* −0.85 −1.51 −0.06
ANQ −1.1 −0.73 −2.56 −3.15 −3.64 −0.86 −3.06 4.68* −0.26 8.21** 1 −1.92
ARQ −1.11 −0.78 −2.38 −1.66 −3.06 −3.18 −1.23 −3.46 1.98* 10.4*** 4.08* −1.94
COQ −0.7 −1.2 −2.09 −2.02 −2.07 −2.02 −1.64 −0.76 −1.64 7.83** 5.6** −1.23
RF 0.3 −2.23 17.78*** −0.85 −3.76 −2.77 −2.77 −2.84 −1.37 −6.57 2.3* −2.47
CPA −0.72 0.57 0.32 2.34* 1.05 0.69 −0.35 −1.74 0.31 −0.34 −1.7 −1.26
SO 10.5*** −0.25 −0.48 −0.27 −0.34 −0.27 −0.35 −0.35 −0.22 −0.44 −0.23 −0.17
Note: * indicates a residual value of 1.96–5, ** indicates a residual value of 5–10, and *** indicates a residual value of >10.
INTERACTIVE LEARNING ENVIRONMENTS
13
 14
S. HE AND Y. LU

Table 8. Descriptive statistics of learning behaviour by performance.

Groups Categorisation Question type Frequency Percentage Categorisation Question type Frequency Percentage
High-performing Group Commencement UT 15 4% 10% ChatGPT- assisted question and answer ANQ 61 16% 46%
RAM 25 6% ARQ 42 11%
Compiling answers CA 77 20% 42% COQ 5 1%
UTA 25 6% RF 61 16%
CTS 37 9% CPA 6 2%
RCM 28 7% Close SO 9 2% 2%
(Grand) total 391/capita: 43.4
Mid- performing Group Commencement UT 15 3% 8% ChatGPT-assisted question and answer ANQ 53 10% 47%
RAM 27 5% ARQ 60 12%
Compiling answers CA 97 18% 41% COQ 23 4%
UTA 35 7% RF 91 17%
CTS 56 11% CPA 28 5%
RCM 28 5% Close SO 14 3% 3%
(Grand) total 527/per capita 37.6
Low-performing Group Commencement UT 9 4% 9% ChatGPT-assisted question and answer ANQ 5 2% 45%
RAM 11 5% ARQ 20 9%
Compiling answers CA 33 15% 43% COQ 24 11%
UTA 17 8% RF 31 14%
CTS 24 11% CPA 20 9%
RCM 21 9% Close SO 7 3% 3%
(grand) total 222/capita 31.7
 INTERACTIVE LEARNING ENVIRONMENTS 15

Figure 4. Network diagram of student behaviour by performance.

Focusing on the use of ChatGPT to assist with answering questions, the network of learning beha­
viours in the high-performing Group was most significant along the following two lines. 1. ANQ → RF
→ CA (ANQ → RF: Z = 6.58 RF → CA: Z = 10.24). 2. ARQ → RF → CA (ARQ → RF: Z = 6.97) RF → CA: Z =
10.24. When combined with the characterisation of learning behaviours detailed in Section 4.2.2, we
can infer that the learning behaviour of the high-performing Group exhibits a combination of inde­
pendent innovation and general mode.
Focusing on the use of ChatGPT to assist in answering questions, the network of learning beha­
viours in the mid-performing Group was most significant along the following lines: ARQ → RF → CA
(ARQ → RF: Z = 8.05 RF → CA: Z = 12.35). These findings suggest that the mid-performing Group
adopt a general mode learning pattern. Adherence to this pattern limits performance to a
medium level without any significant enhancement.
Focusing on the use of ChatGPT to assist in answering questions, the network of learning beha­
viours in the low-performing Group was most significant along the following three lines. ARQ → CPA
(ARQ → CPA: Z = 4.96); COQ → CPA (COQ → CPA: Z = 3.55); and COQ → RF → CA (COQ → RF: Z = 5.88
RF→ CA: Z = 7.67). These findings suggest that the learning behaviour pattern of students in the low-
performing Group is lacking innovation, characterised by a reliance on COQ and CPA. However,
according to the conclusion of Experiment 4.1, the use of ChatGPT to help answer questions does
not significantly boost students’ cognitive development. Consequently, this lacking innovation
behavioural pattern leads these students to overly depend on ChatGPT at the knowledge level,
resulting in lower scores.

5. Discussion
5.1. The limited effectiveness of gen AI in facilitating the teaching and learning of the
humanities and social sciences in the Chinese context
When we compared the significance levels of the experimental data from the three student quiz
Groups, we noted that even though Group A students achieved superior results on the test with
ChatGPT’s assistance, there was a significant difference compared with Group B students, who
16 S. HE AND Y. LU

did not utilise ChatGPT. However, this does not conclusively prove that using ChatGPT as an edu­
cational aid can increase students’ cognitive level. Following the experiment conducted on Group
A2, there was no significant difference in learning level between Groups A and B after ChatGPT
was removed, suggesting no substantial difference in knowledge point cognition between these
two Groups. While ChatGPT may not perform optimally, its potential applications in other areas of
humanities warrant further research. ChatGPT can indeed play a valuable role in rapid content
search and knowledge expansion and alleviate teaching pressure (Arndt, 2023; Liu et al., 2023). Edu­
cators should acknowledge the limited auxiliary effect of ChatGPT (Yu, 2024) when using it for huma­
nities teaching, emphasise its feature of knowledge expansion during the teaching process, and
minimise the “cheating” attribute of ChatGPT to instantly generate answers. This reduces the sub­
jective perception that students have of ChatGPT as an output answer and encourages the inte­
gration of other teaching methods to enhance students’ learning outcomes and cognitive level.
Furthermore, this study suggests that the poor learning effects of ChatGPT in humanities subjects
are directly linked to the training parameters of ChatGPT itself (Hughes et al., 2021). Education,
which is a unique field, requires certain standards, such as knowledge accuracy, ideological correct­
ness, and process interpretability (Hettiarachchilagea, 2023). ChatGPT, as a generalised data model,
still has a long way to go in terms of teaching assistance. Researchers must overcome these difficul­
ties, actively explore the new paradigm of integrating Gen AI and education (Bender et al., 2021), and
create a local Gen AI suitable for the Chinese context to better enable Gen AI, empower education,
and promote the digital transformation of education (Fang Haiguang et al., 2024).

5.2. The three behavioural patterns exhibited by students when using gen AI-assisted
learning
By mapping the behavioural network of all students in Group A who experimented with ChatGPT, we
identified three distinct behavioural patterns: independent innovation, general mode, and lacking
innovation. Independent innovation is characterised by an emphasis on the individual’s initiative
when ChatGPT is utilised for learning. In this context, ChatGPT serves as a tool, providing secondary
roles such as knowledge gathering, broadening thinking, and offering inspiration. This behaviour
suggests that learners already possess a certain level of cognitive understanding of relevant knowl­
edge points. During their interaction with ChatGPT, they connect and respond to existing cognition,
autonomously generating subjective answers to address new problems. In this human‒machine
relationship, the role of the “humans” is dominant. The general mode is characterised by using
ChatGPT to aid learning, with both the individual’s initiative and ChatGPT playing a joint role in
the process. Here, ChatGPT functions as an assistant, providing answers to compilations, searching
for knowledge, correcting errors, and other ancillary tasks. This pattern is observed throughout all
stages of answering questions. This type of behaviour indicates a lack of confidence in the learner’s
level of knowledge points and a balanced human‒computer relationship. Lacking innovation is
characterised by the dominance of the tool in its use. Students in this category rely heavily on
the information feedback function of ChatGPT, leading to a situation where “machine thinking”
replaces “student thinking” (Hamdani et al., 2022; Rahman & Watanobe, 2023). Students lack the
ability to understand and master the questions, merely copying them and mechanically pasting
answers without effectively integrating what they have learned into their cognitive process. The
intricacies of Gen AI’s internal mechanism often result in insufficient detail and explanation regard­
ing its generated results and decisions. This lack of clarity poses challenges for students with weaker
foundational knowledge, hindering their comprehension of what Gen AI produces. Consequently,
these students tend to lag behind their peers who engage in independent innovation, exacerbating
educational disparities (Laupichler et al., 2022).
Consequently, to optimise the use of ChatGPT for assisted learning, learners should engage in the
following activities: (1) Initiate an understanding of the points, not merely by copying and pasting
the questions but also by thoroughly reading and considering the logic and intent embedded
 INTERACTIVE LEARNING ENVIRONMENTS 17

within them. (2) Foster active thinking and problem-solving skills, avoiding excessive reliance on
feedback from ChatGPT. Instead, students should attempt to integrate the knowledge they have
gained with that provided by ChatGPT, recognising the secondary processing of pre-existing knowl­
edge through continuous questioning, modification, and compilation (Desaire et al., 2023). (3) Proac­
tively pose new questions when ChatGPT is utilised for exams or regular learning. Given that the
performance of ChatGPT, in terms of information feedback in Chinese, is suboptimal and may
even yield incorrect answers, its stability as a platform remains debatable (Yilmaz, 2023a). Hence, stu­
dents should view ChatGPT as a tool for expanding information rather than as a direct source of
answers. They should actively engage ChatGPT with new questions and verify their content
against class material to enhance knowledge consolidation.

5.3. The intrinsic link between behavioural patterns and learning outcomes during gen
AI-assisted learning
We investigated the intrinsic relationship between ChatGPT-assisted learning behavioural patterns
and learning outcomes by mapping the network of students’ ChatGPT-assisted question-answering
behaviours with varying levels of performance. We found that students with higher grades were
more likely to adopt independent innovation in their learning behaviours. They utilised ChatGPT
by posing subjective questions, skilfully integrating their acquired knowledge with ChatGPT to
achieve higher grades through continuous rounds of knowledge processing. Mid-performing stu­
dents, on the other hand, tended to adopt a general mode of learning behaviour. They posed
ChatGPT questions related to exam topics and meticulously completed tasks outlined by the
teacher. Despite lacking sufficient knowledge, they effectively collaborated with ChatGPT to
obtain answers through repeated questioning. Low-performing students, however, exhibited
lacking innovation. They only knew how to copy the original question into the ChatGPT page and
paste the feedback into the answer page, using ChatGPT solely to obtain answers. Their knowledge
and ChatGPT operated independently and were unable to communicate effectively and integrate.
Experiment 4.2.3 substantiated that the various modalities of ChatGPT-assisted learning influence
the ultimate learning outcomes for most students. Furthermore, educators, who act as facilitators in
students’ educational journeys (Rogers & Jerome Feriberg, 2015), should commence by fostering
positive learning behavioural patterns to establish an intelligent learning environment characterised
by effective human‒computer interaction.
Consequently, educators should encourage students to take the lead in critical thinking. More­
over, problem-solving is pivotal in addressing the lack of innovation, primarily due to a deficiency
in intrinsic self-confidence (Mitchell & McConnell, 2012). The use of ChatGPT as an aid in instructional
activities can foster this confidence. Students should be urged to integrate their existing knowledge
with ChatGPT capabilities on a regular basis To facilitate a deeper understanding and mastery of the
material through iterative questioning, revision, and compilation. Furthermore, it nurtures good
habits of innovative thinking, even when students operate independently with ChatGPT (Kohnke
et al., 2023).
Teachers should prioritise the accumulation of foundational knowledge (Li & Zhang, 2015). Cur­
rently, ChatGPT is not capable of providing comprehensive support in the Chinese context for huma­
nities. Its content accuracy in teaching humanities, completeness of information, and immediacy fall
short of expectations. The most crucial aspect of education remains the preservation and extraction
of knowledge from the brain rather than the innovation or debugging of tools. As such, educators
must continue to focus on teaching the fundamental aspects of their curriculum.
In the current stage, ChatGPT lacks the capacity to serve as a standalone educational Gen AI
model. Its proficiency in education and teaching remains relatively limited, and its detailed
content might not always be entirely accurate. Therefore, it is imperative for educators to provide
immediate feedback and guidance (Wu, 2023). This involves closely observing the content generated
by ChatGPT, directing students’ learning process, and conducting secondary validation of the
18 S. HE AND Y. LU

information provided by ChatGPT (Jalil et al., 2023). This approach is critical to prevent erroneous
knowledge from interfering with the correct cognitive processes that students have already
established.

5.4 Educational implications of the findings

Unlike other researchers who have focused on exploring the external manifestations (scores) of Gen
AI-assisted education, we explored the internal causes of the impact of Gen AI-assisted education on
students’ cognitive levels from the perspective of students’ behaviours. Innovatively combining the
pedagogical hotspot of Gen AI-assisted teaching with constructivist ideas, we produced research
results similar to those of other scholars (Gen AI is less effective in assisting in the humanities in
the Chinese context) by studying students’ learning behavioural networks via ChatGPT. We com­
bined constructivism and Gen AI research, which provides an effective reference for other research­
ers to carry out investigations in this area. On the other hand, in contrast to similar studies, we
included test score statistics of multiple Groups in the process of using LSA for research on
student behavioural networks. For example, through the longitudinal comparison of the test
results of Groups A1, A2, and A3, we initially determined that Group A did not achieve effective
improvement in cognitive level. Most of the other similar experiments compared only the A2 and
A3 Groups; thus, the use of Gen AI can significantly improve students’ cognitive level. As a result,
these studies are somewhat less rigorous. Our study compensates for this limitation through the
A and B Groups for a total of six comparative tests to validate the conclusion that the use of Gen
AI to assist in the learning of the humanities and sciences in the context of the Chinese language
is the true effect of the study.

6. Conclusions
This paper has foundational findings for the three research questions posed. (1) The integration of
Gen AI into humanities and social sciences teaching and learning in the Chinese context fails to sig­
nificantly improve students’ cognitive level and knowledge construction. (2) The results prove that
students’ learning patterns have distinctive characteristics when ChatGPT is used to support learn­
ing, which can be divided into three categories: independent innovation, general mode, and lacking
innovation. (3) This study also shows that different learning patterns are significantly associated with
final outcomes when learning with ChatGPT, as evidenced by the fact that high performers are
biased towards independent innovation, medium performers are biased towards the general mode,
and low performers are biased towards lacking innovation. These findings are also important for
other educators to implement Gen AI-assisted teaching in subsequent educational activities.
First, educators should be aware that the effectiveness of Gen AI in Chinese contexts still needs to
be improved, and domestic educators should focus on how to do a good job of localising Gen AI or
constructing specialised AI for teaching in Chinese. Second, educators should supervise the use of Gen
AI in teaching to prevent students from falling into the vicious circle of overreliance. Finally, Gen AI
should be an important supplement to education and teaching rather than the dominant educational
tool. Teachers should play a leading role in teaching, guiding students to think positively to develop
an independent innovation learning mode. In the absence of Gen AI-assisted learning conditions, stu­
dents can also assume a subjective awareness to construct knowledge and develop good habits of
active thinking. The different learning behaviours of students presented in this study also provide
new ideas for the development of Gen AI technology: a higher level of customised educational AI.
Based on the premeasured behavioural patterns of students, AI technology can choose different edu­
cational ideas. For example, for students with independent innovation, Gen AI can provide more auth­
ority to answer questions to help students broaden their horizons, and for students who tend towards
lacking innovation, Gen AI can weaken the authority to answer questions via the “Socratic question
and answer” method to help students develop the ability to think on their own.
 INTERACTIVE LEARNING ENVIRONMENTS 19

Importantly, this study has several limitations. First, owing to technical limitations, with the
student learning behaviour network of Gen AI-assisted instruction produced in this study, we stat­
istically analysed student behaviours only when students used ChatGPT to take quizzes in the last
class; this result could not reveal the behavioural patterns of the students in Group A when the stu­
dents used ChatGPT-assisted learning in the first fifteen classes. Second, the sample size was not
large, and only a small class of 30 students was studied; thus, the experimental population and
results lack breadth. Our team is informative about teaching and learning in small classes of approxi­
mately 30 students, and we subsequently sought to increase the sample size to investigate whether
the number of students in the class led to significant changes in the effectiveness of AI-assisted
teaching. Third, the Gen AI model is relatively singular; only the ChatGPT platform has been
studied, and there is no guarantee that other Gen AI tools will cause differences in the results.
The above shortcomings will be continuously and gradually improved in subsequent research. More­
over, we expect that Gen AI—as the most powerful tool expected to change human society—will
eventually integrate deeply with education to create a highly digitalised and intelligent education
that everyone aspires to receive (Giordano et al., 2023), and provide a feasible solution to
promote the realisation of educational equality and lifelong education through continuous improve­
ment and development.

Notes
1. LSA: Lag sequential analysis is a method for analysing time series data to identify patterns and dependencies
between events by evaluating the lagged relationships between occurrences.
2. GSEQ: GSEQ is a specialised software for lag sequential analysis that assists researchers in examining and visua­
lising patterns and dependencies in event sequences.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This research was supported by the Research Project on Legal Issues of Digital Privacy from the Perspective of Artificial
Intelligence (Project No: 21YJC820014), funded by the Ministry of Education of China.

Notes on contributors
Shuai He holds a PhD from the China University of Political Science and Law and is employed at the School of Economics
and Management of Beijing University of Posts and Telecommunications, where he teaches education and economics.
His research interests include education economics and policy and digital economy governance
Yu Lu is studying at the School of Humanities of Beijing University of Posts and Telecommunications, where he is cur­
rently pursuing a master’s degree in education. His research interests include digital education and artificial intelli­
gence-assisted education.

ORCID
Shuai He http://orcid.org/0009-0007-0087-7477
Yu Lu http://orcid.org/0009-0000-1243-3389

References
Agustin. (2023). Examining the role of ChatGPT as a learning tool in promoting students’ English language learning
autonomy relevant to Kurikulum Merdeka Belajar Edukasia. Jurnal Pendidikan Dan Pembelajaran, 4(2), 921–934.
https://jurnaledukasia.org/index.php/edukasia/article/view/
20 S. HE AND Y. LU

Akeman, R., & Quera, V. (1995). Analyzing interaction: Sequential analysis with SDIS and GSEQ[M]. Cambridge University
Press.
Arndt. (2023). AI and Education: An investigation into the use of ChatGPT for systems thinking. Cornell University. https://
doi.org/10.48550/arxiv.2307.14206.
Bahrini, A., Khamoshifar, M., Abbasimehr, H., Riggs, R. J., Esmaeili, M., & Majdabadkohne, R. M. (2023). ChatGPT:
Applications, Opportunities, and Threats. 2023 Systems and Information Engineering Design Symposium (SIEDS),
Charlottesville, VA, USA. pp. 274-279. https://doi.org/10.1109/SIEDS58326.2023.10137850.
Baidoo-Anu, D., & Owusu Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding
the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 1–14. https://doi.org/10.
61969/jai.1311271
Bakeman, R. (1997). Observing interaction: An introduction to sequential analysis[M]. Cambridge university press.
Bender, T. G., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be
too big? FAccT, 21, Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 610–
623. https://doi.org/10.1145/3442188.3445922.
Blikstein, P., Worsley, M., Piech, C., Sahami, M., Cooper, S., & Koller, D. (2014). Programming pluralism: Using learning
analytics to detect patterns in the learning of computer programming. Journal of the Learning Sciences, 23(4),
561–599. https://doi.org/10.1080/10508406.2014.954750
Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable para­
digm shift. Asian Journal of Distance Education, 18(1), 198–204. https://doi.org/10.5281/zenodo.7716416
Dan, S., Chengcong, Z., Zuodong, X., & Guangtao, X. (2024). An analysis of college students’ programming learning
behavior based on generative artificial intelligence. Research on E-Chemical Education, (3), 113–120. https://doi.
org/10.13811/j.cnki.eer.2024.03.016.
Desaire, A. E. C., Isom, M., Jarošová, R., & Hua, D. (2023). Distinguishing academic science writing from humans or
ChatGPT with over 99% accuracy using off-the-shelf machine learning tools. Cell Reports Physical Science, 4(6),
Article 101426. https://doi.org/10.1016/j.xcrp.2023.101426
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja,
M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D.,
Carter, L.. … Wright, R. (2023). Opinion paper: “So what if ChatGPTwrote it?” Multidisciplinary perspective sonopport
unities, challenges and implications of generative conversational AI for research, practice and policy. International
Journal of Information Management, 71, 1–63. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Eryilmaz, E., van der Pol, J., Ryan, T., Clark, P. M., & Mary, J. (2013). Enhancing student knowledge acquisition from online
learning conversations. International Journal of Computer-Supported Collaborative Learning, 8(1), 113–144. https://doi.
org/10.1007/s11412-012-9163-y
Fathi, J., Rahimi, M., & Derakhshan, A. (2024). Improving EFL learners’ speaking skills and willingness to communicate via
artificial intelligence-mediated interactions. System, 121(2024), 103254. https://doi.org/10.1016/j.system.2024.
103254
Giordano, V., Spada, I., & Chiarello, F. (2023). The impact of ChatGPT on human skills: A quantitative study on twitter
data. Technological Forecasting and Social Change, 203, 123389. ISSN 0040-1625. https://doi.org/10.1016/j.techfore.
2024.123389
Haiguang, F., Lili, S., Xianchuang, W., & Xin, H. (2024). Possibilities and feasibility of digital transformation of education in
the era of generative artificial intelligence: Reflections on Sora. Journal of the National Academy of Education
Administration, (4), 69–75.
Hamdani, D. I. S., Marpaung, A. P., Gonggo, R. A., & Sulistyanti, U. (2022). Investigating the students’ behavior towards
the temptation to do academic misconduct in higher education: The moderation of religiosity. Journal of
Contemporary Accounting, 4(1), 10–22. https://doi.org/10.20885/jca.vol4.iss1.art2
Hettiarachchilagea, N. H. (2023). Effective model with personalized online teaching and learning science in the Era of
ChatGPT. Cornell University. https://doi.org/10.48550/arXiv.2306.09545
Hou, H. T. (2012). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing
game (MMORPG). Computers & Education, 58(4), 1225–1233. https://doi.org/10.1016/j.compedu.2011.11.015
Hughes, R. T., Zhu, L., & Bednarz, T. (2021). Generative adversarial networks–Enabled Human–Artificial Intelligence col­
laborative applications for creative and design industries: A systematic review of current approaches and trends.
Frontiers in Artificial Intelligence, 4, 604234. https://doi.org/10.3389/frai.2021.604234
Jalil, S. R., Latoza, T. D., Moran, K., & Lam, W. (2023). ChatGPT and software testing education: Promises & Perils. 2023 IEEE
International Conference on Software Testing, Verification and Validation Workshops ICSTW), pp. 4130–4137. https://
doi.org/10.1109/icstw58534.2023.00078.
Jamaludin, A., Chee, Y. S., & Ho, C. M. L. (2009). Fostering argumentative knowledge construction through enactive role
play in Second Life. Computers & Education, 53(2), 317–329. https://doi.org/10.1016/j.compedu.2009.02.009
Jiang, C., & Duan, Y. (2023). A preliminary study on the reform of computer practice classes in the context of ChatGPT.
Experimental Technology and Management, 12, 1–7 + 23. https://doi.org/10.16791/j.cnki.sjg.2023.12.001.
 INTERACTIVE LEARNING ENVIRONMENTS 21

Karabacak, B., Yildirim, S. O., & Baykal, N. (2016). A vulnerability-driven cyber security maturity model for measuring
national critical infrastructure protection preparedness. International Journal of Critical Infrastructure Protection, 15,
47–59. https://doi.org/10.1016/j.ijcip.2016.10.001
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550.
https://doi.org/10.1177/00336882231162868
Kucuk, S., & Sisman, B. (2017). Behavioral patterns of elementary students and teachers in one-toone robotics instruc­
tion. Computers & Education, 111, 31–43. https://doi.org/10.1016/j.compedu.2017.04.002
Kuhail, M. A., Mathew, S. S., Khalil, A., Berengueres, J., & Hussain Shah, S. J. (2024). Will I be replaced?” Assessing
ChatGPT’s effect on software development and programmer perceptions of AI tools. Science of Computer
Programming, 235, 103111. https://doi.org/10.1016/j.scico.2024.103111
Lai, C. L., & Hwang, G. J. (2015). A spreadsheet-based visualized Mindtool for improving students’ learning performance
in identifying relationships between numerical variables. Interactive Learning Environments, 23(2), 230–249. https://
doi.org/10.1080/10494820.2014.997247
Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence literacy in higher and adult education: A
scoping literature review. Computer & Education, 3, 100101. https://doi.org/10.1016/j.caeai.2022.100101
Lee, D., Arnold, M., Srivastava, A., Plastow, K., Strelan, P., Ploeckl, F., Lekkas, D., & Palmer, E. (2024). The impact of gen­
erative AI on higher education learning and teaching: A study of educators’ perspectives. Computers and Education:
Artificial Intelligence, 6, 100221. https://doi.org/10.1016/j.caeai.2024.100221
Li, H., & Zhang, J. (2015). The ARS interactive teaching model and its application in the smart classroom. China
Educational Technology, 22(11), 103–109. https://doi.org/10.3969/j.issn.1006-9860.2015.11.015
Liping, S., et al. (2024). An analysis of the application of big language modeling in secondary school history. Modern
Educational Technology, 34(2), 62–71.
Liu, J., Shen, L., & Wei, G. W. (2023). ChatGPT for computational topology. Cornell University. https://doi.org/10.48550/
arxiv.2310.07570
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024). Dissociating language
and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.
2024.01.011
Mitchell, A. W., & McConnell, J. R. (2012). A historical review of contemporary educational psychology from 1995 to 2010.
Contemporary Educational Psychology, 37(2), 136–147. https://doi.org/10.1016/j.cedpsych.2011.11.001
Mugruza-Vassallo, C. A., & Suárez, S. M. (2016). Academia and patents at information and communications technology in
South-America productivity. 2016 6th International Conference on Information Communication and Management
(ICICM), Hatfield, UK, pp. 24–29. https://doi.org/10.1109/INFOCOMAN.2016.7784209
Pelletier. (2024). EDUCAUSE Horizon Report 2023, Teaching and Learning Edition. https://library.educause.edu/-/media/
files/library/2023/4/2023hrteachinglearning.pdf?#page = 54
Perlman, A. (2022). The Implications of ChatGPT for Legal Services and Society. Suffolk University Law School Research
Paper. No22-14, https://doi.org/10.2139/ssrn.4294197.
Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies.
Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
Rogers, C., & Jerome Feriberg, H. (2015). Freedom to learn [M] (Wang Yehui Translator). People’s Posts and
Telecommunications Press. 41-42.40.
Sackett, G. P. (1978). Observing behavior: Theory and applications in mental retardation (Vol. 1). University Park Press.
Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 30–32.
Steele, J. L. (2023). To GPT or not GPT? Empowering our students to learn with AI. Computers and Education: Artificial
Intelligence, 5, 100160. http://dx.doi.org/10.1016/j.caeai.2023.100160
Sung, H.-Y., & Hwang, G.-J. (2018). Facilitating efective digital game-based learning behaviors and learning perform­
ances of students based on a collaborative knowledge construction strategy. Interactive Learning Environments, 26
(1), 118–134. https://doi.org/10.1080/10494820.2017.1283334
Van Aalst, J. (2009). Distinguishing knowledge-sharing, nowledge-construction, and knowledge-creation discourses.
Computer Supported Learning, 4(3), 259–287. https://doi.org/10.1007/s11412-009-9069-5
Wang, L. (2023). From evolution to revolution: The legal industry under the influence of ChatGPT-like technologies.
Oriental Law, 04, 56–67. https://doi.org/10.19404/j.cnki.dffx.20230714.009
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-sup­
ported collaborative learning. Computers & Education, 46(1), 71–95. https://doi.org/10.1016/j.compedu.2005.04.003
Wu, Y. (2023). Integrating generative AI in education: How ChatGPT brings challenges for future learning and teaching.
Journal of Advanced Research in Education, 2(4), 6–10. https://doi.org/10.56397/jare.2023.07.02
Xu Dan, D. X. (2024). Artificial intelligence literacy: Challenges and responses in higher education - interpretation and
implications of the horizon report 2024: Teaching and learning edition. Open Education Research, 03, 24–36. https://
doi.org/10.13966/j.cnki.kfjyyj.2024.03.003
Yang, T.-C., Chen, S. Y., & Hwang, G.-J. (2015). The influences of a two-tier test strategy on student learning: A lag sequen­
tial analysis approach. Computers & Education, 82, 366–377. https://doi.org/10.1016/j.compedu.2014.11.021
22 S. HE AND Y. LU

Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023a). Augmented intelligence in programming learning: Examining student
views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans, 1(2),
100005. https://doi.org/10.1016/j.chbah.2023.100005
Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023b). The effect of generative artificial intelligence (AI)-based tool use on stu­
dents’ computational thinking skills, programming self-efficacy and motivation. Computers and Education:
Artificial Intelligence, 4, 100147. https://doi.org/10.1016/j.caeai.2023.100147
Yılmaz, R., & Yılmaz, F. G. K. (2022). Examining student satisfaction with the use of smart mooc International I? Stanbul
Scientific Research Congress, Istanbul, pp. 264–269. http://hdl.handle.net/11772/6929
Yu, H. (2024). The application and challenges of ChatGPT in educational transformation: New demands for teachers’
roles. Heliyon, 10(2), e24289. ISSN 2405-8440. https://doi.org/10.1016/j.heliyon.2024.e24289
Zekaj, R. (2023). AI language models as educational allies: Enhancing Instructional support in higher education.
International Journal of Learning, Teaching and Educational Research, 22(8), 120–134. https://doi.org/10.26803/
ijlter.22.8.7
Zhou, H. (2023). A study on the auxiliary application of ChatGPT in university English vocabulary teaching. Journal of
Social Sciences of Jiamusi University, 41(6), 192–194.