Respectus Philologicus eISSN 2335-2388
2025, no. 47 (52), pp. 50–65 DOI: http://dx.doi.org/10.15388/RESPECTUS.2025.47.4

ChatGPT in Foreign Language Acquisition: a close ally or a distant concept for Macedonian university students?

Silvana Neshkovska
St. Kliment Ohridski University – Bitola, Faculty of Education
Vasko Karangelevski, b.b. 7000 Bitola, Republic of North Macedonia
Email: silvana.neskovska@uklo.edu.mk
ORCID iD: https://orcid.org/0000-0003-4417-7783
Research interests: Pragmatics, Second Language Acquisition, Translation

Lela Ivanovska
St. Kliment Ohridski University – Bitola
Faculty of Information and Communication Technologies
Partizanska b.b. 7000 Bitola, Republic of North Macedonia
Email: lela.andreevska@uklo.edu.mk
ORCID iD: https://orcid.org/0009-0009-3915-334X
Research interests: Methodology of Teaching English as a Foreign Language, Teaching English for Specific Purposes

Vesna Trajkovska
St. Kliment Ohridski University – Bitola, Faculty of Security
1 Kolonija – Idrizovo, 2A, 1000 Skopje, Republic of North Macedonia
Email: vesna.trajkovska@uklo.edu.mk
ORCID iD: https://orcid.org/0009-0007-3114-3265
Research interests: English for Law Enforcement, Law Enforcement Terminology

Milena Kasaposka-Chadlovska
St. Cyril and Methodius University of Skopje, “Blaze Koneski” Faculty of Philology
Goce Delcev, 9A, 1000, Skopje, Republic of North Macedonia
Email: milena.kasaposka@flf.ukim.edu.mk
ORCID iD: https://orcid.org/0000-0003-1617-3750
Research interests: Grammar, Didactics, Interpretation

Abstract. ChatGPT has instigated a tectonic change in foreign language acquisition. The latest research shows that language learners use it to enhance their major language skills and boost their critical thinking, research activities, and exam preparation. Consequently, ChatGPT has significantly impacted all levels of education, particularly the tertiary level. This paper aims to showcase how Macedonian university students fare in employing ChatGPT for foreign language acquisition. The study examines whether they are sufficiently familiar with this chatbot and to what extent they use it in various language-learning contexts. It also explores students’ attitudes toward this novel tool for improving their language abilities. Insights are drawn from qualitative and quantitative analyses of data gathered via a tailor-made survey, which targeted Macedonian university students, the majority of whom took a foreign language course during the academic year 2023/2024. Even though only 180 university students from several higher education institutions in the RN Macedonia showed interest in the study, the findings reveal mixed student perceptions of ChatGPT’s effectiveness, with some recognising its benefits for writing, translation, grammar, and research, while others expressing scepticism, particularly regarding its role in generating high-quality writing ideas, improving speaking skills, and fostering critical thinking, and creativity.

Keywords: ChatGPT; AI; foreign language acquisition; higher education; Macedonia; students.

Submitted 28 August 2024/ Accepted 6 December 2024
Įteikta 2024 08 28 / Priimta 2024 12 06
Copyright © 2025 Silvana Neshkovska, Lela Ivanovska, Vesna Trajkovska, Milena Kasaposka-Chadlovska. Published by Vilnius University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution License CC BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium provided the original author and source are credited.

Introduction

Nowadays, AI is becoming increasingly prevalent across all sectors, including education. The application of special AI-based tools called chatbots has proven very conducive, particularly in second language acquisition. Chatbots are generally defined as “a software program that interacts with users using natural language” (Abu Shawar, Atwell, 2007, p. 29). In other words, a chatbot is an AI system that can simulate a human conversation since its architecture integrates a language model and computational algorithms to emulate informal chat communication between a human user and a computer using natural language (Abu Shawar, Atwell, ibid.). Chatbots, such as Bing AI, CoPilot, Open AI Playground, Bard AI, You Chat, ChatSonic, etc., offer many advantages to language learners. They are new and interesting; students can use them anytime and anywhere. Chatbots provide quick and effective feedback for students’ spelling and grammar; many bots provide both text and synthesised speech, allowing students to practice both listening and reading skills; students tend to feel more relaxed talking to a chatbot than to a teacher, etc. (Haristiani, 2019). Since its release in November 2022, ChatGPT has become one of the most widely used chatbots (Stepanechko, Kozub, 2023), causing a surge of interest, especially in higher education (Hu, 2023). ChatGPT describes itself as “a powerful machine learning software that uses the Generative Pre-trained Transformer (GPT) algorithm to generate human-like responses to text-based inputs”1. ChatGPT has become one of the most preferred AI tools in education because it can perform a broad range of tasks, ranging from question answering to text summarisation, creative writing (poetry or fiction), reacting to conversational prompts, elucidating difficult subjects, concepts, or themes, translating text from one language to another, correcting mistakes etc. (Eke, 2023). Its most recent version, ChatGPT-4, is highly potent as it can carry out even more complicated tasks than its predecessors by accepting image and text inputs (Ray, 2023). Given its vast potential, ChatGPT can be employed as an invaluable assistant for instructors – helping them develop course materials – and as a virtual tutor for students, aiding in answering their questions and fostering teamwork (Farhi et al., 2023). Nevertheless, this AI-driven tool has had a notably divisive effect in higher education. Many universities have already banned it over fears of student plagiarism and cheating in exams (Sabzalieva, Valentini, 2023, p. 10).

Drawing on many recent studies which examine university students’ perspectives on the use of ChatGPT in higher education across different countries, this paper aims to disclose Macedonian university students’ experiences with and attitudes toward ChatGPT in the context of foreign language acquisition. To investigate whether ChatGPT has already become their close ally or if they are still acclimating to using this tool in learning a foreign language, an online tailor-made survey was administered in a number of public and private universities in Macedonia. The upcoming sections provide an overview of previous studies, the research methodology, and the results obtained from the study. The final section highlights some conclusions and offers recommendations about harnessing the full potential of ChatGPT in foreign language acquisition, as well as about future studies.

1. Theoretical background

In recent years, there has been a rapid surge of studies exploring the application of AI tools in education, with a particular focus on ChatGPT. Some researchers have dealt with the benefits and limitations of using AI tools in education. The most frequently mentioned benefits include creating learning assessments, enhancing pedagogical practice, offering virtual personal tutoring, creating an outline, and brainstorming ideas. As to the limitations, scholars primarily link them to academic integrity issues, unfair learning assessment, inaccurate information, and over-reliance on AI (Sok, Heng, 2023).

Other researchers focus on students’ propensity to use these novel AI tools. Malmström et al. (2023) conducted an extensive survey across Swedish universities about students’ use of ChatGPT and other AI language tools. Overall, the majority of students show a positive attitude, claiming that AI makes them more effective learners. Interestingly, only one-third state that they use ChatGPT regularly. Also, although more than half of them characterise the use of chatbots during examinations as cheating, they are decidedly against the prohibition of AI in educational settings.

A similar study which targets the views and concerns of students in the United Arab Emirates was carried out by Farhi et al. (2023). This study shows that students consider ChatGPT a revolutionary technology that helps them in many ways. They also demonstrate awareness that the use of ChatGPT entails some concerning effects on educational integrity. Hence, Farhi et al. (2023) encourage teachers and policymakers to keep the right balance between leveraging AI technology to improve education and upholding ethical practices that promote critical thinking, originality, and integrity among students.

Stepanechko and Kozub (2023) focus on the challenges language teachers face due to the advent of ChatGPT. Given that they often task students with reading and writing assignments, which ChatGPT can complete in seconds, teachers are concerned that students will not develop their critical thinking, analytical abilities, and writing skills. Consequently, Stepanechko and Kozub (2023, p. 301) recommend that teachers check the authenticity of students’ work via debates, discussions, case studies, and inquiry-based tasks. They, too, conclude that it is crucial to strike a balance between harnessing the abilities of ChatGPT to improve students’ ledge and minimising its potential negative consequences (Stepanechko, Kozub, ibid.).

Some studies focus on the impact of ChatGPT on the development of specific language learners’ skills. Thus, for instance, Song and Song’s (2023) study evaluates the impact of AI-assisted language learning on 50 Chinese EFL students’ writing skills and motivation. The quantitative analysis of their data reveals that the students in the experimental group (those who received ChatGPT-based instruction) improved their motivation to learn the foreign language and showed more significant improvements in their writing tasks than those in the control group (these students received only traditional instructions).

Karataş et al. (2024) explored the effects of utilising ChatGPT on the overall foreign language acquisition process. Their study involved interviews with 13 Turkish university students who have engaged with ChatGPT. Their responses show that ChatGPT positively affects students’ learning experiences, particularly their writing, grammar, and vocabulary. It enhances students’ motivation and engagement and fosters creative thinking skills during classroom activities. However, Karataş et al. (2024) also acknowledge students’ concerns about the potential over-reliance on ChatGPT, which could hinder the development of critical thinking and learner autonomy.

This brief overview of the current literature demonstrates that teachers and students worldwide are aware of both the potential and risks of AI in language education. It also validates the relevance of the topic and underscores the importance of pursuing it further.

2. Research methodology

The aim of this paper was to investigate how Macedonian university students fare when it comes to the use of ChatGPT in the context of learning foreign languages. Put differently, we wanted to explore if students have already figured out how, when, and to what extent they can make use of ChatGPT to enhance their foreign language learning experience. Thus, via a tailor-made survey, firstly, we wanted to test whether students prefer using ChatGPT in learning a foreign language over other AI-driven tools (chatbots), and, secondly, we wanted to investigate exactly how they use it in this context.

The survey was designed using Google Forms and partly drew on previous similar surveys conducted in other countries (Farhi et al., 2023; Malmström et al., 2023). It was composed of four sections in which the informants were asked to state: 1) their age, gender, university; study program, chosen foreign language, the period when they had attended a foreign language course (Section 1); 2) their familiarity with and frequency of use of different AI-driven chatbots, including ChatGPT (Section 2); 3) familiarity with and frequency of use of ChatGPT in a variety of language learning contexts to boost some essential language skills such as writing, speaking, grammar, vocabulary and translation as well as other skills such as critical thinking, doing research, curiosity and preparing for exams (Section 3), and 4) their attitude towards the use of ChatGPT in foreign language acquisition (Section 4).

Section 2 featured statements to which students’ possible reactions were: I am familiar and regularly use it; I am familiar but occasionally use it; I am familiar but never use it, and I am not familiar with it, whereas Section 3 featured statements to which students could react by using the following options: Yes; No; I don’t know, and I prefer not to respond. Section 4 contained statements and open-ended questions intended to disclose students’ attitudes.

The survey was distributed to different state and private higher education institutions2 and was open for students to fill in from 20 April to 10 May 2024. Before filling in the survey, they were informed that their answers would be analysed anonymously and treated ethically and merely for the purposes of this scientific endeavour.

A mixed methodology was used to analyse the data obtained, incorporating both qualitative and quantitative approaches. The research questions we sought to answer in this study were as follows:

RQ1: What is ChatGPT’s rate of use and popularity compared to other chatbots in the context of foreign language acquisition among Macedonian university students?

RQ2: What are the most and least preferred ways for Macedonian university students to use ChatGPT in enhancing essential language skills, including writing, speaking, grammar, vocabulary, and translation?

RQ3: Do Macedonian university students find ChatGPT helpful in developing additional skills such as critical thinking, curiosity, research, and exam preparation?

RQ4: What is the attitude of Macedonian university students towards implementing ChatGPT in foreign language acquisition?

In the following section, we address the research questions by discussing the results obtained from this study.

3. Results

A) Section 1 – Respondents’ Demographic Data

In the designated 20-day period, 180 university students from 12 state and private higher education institutions in RN Macedonia filled in our online survey.

Table 1. Respondents’ demographic data

Information about the respondents		Number of students (out of 180 in total)	Percentage
Age	18-30	171	95%
	over 31	9	5%
Gender	Male	57	31.7%
	Female	123	68.3%
Level of tertiary education	first cycle studies	177	98.3%
	second cycle studies	2	1.1%
	third cycle studies	1	0.6%
Period of foreign language instruction	academic year 2023-2024	166	92.2%
	prior to the academic year 2023-2024	14	7.8%
Foreign language	English	164	91%
	French	10	5.6%
	German	4	2.2%
	Other language3	2	1.1%
State and private higher education institutions	Higher Medical School – Bitola	40	22.2%
	Faculty of Security – Skopje	32	17.8%
	Faculty of Education – Bitola	31	17.2%
	Faculty of Information and Communication Technologies – Bitola	26	14.4%
	Faculty of Economics – Prilep	9	5%
	Faculty of Technical Sciences – Bitola	9	5%
	Faculty of Tourism and Hospitality – Ohrid	1	0.6%
	Faculty of Biotechnical Sciences – Bitola	1	0.6%
	Faculty of Philology – Skopje	14	7.8%
	Faculty of Philology – Shtip	4	2.2%
	Business Academy Smilevski – Skopje	9	5%
	University of Skopje	4	2.2%

As shown in Table 1, most respondents were first-year students (98.3%), aged 18–30 (95%). A vast majority of them were female individuals (68.3%). Almost all of the respondents had a foreign language as part of their curriculum in the academic year 2023/2024 (92.2%), and many listed English as their chosen foreign language (91%). The gathered data were generated by respondents from 12 higher education units from 2 private higher education institutions (1 university and one academy) and three state universities. However, the majority of the respondents were from the second oldest state university in Macedonia – University St. Kliment Ohridski – Bitola (Higher Medical School – Bitola (22.2%); Faculty of Information and Communication Technologies – Bitola (14.4%); Faculty of Education – Bitola (17.2%); and Faculty of Security – Skopje (17.8%)).

B) Section 2 – Familiarity with and Frequency of Use of AI chatbots

The second section of the survey presented students with ten statements about a number of chatbots (ChatGPT, Bing AI, CoPilot, Open AI playground, Bard AI, You Chat, ChatSonic, DialoGPT, Socratic, and Jasper Chat). These statements were adapted from a similar, much broader survey on AI and chatbots conducted among university students in Sweden, authored by Malmström et al. (2023). The students were instructed to state whether they are familiar with these AI chatbots and how often they use them when learning foreign languages. This section aimed to answer the first research question (RQ1. What is ChatGPT’s rate of use and popularity compared to other chatbots in the context of foreign language acquisition among Macedonian university students?).

As shown in Table 2 below, ChatGPT is the most popular student chatbot. More precisely, 84.4% of the respondents professed familiarity with it, whereas 15.6% stated they were not familiar with it. This result starkly contrasts with the very high percentage of unfamiliarity students show with the rest of the chatbots, ranging from 84.4% in the case of Bing AI up to 89.4% in the case of Jasper Chat (Table 2). This result is completely in line with Malmström et al.’s (2023) results, according to which “ChatGPT stands out from the rest of the chatbots, with an overwhelming 95% of respondents claiming to be familiar with it” (p. 5).

Overall, these results point to a serious lack of familiarity with chatbots on the part of Macedonian students (except for ChatGPT). This, in turn, suggests that they are also very much unacquainted with the merits of these tools in foreign language acquisition.

Table 2. Familiarity with and frequency of use of chatbots

	Not familiar with this chatbot	Familiar with this chatbot
		regular use	occasional use	rare use	Total
ChatGPT	15.6%	18.9%	20.6%	44.9%	84.4%
Bing AI	74.4%	2.8%	2.2%	20.6%	25.6%
CoPilot	76.7%	1.7%	3.9%	17.7%	23.3%
Open AI playground	75%	1.7%	3.3%	20%	25%
Bard AI	81.7%	2.2%	1.7%	14.4%	18.3%
You Chat	76.1%	1.7%	3.3%	18.9%	23.9%
ChatSonic	82.8%	1.1%	1.1%	15%	17.2%
DialoGPT	85.6%	1%	1.7%	11.7%	14.4%
Socratic	87.2%	1.1%	1.1%	10.6%	12.8%
Jasper Chat	89.4%	1.1%	0.6%	8.9%	10.6%

Even though ChatGPT has established its primacy over the rest of the AI-driven tools, a very interesting discrepancy can still be spotted in the frequency of its use among those who claim familiarity with it, namely, of all the students who know and use this AI tool, only 18.9% state that they resort to it regularly, which is a very unsatisfactory percentage. The rest profess only occasional (20.6%) or very infrequent usage (44.9%) (Figure 1). Again, the results are similar to Malmström et al.’s (2023) finding that only over a third (35%) of the students who professed to use ChatGPT use it regularly.

Fig. 1. Ss’ responses regarding their familiarity with and frequency of use of ChatGPT

This finding suggests that although the students have experienced the benefits of using this chatbot in certain language learning contexts, they still have not reached a stage to harness its full potential. Likely, they are inadequately acquainted with its diverse modes of application in foreign language learning.

As to the other AI tools mentioned in the survey, it is worth pointing out the low percentage of familiarity, which ranged from 10.6% in the case of Jasper Chat (lowest) to 25.6% in the case of Bing AI (highest), is compounded even further by the high percentage of infrequent usage on the part of those who claim familiarity with these chatbots (Table 2). Malmström et al. (2023) also note that the Swedish students were unfamiliar with most other chatbots except for Bing, Bard, and OpenAI playground.

C) Section 3 – Students’ Use of ChatGPT in Different Foreign Language Learning Contexts

The third section of the survey contains 18 statements related to the students’ experiences with ChatGPT. Namely, the first 10 statements refer to the assistance students receive from ChatGPT in acquiring specific language skills. These statements were partly adapted from Malmström et al. (2023) and Farhi et al.’s (2023) surveys on AI and chatbots. With these statements, the students were prompted to indicate whether ChatGPT helped them with their speaking, writing assignments, translation tasks, grammar explanations, and acquisition of new words and expressions. With these statements, the intention was to respond to the second research question (RQ2. What are the most and least preferred ways for Macedonian university students to use ChatGPT to enhance essential language skills, including writing, speaking, grammar, vocabulary, and translation?).

Thus, with respect to the statement that students rely on ChatGPT when writing articles, essays, and poetry, most of the students answered negatively (Figure 2). However, a significant percentage confirmed that this AI-driven tool has helped them write articles (26.7%) and essays (33.3%). Interestingly, some respondents chose the I don’t know or prefer not to say option. These results might suggest that students don’t feel comfortable openly admitting that they turn to ChatGPT for help when they do their writing assignments or are possibly ashamed to admit that they do not keep pace with these latest technologies.

Fig. 2. Ss’ responses regarding whether ChatGPT has helped them in composing written materials (articles, essays, poetry)

The tendency not to use the full potential of ChatGPT in writing assignments can also be noted in the fact that 52.2% disagree that ChatGPT gives them the best possible ideas for writing. In contrast, about the same percentage of students as the percentage of those who professed using ChatGPT for writing essays and articles believe that ChatGPT provides them with the best writing ideas (26.1%) (Figure 3). This finding suggests that those who already used ChatGPT have seen its benefits first-hand and trust the tool to help them generate high-quality ideas. Conversely, in Farhi et al.’s (2023) study, the majority of United Arab Emirates students consider ChatGPT as an advanced writing model that encourages their productivity, offers practical guidance and fosters their creative expression.

Fig. 3. Ss’ responses regarding whether ChatGPT has provided them with the best possible writing ideas

Interestingly, the number of respondents who believe that ChatGPT gives them support and feedback when writing is significantly higher (38.9%) compared to the previous answers that referred to using ChatGPT for their writing assignments. That could mean some students do not equate “getting feedback” from ChatGPT with “getting assistance” in writing assignments (Figure 4). Most probably, “getting feedback” is a concept student mainly associated with the aspects of writing related to vocabulary, grammar, punctuation and spelling, whereas “getting assistance” is primarily linked to obtaining ideas or arguments about their topic.

Fig. 4. Ss’ responses to whether ChatGPT has helped them with feedback in doing writing assignments

The percentage of respondents who claim that ChatGPT has not helped them translate texts is relatively high (60%). Within the remaining 40% of the respondents, only 27.2% acknowledge having received help from this AI tool during translation assignments; 7.8% chose the I don’t know option, and 5% of respondents chose not to answer this question (see Fig. 5). The students have not used ChatGPT to translate texts, possibly, because they may not have been aware of the tool’s translation feature or they may have been doubtful about the quality of the translations. The latter reason was emphasised in Klimova et al.’s (2024) study, in which Czech English language majors qualified the translated output generated by ChatGPT as unnatural, full of mistakes and difficult to understand. Contrary to these findings, Farhi et al.’s (2023) study reveals a very positive attitude of UAE students towards ChatGPT’s translating capabilities.

Fig. 5. Ss’ responses to whether ChatGPT has helped them to translate texts

Fig. 6. Ss’ responses regarding their use of ChatGPT to help them with grammar, vocabulary and speaking

When asked whether ChatGPT has helped them with difficult grammatical content, most respondents (56.1%) answered negatively. However, the number of students who admit that ChatGPT has assisted them with grammar is quite notable, too (35%). A significant percentage of the respondents (8.9%) did not answer this question. This result is very similar to the result obtained regarding the following statement, which targeted vocabulary. It was intended to investigate whether students find ChatGPT helpful when they need explanations for words and expressions they are unfamiliar with. Namely, 41.7% of their answers were negative, 36.1% were positive, 13.3% selected the I don’t know option, and 8.9% chose the I prefer not to say option. As to speaking, more than half of the respondents (61.1%) do not agree with the premise that ChatGPT has helped them to improve their speaking skills. Surprisingly, only 18.9% admit to this claim (Figure 6). In Klimova et al.’s (2024) study, Czech students assessed ChatGPT’s usage very highly with respect to resolving their language dilemmas related to grammar and vocabulary. According to Klimova et al. (2024, p.8), although ChatGPT primarily operates in written format, its conversational capabilities can still provide opportunities for speaking practice, and students can practice their pronunciation, fluency, and conversational skills.

What is interesting to note here is that in all these contexts – grammar, vocabulary, and speaking, the percentage of the students who avoid answering either by choosing the I don’t know or I prefer not to say the option is not negligible (Figure 6).

This section’s next set of statements refers to the respondents’ perceptions of ChatGPT’s role in ameliorating their ability to think critically, research, be curious and prepare better for exams. With this set of statements, we attempted to answer the third research question (RQ3. Do Macedonian university students find ChatGPT helpful in developing additional skills such as critical thinking, curiosity, doing research and exam preparation?).

Fig. 7. Ss’ responses regarding the role of ChatGPT in enhancing their other abilities

Students’ viewpoints regarding the role of ChatGPT in ameliorating their critical thinking ability, their ability to prepare well for exams, and their curiosity about exploring new topics are not very favourable, either. The students do not profess high reliance on ChatGPT in such contexts. On the contrary, the percentage of those who do not find ChatGPT very helpful significantly surpasses those who do (Figure 7). Only when it comes to doing research is the percentage of students with a favourable attitude towards ChatGPT visibly higher (43.9%) than that of students who claim independence from this tool (33.9%). Again, the percentage of students who avoid voicing their opinion by choosing the I don’t know and I prefer not to say options is significant (Figure 7).

D) Section 4 – Students’ attitude towards the use of ChatGPT in foreign language acquisition

The last section of the survey contained three statements and two open-ended questions aimed at investigating whether the students had a positive or negative attitude towards implementing ChatGPT in foreign language acquisition. Students’ responses in this section were supposed to assist in answering the last research question (RQ4. What is the attitude of Macedonian university students towards the implementation of ChatGPT in foreign language acquisition?).

Fig. 8. Ss’ responses regarding the possible adverse effects of using ChatGPT in foreign language acquisition

Students’ reactions to these three statements unveil their pronounced suspicion regarding the effects that the use of ChatGPT can have on their critical thinking ability, creative writing capability, and the reliability of the data ChatGPT provides for them. As can be seen in Figure 8, the majority support the claim that this AI tool has adverse effects on their critical thinking ability (43.3%) and creative writing capability (45%) and that they do not find the data generated by it reliable and truthful (42.2%). The percentage of those with a positive stance is surpassed not only by those with a negative stance but also by those who do not wish to reveal their opinion and choose either the I don’t know or prefer not to say option. In Farhi et al.’s (2023) study, UAE students, too, express concerns that heavy reliance on ChatGPT could undermine their creative and critical thinking abilities.

In the penultimate open-ended question in this section, the respondents were asked to suggest ChatGPT-based activities that can be implemented in foreign language classes. In the analysed 153 responses, the most frequent suggestions included activities such as searching for ideas and suggestions (for essay topics and projects); translating and checking translations (of texts or isolated terms); mastering grammatical content; writing essays and term papers; making dialogues (using ChatGPT as an interlocutor); creating grammar exercises and exercises for acquiring new terminology; checking answers and spelling; quizzes and games; comparison of generated texts and translations; human and ChatGPT generated text analysis; grading essays, pronunciation exercises, etc. Interestingly, despite the reservations expressed in their previous responses, these answers demonstrate students’ awareness of this tool’s wide range of applications in foreign language acquisition.

The last question in this section, which was also open-ended, was intended to address why students avoid using ChatGPT. From the answers obtained, it can be concluded that most reasons for not using this tool are related to the absence of the need to use it, the uselessness of this tool, and the conviction that independent learning is much more effective and gives more significant results. All these reasons are succinctly subsumed in the following answer provided by one of the respondents: “I believe in real knowledge in a certain area and the effort invested in doing tasks, writing and studying. The main reason is to be independent of tools that can completely do the job for us because the tools are not innovative or thinking “. The opinion that resorting to this tool makes students passive, reduces their creativity, slows down cognitive development and lowers the feeling of satisfaction from the learning process surfaced in a number of answers, too, and is visible in the answer given by one of the respondents: “I don’t use it regularly because I think that the feeling of fulfilment due to the final result after completing a project is greater when we know that we have invested effort and that for me personally is worth a lot more than an essay perfectly put together with the help of AI”.

These reactions appear to align closely with the views of university students from other countries, who have also expressed concerns about the potential negative consequences of over-reliance on ChatGPT for educational tasks (Farhi et al., 2023; Welding, 2023; Malmström et al., 2024).

Conclusion

This study revealed that ChatGPT is the most recognised and used AI tool among Macedonian university students. The familiarity with and utilisation of other chatbots remains notably low. Also, while students recognise ChatGPT’s potential, they have yet to integrate its capabilities into their language learning practice fully. This finding suggests that Macedonian university students are similar to their foreign counterparts.

The results indicate mixed responses regarding students’ perceptions of ChatGPT’s effectiveness in enhancing various language skills. While some students acknowledge its benefits in writing and translation tasks, grammar and vocabulary, others express scepticism, particularly its role in providing high-quality writing ideas and improving their speaking skills. Additionally, students lack confidence in ChatGPT’s utility for developing critical thinking, curiosity, and exam preparation. However, they feel slightly more confident about its usefulness in boosting their research skills. Overall, it can be concluded that a notable level of scepticism characterises students’ attitudes towards ChatGPT. Confirmation for this can be found in the fact that they avoided revealing their position concerning most of the statements by choosing either the I don’t know, or I prefer not to say option. Also, they unequivocally expressed concerns about the adverse impact of ChatGPT on critical thinking, creative writing, and data reliability. The open-ended responses indicate that while students have numerous ideas about how ChatGPT can be implemented in learning a foreign language, they value independent learning and personal efforts and, consequently, perceive AI tools as a potential threat.

In summary, while ChatGPT is the most familiar AI tool among Macedonian university students, its actual use in language learning contexts is less pronounced than expected. This discrepancy points to a need for increased awareness and training on the effective utilisation of AI tools. Their institutions should encourage language instructors to stay updated on technology trends that could revolutionise teaching through conferences, seminars, and workshops on effective AI integration in the classroom. They should then share this knowledge with students, modelling how AI can enhance language skills. For ChatGPT, instructors can offer training on writing clear and ethical prompts, detecting “hallucinations”, and properly citing AI-generated texts. Lastly, as Farhi et al. (2024) pointed out, policymakers should provide clear guidelines sooner rather than later, clarifying appropriate uses of AI tools, restrictions, and ethical boundaries.

Future research should further explore the barriers to greater engagement with AI chatbots in educational settings, including the underlying reasons for students’ reluctance and the manners in which AI tools can be integrated into language learning curricula more effectively. Addressing these aspects could lead to a more balanced and informed use of AI technologies, which, in turn, would undoubtedly enhance students’ learning experiences and outcomes. Moreover, large-scale studies are recommended in the Macedonian educational setting, involving a representative sample of university students from various institutions, to provide more objective and reliable insights. Future research could also include follow-up interviews, comparison with traditional teaching methods and longitudinal studies to reveal the actual state of the issue, which will, undoubtedly, continue to attract researchers’ attention in the years to come.

References

Abu Shawar, B., Atwell, E., 2007. Chatbots: Are they Really Useful? Journal for Language Technology and Computational Linguistics 22(1), pp. 29–49. https://doi.org/10.21248/jlcl.22.2007.88. Available at: <https://www.researchgate.net/publication/220046725_Chatbots_Are_they_Really_Useful>. [Accessed 11 July 2024].

Eke, D. O., 2023. ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology 13, 100060, pp. 1–4. https://doi.org/10.1016/j.jrt.2023.100060. Available at: <https://www.sciencedirect.com/science/article/pii/S2666659623000033>. [Accessed 11 July 2024].

Farhi, F., Jeljeli, R., Aburezeq, I., Dweikat, F. F., Al-shami, S. A., Slamene, R., 2023. Analysing the students’ views, concerns, and perceived ethics about chat GPT usage. Computers and Education: Artificial Intelligence, 5, 100180, pp. 1–8. https://doi.org/10.1016/j.caeai.2023.100180. Available at: <https://www.sciencedirect.com/science/article/pii/S2666920X23000590>. [Accessed 11 July 2024].

Haristiani, N., 2019. Artificial Intelligence (AI) Chatbot as Language Learning Medium: An inquiry. Journal of Physics: Conference Series, 1387, pp. 1–6. https://doi.org/10.1088/1742- 6596/1387/1/012020. [Accessed 11 July 2024].

Hu, K. H., 2023. An exploration of the key determinants for the application of AI-enabled higher education based on a hybrid soft-computing technique and a DEMATEL approach. Expert Systems with Applications, 212. https://doi.org/10.1016/j.eswa.2022.118762.

Karataş, F., Abedi Y. F., Gunyel, O. F., Karadeniz, D., Kuzgun, Y., 2024. Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners. Education and Information Technologies, 29, pp. 19343–19366. https://doi.org/10.1007/s10639-024-12574-6. Available at: <https://link.springer.com/article/10.1007/s10639-024-12574-6>. [Accessed 11 July 2024].

Klimova, B., Pikhart, M., Al-Obaydi, L. H., 2024. Exploring the potential of ChatGPT for foreign language education at the university level. Frontiers in Psychology, 15. https://doi.org/10.3389/fpsyg.2024.1269319. Available at: <https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2024.1269319/full>. [Accessed 11 July 2024].

Malmström, H., Stöhr, C., Ou, A. W., 2023. Chatbots and other AI for learning: A survey of use and views among university students in Sweden. Chalmers Studies in Communication and Learning in Higher Education, 1. https://doi.org/10.17196/cls.csclhe/2023/01. Available at: <https://research.chalmers.se/publication/535715/file/535715_Fulltext.pdf>. [Accessed 11 July 2024].

Ray, P., 2023. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, pp. 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003. Available at: <https://www.sciencedirect.com/science/article/pii/S266734522300024X>. [Accessed 11 July 2024].

Sabzalieva, E., Valentini, A., 2023. ChatGPT and Artificial Intelligence in higher education: Quick start guide. UNESCO. Available at: <https://unesdoc.unesco.org/ark:/48223/pf0000385146>. [Accessed 11 July 2024].

Sok, S., Heng, K., 2023. ChatGPT for Education and Research: A Review of Benefits and Risks. Cambodian Journal of Educational Research, 3(1), pp. 110–121. https://doi.org/10.2139/ssrn.4378735. Available at: <https://ssrn.com/abstract=4378735>. [Accessed 11 July 2024].

Song, C., Song, Y., 2023. Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, pp. 1–14. https://doi.org/10.3389/fpsyg.2023.1260843. Available at: <https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1260843/full>. [Accessed 11 July 2024].

Stepanechko, O., Kozub, L., 2023. English teachers’ concerns about the ethical use of ChatGPT by university students. International Scientific Journal Grail of Science, 25, pp. 297–302. https://doi.org/10.36074/grail-of-science.17.03.2023.051. Available at: <https://archive.journal-grail.science/index.php/2710-3056/article/view/1040/1061>. [Accessed 11 July 2024].

Welding, L., 2023. Half of college students say using AI on schoolwork is cheating or plagiarism. Best Colleges. Available at: <https://www.bestcolleges.com/research/college-students-ai-tools-survey/>. [Accessed 11 July 2024].

Author contributions

Silvana Neshkovska: conceptualisation, investigation, resources, methodology, writing –original draft, writing – review and editing.

Lela Ivanovska: conceptualisation, investigation, resources, methodology, writing – original draft.

Milena Kasaposka Chadlovska: investigation, formal analysis, visualisation, writing – original draft.

Vesna Trajkovska: investigation, formal analysis, visualisation, writing – original draft.

---

1. 1 https://chatgpt.com/

2. 2 Some students who took part in the survey were majoring in foreign languages (e.g. English, French, German, etc.), whereas some were majoring in other disciplines (IT, security, law, economics, biotechnology, etc.). Despite the fact that many of the respondents were not language students, most оf them attended a foreign language course in the academic year 2023/2024.

3. 3 Under the category “other language” one of the students added Macedonian, while another one added English and French (although both languages were already provided as options).