Acta Paedagogica Vilnensia ISSN 1392-5016 eISSN 1648-665X
2022, vol. 48, pp. 26–46 DOI: https://doi.org/10.15388/ActPaed.2022.48.2
The Impact of Metacognitive Reading Strategies on Master Students’ EFL Reading Proficiency and Academic Achievement
Iryna Levchyk
Foreign Languages Department
Ternopil Volodymyr Hnatiuk National Pedagogical University, Ukraine
iralevchyk@tnpu.edu.ua
Hanna Chaikovska
Department of Pedagogy and Methods of Primary and Preschool Education
Ternopil Volodymyr Hnatiuk National Pedagogical University, Ukraine
chaicov78@tnpu.edu.ua
Oksana Mazur
Foreign Languages Department
Volodymyr Hnatiuk Ternopil National Pedagogical University, Ukraine
mazur@tnpu.edu.ua
Zoriana Adamska
Department of Developmental Psychology
Ternopil Volodymyr Hnatiuk National Pedagogical University, Ukraine,
zoryanada@ukr.net
Nataliia Zakordonets
Foreign Languages Department
Ternopil Volodymyr Hnatiuk National Pedagogical University, Ukraine
nataliia.zakordonets@tnpu.edu.ua
Abstract. The paper investigates correlation of learners’ academic achievements with the application of metacognitive reading strategies (MRS) by the master students in EFL academic reading activities. The study was conducted in Ternopil Volodymyr Hnatiuk National Pedagogical University. Data was collected by means of focus groups interview and adapted scales of the MARSI questionnaire. The highlighted research is of a descriptive character; it has been completed with the aim of collecting the information at the initial stage of the experiment on master students’ usage of MRS in EFL academic reading activities.
Key words: metacognitive reading strategies, reading comprehension, academic achievement.
Metakognityvinių skaitymo strategijų įtaka magistrantūros studentų anglų kaip užsienio kalbos skaitymo įgūdžiams ir akademiniams pasiekimams
Santrauka. Straipsnyje nagrinėjama magistrantų akademinių pasiekimų ir metakognityvinių skaitymo strategijų (MSS) taikymo anglų kaip antrosios kalbos akademinio skaitymo užduotims koreliacija. Tyrimas atliktas Ternopilio Volodymyro Hnatiuko nacionaliniame pedagoginiame universitete. Duomenys buvo renkami tikslinių grupių interviu ir adaptuotomis MARSI klausimyno skalėmis. Straipsnyje pristatomas tyrimas yra aprašomasis, jis atliktas siekiant surinkti informaciją pradiniame eksperimento etape – eksperimentu tiriamas magistrantų MSS naudojimas atliekant anglų kaip antrosios kalbos akademinio skaitymo užduotis. Tyrimo dalyvių palankiausiai vertinamas MSS tipas buvo problemų sprendimo strategija, jos pritaikymo rodiklis didžiausias (bendras įverčio vidurkis 3,7), be to, keturios iš penkių dažniausiai pasirinktų strategijų irgi buvo problemų sprendimo tipo. Lyginamoji tyrimo rezultatų analizė atskleidė, kad aukštus ir žemus akademinius pasiekimus turintys magistrantai dažniausiai naudoja panašias MSS, skyrėsi tik šių dviejų respondentų grupių MSS taikymo dažnis anglų kaip antrosios kalbos akademinio skaitymo užduotims. Gerai besimokantys magistrantai (72 respondentai) MSS taikė vidutiniškai dažnai ir dažnai, jų įverčių vidurkis buvo nuo 3,0 iki 4,2, o prastai besimokantys studentai (58 dalyviai) strategijas taikė vidutiniškai dažnai ir retai, jų įverčių vidurkiai varijavo nuo 2,0 iki 3,2. Tyrimo teorinė nauda ta, kad pristatomi svarbūs rezultatai apie MSS taikymo poveikį Ternopilio Volodymyro Hnatiuko nacionalinio pedagoginio universiteto magistrantų skaitymo gebėjimams. Praktinė tyrimo implikacija ta, kad į magistrantams skirtas mokymo programas turėtų būtų įtraukiami pagrindinius arba pasirenkamuosius dalykus papildantys teksto supratimo kursai, ir jie turėtų būti kuriami vadovaujantis metakognityvinių strategijų naudojimo mokant anglų kaip antrosios kalbos teksto supratimo praktiniais mokymo vadovais.
Pagrindiniai žodžiai: metakognityvinės skaitymo strategijos, teksto supratimas, akademiniai pasiekimai.
__________
Received: 20/12/2020. Accepted: 18/05/2021
Copyright © Iryna Levchyk
1. Introduction
Comprehension of English academic texts is typically regarded as one of the hardest challenges for the master students in Ukraine. Despite the limited number of hours for the practical classes of Academic English master students are supposed to use strategies for effective reading and to search for the data in various sources of information, including Internet search in order to understand the content of written scientific information and the level of its relevance for their own research topic. They are expected to analyze and synthesize information presented in scientific sources, use it to write their own articles, as well as, annotate and summarize the sources of scientific information at the appropriate level of grammatical and academic correctness. They are supposed to analyze and synthesize the information for producing their own academically oriented texts, using typical for this style grammatical structures and functional patterns and understand and describe graphs, tables, charts, etc., using language forms and grammatical structures inherent in describing visualization tools (Levchyk, Turchyn 2020, p. 3). Therefore, master students are expected to have high achievements in learning foreign language, they need to be equipped with proper reading strategy use in order to meet the requirements during their study at university. The presented study is focused on the problem of student achievements and MRS application, particularly for developing reading proficiency in Academic English.
The relevance of the investigated problem of development of metacognitive awareness in the ESL learners is caused by the inconsistency in the expected and real level of language proficiency in master students in Ukraine and a strong positive correlation of application of MRS with the improved academic performance. Significance of the problem analyzed by the authors of the research in the field of methodology of teaching foreign languages is revealed due to the facilitative function of metacognitive strategies, it is of great theoretical and practical interest because it contributes to better academic achievement of master students.
The research object is to discover the relationship between the use of MRS by master students and their academic achievement in EFL.
The research goals imply:
1) definition of MRS categories most frequently used by master students in their EFL academic reading activities according to their application rate;
2) correlation of the results of the MARSI questionnaire with the students’ academic performance in EFL;
3) comparative analysis of choice and frequency of application of MRS between high and low achieving students;
4) composing the list of MRS used by high performing students in order to instruct and facilitate low performing students in their EFL academic reading activity.
2. Literature Review
As noted by Azher et al (2015) in their research of Pakistan post graduate students’ awareness of metacognitive reading strategies and reading comprehension, they need to apply in their reading activity such strategies like making predictions, skimming, understanding and engrossing, analyzing and recognizing particular issues, queries, summarizing and reviewing information (Azher et al, 2015, p.20). Being very critical for the second language learners, metacognitive reading strategy, as suggested by Ahmadi et al (2013), can solve the problems they usually face due to facilitation of the reading comprehension in the field of English as Foreign Language studies (Ahmadi et al., 2013, p. 237). The application of metacognitive strategy is generally considered to be an important effort that students can use in improving their reading skill as far as it can be implemented differently by low performance students to result high performance ones (Aziz et al., 2019, p. 140; Ismail & Tawalbeh, 2015, p. 75). The findings of the many studies have revealed that metacognitive strategies are positively correlated with reading achievement, so successful learners show a higher degree of metacognitive awareness, which enables them to use reading strategies more frequently then unsuccessful learners (Zhang & Seepho, 2013, p. 64). Models of teaching students metacognitive strategies in order to perform better in comprehension test have been addressed in recent studies, particularly by Phakiti (2003) on the example of the Thai students’ questionnaire results, retrospective interviews and an EFL achievement test. Similarly, a theoretical metacognitive strategies application model of reading comprehension was suggested by Riyadi et al. (2017), who highlighted underlining, note-taking, abridging, and idea mapping as significantly facilitative techniques. The findings from the recent study by Davoud Amini1, Mostafa Hosseini Anhari and Abolfazl Ghasemzadeh (2020) provide further conceptual support for dynamic models of self-regulated action and practical implications regarding the complementary role of metacognitive and self-regulatory engagements in reading development. The issue of development of metacognition training programs in order to increase students’ EFL reading comprehension gained scientific interest and was revealed in studies by Karbalaei (2011); Aghaie & Zhang (2012); Jafari & Ketabi (2012); Seifoori (2018). Different models of strategy use, in particular, for students with and without history of reading difficulties was reported in study by Chevalier et al. (2015), additionally, their findings indicated that the use of these strategies predicted academic success in students with history of reading difficulties. The metacognitive reading instructions gain scientific interest due to their significance for EFL learners in formation of reading comprehension skills, in this context a good reader is considered to develop a specific strategy which implies motivation to set goals for reading and abilities to identify and remember information, monitor and evaluate the perceived information (Teng, 2019, p. 29; Afflerbach & Cho, 2009; Duke & Pearson, 2002; Bаkеr & Bеаll, 2009). Following this conception, Zhussupova & Kazbekova (2016) in their pursuit to investigate the impact of metacognitive training on reading performance, have defined the aim of metacognitive strategies as formation of three skill techniques – planning, monitoring and evaluation – to contribute reading comprehension of EFL students in Kazakhstan. Basically, metacognitive awareness is referred to the effective state of a person that involved motivational and attitudinal states as well as ability to evaluate certain activity (Brown, 1987, p. 85), different studies proved evidence about application of metacognitive approach in language learning (Fleming and Walls, 1998). Clear and splendid explanations on instructions how to use MRS are crucial for development of reading proficiency, particularly for learners without English speaking surrounding (Teng, 2019; p. 29; Pressley, 2000, p. 33). Overall, readers’ metacognitive awareness of reading strategies is reported to be closely linked to their language proficiency explaining that poor readers rarely use reading strategy, whereas it is crucial for them to improve their reading ability (Zhang & Seepho, 2013, p. 62). The differences in the observed use of reading strategies of native and non-native English speakers when reading academic materials indicate that student’s reading ability is related to their metacognitive awareness and the use of reading strategy while reading, and demonstrate that both proficient native and non-native English learners have higher degree of metacognitive awareness as compared to the non-proficient ones (Sheorey and Mokhtari, 2001, p. 445-447). A strong correlation was revealed between the number of metacognitive comments students made during their interviews and higher science assessment posttests scores in study by Grotzer and Mittlefehldt (2012) who associated metacognitive knowledge and metacognitive regulation with comprehension and problem solving in reading (Grotzer and Mittlefehldt, 2012, p. 91). The benefits of metacognition as self-regulative strategies used by students to supervise their own learning were indicated by Chi et al. (1989, p. 176) who pointed out that application of the named strategies in self-explanation and self-monitoring activities gave significant advantage over poor learners who didn’t use them especially in problem solving. In order to support this theory, it is expedient to mention the research by White and Frederiksen (1998) showing the reserved potential of the low achieving students when engaged in metacognitive reflection, as well as, the crucial for assessment methods in this particular field work by Taraban et al. (2000) demonstrating a strong correlation between the use of MRS and academic achievement as measured with help of Metacognitive Reading Strategies Questionnaire (MRSQ). In order to explore the tertiary level students’ reading awareness a satisfactory self-report instrument from a measurement perspective was developed by Chen et al. (2009). The Metacognitive Reading Awareness Inventory was intended to assess college students’ reading awareness in reading academic or school-related materials. The authors named five components or subscales of reading: Phonemic Awareness, Phonics, Reading Fluency, Vocabulary Development, and Reading Comprehension. Overall, metacognitive awareness is considered by the mainstream (Mokhtari & Sheorey, 2002; Anderson, 2002; Yüksel & Yüksel, 2011; Ahmadi et al., 2013; Chevalier et al., 2015 and others) to be the key factor for proficient strategic reading since EFL students can consciously control the cognition and reasoning processes applying strategies effectively while being engaged in reading activity. In particular, Yüksel & Yüksel (2011) report about the shift in focus of scientific interest to academic reading sphere, making accent on the proved fact in multiple studies (Carrell, 1989; Sheorey & Mokhtari, 2001; Anderson, 2002; Mokhtari and Reichard, 2004) that much of the research about metacognition in L2 reading strategies suggested that readers’ metacognitive awareness is related positively to their success in L2 reading comprehension and performance and that both reading proficiency and L2 overall proficiency are connected to readers’ development of metacognition (Yüksel & Yüksel, 2011, p. 895).
Though metacognition is considered to be a chief component of self-regulated learning process (Butler et al., 2011, p. 97), especially effective in overcoming the typical for EFL students reading difficulties, by research relevant to this topic, it is still insufficiently known about the impact of metacognitive strategies in EFL reading proficiency by master students. The present study describes data collection on preference in usage of MRS by high achieving and low achieving master students in their university course of Academic English.
3. Metacognitive reading strategies
Reading strategies are of great importance in mastering EFL because they may help students to improve the reading comprehension in their efforts to make sense of reading text and meet the demands of reading task. It derives from the three components of reading strategy, composed by Mokhtari and Sheorey (2002, p. 4), namely: intentional well-planned techniques by which readers manage their reading process; actions and procedures utilized by readers in their direct text processing; fundamental support mechanisms intended to aid in text decoding. According to Chamot and O’Malley (1990, pp. 114-150) reading strategies may be classified into cognitive, metacognitive and social affective ones. Cognitive strategies imply direct interaction with the text that contributes to facilitate comprehension, operate directly with the oncoming information and manipulate it in different ways to enhance learning. Metacognitive strategies deal with the sequence of the processes used by a reader to control one’s cognitive abilities. While social affective strategies cover the areas of mediating and interacting with other people. Thus, regarding metacognitive strategies as high order executive skills scientists (Aziz et al, 2019; Zhang and Seepho, 2013; Pressley, 2000) emphasize their role in regulation of one’s learning by means of involving activities aimed to plan, monitor and simultaneously evaluate the reading process. Following the definition of metacognitive reading strategies by Olson, Platt, and Dieker (2008), Chevalier et al (2015) presuppose engaging of awareness and monitoring of the relationship between the cognitive resources and reading task requirements – the process which is called metacognition. As it is explained further by the investigators practically these may include planning reading tasks, monitoring that information is processed correctly and integrating the new information with the background one, while in the context of university EFL study reading comprehension is usually associated with English vocabulary related to the topic and the previous knowledge of the subject (Chevalier et al., 2015, p. 2). Taking to consideration this tendency, it was investigated by Cromley et al. (2010) that such strategies which encourage inference from text like summarizing information, self-questioning and linking new to previously learning material have a significant and strong impact on reading comprehension (Cromley et al., 2010, p. 689). Together with great number of other studies on the discussed issue (Sheorey & Mokhtari, 2001; Mokharti and Reichard, 2002; Anderson, 2002; Chen et al, 2009; Yüksel & Yüksel, 2011; Ahmadi et al., 2013; Chevalier et al., 2015 and others) it demonstrates a strong link between the effective reading comprehension and metacognitive reading awareness in the ways the students arrange their interaction with the context and use the related strategies. In the presented study, when measuring the level, frequency and choice of master students’ application of MRS in correlation to their academic performance in EFL, the authors followed the categories developed by Mokharti and Reichard (2002), who identified global reading, problem solving and support ones. The global reading strategies are characterized by scholars like generalized and intentional ones, analysis-oriented with the aim to set stage for the reading act and well-planned techniques by a reader to monitor their reading process. They include purpose setting, text structure preview, content prediction. The second group, problem solving strategies, is mainly aimed at solving problems when the text turns out to be difficult to read. They are called localized, focused techniques because they are used when reader faces problems of comprehension of textual information. Examples of problem solving strategies are adjusting one’s speed of reading when the materials become difficult or easy, guessing the meaning of unknown words, and rereading the text to improve comprehension (Par, 2020, p. 227). Finally, support strategies deal with comprehension of the text, for example, using reference materials, dictionary, making notes, underlining, highlighting parts of text. They are considered to be basic support mechanisms intended to aid the reader in the process of comprehending, constructing, and reconstructing the meaning of the text. The recommended division into categories has been widely adopted among the researchers in multiple efforts to identify which MRS type is most favored by language learners relatively to their success in reading proficiency in EFL (Madhumathi & Ghosh, 2012; Zhang and Seepho, 2013; Barrot, 2016; Aziz et al, 2019; Chevalier et al., 2015; Par, 2020 and others).
4. Methodology
4.1. Objective
The research objective was to reveal most favorable MRS master students apply, particularly, we were interested in the choice of strategy by students with high academic performance and identification of the degree to which metacognitive awareness influences students’ academic performance. Comparative analysis of usage of MRS by the high and low achieving master students resulted in composing a list of the most frequently applied strategies by successful students This may contribute to solution of the reading comprehension problem through providing low achieving students with effective tools to overcome difficulties in reading activities.
4.2. Participants
Participants (n = 130) were 1st-year master students of full-time department of the faculties of Pedagogy and Psychology (24), Primary Education (50), of Philology and Journalistic (41), Chemistry and Biology (5), Geography (5) and Physical Training (5) of Ternopil Volodymyr Hnatiuk National Pedagogical University.
The participants age varied from 21 to 22, predominantly they were females – 88, and there were 42 male students.
The average entering grade of 1st year Master course students is approximately 91% according to the results of the external independent evaluation.
4.3. Procedure
All incoming 1st-year Master course students in a single academic year 2019-2020 were also divided into two groups according to the mean score of the reading comprehension tests they had passed at the Academic English classes at the end of the first term. The validity of the reading comprehension test chosen for the survey has been already approved (TOEFL iBT reading comprehension test). According to the results they were classified into high performance and low performance students. Those students who reached the passing grade of 75 out of 100 were considered of high performance, and the others who didn’t reach the minimal passing score in the test were classified into low performance group. All the participants were invited by e-mail to complete the adapted MARSI questionnaire as the online survey with help of ‘Moodle’, the university server of the electronic courses for distant education.
Surveys were completed between September 15 and November 10. The discussed data was collected with help of the results obtained from the MARSI questionnaire passed by the participants online. It contributed to detecting the types of MRS and their priority in application both by high and low performance students.
4.4. Measure
The first task of the research implied application of the Metacognitive Awareness of Reading Strategies Inventory (MARSI) questionnaire (Mokhtari and Reichard, 2002) in order to measure the predominant types of MRS and frequency of their application by master students. It suggests to answer 30 statement, covering global reading strategies, problem solving strategies and support reading strategies. Each item is attributed with a 5 point Likert type scale with responses ranging from never to always. Global type of strategy is characterized with setting a purpose for reading, and previewing content; problem solving strategies aim to correct understanding, they are usually applied when the text becomes problematic. Support strategies deal with tools or mechanisms to support a reader like dictionaries or other reference materials. A number of studies in metacognitive awareness and positive dynamics in EFL proficiency have investigated the reliability and validity of this scale. Mokhtari and Reichard (2002) reported a Chronbach’s alpha of .89. Prior to completing the questionnaire, students were asked for consent to access their responses and to follow their academic progress, including the mean score of the academic achievement at the University. According to the research goals, the correlation of the results of the MARSI questionnaire with the students’ academic performance was necessary to enable the comparative analysis of findings regarding the possible links between master students’ achievements in EFL reading proficiency and metacognitive awareness.
The data were analyzed with help of the MARSI scoring rubric. The participants were supposed to write their response to each statement (i.e., 1, 2, 3, 4, or 5) in each of the blanks; add up the scores under each column and divide the score by the number of statements in each column to get the average for each subscale. Global reading strategies subscale includes 13 statements, problem solving – 8 and support – 9. Overall mean demonstrates how often a respondent uses reading strategies when reading academic materials. The average for each subscale of the inventory indicates which group of strategies (i.e., global, problem solving, and support strategies) language learner uses most often when reading. However, MARSI developers (Mokhtari and Reichard, 2002) note that the best possible use of MRS depends on learner’s reading ability in English, the type of the reading material, and purpose of reading it.
The mathematical and statistics analysis was completed with help of data processing computer software Microsoft Exel XP, IBM SPSS Statistics 21.
Grade point average. Volodymyr Hnatiuk Ternopil National Pedagogical University at which the study took place measures GPA on a scale ranging from 0 to 5. Students included in this study provided consent to access their GPA through the office of the registrar. Average GPA was calculated across the two semesters of each student’s 1st year. It must be noted that 72 master students with high achievements in EFL academic reading activities, who participated in the presented research in 2019-2020, had GPA fluctuating from 4.5 to 5.0.
5. Results
The results of the questionnaire demonstrated the frequency and priority of students’ application of the three suggested MRS, namely global, problem solving and support ones. The faculty average scores have been taken from answers of respondents studying at the same department. The results are presented in the table 1 below.
Table 1. Disposition of master students application of the MRS in EFL academic reading activity
|
Faculty |
Global |
Problem solving |
Support |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Source: own research
Obviously, problem solving category was reported most popular and global MRS type turned out to be the least popular among all the master students no matter what department they study at. That is why we will not focus at department distribution of students but only at their academic performance in EFL academic reading activities. Besides, calculation of the mean score of each respondent’s in three subcategories presents the most and the least frequently chosen reading strategy. Having summarized all item’s mean scores in each subcategory and divided by the number of items shows the mean scores for each strategy type as presented in table2.
Table 2. Frequency of application of MRS
|
Category |
Mean score/ Disposition in percent |
|
|
|
|
|
|
|
|
|
Source: own research
According to the results, demonstrated in the table 2, the problem solving strategy with the highest average score was chosen by 55 respondents, that was 43% of the total number of participants, it appeared to be the most preferred strategy type, followed by the support one that was used by 43 respondents and global one which was applied by 32 participants. It can be explained that students had tendency to apply the problem solving strategy when overcoming the difficulties in reading more often than the other two types. This conclusion is supported by the results of the calculation of the mean score for each statement from the MARSI questionnaire. It was made from the total score of each statement divided by the total number of the participants. The information on master students’ mean scores regarding MARSI questionnaire statements is presented in the Appendix I. The idea of priority of problem solving category among the respondents can be clearly expressed with help of the analysis of the top five MRS used by the master students in their EFL academic reading as displayed in table 3 below.
Table 3. Mean scores for the top five MRS used by master students in EFL academic reading activities according to MARSI questionnaire
|
Statements/Category |
Mean Scores |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Source: own research
The results presented in the table 3 proved that students mostly used to pay closer attention to what they were reading when they faced some difficulties in text, also they were inclined to use reference materials in particular online dictionaries in order to check the meaning of the unknown vocabulary; with the equal tendency respondents used to slow down the reading speed, re-read difficult parts and finally to guess the meaning of the unknown vocabulary with help of context.
According to the scoring rubric of MARSI there are three levels of strategy usage, namely low with average score of 2.4 and lower; medium with average score from 2.5 to 3.4; and high with average score of 3.5 and higher. As it can be vividly seen from the table 3, the high usage strategy group includes the first four problem solving techniques. Additionally, the support type strategy ranked the second with the mean score of 3.85 – “I use reference materials such as dictionaries to help me understand what I read.” That was caused by lack of specific vocabulary knowledge and consequently the arousing need to appeal to dictionaries in order to understand the text. The analysis of the questionnaire results showed both high and low usage metacognitive reading strategies. Students’ low preference of some strategies for text comprehension may be explained by their ignorance in this sphere and lack of practical skills.
Comparison of the students’ academic achievement with the frequency of usage of MRS shows that high performing students (72 respondents) used to apply them at medium and high rates with the mean scores from 3.0 to 4.2. While the low performing students (58 participants) demonstrated medium and low strategies application with the mean scores from 2.0 to 3.2 as presented in the table 4 below.
Table 4. Correlation of Preference in MRS type with MRS application rate of students with high and low academic achievement
|
Academic Achievement |
Strategy Type/ Application Rate |
Mean Score |
Overall Mean |
|
|
|
|
|
|
|
|
||
|
|
|
||
|
|
|
|
|
|
|
|
||
|
|
|
Source: own research
Besides, when interpreting scores, attention should be paid to the overall mean, because it indicates how often master students applied MRS working with English academic texts. The comparative analysis revealed significant difference in frequency of MRS usage between high achieving and low achieving language learners, while the choice of MRS category coincided for both groups of respondents (fig.1).
Source: own research
Fig. 1. Comparative distribution of indicators of high achieving (n=72) and low achieving (n=58) master students
Both groups of respondents preferred the problem solving type but high performing master students used it more often than their low performing mates (table 4). Additionally, correlation analysis by way of Pearson’s product-moment correlation coefficient also proved that problem solving category of MRS ranked the first among high achieving and low achieving participants of research. But it highlighted strong correlations between the indicators of EFL reading proficiency and MARSI subscales of support (r=0,840, at p≤0,01), global (r=0,729, at p≤0,01) and their rate of application (r=0,926, at p≤0,01), the findings are presented in table 6.
Table 6. Correlation between MARSI subscales, MRS application rate and EFL reading proficiency
|
Category |
Problem Solving |
Support |
Global |
Application Rate |
EFL_Reading _Proficiency |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
,838** |
,944** |
,840** |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
,838** |
|
,926** |
,729** |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
,944** |
,926** |
|
,812** |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
,840** |
,729** |
,812** |
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
||||||
Source: own research
The relations between the MRS application rate and academic achievement of language learners was also revealed with help of t-test statistics analysis (table 7).
The results of the t-test for independent samples indicate that master students with different level of academic achievement use MRS with different application rate (t= 40.116), especially it is related to the indicators of less popular categories among the respondents, such as support (t=30.220) and global (t=21.808), which were applied at high and medium levels of frequency respectively by high achieving students, however, only at low level of frequency by low achieving students. There were no significant differences in application of problem solving MRS (t=1.296), because it was equally preferred by all participants of the research fluctuating from high to medium level of application rate (table 8).
Table 7. Descriptive statistics for MARSI subscales, application rate and EFL reading proficiency
|
Category |
Achievement level |
N |
Mean |
SD |
Standard error mean |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Source: own research
Table 8. Independent Sample t-test for high and low achieving master students
|
Levene’s test of equal variances |
t-test for Equality of Means |
|||||||||
|
F |
Sig. |
t |
df. |
Significance (2-tailed) |
SD |
SE M |
95% confidence interval of the differences |
|||
|
lower |
upper |
|||||||||
|
|
|
|
|
1,296 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
30,220 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
21,808 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
40,116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||
Source: own research
The outcomes of the current study correlate with the results of the other research carried out by Ahzar M. et al (2015), and Aziz et al. (2019) about MRS use and reading comprehension of the academic material by EFL students. The latter investigated the differences in application of metacognitive strategies by low performance students and high performance students. “Although both groups fell into medium usage, none of the low performance students used the strategies in high usage. On the contrary, none of the high performance students reported to use the strategies in low usage. In line with this, it can be argued that high performance students showed high awareness of metacognitive strategies which leads them to use reading strategies frequently, whereas low performance students find it hard to use those strategies in enhancing their reading proficiency. It suggests that the students’ awareness of applying MRS is closely linked to their reading proficiency. Therefore, those who have problems in reading can take into consideration to apply MRS since these strategies will be effective to assist them to cope with the problems like experienced by the high performance students” (Aziz et al. 2019, p. 152). While there is a deviation observed from the results of the research by Azher et al (2015) who reported an inclination toward global strategies, followed by support and problem solving types respectively. Additionally, the authors found out the correlation between higher scores of comprehension tests and application of problem solving strategies, compared to scores of respondents who preferred global and support strategies. Thus, problem solving strategies proved to be more effective in increasing reading comprehension and have positive relation to higher academic performance in reading activity at EFL classes. Besides the present study outcomes allow to state that both high and low performance students used almost the same MRS, preferring problem solving type but the frequency of using these strategies differed considerably for these of groups.
6. Discussion
Ukrainian master students have to meet the demands of Academic English course during their university study. Reading skills, being essential for EFL learners in process of mastering English, require application of reading strategies in order to succeed and achieve high academic performance. High reading proficiency provides master students with a range of necessary skills, namely understanding the content of written scientific information according to relevance for their own research topic; abilities to analyze and synthesize information presented in scientific sources, annotate and summarize the sources of scientific information at the appropriate level of grammatical and academic correctness. Consequently, students will be able to produce their own academically oriented texts, using typical grammar constructions and vocabulary patterns; describe graphs, tables and charts, etc., that corresponds to the planned outcomes of study of the professionally oriented foreign language course for the master students.
The way students use reading strategies to meet the reading task goal reflects their metacognitive skills. Metacognitive reading skills are thought by Van Gelderen et al. (2004) to be learnt in native language and subsequently transferred to foreign language reading. Investigating linguistic knowledge, processing speed and metacognitive knowledge in first and second language reading comprehension the scientists defined a necessary condition for successful application of these skills; it implies possession of sufficient metacognitive awareness about reading strategies and the text characteristics. The importance of MRS for developing of reading proficiency has been proved by a number of studies devoted to overcoming EFL reading difficulties.
Discussing the top five MRS it can be suggested that traditionally EFL teachers demand that language learners stay focused when they are involved in reading activity and instruct them to use dictionaries in case there are any unknown vocabulary. So, it was quite predictable that MARSI statements “When text becomes difficult, I pay closer attention to what I’m reading” and “I use reference materials such as online dictionaries. to help me understand what I read” received the largest mean scores. Besides, the current research demonstrated that four out of five most often used strategies belong to problem solving type. Using dictionaries as a support reading strategy ranked the second among both high achieving and low achieving students. Differences in application in both groups of participants refer to the frequency of usage not the choice of reading strategies. The outcomes of the present study are similar to the research carried out previously in this field by Genc (2011, p. 654), Aziz et al. (2019, p. 138-159) which also informed about high usage of problem solving strategies as well as using reference materials to be the top five strategies students applied. The approach to strategy grouping may vary, but in general the results of this study don’t contradict conclusions made by Channa et al. (2018) who generated the research data using images of the text, selecting the main ideas, selecting the topic sentences, scanning of the text, summarizing of the text and questioning, e.g. with a different approach to categorizing reading strategies. The priority was also given by authors to the problem solving strategy (visualization) and the lower rank of global type (selecting the main ideas, selecting the topic sentences, scanning of the text, summarizing of the text), and the use of support type (questioning) was explained by lack of students’ knowledge and practice in searching out the key details, or other necessary skills like proper time management. In accordance with the statistical data concerning the choice and frequency of strategies application the presented study agrees with Alquahtani (2019) who confirmed the most often usage of problem solving type strategies by both male or female students, followed by the support strategy and finally the global reading strategy. Additionally, other scientists who interested in degree of metacognitive awareness of learners, like Sheorey and Mokhtari (2001); Zhang & Sheepho (2013); Aziz et al. (2019, p. 156), declared that students with high academic achievement showed higher degrees of metacognitive awareness than low achieving learners, what consequently enabled them to use more strategies. This statement has been proved by the results of the statistical data analysis in our study, which revealed high and medium levels of MRS application rate among high achieving students, that facilitated their EFL learning. Low achieving students were limited with medium level of application of the problem solving type, didn’t have necessary knowledge and skills to use other MRS categories. The lack of metacognitive awareness in the studied case resulted in a strong disadvantage position of low achieving master students when compared to their high achieving peers. It was remarked that majority of students had tendency to use the same strategy types, however, a significant dissimilarity in frequency was indicated. The findings of the research don’t allow us to compose a list of MRS preferred by high performing students. The choice of MRS categories turned out to be the same for the both groups of language learners. The link between the metacognitive awareness and reading proficiency in this and other similar research can be explained by the tendency of the successful students to show higher degree of metacognitive awareness. Therefore, they demonstrated more frequent application of the discussed strategies in reading, when compared to their low achieving peers.
The received data analysis allows to suggest positive influence of metacognitive awareness of applying reading strategies on EFL learner’s reading proficiency as well as it finds its support in studies conducted by Karbalaei (2011); Aghaie & Zhang (2012); Jafari & Ketabi (2012); Zhang and Seepho’s (2013); Ismail & Tawalbeh (2015); Riyadi et al. (2017); Shih & Huang (2018); Teng (2019); Aziz et al.(2019); Mohseni et al. (2020) who indicated that there was a significant positive correlation between use of metacognitive strategy and English reading achievement.
The contradictory research findings regarding the effect of metacognitive training between this and previous studies (Mehrdad, Ahghar, and Ahghar, 2012; Pammu, Amir, and Maasum, 2014) are clearly explained by Mohseni et al. (2020, p. 16) from the perspective of Adaptive Control of Thought (ACT) Model (Anderson, 1983) with regard to the degree of proceduralization of the achieved strategic knowledge. The scholars assumed that strategy training, like any other kind of learning, was governed by the same principles and it demanded principal attention while processing information during the cognitive stage of skill development.
Overall, metacognitive strategy awareness is considered to be an effective means to deal with EFL students’ reading difficulties, teaching MRS application not only increases text comprehension but also it has a positive relation to students’ academic achievement.
7. Conclusions
The detailed analysis of the received data with comparison to the results of the other investigations carried out in this field allows to state that at Volodymyr Hnatiuk Ternopil National Pedagogical University majority of master students use MRS at high and medium levels of frequency. The most often applied type of strategy among both high achieving and low achieving students turned out to be the problem solving one. While the support and global strategies were usually applied at medium level. More precise investigation of the problem solving strategy application showed that participants preferred the following techniques: to pay closer attention to what they were reading, to slow down the reading speed, to re-read difficult parts and guess the meaning of the unknown vocabulary with the help of context. The top five most often used strategies include also one strategy of support type – to use reference materials such as dictionaries. Thus, it can be concluded that majority of EFL master students have problems with time management in reading activity because they need more time to focus on the text for better comprehension. That is why they tend to read slowly and carefully, as well as they often re-read text when losing concentration. The other major comprehension difficulty students face in their reading is related to vocabulary gaps, thus dictionaries play very important role in overcoming translation problem.
Discovering the relationship between the use of MRS by master students and their academic achievement in EFL exposed the problem solving category as the most frequently used MRS by master students in their EFL academic reading activities according to their highest application rate (3.7 – for high achieving and 2.5 - for low achieving students); correlation of the results of the MARSI questionnaire with the students’ academic performance in EFL demonstrated significant difference in application rate, but similarity in choice of the MRS category due to the comparative analysis of the received data; the findings of the carried out research didn’t make it expedient to compose the list of MRS used by high performing students in order to instruct and facilitate low performing students in their EFL academic reading activity. The findings of the research can be practically implemented in the process of overcoming the typical learning difficulties of EFL students as an effective tool to increase the reading proficiency. Students should be informed about MRS and encouraged to develop the practical skills based on application of problem solving, support and global strategies. With proper instructions teachers may guide EFL learners to cultivate their own metacognitive strategies for reading comprehension, after they have been exposed to the wide range of problem solving, support and global strategies. Interpretation of the data received from the presented research supported with the findings of other investigations in this field allow to recommend development of elective EFL academic reading proficiency courses for master students, complement of EFL teaching manuals with instructions based on MRS application in learning. With metacognitive awareness EFL learners better understand the necessity of reading proficiency, it gives them a serious advantage of self-regulation of their own reading activity by application of MRS to enhance their comprehension of academic texts.
8. Limitations and future research
Limitations of the present study are mainly related to its descriptive character. As it has been predicted, quantitative designed research contributes to statistical analysis but it usually does not cover the whole field of investigation. The future research should be focused on the qualitative data collection with possible objective of developing our own model to rise metacognitive awareness among EFL students and comparison of it with the existing ones. Besides, the effectiveness of the programs on application of metacognitive strategies needs to be assessed in terms of EFL reading proficiency and speaking proficiency. Due to the positive link of metacognition with academic success, such programs need to be in the focus of future research as possible factors to increase students’ performance. The following studies have to be carried out with larger samples of EFL students and with use of different tools of assessment including self-reporting tests, reading comprehension tests, interviews, questionnaires representing various approaches to the issue of implementation of MRS into learning process before arriving at more definite conclusions.
References
Afflerbach, P., Cho, B.-Y. (2009). Identifying and describing constructively responsive comprehension strategies in new and traditional forms of reading. In S. Israel, G. Duffy (Eds.) Handbook of Research on Reading Comprehension. (pp.69–90) New York: Routledge
Aghaie, R. & Zhang, L. J. (2012). Effects of explicit instruction in cognitive and metacognitive reading strategies on Iranian EFL students’ reading performance and strategy transfer. Instructional Science, 40(6), 1063–1081. Retrieved from http://doi.org/10.1007/s11251-011-9202-5
Ahmadi, M. R., Ismail, H. N., & Abdullah, M. K. K.. (2013). The Importance of Metacognitive Reading Strategy Awareness in Reading Comprehension. English Language Teaching, 6 (10), 235–244.
Alqahtani, S. M. A. (2019). Investigating the Relationship between Metacognitive Strategies and Reading Proficiency among the University of Jeddah Learners. Acta Scientiae et Intellectus, 5(4), 38–52. Retrieved from http://actaint.com/issue.php?go=december-2019t4
Amini, D., Hosseini, A. M. & Ghasemzadeh, A. (2020). Modeling the relationship between metacognitive strategy awareness, self-regulation and reading proficiency of Iranian EFL learners. Cogent Education, 7(1), 1787018. Retrieved from https://doi.org/10.1080/2331186X.2020.1787018
Anderson,N. J. (2002). The role of metacognition in second/foreign language teaching and learning. ERIC Digest. Washington, DC: ERIC Clearinghouse on Languages and Linguistics. Retrieved from https://www.govinfo.gov/content/pkg/ERIC-ED463659/pdf/ERIC-ED463659.pdf
Azhar, M., Awan, R. N.& Khalid, S. (2015). University Students’ Awareness of Meta-cognitive Reading Strategies and Reading Comprehension. Journal of Educational Sciences & Research Spring, 2(1), 19–32. Retrieved from https://www.researchgate.net/publication/323676379_University_Students’_Awareness_of_Meta-cognitive_Reading_Strategies_and_Reading_Comprehension
Aziz, Z. A., Nasir, C., & Ramazani, R. (2019). Applying metacognitive strategies in comprehending English reading texts. Celt: A Journal of Culture, English Language Teaching & Literature, 19(1), 138–159. Retrieved from: https://doi.org/10.24167/celt.v19i1.1863
Brown, A. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. Weinert & R. Kluwe (Eds.), Metacognition, motivation, and understanding. (pp. 65–116), Hillsdale, NJ: Erlbaum.
Butler, D. L., Cartier, S. C., Schnellert,L, Gagnon, F. & Giammarino, M. (2011). Secondary students’ self-regulated engagement in reading: researching self-regulation as situated in context. Psychological Test and Assessment Modeling, 53(1), 73–105.
Bаkеr, L., Bеаll, L. C. (2009). Mеtаcоgnіtіvе prоcеssеs аnd rеаdіng cоmprеhеnsіоn. Іn S. Е. Іsrаеl & G. G. Duffу (Еds.), Hаndbооk оf rеsеаrch оn rеаdіng cоmprеhеnsіоn. (pp. 373–388), Nеw Уоrk: Rоutlеdgе
Barrot, J. S. (2016). ESL learners’ use of reading strategies across different text types. The Asia-Pacific Edu. Researcher, 25(5), 883–892. Retrieved from: https://doi.org/10.1007/s40299-016-0313-2.
Carrell, P. L. (1989). Metacognitive Awareness and Second Language Reading. Modern Language Journal, 73, 121 –134.
Chamot, A. & O’Malley, J. M. (2012). Learning Strategies in Second Language Acquisition. Oxford: Oxford University Press. Retrieved from https://doi.org/10.1017/CBO9781139524490
Channa, M. A., Abassi, A. M., John S., Mohammad, J. K., Memon, M. A., Nordin, Z. S. (2018). Reading Comprehension and Metacognitive Strategies in First Year Engineering University Students in Pakistan. International Journal of English Linguistics, 8(6), 78–87. Retrieved from http://www.ccsenet.org/journal/index.php/ijel/issue/view/2121
Chen, M. H., Gualberto, P. J., Tameta, C. L. (2009). The Development of Metacognitive Reading Awareness Inventory. TESOL Journal, 1, 43–57. Retrieved from http://www.tesoljournal.com
Chevalier, T. M., Parrila, R., Ritchie, K. C., Deacon, S. H. (2015). The role of metacognitive reading strategies, metacognitive study and learning strategies, and behavioural study and learning strategies in predicting academic success in students with and without a history of reading difficulties. Journal of Learning Disabilities, 50(1), 34–48. Retrieved from https://doi.org/10.1177/0022219415588850
Chi, M. T.H., Bassok, M., Lewis, M. W., Reimann, P.& Glaser, R. (1989). Self-Explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182. Retrieved from https://doi.org/10.1207/s15516709cog1302_1
Cromley, J. G., Snyder-Hogan, L. E., & Luciw-Dubas, U. A. (2010). Reading comprehension of scientific text: A domain specific test of the direct and inferential mediation model of reading comprehension. Journal of Educational Psychology, 102(3), 687–700. Retrieved from http://doi.org/10.1037/a0019452
Duke, N. K. & Pearson, P. D. (2002). Effective practices for developing reading comprehension. In A. E. Farstrup, S. J. Samuels (Eds.) What Research Has to Say about Reading Instruction, (pp. 205–242). Newark, DE: International Reading Association, Inc.
Fleming, F., & Walls, G. (1998). What pupils do: The role of strategic planning in modern foreign language learning? The Language Learning Journal, 18(1), 14–21. Retrieved from https://doi.org/10.1080/09571739885200201
Genc, H. (2011). Paper and Screen: Reading Strategies Used by Low-Proficient EFL Learners. Sino-US English Teaching, 8(10), 648–658.
Grotzer, T., & Mittlefehldt, S. (2012). The role of metacognition in students’ understanding and transfer of explanatory structures in science. In A. Zohar & Y. J. Dori (Eds.), Metacognition in Science Education: Trends in Current Research, (pp. 79–99). Springer. Retrieved from https://doi.org/10.1007/978-94-007-2132-6_5
Ismail, N. M., & Tawalbeh, T. I. (2015). Effectiveness of a metacognitive reading strategies program for improving low achieving EFL readers. International Education Studies, 8(1), 71–87.
Jafari, D., & Ketabi, S. (2012). Metacognitive strategies and reading comprehension enhancement in Iranian intermediate EFL setting. International Journal of Linguistics, 4(3), 1–14. Retrieved from http://doi.org/10.5296/ijl.v4i3.1684
Karbalaei, A. (2001). Assessing reading strategy training based on CALLA model in EFL and ESL context. Íkala, Revista De Lenguaje y Cultura, 16(27), 167–187.
Levchyk, I., Turchyn, A. (2020). Curriculum of the Master Course of English for Academic Purposes. Ternopil: Volodymyr Hnatiuk Ternopil National Pedagogical University
Madhumathi, P., & Ghosh, A. (2012). Awareness of reading strategy use of Indian ESL students and the relationship with reading comprehension achievement. English Language Teaching, 5(12), 131–140.
Mehrdad, A. G., Ahghar, M. R., & Ahghar M. (2012). The effect of teaching cognitive and metacognitive strategies on EFL students’ reading comprehension across proficiency levels. Social and Behavioral Sciences, 46(7), 3757–3763.
Mohseni, F., Seifoori, Z., & Ahangari, S. (2020). The impact of metacognitive strategy training and critical thinking awareness-raising on reading comprehension. Cogent Education, 7(1), 1–22. Retrieved from https://doi.org/10.1080/2331186X.2020.1720946
Mokhtari, K. & Reichard, C. (2002). Assessing Students’ Metacognitive Awareness of Reading Strategies. Journal of Educational Psychology, 94(2), 249–259. Retrieved from https://doi.org/10.5539/ijel.v8n6p78
Mokhtari, K., & Reichard, C. (2004). Investigating the strategic reading processes of first and second language readers in two different cultural contexts. System, 32, 379–394.
Mokhtari, K., & Sheorey, R. (2002). Measuring ESL students’ awareness of reading strategies. Journal of Developmental Education, 25(3), 2–10.
Olson, J. L., Platt J. C., & Dieker L. A. (2008). Teaching children and adolescents with special needs (5th ed.). Columbus, OH: Prentice Hall.
Pammu, A., Amir, Z., & Maasum, T. (2014). Metacognitive reading strategies of less proficient tertiary learners: A case study of EFL learners at a public university in Makassar, Indonesia. The Journal of Teaching Language Skills (JTLS), 8(2), 115–132.
Par, L. (2020). The Relationship between Reading Strategies and Reading Achievement of the EFL Students. International Journal of Instruction, 13(2), 223–238. https://doi.org/10.29333/iji.2020.13216a
Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test performance. Language Testing, 20(1), 26–56. Retrieved from http://doi.org/10.1191/0265532203lt243oa
Pressley, M. (2000). Comprehension instruction in elementary school: A quarter-century of research progress. In B. Taylor, M. F. Graves, & P. van den Broek (Eds.), Reading for meaning: Fostering comprehension in the middle grades. (pp. 32–51). New York, NY: Teachers College Press.
Riyadi, I., Hersulastuti, H., & Nugrahaningsih, T. K. (2017). Metacognitive learning strategy: In search of theoretical model for reading comprehension. SEEDS, 1(1), 125–142.
Seifoori, Z. (2018). Postgraduate English students’ metacognitive awareness of reading strategies and their reading comprehension: A comparative study. The Journal of Applied Linguistics and Discourse Analysis, 3(1), 117–134.
Sheorey, R., & Mokhtari, K. (2001). Differences in The Metacognitive Awareness of Reading Strategies Among Native and Non-Native Readers. System, 29, 431–449.
Shih, H. C. J., & Huang, S. (2018). EFL learners’ metacognitive strategy use in reading tests. English Teaching & Learning, 42(2), 117–130. Retrieved from http://doi.org/10.1007/s42321-018-0007-3
Taraban, R., Rynearson, K., & Kerr, M. (2000). College students’ academic performance and self-reports of comprehension strategy use. Reading Psychology, 21(4), 283–308. Retrieved from http://doi.org/10.1080/027027100750061930
Teng, M. F. (2019). The benefits of metacognitive reading strategy awareness instruction for young learners of English as a second language: Metacognitive instruction in reading. Literacy, 54(5), 29–39. Retrieved from https://doi.org/10.1111/lit.12181
Van Gelderen, A., Schoonen, R., De Glopper. K., Hulstijn. J., Snellings.P., Simis. A., & Stevenson, M. (2004). Linguistic knowledge, speed of access and metacognitive knowledge in first and second language reading comprehension. Journal of Educational Psychology, 96(1), 19–30. Retrieved from http://doi.org/10.1037/0022-0663.96.1.19
Wade, S. E., Trathen,W. & Schraw, G. (1990). An analysis of spontaneous study strategies. Reading Research Quarterly, 25(2), 147–166. Retrieved from http://www.jstor.org/stable/10.2307/747599
White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3–118. Retrieved from https://doi.org/10.1207/s1532690xci1601_2
Yüksel, İ., Yüksel, İ. (2011). Metacognitive Awareness of Academic Reading Strategies. Procedia – Social and Behavioral Sciences. 31, 894–898. Retrieved from https://doi.org/10.1016/j.sbspro.2011.12.164
Zhang, L. & Seepho, S. (2013). Metacognitive Strategy Use and Academic Reading Achievement: Insights from a Chinese Context. Electronic Journal of Foreign Language Teaching, 10(1), 54–69.
Zhussupova, R., Kazbekova, M. (2016). Metacognitive strategies as points in teaching reading comprehension. Procedia – Social and Behavioral Sciences, 228, 593–600. Retrieved from https://doi.org/10.1016/j.sbspro.2016.07.091
Appendix I
This Appendix contains the table presenting the information on the master students’ mean scores of the statements from MARSI questionnaire that was used by the authors of the research.
Table 9. Mean scores for 30 MRS statements used by master students in EFL academic reading activities according to MARSI questionnaire in priority of choice
|
Statements/Category |
Mean Scores |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|