English for Specific Purposes
Wrld

STUDENTS REFLECTIONS ON LEARNING ENGLISH FOR SPECIFIC PURPOSES

Galina Kavaliauskienė

Mykolas Romeris University, Vilnius, Lithuania

Abstract

Active participation of learners in the self-evaluation of their learning and the development of reflective thinking are the key features of alternative assessment. Alternative assessment is often defined as any type of assessment in which learners create a response to an assignment. The success of alternative assessment in language teaching is predetermined by students performance which demonstrates what learners can do with language in contemporary communicative classrooms.

This research aimed at investigating influence of students assessment of utility of various assignments for their linguistic development in English for Specific Purposes. The study employed a survey on the usefulness of different assignments and learners' written reflections on the perceptions of benefits to language mastery. Learners assignments included various contributions to portfolios such as essays, summaries of professional texts, outlines of oral presentations, creative computer tasks, tests as well as students written self-assessment notes, i.e. their reflections on various classroom activities. The research involved the students who study Social Sciences at Mykolas Romeris University.

The results demonstrated that self-assessment was beneficial for learners linguistic development. The learners reflections reveal their attitudes to various assignments and judgments of their usefulness in learning. Reflective practice might help teachers develop ways of dealing with specific difficulties and improve the quality of teaching. Training learners to reflect on learning outcomes is beneficial from the perspective of lifelong learning.

Key words: English for Specific Purposes, self-assessment, assignments, portfolio, reflections.

Introduction 

Recently self-assessment has become part of learning and teaching foreign languages. Methods of assessment are based on learners' evaluation of their own learning and linguistic development. The importance and relevance of self-assessment is demonstrated by a multitude of publications in this area. Portfolio assessment as one of the ways of self-assessment has become widely used in educational settings. The key features of alternative assessment are: an alternative to traditional testing, ways to self-evaluate knowledge, skills and progress in learning and active participation of learners in the evaluation and development of reflective and critical thinking. Success in learning often depends on learners interest and subsequent motivation to study. However, the learners' perception of the usefulness of various class activities has not been sufficiently explored. There seems to be a discrepancy between teachers' and learners' views on benefits of the same activity. Quite often unpopular activities among teachers like grammar exercises or translation from L2 into L1 are perceived by learners as beneficial to their learning.

The aims of this research are to: 1. study learners' attitudes to the usefulness of various class activities, 2. introduce self-evaluation and self-assessment of learning, and 3. analyze learners' reflections on success or failure in their learning. The research methods used in this study are: 1. a survey on the usefulness of class activities, 2. learners' written reflections on learning outcomes, and 3. statistical treatment of their responses. The activities in English for Specific Purposes (ESP) class included computer tasks, creative assignments, discussions, Power Point Presentations, listening practice, summary and essay writing. Traditional testing of learner performance in class activities was replaced by self-assessment.

Literature review 

Alternative assessment has been widely used in assessing the effectiveness of education at tertiary level (Douglas, 2000). It refers to the purposeful, selective collection of learner work and reflective self-assessment which documents progress and achievements over time. The key features of alternative assessment are learners active participation in the evaluation of their own learning and the development of reflective thinking. The success of alternative assessment depends on the performance that shows what learners can do with language in communicative classrooms. Alternative assessment is often understood as the utilization of non-traditional approaches in judging students' performance.

Differences between traditional and alternative assessment were analyzed by Douglas- Brown (2003), who noted that many forms of assessment fall in between the two, and some combine the best of both. Our Google search produced 71,700,000 hits for alternative assessment' (March 2007). Portfolios have been known as a means of alternative assessment for evaluating the effectiveness of learning since the late 1980s. Another Google search produced 3,840,000 for language portfolio' (March 2007). The amount of online references demonstrates the importance and relevance of these forms of assessment in teaching and learning a foreign language.

The portfolio, as an element of authentic assessment, has captured the interest of many instructors who want a more comprehensive way to assess their students' knowledge and skills, to have students actively participate in the evaluation process, and to simultaneously develop students' skills of reflective thinking. Five key characteristics of portfolio are: an alternative to traditional testing, comprehensive ways to assess students' knowledge and skills, authenticity of assessment, students' active participation in the evaluation process, simultaneous development of students' reflective thinking. For some teachers, the portfolio is part of an alternative assessment program, and it can either include a record of students' achievements or simply document their best work. For others, the portfolio documents the students' learning process, and can be used as a means of promoting learner reflection. The definition of the portfolio can shift from product to process according to the context and design of its development (Nunes, 2004).

The European Language Portfolio (Language passport + Language biography + Dossier) was developed and piloted by the Modern Languages Division of the Council of Europe. It is a well known document in which learners can record their language learning and cultural experiences. The portfolio has pedagogic and reporting functions. Portfolios provide teachers with a wealth of information upon which to base instructional decisions and to evaluate student progress. Portfolios can serve as a means of motivating students and promoting their self-evaluation and self-understanding. Central to portfolios are the reflections or commentaries on the entries which are typically present in portfolios. Through reflections students can develop metacognitive awareness of texts and situations, can improve their strategies dealing with various tasks as well as may judge their own work and compare performance in different assignments. The European Language Portfolio (ELP) for Higher Education (Vosicki, 2003) conforms to the common Principles and Guidelines of the Council of Europe, and consists of three parts like its companion piece The European Language Portfolio within the Common European Framework of Reference for Languages. According to Little (2005), there are three reasons for engaging learners in self-assessment and taking account of the results: firstly, a learner-centerdness; secondly, self-assessment, and thirdly, a tool for lifelong language learning. The most important feature of the ELP is that it supports reflective learning in which self-assessment and goal setting play a central role (Little, 2005). Generally speaking, portfolios capitalize on students' natural tendency to save work and become an effective way of getting students to take a second look and think about how they could improve future work. Portfolios can provide structure for involving students in developing and understanding criteria for good efforts and in applying the criteria to their own work (McCabe, 2004). The key drawback of portfolios is that they place additional demands on teachers and students. Teachers need additional time for planning, developing strategies and materials, meeting with individual students and small groups, and reviewing and commenting on student work. Portfolios have been characterized by some teachers as a worthwhile burden with tangible results in instruction and student motivation.

Portfolio assessment is closely linked to instruction because they can reveal weaknesses in instructional processes. Portfolios offer the teacher an in-depth knowledge of the student as a learner and allow the teacher to individualize instruction for the student. However, without reflections, the portfolio remains a folder of the accumulated papers (Coombe & Barlow, 2004). The most common areas of student reflections noted by Nunes (2004) are: syllabus (7%), instruction (36%), learning (43%), and assessment (14%). The students' experiences using the portfolio framework were investigated by examining learners' written reflections (Beckett & Slater, 2005). It was found that only one fifth of the 73 participants enjoyed project work; one quarter had mixed feelings, and 57% perceived it negatively. Moreover, the high drop-out rate from the course existed because some students found the course too difficult or believed English classes should be limited to the study of language and resented being asked to accomplish non-linguistic tasks. It has been claimed that the challenges of portfolio assessment to language learners include lower comparability and reliability and difficulty ensuring standardized testing conditions. They also pose a scoring problem because criteria requires staff training and is more time consuming than scoring a single norm-referenced test (Gomer, 2001). According to Nunan (1988), there is a considerable extent of the mismatch between teacher and learner perceptions of the usefulness of different activities. Learners rated grammar exercises, pronunciation, and error correction more highly that their teachers did, who were more likely to feel comfortable with pair / group work and communication tasks. Interestingly, a study by Spratt (1999) was conducted to compare learners' preferred activities with teachers' perceptions of what those preferences were, and only a roughly 50% correlation was found. Similarly, another researcher (McDonough, 2002), reported learners' dislikes such as the listening to tapes and course book dialogues, and preferred priorities - grammar exercises, reading aloud, translation, and so on as being useful to learning.

Summing up, it is important for ESP practitioners to clarify what learners' attitudes are towards various assignments, constantly analyze learners' feedback and adjust teaching methods to learners changing needs. 

Respondents and research methods 

The respondents in this research were the students specializing in Social Sciences at Mykolas Romeris University and studying English for Specific Purposes. There were 60 participants aged 19 to 22 in this project. They were predominantly females at the pre-intermediate and intermediate levels. The amount of time spent in L2 environment was 4 hours a week for 3 semesters, which amounts to about 190 hours of English instruction. The data obtained for researched groups was compared to the findings for the control group of 20 students.

In this research we administered a brief survey, which was designed according to the scientific standards (Dornyei, 2003) and reproduced in Appendix 1. Our aim was to get insights into the learners attitudes to the usefulness of various activities and assignments in English for Specific Purposes classes. The written self-reflections on the performance and achievements in various activities were analyzed and treated statistically with the aim of evaluating the significance level of obtained data. A few excerpts from students' reflections are reproduced in Appendix 2.

Results

In accordance with the scientific requirements to the surveys in Social Sciences (Dornyei,2003), all statements were rated on the scale of five possible answers. The students rated each statement by circling or writing the appropriate number: 1 - strongly disagree, 2 disagree, 3 - not sure, 4 - agree, 5 - strongly agree. The learners responses are summarized in Table 1. For the sake of clarity, positive answers (strongly agree and agree) and negative answers (strongly disagree and disagree) are added up. However, in statistical treatment of the data, which was conducted and described below, the responses were treated separately.

The first row in Table 1 shows the respondents' perceptions of the utility of learning vocabulary definitions for linguistic development. 73% of the students feel that vocabulary definitions are useful to learning. There were very few negative responses (8%). About the fifth of the respondents (19%) are not sure about the usefulness of this activity in language classes. 

The attitudes to creative tasks are shown in the third row of Table 1. As many as 62% of students enjoy doing creative assignments and assess them as beneficial to linguistic development. 12% of learners are negative about this type of tasks, and 26% are not sure.

Perceptions of utility of classroom discussions that usually involve impromptu speaking are presented by the fourth row in Table 1. The majority of learners are positive about this activity (88%), and 12% are not sure. None of the students perceive this activity negatively. Majority of learners are active and prolific at generating ideas and enjoy debating.

Preparation and delivery of Power Point Presentations are positively assessed by less than half of the respondents (46%). The rest are either not sure of the usefulness of this assignment (26%) or negative (27%). This is quite understandable because preparation of presentations is time consuming and delivery is daunting for some students.

Table 1. Students responses to the questionnaire on the usefulness of various class activities.

Survey statements	Agree, %	Not sure, %	Disagree, %
1. Writing definitions of professional vocabulary terms is beneficial for linguistic development.	73	19	8
2. Writing summaries of professional texts is useful for developing writing skills.	56	29	15
3. Doing creative tasks is useful for linguistic development.	62	26	12
4. Classroom discussions on various topics are useful for developing speaking skills.	88	12	0
5. Power Point Presentations are beneficial for developing speaking skills.	47	26	27
6. Formal testing of professional vocabulary is beneficial to learning.	79	21	0

The second row in Table 1 shows respondents attitudes to writing summaries. 56% of students found writing activities beneficial while 15% disagree, and 29% are not sure. Generally speaking, students do not enjoy writing formal texts, mainly because they find this activity time consuming and frustrating due to difficulties in expressing ideas formally, accurately, briefly and clearly.

Preparation of Power Point Presentations (the fifth row) is a time consuming activity, and their delivery in front of the audience is intimidating to some learners. Consequently, only 47% agree with the statement, and the rest either disagree (27%) or not sure (26%).

The last row in Table 1 shows the students attitudes to formal testing. Surprisingly, as many as 79% of the students are positive about formal testing while the rest are not sure, with none of the learners being negative about it. Moreover, learners expressed their multiple requests for being tested claiming that tests prompt them to learn harder.

Statistical processing of the data 

The results that are shown in Table 1 have been processed statistically in order to determine how significant the obtained data are. Such processing is essential in the case of a small sample of respondents as it is the case here (60 respondents).

Internal consistency reliability was met by satisfying two conditions: multi-item scales were used, and items measured the same target area (Dornyei, 2003). Internal consistency reliability was evaluated by computing Cronbach Alpha coefficient. Normally Cronbach Alpha coefficient is ranged between zero and +1. For well-developed scales containing a few items it ought to approach 0.80 (Dornyei, 2003). In this study, Cronbach Alpha coefficient was equal to 0.83, which ensures reliability of internal consistency.

Generally, formal experiments are analyzed using inferential statistics in order to make inferences on research data. For this purpose, the Means and Standard Deviations were computed and the t-test was applied in data analysis. The t-test is the most frequently used measure in L2 research when comparing mean scores for two samples. The sample group was compared with the control group. The adjustment for group size was made by using the theoretical values in Table 7.5 (from Brown & Rodgers, 2002) and critical values for the t-test statistics from the same source. If the exact number of a degree of freedom df is not shown in the table, the closest value is usually taken. This procedure is known as adjustment for the group size. Here, the group size was adjusted for by calculating the degrees of freedom, which are determined by subtracting 1 from the number of participants in each group and then adding the two resulting numbers together. The data of statistical processing of the obtained results are presented in Table 2.

Table 2. Means, Standard Deviations (SDs), t-test coefficients and two-tailed significance levels. There were 60 participants in the sample, and 20 participants in the control group. The degree of freedom df = 58.

Discussion of results

The research was conducted into the usefulness of various activities in ESP classes, and the data of statistical processing are shown in Table 2. The comparison of the Means for various activities (columns 2 and 3) shows that the respondents in the studied sample had more favorable views on the utility of classroom activities than the respondents in the control group: the Mean values are greater for all activities. The values of Standard Deviations (SDs), which show the scattering of the data, are smaller except for the activity of summary writing. A t-test analysis between means for each activity yielded t-values that are shown in the fourth column of Table 2. The comparison of calculated t-coefficients with the tabled t-values (the fifth column) gives the two-tailed significance levels which are shown in the sixth column. The closest value for the degree of freedom df=58 that is found in the theoretical Table 7.5 (Brown & Rodgers, 2002) is 60, and it was used for the determination of the level of significance. It should be noted that significance levels differ for various activities. For vocabulary definitions and vocabulary tests the significance level p < 0.02, while for summary writing and creative tasks p < 0.05. Discussion tasks yield p < 0.01. Interestingly, for Power Point Presentations calculated t coefficient is below the critical value 1.67, which implies the non-existence of significance.

Learners' reflections on class activities 

As it has already been mentioned, the reflection on the utility of class activities to learning during the academic year makes an important contribution to alternative assessment. The excerpts from the reflections of some students are reproduced in Appendix 2. The language has not been corrected for the sake of authenticity, and the names of students are omitted in order to preserve anonymity. It is obvious from these reflections that learners sincerely describe their experiences in conducting various tasks because they have come to realize the importance of self-evaluation in learning. Critical thinking about ones achievements or failures encourages a learner to make more efforts to learn better.

Conclusions

Learners' reflections on the utility of various tasks in English classes offer insights into learning progress. Given the disparity of usefulness scores, it is apparent that only statistical processing might provide further clarification. The computation of t-coefficients and significance levels allows us to clarify the uncertainty in utility scores that are displayed visually in Table 2 for a small sample of 60 students-respondents. Usefulness is questionable for the activity of making Power Point Presentations due to the higher value of significance level (Table 2, p < 0.10).

In teaching English in higher education teachers need to examine learners' ongoing self-evaluation which can help teachers cater to changing students' needs. For this purpose, learner's individual written reflections and self-assessment in the form of their contributions to portfolios might serve best due to opportunity to analyze students individual difficulties and dislikes, i.e. attitudes to grammar, writing essays or listening to long passages. Individualized approach allows teachers to gain each learners trust and think of the ways of fostering their linguistic development. 

References

1. Douglas, D. (2000). Assessing Language for Specific Purposes. Cambridge: Cambridge University Press.

2. Douglas-Brown, H. (2003). Language Assessment: Principles and Classroom Practices. New York: Pearson Education Inc.

3. Nunes, A. (2004). "Portfolios in the EFL Classroom: Disclosing an Informed Practice". ELT Journal 58/4:327-335. 

4. Vosicki, B.F. (2003). "European Language Portfolio for Higher Education".

http://web.fu-berlin.de/elc/portfolio/en.html (Retrieved February 2007). 

5. Little, D. (2005). The Common European Framework and the European Language Portfolio: Involving Learners and Their Judgements in the Assessment process. Language Testing, 22/2: 321-336. http://ltj.sagepub.com (Retrieved March 2007).

6. McCabe, M. (2004). "Student Portfolios: Building Fluency". http://www.ed.gov/pubs/  (Retrieved March 2007).   

7. Coombe, C. & Barlow, L. (2004). "The Reflective Portfolio: Two Case Studies from the United Arab Emirates". English Teaching Forum. 42/1: 18-23.

8. Beckett, G. H., and Slater, T. (2005). The Project Framework: a Tool for Language, Content, and Skills Integration. ELT Journal 39/2: 108-116.

9. Gomer, E. (2001). "What Are the Challenges of Assessment Portfolios that Include English Language Learners?" http://www.ericdigests.org/2001-3/large.htm (Retrieved February 2007). 

10. Nunan, D. (1988). The Learner-Centered Curriculum. Cambridge: Cambridge University Press. 

11. Spratt, M. (1999). "How good are we at knowing what learners like?" System 27/2:141-155. 

12. McDonough, J. (2002). "The Teacher as Language Learner: Worlds of Difference?" ELT Journal 56/4: 404-411. 

13. Dornyei, Z. (2003). Questionnaires in Second Language Research. Lawrence Erlbaum   Associates, Inc., Publishers. New Jersey. USA. 

Appendix 1. Survey: Students' Attitudes to the Utility of Classroom Activities.

Rate the following statements according to the scale by writing the appropriate number:

1 - strongly disagree, 2 disagree, 3 - not sure, 4 - agree, 5 - strongly agree.

1)  Writing definitions of professional vocabulary terms is beneficial for linguistic development.  

2)  Writing summaries of professional texts is useful for improving writing skills. 

3)  Doing creative tasks is useful for improving language skills.

4) Classroom discussions on various topics are useful for linguistic development.  

5) Preparing PowerPoint Presentations and delivering them in front of the audience is beneficial for developing speaking skills.  

6)  Formal testing of professional vocabulary is useful for learning English.

Appendix 2. Learners Reflections on Class Tasks.  Students identity has not been revealed for the sake of anonymity.

Student 1. Making portfolio contributions was easy because I knew exactly what and how I should do them. I think it is useful to keep the contributions in a dossier because the portfolio content shows what I have done during the course and how I progressed.

Student 2. All materials that we are collecting in portfolio helps us to observe our progress or regress. I can evaluate my work of several months and decide if I make enough efforts to improve my language skills. I am pleased with my progress I work hard.

Student 3. Listening activities are the hardest tasks for me. I want an absolute silence in class while listening, classmates' remarks distract me and I find it difficult to do my best. In speaking activities, I have some ideas, but need time to phrase them. Somebody else speaks out before me it is frustrating.

Student 4. Creative tasks for me are unclear and sometimes strange. Even if I can find some information, it is hard to say what the key meaning is. Moreover, I want to say something interesting, but fail to do it.

Student 5. Grammar and listening were my weak spots. Now I feel I've improved them. I succeeded because I liked what and how we were learning. I think that writing essays was good for me. I was able to express my own viewpoints on subjects of interest in my own way. That is why I succeeded in doing such tasks.

Student 6. I like computer activities because we can produce so ething unusual, different from what we've done before and make our day special. Power Point Presentations help to present information in an interesting way and attract attention of other students, so I enjoy making Power Point Presentations and I have improved my skills of presenting, which will be of great benefit in my professional life.

Student 7. Portfolios were very useful to me. I have improved my writing & reading skills, built up vocabulary; writing definitions and vocabulary tests helped me to remember new terminology. I learned to use different dictionaries, on paper and online. Computer tasks were the most interesting, entertaining and exciting. They helped me to improve my speaking & listening skills.

Student 8. I learned to use different dictionaries, on paper and online. Computer tasks were the most interesting, entertaining and exciting. They helped me to improve my speaking and listening skills.

Student 9. Discussions are one of the most important and interesting tasks in English classes. It is the only way I can improve my speaking skills, express my own opinion & ideas about the object of discussion, and find a solution. Creation of presentations has a lot of advantages. I can use visual aids to say what I want, and it is very exciting.

Student 10. I do not like making PPPs because it is not interesting. For me, it is a very difficult task. And I am afraid to talk in front of the audience. I did not like writing summaries, it was a new task, and I have never done it before. I improve my grammar and writing skills as a result of this training.

Student 11. Listening to authentic English is still the most difficult thing for me, but I've bettered my listening skills and learned the strategies of listening. The main reason for my difficulty is that I mentally translate while I listen to recordings and miss further information.