Assessment of the writing task in advanced-level language tests
The assessment of writing tasks in the second national language and foreign language tests of the Matriculation Examination is of considerable interest to teachers and candidates. This is understandable, not least because the writing task is the only extended text production part in the test and accounts for approximately one third of the total score. At times, differences between the preliminary and final assessment also raise questions. Another issue that repeatedly emerges in discussions with teachers concerns which assessment criteria are emphasised. This article examines the assessment of writing tasks in tests based on the advanced syllabus, focusing in particular on communication and compliance with the task instructions.
In the advanced syllabus writing task, candidates write an extended text based on a task of their choice, which determines both the topic and the approach. In addition to specifying the topic, the task instructions may also describe the intended audience, purpose and context of the text, as well as the role in which the candidate is expected to write. Candidates are required to take these aspects into account in their response. Typical tasks involve, for example, expressing an opinion or presenting arguments for or against a particular issue, or providing information about or describing a given topic. Candidates choose one of four task options. The required length of the response is defined by both a minimum and a maximum word limit.
The writing task assesses the candidate’s ability to produce, independently and on the basis of the given task instructions, a text that is appropriate to the specified communicative situation. The most important aspect of the response to be assessed is communication, sometimes also referred to as communicative effectiveness or simply whether the message is conveyed successfully. Communication concerns the extent to which the response fulfils the task instructions. It is shaped by several factors, including the content and structure of the text as well as the writer’s linguistic expression. The emphasis on communication is linked to an action-oriented view of language proficiency, which is based on the Common European Framework of Reference for Languages (CEFR) and the National Core Curriculum for General Upper Secondary Education.
The assessment criteria for writing performances are outlined at a broad level in the test-specific regulations and guidelines and in greater detail in the evaluation criteria for the advanced syllabus writing task. The columns of the table describe the aspects taken into account in the assessment. From left to right, these are: Communication (the primary criterion); Content of text and/or dealing with the topic; Text structure associated with text type; Linguistic range; and Grammatical correctness and severity of errors.
The rows of the table describe the typical characteristics of responses at different levels using verbal descriptors and are divided into ten categories (score categories 99, 94, 84, 74, 64, 54, 44, 28, 14 and 0). For example, in a response awarded 99 points, “the message is communicated extremely clearly, naturally, fluently and with finer shades of meaning.” At this level, the range of vocabulary and expressions is also typically broad, idiomatic and highly appropriate to the context.
Without examining each criterion in detail, it is worth considering Communication, the primary assessment criterion. According to the table, at the 64-point level, “the message is not conveyed clearly at all times”, but it is nevertheless conveyed. At the 54-point level, by contrast, parts of the message are no longer conveyed at all. At the 44-point level, the message is no longer conveyed in many places. There is therefore, in a sense, an imaginary boundary between the 64-point and 54-point categories: above this boundary, the message is conveyed, whereas below it, the message is no longer fully conveyed. The higher score categories of 64, 74, 84, 94 and 99 points may, in turn, be seen as distinguishing how successfully the message required by the task is conveyed.
Communication concerns how successfully the text fulfils the communicative task described in the task instructions. This means that the text succeeds in terms of both its content and its treatment of the topic, that it has an effective structure appropriate to the text type, and that the language used is suitable for the communicative situation and sufficiently broad and accurate. The concept of conveying the message may also be understood in a narrower sense, referring to individual passages in the text that are not comprehensible. An example can be found in the Communication criterion of the evaluation criteria at the 54-point level, where “the message is not conveyed clearly and at times not at all.” Breakdowns in communication in individual parts of the text naturally affect the extent to which the text, as a whole, is successful and effective in accomplishing its communicative purpose.
In all advanced syllabus tests, the writing tasks are assessed according to the same criteria. It should be noted, however, that the level of proficiency defined as good performance in the National Core Curriculum for General Upper Secondary Education varies between languages and syllabi. For example, in advanced English (A syllabus), the target level for written production is B2.1, whereas in other advanced language syllabi the target level is B1.2. There are also differences in the objectives for interaction skills. The assessment criteria must therefore be interpreted in relation to the required proficiency level. In that case, the description “the message is communicated with finer shades of meaning” in the 99-point category must necessarily mean different things at different target proficiency levels.
The assessment of writing tasks is not always straightforward. During the preliminary assessment, questions may arise, for example, about how great a quantity is implied by the description “some linguistic errors” in the 74-point category of the evaluation criteria, or how this differs from the description “several linguistic errors” in the 64-point category. Questions may also be raised about the point at which errors become so numerous that they begin to make it difficult to convey the message. Another frequently asked question is whether there is a list of particularly serious error types or especially important linguistic structures that candidates are expected to master.
There are no precise, universally applicable answers to these questions. The assessment of texts should not be based on counting errors, error types or the number of structures used. Rather, it should be based primarily on an evaluation of the extent to which the message defined in the task instructions is conveyed. This overall judgement can then be compared with the descriptors in the evaluation criteria, which serve as supporting criteria and help determine the final score awarded to the performance. There are no “official” lists issued by the Matriculation Examination Board of particularly important words or structures. Nor are there any authorised learning materials on which the tasks are based. Instead, the target proficiency level specified for each language and syllabus in the National Core Curriculum for General Upper Secondary Education, that is, the level defined as good performance, determines both the level of the writing task and the level of competence at which a learner is expected to perform well in it.
In the assessment of writing tasks, it is essential that the candidate’s text conveys the specific message required in the task instructions. If the text does not correspond to the task instructions, it cannot convey the intended message. Minor deviations from the instructions, such as omitting a required salutation, closing formula or title, may lead to a deduction of fewer than ten points, as determined at the censors’ meeting. More substantial deviations are assessed primarily on the basis of the criteria Communication, Content of text and/or dealing with the topic, and Text structure associated with text type. If compliance with the task instructions is neglected or plays only a minor role in the response, even a linguistically accomplished text cannot achieve the highest scores.
Compliance with the task instructions guides candidates to demonstrate their ability to produce a text related to specific communicative situations and subject areas targeted by the assessment, using language that is appropriate to the context. The task instructions are confidential test materials. They are designed to be sufficiently focused in content to ensure that each task is unique and that the response is produced independently during the test.
It is also important to consider the structure of the text as a whole. Vocabulary and grammatical knowledge alone are not sufficient if the text is not organised into a logically progressing, coherent whole that communicates clearly and is easy for the reader to follow. These two aspects are central to conveying the message.