Role of Grammar Errors in Assessing English Writing: Insights from Austrian Educational Standards
Explore the significance of grammar errors in evaluating English writing through the Austrian Educational Standards Baseline Test. Analysis of error frequencies and correlations with dimensions like Task Achievement, Coherence, Grammar, and Vocabulary reveals the critical role of grammar in assessing writing performances.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
The Role of error in assessing English Writing in the Austrian Educational Standards Baseline Test Florian Pibal, G nther Sigott & Hermann Cesnik Identifying errors: error analysis 100 writing performances E8 Rated on four dimensions: Task Achievement, Coherence and Cohesion, Grammar, Vocabulary. Fair measures resulting from MFRA Correlations of error counts with the ratings on the four dimensions Language Testing in Austria - 1 - Pibal, Sigott, Cesnik
Broad category Subcategory Text extract Comment Grammar error Subject-verb agreement Example Example The clothes of Hitman [looks] very cool. I promise I [bring] it back tomorrow. Use of present tense to refer to the Addition of 3rd-person s Time, tense, and aspect future. Use of general interrogative structure for modal verbs. Incorrect use of personal pronouns according to German gender. Modal verbs [Do you can] swim? Gender My watch is broken. Can you mend [her]? we missed the bus, so we had to go (German 'gehen' = 'go' or 'walk' Lexical error False friends according to context) Use of artists to express actors . Lexical choice Another reason is that the [artists] are very good. Collocations (incl. idioms) [At] first we went to the dinosaurs. Use of at first to express first . Word formation the [head song] in this album. head song to mean main song . Discourse-level error Coherence Lack of connectors / coherence devices where there should be some in order to maintain meaning and logical progression. Repetition of information impacting readability Use of wrong coherence device(s) or readability breakdown because information presented does not fit together Temporal, relational, sequential misordering of information Omission of form of address in letter Pragmatics Addition of text-type inappropriate formulae / headings / closings, etc. god instead of good . Orthography & punctuation error Spelling the artists were very [god]. [schopping] Use of German sch instead of English sh . Capitalization for nouns. Capitalization I bought my [Car] from a [Friend] Punctuation Please write me back soon [ ] I miss you! Omission of comma or full stop. Language Testing in Austria - 2 - Pibal, Sigott, Cesnik
Fig. 3: Error frequencies (linguistic categories) Tab. 5: Pearson r between Error Ratios and Fair Measures. All correlations sig. at p=0.01 (N=100). COHERENCE & COHESION Fair Measure -.719 -.393 -.471 -.588 -.370 -.735 TOTAL GRAMMAR Fair Measure -.766 -.346 -.523 -.576 -.431 -.770 VOCABULARY Fair Measure -.738 -.399 -.525 -.614 -.398 -.773 TASK ACHIEVEMENT Fair Measure -.618 -.359 -.423 -.612 -.502 -.672 Fair Measure -.766 -.405 -.527 -.652 -.462 -.799 Grammar Error Ratio Lexical Error Ratio Orth. & Punct. Error Ratio D-Coherence Error Ratio D-Pragmatics Error Ratio Total Error Ratio Language Testing in Austria - 3 - Pibal, Sigott, Cesnik
- 4 - Pibal, Sigott, Cesnik Language Testing in Austria
- 5 - Pibal, Sigott, Cesnik Language Testing in Austria
Conclusions Grammar plays the most important role in the assessment of writing in the E8 Test. Grammar errors are also the most frequent errors. Therefore, an emphasis on grammar in the teaching of writing is justified. After Grammar Errors, Coherence Errors are the second strongest predictor of the Task Achievement ratings. Task Achievement is primarily seen as a matter of coherence. Consequently, the conceptual distinction between coherence, cohesion and task achievement needs to be sharpened, or if this turns out to be impossible, abandoned. The VOCABULARY rating dimension needs to be examined carefully. It does not seem to be sensitive to lexical errors per se but rather to issues to do with grammar and coherence. Impossible to distinguish between grammar and vocabulary in a principled way (cf. Alderson and Kremmel 2013)? Raters may be particularly sensitive to positive features in the VOCABULARY rating. Orthography & Punctuation Errors influence the ratings on all four dimensions, but always in a subsidiary role. Qualitative research into raters decision-making behaviour is needed to supplement the results. Language Testing in Austria - 6 - Pibal, Sigott, Cesnik
