Can You Only Diagnose What You Know? The Relation Between Teachers’ Self-Regulation of Learning Concepts and Their Assessment of Students’ Self-Regulation

Cognition, Psychology 3 Minutes

You’ve often heard from me about the problem that I have with using both self-report as a measure when doing research as well as using teacher-report as a measure. My reasons are simple, namely that such reports are subjective and not objective and that they are usually neither valid nor reliable. These two problems are compounded by the results of this study, namely that a person can only make a judgement if they really understand the concept. This, for example, is a prominent problem with the most often used measure of cognitive load; the one item scale by Paas and colleagues. The learner is often asked to judge how much mental effort a task required. Unfortunately, most people (1) have no basis with which they can judge their mental effort as they do for physical effort (e.g., whether or not they are out of breath, are sweating, have an increased heart rate, feel their arms or legs getting tired or cramping) and (2) confuse how much mental effort a task required with how difficult the task was, how much trouble they had when carrying out the task, and so forth.

The authors of this paper (Charlotte Dignath and Lara Sprenger) encountered this problem with respect to teacher assessment of self-regulated learning in their students. In the article ‘Can you only diagnose what you know? The relation between teachers’ self-regulation of learning concepts and their assessment of students’ self-regulation’ they conclude:

We conducted an examination of educators’ conceptualization of SRL, identified three patterns to classify these conceptualizations, and investigated how such conceptualization is associated with teachers’ assessment of SRL. We found that, in particular, teachers who conceptualize SRL as student autonomy and self-directedness might be at risk for using cues that are not diagnostic of SRL when attempting to identify their students’ self-regulation skills. This raises questions about the impact that teachers’ conceptualization of SRL has on their assessment accuracy for SRL, which might, in turn, affect teachers’ adaptive teaching and promoting SRL in the classroom.

The answer to the question is the article’s title is: No

Here the abstract:

Self-regulation of learning (SRL) positively affects achievement and motivation. Therefore, teachers are supposed to foster students’ SRL by providing them with strategies. However, two preconditions have to be met: teachers need to diagnose their students’ SRL to take instructional decisions about promoting SRL. To this end, teachers need knowledge about SRL to know what to diagnose. Only little research has investigated teachers’ knowledge about SRL and its assessment yet. Thus, the aim of this study was to identify teachers’ conceptions about SRL, to investigate their ideas about how to diagnose their students’ SRL, and to test relationships between both. To this end, we developed two systematic coding schemes to analyze the conceptions about SRL and the ideas about assessing SRL in the classroom among a sample of 205 teachers. The coding schemes for teachers’ open answers were developed based on models about SRL and were extended by deriving codes from the empirical data and produced satisfactory interrater reliability (conceptions about SRL: κ = 0.85, SE = 0.03; ideas about assessing SRL: κ = 0.63, SE = 0.05). The results showed that many teachers did not refer to any regulation procedure at all and described SRL mainly as student autonomy and self-directedness. Only few teachers had a comprehensive conception of the entire SRL cycle. We identified three patterns of teachers’ conceptualizations of SRL: a motivation-oriented, an autonomy-oriented, and a regulation-oriented conceptualization of SRL. Regarding teachers’ ideas about assessing their students’ SRL, teachers mainly focused on cues that are not diagnostic of SRL. Yet, many teachers knew about portfolios to register SRL among students. Finally, our results suggest that, partly, teachers’ ideas about assessing SRL varied as a function of their SRL concept: teachers with an autonomy-oriented conceptualization of SRL were more likely to use cues that are not diagnostic of SRL, such as unsystematic observation or off-task behavior. The results provide insights into teachers’ conceptions of SRL and of its assessment. Implications for future research in the field of SRL will be drawn, in particular about how to support teachers in diagnosing and fostering SR among their students.

Source: Can You Only Diagnose What You Know? The Relation Between Teachers’ Self-Regulation of Learning Concepts and Their Assessment of Students’ Self-Regulation

Gepubliceerd door Paul A. Kirschner

Nederlands: Paul A. Kirschner (1951) is Emeritus hoogleraar Onderwijspsychologie aan de Open Universiteit (Nederland), eredoctor (doctor honoris causa) aan Oulu University (Finland), Gastprofessor aan de Thomas More Hogeschool (België) en eigenaar van kirschner-ED. Hij was eerder Universiteitshoogleraar en hoogleraar Onderwijspsychologie aan de Open Universiteit, Visiting Professor Onderwijs met een leerstoel in Leren en Interactie in de Lerarenopleiding aan Oulu University, hoogleraar Onderwijswetenschappen aan de Universiteit Utrecht, hoogleraar Contact- en Afstandsonderwijs aan de Universiteit Maastricht en Visiting Professor aan de Open University of Catalonia (Spain). Hij is een internationaal erkende expert op zijn gebied met meer dan 350 wetenschappelijke publicaties. Hij heeft zitting gehad in de Onderwijsraad in de periode 2000-2004 en de Wetenschappelijk Technische Raad van SURF van 2009-2019. Hij is Fellow van de American Educational Research Association (NB de eerste Europeaan aan wie deze eer werd toegekend), de International Society of the Learning Sciences en de Netherlands Institute for Advanced Study in the Humanities and Social Science of the Royal Dutch Academy of Sciences (NIAS-KNAW). Hij was president van de International Society of the Learning Sciences in de periode 2010-2011. Hij is hoofdredacteur van de Journal of Computer Assisted Learning en commissioning editor van Computers in Human Behavior. Hij heeft veel boeken (mede)geschreven, o.a. Ten steps to complex learning (Routledge/Erlbaum), Op de Schouders van Reuzen en Wijze Lessen: Twaalf Bouwstenen voor Effectieve Didactiek (beiden gratis verkrijgbaar op het web), twee boeken over mythes in het onderwijs Jongens zijn Slimmer dan Meisjes XL en Juffen zijn Toffer dan Meesters (beiden ook in het Engels verschenen), Evidence Informed Learning Design, and How Learning Happens: Seminal Works in Educational Psychology and What They Mean in Practice. Hij schrift ook regelmatig voor Didactief (de kolom KirschnerKiest over wat docenten kunnen met wetenschappelijke resultaten). Hij wordt gezien als expert op veel gebieden en vooral het ontwerpen van effectief, efficiënt en bevredigend onderwijs, computerondersteund samenwerkend leren (CSCL), mediagebruik in het onderwijs en het verwerven van complex cognitieve vaardigheden. English: Paul A. Kirschner, dr.h.c. (1951) is Emeritus Professor Educational Psychology at the Open University of the Netherlands, Guest Professor at the Thomas More University of Applied Science in Mechelen, Belgium, Honorary Doctor (Doctor Honoris Causa) at the University of Oulu, Finland, and owner of kirschner-ED which carries out educational consultancy, masterclasses for teachers, school heads and educational policy makers, and keynotes/presentations at conferences and other educational get-togethers. He is a Research Fellow of the American Educational Research Association, the International Society of the Learning Sciences, and the Netherlands Institute for Advanced Study in the Humanities and Social Science. He is a past President (2010-2011) of the International Society of the Learning Sciences and former member of the Dutch Educational Council and the Scientific Technical Council of the Foundation for University Computing Facilities (SURF WTR). He is chief editor of Journal of Computer Assisted Learning and commissioning editor of Computers in Human Behavior. He has also published more than 350 scientific articles as well as many popular articles for teacher journals. As for books, he is co-author of How Learning Happens: Seminal Works in Educational Psychology, Evidence Informed Learning Design, Urban Myths about Learning and Education and More Urban Myths about Learning and Education as well as of the highly successful book Ten Steps to Complex Learning, and editor of two other books (Visualizing Argumentation and What we know about CSCL). He is seen as an expert in many areas and in particular the design of effective, efficient and enjoyable education, computer-supported collaborative learning (CSCL), media use in education, and the acquisition of complex cognitive skills.

Gepubliceerd

Plaats een reactie