Feedback as Dialogue: Exploring the Links between Formative Assessment. and Social Software in Distance Learning

Learning, Media and Technology (2009). In Press. Feedback as Dialogue: Exploring the Links between Formative Assessment and Social Software in Distan...
Author: Jonas Small
5 downloads 0 Views 116KB Size
Learning, Media and Technology (2009). In Press.

Feedback as Dialogue: Exploring the Links between Formative Assessment and Social Software in Distance Learning

Stylianos Hatzipanagos & Steven Warburton King's College London, UK

Dr. Stylianos Hatzipanagos, King’s Learning Institute, KING’S COLLEGE LONDON, FranklinWilkins Building, Waterloo Road, London, SE1 9NH, Tel.: +44 (0)20 7848 3716, Fax: +44 (0)20 7848 3253, E-mail: [email protected]

Dr. Steven Warburton, School of Law, KING’S COLLEGE LONDON, Tel.: +44 (0)20 7848 2457, Fax: +44 (0)20 7848 2456, E-mail: [email protected]

Learning, Media and Technology (2009). In Press.

Feedback as Dialogue: Exploring the Links between Formative Assessment and Social Software in Distance Learning

Abstract: The paper explores the relationship between formative assessment and social software. Formative assessment practices though beneficial for student learning become marginalised and constrained in open and distance learning environments in higher education. Feedback is a key factor in formative assessment and learners can benefit from the deployment of emerging technologies and the opportunities for participation and dialogue afforded by social software. This paper explores and proposes a conceptual framework for this relationship. The claim is that the social dimensions of emerging technologies allow for formative assessment practices to be re-invented or at the very least facilitated by essentially participative and student focussed interventions. A comparison of these technologies against formative assessment mechanisms identifies the types of processes that these new tools might best support to encourage effective feedback approaches that both empower the learner and enhance their learning experience.

Learning, Media and Technology (2009). In Press.

Introduction There is, according to Rust (2002), a greater variety of assessment practices being used in universities today compared with 10 or 15 years ago, many of which could be justified with reference to what the research literature tells us about the impact of assessment on student learning. The evidence is compelling that nature and form of assessment have a significant impact upon the student learning experience, approaches to learning, motivation, and retention rates (Ridgway, McCusker & Pead, 2004). As higher education institutions change, two basic types have emerged as dominant: Open and Distance Learning (ODL) and Resource-based learning (RBL) institutions. Here, RBL is defined as an integrated set of strategies to promote student-centred learning in a mass education context, through a combination of specially designed learning resources and interactive media and technologies (NCODE, 2004). In higher education, the nature of an institution often dictates how assessment practices have been developed. For example, ODL environments have emphasized the necessity for formative assessment practices. Distance education in general has been proactive in formative assessment practices out of the need to find ways to provide systematic feedback to students in the absence of the immediate contact and interaction that students have enjoyed with tutors in a campus setting. However in both types of environments, the impact of assessment on learning can be moderated by the use of appropriate assessment methods by teaching practitioners and practices have been supported/complemented by the use of computer assisted learning resources (GuriRosenblit 2005).

Learning, Media and Technology (2009). In Press.

Computer-assisted assessment (most commonly in the form of automated quizzes or online objective tests) has been used by academic tutors. In a virtual learning environment (VLE) they can be used to monitor student understanding, and sometimes for course evaluation purposes. Text-based discussion fora (mainly asynchronous) can also be used for self and peer assessment purposes. Computer Mediated Communication in learning environments facilitates the creation of learning communities (Hatzipanagos, 2006) by the use of email, discussion list conferencing, mobile texting, podcasting, publishing via blogs and wikis, aggregation and newsfeeds, audio, chat, instant messaging and videoconferencing. Two major characteristics of new tendencies in relation to online social tools are a shift towards ‘user-generated content’ where collaborative information is gathered and shared, and the collective intelligence of the users is harnessed (O’ Hear, 2005). In formative feedback, dialogue forms the mechanism by which the learner monitors, identifies and then is able to ‘bridge’ the gap in the learning process. Social software is also designed to engender dialogue, and yet the link between these two has not been explored. The paper explores the relationship between formative assessment and social software, i.e. software that supports group interaction (JISC InfoNet, 2006) in light of the possibilities that blogs and wikis afford.

Formative assessment

Learning, Media and Technology (2009). In Press.

All forms of assessment should primarily be concerned with providing guidance to the learner (Nicholls, 2001). However, while tutors may declare that assessments serve formative purposes, the feedback provided may not have the intended positive impact on learning (MacLellan, 2001). A naïve view of assessment is that it logically follows teaching and learning interventions and tests their outcomes (Bostock, 2004). Formative assessment aims to go beyond the mere ascription of grades associated with the more traditional, 'summative' forms, promoting feedback dialogue between the assessors and students. As studies in this area have emphasised, learners derive greater benefit from receiving feedback, in terms of motivation and learning gains, when the feedback is linked closely to student learning and learning outcomes (Black & Wiliam, 1998). This learning-based focus is increasingly viewed as an essential component of good course design since, if properly implemented, it can point to ways in which the course can be adapted and developed to enhance student learning. As Stefani & Nicol (1997) note, short taskperformance-feedback cycles can allow students and tutors to regularly share conceptions about the processes and goals of learning. Table 1 (adapted from Bull & McKenna, 2004 to accommodate our views on assessment types) presents the perceived dimensions and functions of assessment types. Comment [SH1]: Table 1

Insert Table 1 This focus on formative assessment, including such practices as feedback, self and peer assessment, has positive implications for student learning. They can be seen as part of a process, or strategy, that allows students to play a more active role in the management of their own learning (Nicol, 1997). In addition, they can play a pivotal role in helping equip the student with evaluative skills that promote the ability to self-assess and self-correct (Sadler, 1989; McConnell 2006).

Learning, Media and Technology (2009). In Press.

The view of feedback as an active, participative process, contrasts with the notion of feedback as simply being a transmissive process that involves ‘telling’ or passing on information. The latter can be perfunctory and does not assist with students’ learning (QAA, 2002). In other words, communication becomes a vital part of the feedback cycle that enables students to actively construct their own understanding of what can be complex and difficult messages to decipher (Higgins, Hartley & Skelton, 2001).

Feedback as dialogue This understanding of feedback as dialogue is fundamental to the process or 'closing the loop' in assessment (Sadler, 1989). Communication forms part of the mechanism by which the learner identifies and then bridges the gap between the current learning achievements and the goals set by the tutor. Juwah et al (2004) have developed a conceptual model of formative assessment that represents a synthesis of current thinking by key researchers, is based on the original model of self-regulated learning by Butler and Winne (1995) and includes work by Sadler (1983, 1989); Black and Wiliam (1998); Torrence and Pryor (1998) and Yorke (2003). The learner is at the centre of this model and an active participant in monitoring their performance. The seven key principles of formative assessment, identified by Juwah et. al. (2004), and the descriptors from the literature address issues such as dialogue, motivation, student participation and intrinsic/extrinsic performance indicators. The encouragement of dialogue around learning is viewed as fundamental to effective feedback practices and one that naturally becomes problematic within ODL settings.

Learning, Media and Technology (2009). In Press.

In traditional RBL courses, test-feedback cycles and peer assessment pathways have always existed - within the formal face-to-face classroom and the informal tutor-learner and peer-to-peer interaction. But within an ODL environment these are reformulated and occluded for learners whose interactions (mostly online) are understood to be constrained by both time and space.

Attributes of feedback Gibbs’ theory of feedback (2002) highlights the significance of certain attributes. These attributes are related to both the quantity (frequency) and the quality of feedback, as it should focus on learning rather than on marks or the students themselves. Finally, feedback should be timely in that it should be provided quickly enough to be useful to students. By investigating the broad range of literature in the area of formative assessment, it is possible to discern other attributes that can be summarised under a number of key processes or dimensions (Table 2). Comment [SH2]: Table 2

Insert Table 2 It is particularly worth noting that in formal learning situations, knowledge is produced in the context of power (Giroux, 1992). As McConnell (1999) points out, tutors strive to engender a ‘learning relationship’, yet they are nevertheless constantly engaged in a power relationship. Learners should be encouraged to make decisions about their own learning including how they are assessed (Leach, Neutze & Zepke, 2001). One of the questions of this paper is to ask whether emerging technologies offer the potential to promote a wider more democratic approach to teaching and learning. Particularly, to investigate how they might play an important role in

Learning, Media and Technology (2009). In Press.

repositioning student/tutor barriers and allowing students to take ownership of the assessment process. Assessment activities within open and distance education are used to improve dialogue, interaction and collaborative work and consequently improve student learning. Because of the restrictions that ODL environments impose, formative assessment practices can benefit from the use of new and emerging technologies through the increased opportunities for participation and dialogue that tools such as social software can offer.

Social software and e-learning Web 2.0 technologies, particulary those that might be characterised as social software, are becoming increasingly pervasive in e-learning settings, as they are seen to offer benefits relating to widening access, opportunities for personal ownership and collaboration. Anderson (2006) describes a number of dimensions of social software that are likely to support emerging user needs such as the freedom to negotiate the time and place of learning, the content, the pace and the type of relationship with other learners and the teacher. Social software tools provide a set of low threshold mechanisms for users to engage in self publishing, sharing and collaborative activity. These second generation technologies move beyond what can be described as more traditional or first generation social tools such as discussion fora, discussion lists and internet relay chat (a form of real-time synchronous conferencing), by providing a host of participative and user-driven content production opportunities. Social software can provide a dynamic environment for communication and

Learning, Media and Technology (2009). In Press.

collaboration that includes social networking spaces, social bookmarking and citation sites, user recommendation systems, and collective knowledge production via wikis and blogging communities (Kervin, Mantei and Herrington 2009, Wheeler 2009). The first use of the term social software has been attributed to Shirky (2003) and has since been subject to a broad number of definitions yet one can effectively summarise the key attributes of social software as providing support for: •

user driven content production



conversational interaction between people or groups;



social feedback;



social networks.

Social software tools and services are largely built on open standards, are freely available and according to Anderson (2006) can provide encouragement and support for individuals to learn together while retaining individual control over their time, space, presence, activity, identity and relationship. All these attributes are pertinent to ODL environments. Yet despite the often celebratory rhetoric surrounding the use of social software, it is in reality not free from its own sets of complexities. As Crook et al. (2008) indicate, these can include significant demands on new teacher skills as well as student digital literacy and competencies. In this paper, we focus on blogs and wikis as the two most prominent examples of the educational use of social software. Both blogs and wikis have become increasingly adopted by institutions and the growing body of evidence on deployments in learning and teaching settings makes them prime for analysis (Kerawalla et al. 2009, Wheeler 2009), particularly when compared to other high profile technologies such as social network services, exemplified by

Learning, Media and Technology (2009). In Press.

Facebook, Myspace and Bebo, where research evidence on institutional engagement is as yet sparse.

Blogs A blog is a personal publishing tool that functions as an online journal. Blogs allow individuals to create posts or entries in reverse chronological order on particular topics, and to archive these posts over time. Each post is represented by a web page with a unique URL, and can be tagged with keywords that allow content to be categorised and referenced by others. In learning and teaching settings, students can use blogs to participate in knowledge sharing, keep a chronological record of their own thinking over time and can facilitate critical feedback by letting readers add comments (Kervin, Mantei and Herrington 2009). The act of writing posts or blogging can be interpreted in terms of social action (Miller, 1984), where blogs represent a site of identity performance and persistence, dialogue, creativity and community formation, via the use of blogrolls (a collection or list of links to other blogs and websites featured on blogs) and RSS newsreader aggregators (news updates from various websites). For students a blog can be used as a current record of their learning: a place to pose questions, publish work in progress and provide links to relevant web services. The emergence of this new ‘ecology of participation’ continues to blur the boundaries between formal and informal learning spaces (Downes 2005; Warburton 2006) and challenges the dominant model of hierarchical, tutor-centred education to embrace an ideology of openness, dialogue, ownership and democracy. Blogs have been introduced into a variety of educational settings though the translation of what represents a

Learning, Media and Technology (2009). In Press.

fundamentally informal tool into formal settings has not always been successful (Beuschel 2009).

Wikis

A wiki (from the Hawaiian word for ‘quick’ or to ‘hurry’) is a group publishing tool in the form of a website whose content can be readily edited by anyone that has access to it. Perhaps the best known example on the internet today is Wikipedia (2008) – ‘The Free Encyclopedia’ that has become an artefact of study as a prime example of the phenomenom of collective intelligence. Wikis have recently come to prominence within the educational sphere for collaborative group activities providing a content production and sharing tool that is both democratic (as anyone can edit a wiki page) and simple to use. Klobas, (2006) describes wikis as social information spaces that are modeled on shared ownership and investment in a community of shared goals and aspirations. Wikis provide a range of features beyond collaborative page editing that include the ability to maintain a recorded history of content changes that can be used to return to previous page incarnations alongside a discussion sheet that is attached to each page. The openness and visilbility of wikis has created new and interesting possibilties for group work and knowledge creation with many wikis being used as research community tools, sites for student shared knowledge production and spaces for emerging communties of practice (Leuf and Cunningham, 2001, Wheeler 2009).

Learning, Media and Technology (2009). In Press.

Mapping feedback descriptors and processes This section analyses the dimensions of the two forms of social software and a selection of more traditional, first generation asynchronous ODL technologies and maps these against the key processes identified from the descriptors of formative assessment described earlier in this paper. We selected a range of representative first generation technologies based on their suitability to provide opportunities for participation, dialogue and feedback: •

Electronic quiz: the most commonly used form of quizzes is multiple choice, however, other formats include short answer, true/false, drag and drop, numerical and matching question quizzes. Feedback can vary, however it is generally automatic and occasionally adaptive to the needs of the learners, giving hints and tips for further improvement or model answers.



Email: one of the earliest and still widely used electronic media forms for asynchronous textual communication across the Internet.



Discussion list or mailing list: an asynchronous discussion, where subscribers post and receive contributions to a discussion in the form of emails.



Discussion forum: an application that is accessed via a web browser, and may be part of a Virtual Learning Environment (VLE), that provides a virtual space for holding discussions and posting user-generated content.

For each of the descriptors we have used indicators to mark those tools that have an impact and added comments to describe how first generation traditional technologies (email, discussion lists,

Learning, Media and Technology (2009). In Press.

fora and quizzes) that can be used for formative assessment purposes and social software (blogs and wikis) differ in their relation to these formative assessment descriptors (Table 3).

Comment [SH3]: Table 3 (a to d)

Insert Tables 3 This mapping exercise provides a valuable starting point in uncovering the underlying features of social software that impact on the possibilities for appropriate and purposeful formative assessment activities.

Discussion Participatory communication-based technologies offer opportunities for dialogue in relation to feedback and self and peer assessment. Student autonomy coupled with increased visibility and the expanded opportunities for sharing and reflection are vital aspects of closing the loop within the constrained environment of ODL. In this preliminary analysis of first and second generation web technologies we suggest that first generation synchronous or asynchronous technologies (such as quizzes, email, discussion lists and fora) appear to suffer from limitations in value when mapped across the dimensions of formative assessment. Across these technologies we find recurring weaknesses that include scalability issues, low visibility and limited support for autonomy and ownership. The variability of these tools in providing system-wide value to closing the feedback loop makes any single tool choice almost impossible. By contrast, social software as exemplified by blogs and wikis, appears to have much potential in addressing the formative assessment needs identified in this paper. The analysis and commentary

Learning, Media and Technology (2009). In Press.

provided in Table 3 shows that these tools have a potential positive impact in the following areas, summarised here: (a) Power (autonomy and ownership): Blogs provide a high degree of personalisation, visibility and persistence over time and offer potential synergies between personal and community based learning. (b) Dialogue: the nature of dialogue changes in blogs and wikis from formal to informal and from closed to open conversations. Blogs add value to a learning group as learners may benefit from observing meaningful communication between peers and tutors. Wikis offer the possibility of sharing consensus on assessment criteria and building these criteria in a cooperative manner. (c) Timeliness: the informal and open nature of social software, exemplified by blogs and wikis, allows for the uncovering of the learning processes and in so doing provide opportunities for interventions that are both timely and dialogic in form. (d) Visibility: Individually owned social tools such as blogs may motivate learners to respond to feedback due to the increased visibility of the medium. Individually targeted feedback in blogs is also more likely to engage students in the task-performance-feedback cycle and motivate. Blogs can be deployed as a learning journal or adapted to work as a personal e-portfolio. The high degree of visibility allows blogs and wikis to be records of achievement. (e) Reflection: Blogs are vehicles for reflexivity and for raising self-awareness. The essentially informal tone and diary-style of a blog is an ideal setting for stimulating reflection and addressing meta-cognitive processes. A blogging tool offers an open and visible (often public) reflective space where comments from both peers and tutors can be

Learning, Media and Technology (2009). In Press.

added with ease and immediacy. Wikis can be used to share or distribute exemplars and observe student work over time. Wikis and blogs, as open systems, raise issues around privacy and control. In addition, they score moderately as tools that can improve the quality of feedback, as the attributes of effective feedback are mainly related to the tutors’ role rather than to a particular technology.

Conclusion Engagement with social software in Open and Distance Learning can provide the support mechanisms and tools that are required for effective formative assessment. Within such environments, it can have a positive impact on students by encouraging ownership of learning activities and promoting a rich dialogue in relation to feedback and peer and self assessment activities which, by their nature, place the student at the centre of the educational process as an active participant in constructing knowledge. In addition, engagement can benefit ODL teaching practitioners by encouraging reflection on assessment practices. A comparison of current technology-supported, formative assessment mechanisms has identified the types of processes that these new tools might best support to further encourage effective feedback approaches that both empower and enhance the learning experience for the distance learner. Although this paper has addressed only two forms of social software, blogs and wikis, in an institutional ODL landscape, it will be interesting to see whether social network services and innovative tools of communication and interaction will offer further pathways for supporting student learning.

Learning, Media and Technology (2009). In Press.

References Anderson, T. (2006) ‘Higher education evolution: Individual freedom afforded by educational social software.’ In Beaudoin, M. (Ed.), Perspectives on the Future of Higher Education in the Digital Age. (pp. 77-90). New York, USA: Nova Science Publishers. Beuschel, W. (2009). Weblogs in Higher education. In Handbook of Research on Social Software and Developing Community Ontologies. Eds. Hatzipanagos, S. and Warburton, S. (2009) in press. Pennsylvania: IGI Global. Black, P. and Wiliam D. (1998) Assessment and Classroom Learning, Assessment in Education: Principles, Policy and Practice 5(1): 7-73. Black, P., Harrison, C., Lee, C., Marshall, B. and William, D. (2003) Assessment for learning, Berkshire, UK: Open University Press. Bostock, S. (2004). Motivation and electronic assessment. In A. Jones and S. Alexander (Eds), Effective Learning in Teaching and Computing (pp. 86-99. London and New York: Routledge Falmer:. Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London, Kogan Page. Bull J. and McKenna C. (2004) CAA Blueprint for Computer-Assisted Assessment, London, Routledge Falmer. Butler, D.L. and Winne, P.H. (1995) ‘Feedback and self-regulated learning: a theoretical synthesis’, Review of Educational Research 65 (3): 245-281.

Learning, Media and Technology (2009). In Press.

Conole, G., de Laatb, M. Dillon, T., Darby, J. (2008). ‘Disruptive technologies’, ‘pedagogical innovation’: What’s new: Findings from an in-depth study of students’ use and perception of technology, Computers & Education, 50(2): 511-524. Crook, C. Cummings, Fisher, J. Graber, T. Harrison, C., Lewin, C., Logan, K., Luckin, R. London Oliver, M. Sharples, M. Web 2.0 technologies for learning: The current landscape – opportunities, challenges and tensions. Becta Research Report. Available HTTP: http://partners.becta.org.uk/uploaddir/downloads/page_documents/research/web2_technologies_learning.pdf (17 December 2008).

Downes,

S.

(2005)

E-learning

2.0

ACM

eLearn

Magazine.

Available

http://www.elearnmag.org/subpage.cfm?section=articles&article=29-1

(18

HTTP:

November

2008). Ducate, L.C., Lomicka, L. L. (2008) Adventures in the Blogosphere: From Blog Readers to Blog Writers, Computer Assisted Language Learning, 21(1): 9-28. Efimova, L. and Fiedler, S. (2004) Learning webs: learning in weblog networks. In Isaias, P. and Nunes, M.B. (eds.) Proceeding of the IADIS International Conference on Web Based Communities 2004 (pp 490-494) Lisbon Portugal, IADIS Press EDOL Report (2008). Learning to document Learning Online - an introduction to edoL. Available HTTP: http://wcmprod1.ucalgary.ca/fieldnotes/files/fieldnotes/fieldnotes.pdf p.8-9 (17 December 2008). Farabaugh R,. (2007) ‘The Isle Is Full of Noises": Using Wiki Software to Establish a Discourse Community in a Shakespeare Classroom’ Language Awareness, 16(1) p41-56

Learning, Media and Technology (2009). In Press.

Gibbs, G. (2002, August) Evaluating the impact of formative assessment on student learning behaviour. Paper presented at the European Association for Learning and Instruction Assessment Conference, Newcastle, UK. Guri-Rosenblit, S. (2005) ‘Distance education’ and ‘e-learning’: Not the same thing. Higher Education, 49 p467-493. Gibbs, G. and Simpson, C. (2004) Conditions under which assessment supports students’learning. Learning and Teaching in higher Education, Issue 1, 2004-05. Giroux, H. (1992). Border crossings: Cultural workers and the politics of education, New York, Routledge, Chapman, and Hall. Hammond, M. (2006), Blogging within Formal and Informal Learning Contexts: Where Are the Opportunities and Constraints? Networked Learning. Available HTTP: http://www.networkedlearningconference.org.uk/past/nlc2006/abstracts/pdfs/P03%20Ha mmond.pdf (18 November 2008). Hatzipanagos, S. (2006) ‘HOT and Flaming Spirals: Learning and Empathic Interfaces in Textbased Discussion Forum Dialogues’ European Journal of Open, Distance and eLearning 2006/I. Higgins, R., Hartley, P. and Skelton, A. (2001) ‘Getting the message across: the problem of communicating assessment feedback’ Teaching in Higher Education, 6 (2): 269-274. Huffaker, D., A. and Calvert, S., L. (2005) ‘Gender Identity, and language use in teenage blogs’ Journal of Computer Mediated Communication, 10 (2). JISC InfoNet (2006). Guide to Social Software. Available HTTP: http://www.jiscinfonet.ac.uk/infokits/social-software (18 November 2008).

Learning, Media and Technology (2009). In Press.

Juwah, C., Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D. and Smith, B. (2004) Enhancing Student Learning Through Effective Formative Feedback HEA Resource Guide. Available HTTP: http://www.heacademy.ac.uk/resources/detail/id353_effective_formative_feedback_juwa h_etal (18 November 2008). Kervin, L., Mantei, J. and Herrington, A. (2009). Blogs as a Social Networking Tool to Build Community. In Handbook of Research on Social Software and Developing Community Ontologies. Eds. Hatzipanagos, S. and Warburton, S. (2009) in press. Pennsylvania: IGI Global. Hussey, T. & Smith, P. (2003). The Uses of Learning Outcomes. Teaching in Higher Education, 8(3): 357–368. Kerawalla, L., Minocha, S., Kirkup, G. and Conole, G. (2009). Supporting Student Blogging in Higher Education. In Handbook of Research on Social Software and Developing Community Ontologies. Eds. Hatzipanagos, S. and Warburton, S. (2009) ) in press. Pennsylvania: IGI Global. Klobas, J. (2006) Wikis: Tools for Information Work and Collaboration, Oxford, Chandos Publishing. Lameras, P., Paraskakis, I. and Levy, P. (2009). Using social sofware for teaching in Higher Education. In Handbook of Research on Social Software and Developing Community Ontologies. Eds. Hatzipanagos, S. and Warburton, S. (2009) in press. Pennsylvania: IGI Global.

Learning, Media and Technology (2009). In Press.

Leach, L. Neutze, G. and Zepke (2001) ‘Assesment and empowerment: some critical questions’ Assesment and Evaluation in Higher Education 26(4): 293-305. Leuf, B., and Cunningham, W. (2001). The Wiki way: Quick collaboration on the Web, Upper Saddle River, NJ, USA, Addison Wesley. MacLellan, E. (2001) ‘Assessment for learning: The Differing Perceptions of Tutors and Students’ Assessment and Evaluation in Higher Education 26(4): 307-318. McConnell, D. (1999) ‘Examining a collaborative assesment process in networked lifelong learning’ Journal of Computer Assisted Learning, 15: 232-243. McConnell, D. (2006) E-learning Groups and Communities, SRHE & Open University Press. Miller, C. R. (1984) ‘Genre as Social Action’ Quarterly Journal of Speech, 70: 151-176. Moor,

A.

and

Efimova,

L.

(2004)

‘An

argumentation

analysis

of

weblog

conversations’Proceedings of the 9th International working Conference on the Language Action Perspective on Communication Modeling. NCODE - Flexible Learning Australasia. Available HTTP: http://ncode.uow.edu.au/ (7, June 2007). Nicol, D.J. (1997) Research on Learning and Higher Education Teaching, UCoSDA, Briefing Paper 45, Universities and Colleges Staff Development Agency, Sheffield. Nicholls, G. (2001). Professional development in Higher Education, London, Kogan Page. O'Hear, S (2006). Web's second phase puts users in control, Tuesday June 20 2006, The Guardian Education. Available HTTP: http://education.guardian.co.uk/elearning/story/0,,1801086,00.html (18 November 2008).

Learning, Media and Technology (2009). In Press.

QAA, (2002) Distance Learning Guidelines. Available HTTP: http://www.qaa.ac.uk/academicinfrastructure/codeofpractice/section2/default.asp (18 November 2008). Ridgway, J., McCusker, S. and Pead, D. (2004) Literature Review of E-assessment. NESTA FutureLab Series. Available HTTP: http://www.nestafuturelab.org/research/reviews/10_01.htm (18 November 2008). Rust, C. (2002) ‘The Impact of Assessment on Student Learning: How Can the Research Literature Practically Help to Inform the Development of Departmental Assessment Strategies and Learner-Centred Assessment Practices?’ Active Learning in Higher Education, 3(2): 145–158. Sadler, D. R. (1983) ‘Evaluation and the improvement of academic learning’ Journal of Higher Education 54 (1): 60-79. Sadler, D.R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science 18: 119-144. Shirky, C. (2003) ‘A Group Is Its Own Worst Enemy’ Economics & Culture, Media & Community, Open Source. Available HTTP: http://www.shirky.com/writings/group_enemy.html (18 November 2008). Stefani, L. and Nicol, D. (1997), ‘From teacher to facilitator of collaborative enquiry’ in S. Armstrong, G. Thompson and S. Brown (eds.), Facing up to Radical Changes in Universities and Colleges, London, Kogan Page. Torrence, H. and Pryor, J. (1998) Investigating formative assessment: teaching, learning and assessment in the classroom. Philadelphia, PA, Open University Press.

Learning, Media and Technology (2009). In Press.

Utecht, J. (2007) Blogs Aren't the Enemy; How Blogs Enhance Learning. Voices from Techlearning.com, Technology & Learning, 27(9): 32. Voiskounsky, A. (1997) Telelogue conversations, Journal of Computer-MediatedCommunications, 2(4). Yorke, M. (2003) ‘Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice’, Higher Education 45 (4): 477-501. Warburton, S. (2006) Everything 2.0? Exploring the potential of disruptive technologies in learning and teaching environments. ALT-C 2006: Proceedings of the 13th International Conference of the Association for Learning Technology, September 5-7, 2006, Manchester, UK. West, R., E.; Wright, G.; Gabbitas, B., Graham, C., R. (2006) Reflections from the Introduction of Blogs and RSS Feeds into a Preservice Instructional Technology Course. TechTrends: Linking Research and Practice to Improve Learning, 50(4):54-60 Wikipedia (2008). Available HTTP: http://www.wikipedia.org/ (11 April 2008). Wheeler, S., Yeomans, P. and Wheeler, D. (2007), The Good, The Blog and The Wiki: Evaluating Collaborative Writing in Teacher Education, eLearning Research Seminar Series, University of Plymouth. Wheeler, S. (2009). Destructive creativity on the Social Web: Learning through Wikis in Higher Education. In Handbook of Research on Social Software and Developing Community Ontologies. Eds. Hatzipanagos, S. and Warburton, S. (2009) ) in press. Pennsylvania: IGI Global.

Learning, Media and Technology (2009). In Press.

Type of

Feedback to

Feedback to

assessment

student

lecturer

diagnostic

2

2

self/peer

2

formative

1

summative

Self testing

Determining level/progression 1

1 2

2

2

1

Primary purpose: 1, Secondary purpose: 2 Table 1. Purposes of assessment types (adapted from Bull and McKenna 2004).

Learning, Media and Technology (2009). In Press.

Dimension Power (autonomy and ownership)

Dialogue

Timeliness Visibility

Appropriateness

Action Community

Reflection

Identified attributes of feedback* • • • • • • • • • •

Support management of own learning (self-regulated learning) Improve levels of (student) confidence Increase responsibility and autonomy Ensure feedback is provided often enough and in adequate detail Support peer/tutor dialogue Allow students to respond to feedback Support questioning Share assessment criteria Quantity and timing of feedback The feedback is prompt (provided quickly enough to be useful to students) • Discern student learning needs/prior knowledge • Be able to “spot” unpredicted achieved outcomes Feedback: • is understandable to students • is linked to learning outcomes (constructive alignment) • is linked to the assessment criteria • focuses on learning rather than on marks or students themselves • Feedback is received by students and is acted upon • Task-performance-feedback cycles are facilitated • Help students set personal goals • Support the learning communities • Support peer assessment • Encourage reflection on the work • Compare actual performance with a standard and take action • Provide information to tutor to help shape teaching (reflection in action/on action) • Develop skills in self-awareness

*Sources: Black et al. (2003), Gibbs (2004), Giroux (1992), Juwah et al. (2004), McConnell (2006). Table 2: Identifying the dimensions of formative feedback and assessment.

Learning, Media and Technology (2009). In Press.

Tables 3a to 3d. Mapping the impact of first generation (self assessment quizzes, email, discussion lists and fora) and social technologies (blogs and wikis) on the identified key processes in formative assessment as outlined in Table 2. The indicators distinguish between tools naturally suited to the identified feedback process (z) and those that can perform a particular function but are not normally or naturally suited to the role or context of use (œ ).

Social software Wiki

Comments Blog

Forum

Discussion list

Email

Key processes of formative assessment

Electronic quiz

First generation technologies

Power & autonomy:

Ownership of learning

z

z

Confidence level

Responsibility and autonomy

z

z

z

Blogs and email represent individually owned tools. Blogs provide a high degree of personalisation, visibility and persistence over time. Both tools raise issues around privacy and control over who can view feedback for example placed in the comments section of a blog post. As Efimova and Fiedler (2004) point out, blogs offer a synergy between self-organised and community learning.

z

Blogs and wikis as open systems can also act as a record of achievement. This can be a key factor in promoting confidence levels (EDOL report 2008).

z

z

As individually owned tools, both blogs and email offer higher levels of responsibility and autonomy. Evidence by Ducate and Lomicka (2007) suggests that blogs deployed in a language learning context are capable of fostering both ownership and creativity.

Learning, Media and Technology (2009). In Press.

Wiki

Blog

Forum

Discussion list

Email

Key processes

Electronic quiz

Tables 3a. Power & autonomy.

Comments

Dialogue:

Feedback is provided frequently and in adequate detail

Supports dialogue

z

z

œ

z

œ

z

z

Even though this function is tied to the tutor’s role, there are tools that can facilitate dialogue in the feedback process. The tension between time and effort should be acknowledged, where small cohorts may receive individually targeted feedback through personal tools, however feedback to larger cohorts is limited to the use of group tools such as fora and discussion lists. Conole et al. (2008) have highlighted student expectations for quick response times in computer mediated communication between tutors and students.

z

The nature of dialogue changes across these tools from formal to informal and from closed to open conversations, according to the social nature of the technology used. The structure of blogs, wikis and discussion lists allows retracing conversation threads. Utecht (2007) notes that the power of blogging derives from the conversational threads that are supported when blogs act as vehicles for dialogue, producing what he describes as thoughtful discussion extending beyond the classroom.

Learning, Media and Technology (2009). In Press.

Share assessment criteria

œ

z

z

z

z

z

œ

A tension between private and public is once again present here, for example email offers a closed dialogue. By contrast blogs and discussion lists can add value to group activities as others may benefit from observing meaningful communication between peers and tutors ((Lameras, Paraskakis and Levy 2009).

z

z

Discussion lists and fora can be used for distributing and sharing consensus on assessment criteria whereas wikis offer the possibility of negotiating/making explicit these criteria in a cooperative manner (Wheeler, Yeomans, and Wheeler 2007).

Wiki

z

Support questioning

œ

Forum

z

Discussion list

Allow students to respond to feedback

All these technologies (with the exception of quizzes) offer the possibility of responding to feedback, however individually owned social tools such as blogs may motivate learners to respond to feedback due to increased visibility (Lameras, Paraskakis and Levy 2009).

Comments

Timeliness:

Blog

Email

Key processes

Electronic quiz

Table 3b. Dialogue

Learning, Media and Technology (2009). In Press.

Quantity and timing of feedback

The feedback is prompt

z

z

z

z

A factor not exclusively associated with any tool. Timeliness can also be viewed as a feature of successful dialogue where quick response time is one of the main attributes (Voiskounsky, 1997). Push technologies such as discussion lists may have faster response times than blogs but the work of Moor and Efimova (2004) on blog conversations shows that bloggers can respond rapidly too.

œ

Promptness is a property of automated quizzes but choice of other tools will again be influenced by the size of the cohort where discursive collaborative tools may become more attractive. The co-authoring potential of wikis can provide opportunities for feedback as process, where feedback is ongoing and iterative during a task or activity (Wheeler 2009).

z

Records or traces of activity for example in blogs can provide valuable insights into learning needs and establishing prior knowledge.

z

Blogs as personally maintained sites can form a coherent and developmental archive of work over time (Kervin et al. 2009).

Visibility: Discern student learning needs/prior knowledge

Be able to “spot” unpredicted achieved outcomes (Hussey & Smith 2003)

z

z

Appropriateness of feedback: linked to learning outcomes; linked to assessment criteria; focuses on learning. Table 3c. Timeliness and visibility.

z

z

z

z

These criteria are understood as qualitatively related to the tutor’s role rather than to a particular technology.

Wiki

Blog

Forum

Discussion list

Email

Key processes

Electronic quiz

Learning, Media and Technology (2009). In Press.

Comments

Reflection:

Encourage reflection on work

Compare actual performance with standard and take action

Provide information to tutor to help shape teaching

Develop selfawareness

Table 3d. Reflection

œ

œ

z

z

z

z

œ

œ

z

z

The essentially informal tone and diary style of blogs is an ideal setting for stimulating reflective thought and addressing meta-cognitive processes. West et al. (2006) describe the largely successful use of blogs and RSS feeds to facilitate reflection amongst preservice teachers, while Hammond (2006) has reported positive responses from student appreciation of blogging as a reflective tool.

z

z

z

Quizzes can be matched against specific competency frameworks while wikis and fora can be used to share or distribute exemplars.

z

Wikis and fora can be used to observe student work over time while automated quizzes can provide snapshots of current student progress and competency levels. Quizzes can provide immediate feedback on levels of attainment, however individually owned tools, particularly blogs and to a lesser extent email, are vehicles for reflexivity and for raising self-awareness. Huffaker and Calvert’s (2005) study of teenage blogs and gender performance has examined the positive impact of blogging on identity formation.

Suggest Documents