Assessing the #edcmooc digital artefact

“Close up of The Thinker” by Brian Hillegas. Creative Commons licence CC BY
“Close up of The Thinker” by Brian Hillegas. Creative Commons licence CC BY

Hmmm, I still find myself not quite walking away from my phenomenal #edcmooc experience – though this has not been the experience for everyone who have not liked the laissez-faire approach of this course (Young, 2013). At 1.00am on Monday 4th March 2013, my grade and feedback on my digital artefact had been released to me. I was extremely happy to have received the maximum grade of 2 (it ranges from 0 to 2) and to have received four largely positives comments. More on this later.

Defining the ‘digital artefact

For the course tutors, a ‘digital artefact‘ is something that can be “experienced digitally on the web“. They go on to suggest that the artefact is imbued with the following characteristics:

  • it will contain a mixture of two or more of: text, image, sound, video, links.
  • it will be easy to access and view online.
  • it will be stable enough to be assessed for at least two weeks.

I have seen some fantastic artefacts that the EDC-MOOC participants have shared with people via the EDC-MOOC Facebook page (requires a FB account and access to the EDC-MOOC FB group); a Padlet wall (formerly Wallwisher); and a Google Docs spreadsheet. One of the artefacts that was presented to me to feedback on was simply a photograph with some text embedded in a Google Docs presentation. It ticked all the boxes, but as a reviewer I felt somewhat ‘cheated’ as this particular individual did not appear to have put much effort in it, compared to some of the artefacts that I had seen outside of the formal evaluation process.

Considering the themes

Whilst developing and constructing the artefact, the participants are invited to consider some of the ‘big’ themes raised by the course, whether it is on ‘utopias and dystopias’, ‘being human’ or expressing a “question, an idea, a problem, a hope, a worry or a provocation that the course has raised for you“, as well as the impact the EDC-MOOC course has had on our  “understanding of e-learning“. Some other themes that were mooted include:

  • humans, machines and animals
  • surveillance
  • communication technologies
  • ubiquity
  • open (and opening) education
  • human nature
  • the meaning(s) of learning
  • the evolution of information technology
  • the future of learning institutions

For my artefact, I chose not to address the ‘big’ themes, but to look at ‘open education‘, ‘human nature‘ and the ‘evolution  of information technology‘ through the lens of my own (and others) experience with the EDC-MOOC and presented it in a form of a ‘love letter’.

The assessment criteria

The course tutors presented us with a ‘criteria‘ (or a series of “prompts“) in which to critically evaluate and consider the work of the other participants and how we “engage with their artefact“:

  1. The artefact addresses one or more themes for the course.
  2. The artefact suggests that the author understands at least one key concept from the course.
  3. The artefact has something to say about digital education.
  4. The choice of media is appropriate for the message.
  5. The artefact stimulates a reaction in you, as its audience, e.g. emotion, thinking, action.

On the surface, the above criteria looks fine until you actually have to evaluate artefacts against it and that is where myself and others have struggled. It is possible that we are not use to the process of marking and grading of work. For some, they suggested that the criteria was far too broad and needed to be more tighter and focused. A good example of this ‘misinterpretation‘ of the criteria came from both a Facebook and Google+ discussion around referencing. Some of the artefacts had used quotes but did not include a reference to it, this led to a number of participants ‘marking down‘ the work. Others, myself include, argued that the artefacts were not pieces of academic work, as potentially, a number of the participants may not have come from a Higher Education background, and secondly there was nothing explicitly mentioned in the criteria that references should be included.

I guess from the tutors’ perspective they were hoping that the participants would develop a critical thinking approach in the evaluation of others’ work. The Peeragogy Handbook by Rheingold et al. (2012) perhaps addresses the peer-assessment process more meaningfully.

The feedback process

As part of the assessment criteria, the EDC-MOOC participants were expected to provide peer feedback to a minimum of three randomly selected artefacts, though we could have done more if we wanted to (I managed to complete five during my allocated time, roughly 3 hours). We were told that:

You have up to 250 words to show your analysis for all the criteria. Giving reasons is likely to involve you in commenting on some of the qualities of the artefact. These might relate to its message, media and structure but also could include thoughts about its aesthetic appeal, humour, attention to detail, inventiveness, persuasiveness, educational potential or its shock value, for example. You could also indicate whether there is a lack of a significant quality, and say what would help the author to improve the artefact in this respect.

There was clearly a lot to consider and it took me a good 30 minutes or so on each artefact. If people had invested in the time to create something that the course had inspired in them, then it was only fair that their artefact received the same care and consideration from me. I did not find this an easy task (I don’t think it was meant to be) and wanted to come across as a fair, positive critical evaluator. I eventually award two x 2 scores, two x 1 scores and a 0 score. It was perhaps the 0 score that vexed me the most (the single photograph with some text embedded in a Google Docs presentation). This is probably where I would have appreciated a more detailed, or at least, nuanced criteria to work from.

Received my feedback /evaluation

As mentioned earlier, I received four largely positive comments on my artefact. I have also been receiving some fantastic feedback about my artefact from other sources (Facebook, Google+, Twitter and Vimeo). It was quite clear from reading the feedback how the evaluators were trying to interpret the criteria. Whilst one of the reviewers made the suggestion that I had made references to ‘utopias and dystopias’ and various other themes, but felt that I should had addressed the ‘being human’ theme as well – which just goes to show how subjective we are in interpreting such a wide and broad brief.

I think it was the feedback from my last evaluator (forever known to me as ‘Peer 4‘) that I found the most satisfying as there was a serious attempt at critiquing my work. I also charmed by the “school report” style of writing referring to me as “the student Wayne“. It was also clear in the feedback that English was not the first language of my evaluator, their “mother tongue” being French, which kind of couched the feedback belonging to a much different culture to my own. ‘Peer 4’ made the salient observation that my artefact was about “the course and not for the course content” and would have liked to me to have developed one of the themes from the course content and have an opinion of that. ‘Peer 4’ was of the opinion that it was the role of the EDC-MOOC team to criticique the course and not me – I wholly disagree with that. My (and others) positive critique is what the EDC-MOOC team are looking for in order to make changes to the next iteration of  the EDC-MOOC (Knox, 2013) – it is part and parcel of student feedback for any type of course. All-in-all, ‘Peer 4’ found my artefact to be a “strange love letter” indeed – this I think reflects the cultural differences between students. I should like to thank ‘Peer 4’ for the wonderful feedback, it certainly caused me to think and reflect upon my artefact.

My reason for doing the EDC-MOOC was more to do with the experiencing of the MOOC itself rather than the content. The content, for me, was secondary and to some extent I had engaged with it in various different ways in my Master’s degree. As such, much of my engagement with the content was reflected in my weekly blog posts – but how were my evaluators suppose to know that?

The next chapter…

This will be the last post concerning my experiences on the EDC-MOOC, and MOOCs in general, as I bid them a fond adieu. A new educational adventure begins for me next week as I embark on a Doctorate in Education (EdD) for the next 4-5 years. It may well be that the focus of my thesis will be around this notion of ‘open education‘ that has been so exemplary of this particular course.

My blog posts, for the foreseeable future at least, will be concentrating on my new Doctorate experience and the research that I will undoubtedly undertake.

References

Knox, J.K. (2013). “Staying the course…but doing #edcmooc differently”. Teaching ‘E-learning and Digital Cultures’ Blog, 18.2.2013. Available at: http://edcmoocteam.wordpress.com/2013/02/18/staying-the-course-but-doing-edcmooc-differently/ [Accessed 4.3.2013].

Rheingold, H. et al. (2012). The Peeragogy Handbook. Available at: http://peeragogy.org/ [Accessed 26.2.2013].

Ross, J. (2013). “Assessment, #edcmooc style”. Teaching ‘E-learning and Digital Cultures’ Blog, 19.2.2013. Available at: http://edcmoocteam.wordpress.com/2013/02/19/assessment-edcmooc-style/ [Accessed 4.3.2013].

Young, C. (2013). “Meditations on a MOOC – Week 5”. e-Learning Environments Team Blog (ULC), 3.3.2013. Available at: http://blogs.ucl.ac.uk/ele/2013/03/03/meditations-on-a-mooc-week-5-edcmooc/ [Accessed 4.3.2013].

5 thoughts on “Assessing the #edcmooc digital artefact

  1. Nice summary of the final process, Wayne, and like you, I reviewed 6 artifacts and gave five 2’s and one 1 – for the same reason as you – an artifact that met the minimums but paled in comparison to the richness I had seen in other artifacts as I strolled through Twitter and Facebook.

  2. A very good comment on the assessment process and the troubles associated with it. Also I was struggling a little with the evaluations and felt that I should give them enough time. And, as with grading my students in the class I recently taught as part of my PhD studies, I was being cautious to be too critical. After all, people came to this course with different levels of technical proficiency and, as you say, the assignment and the evaluation criteria were very open.

    Also one of my reviewers gave me a 1 because my artefact was about the course not the course content, saying I should have paid closer attention to the content, less to the subject of MOOC. Which, to be honest, is puzzling me a little as MOOCs are a form of digital education and some of the readings were specifically about them. Oh well, I guess I’m missing some more direct interaction in this anonymous feedback process – to say thanks and to discuss with my reviewers.

    P.S. I’m glad you’ve enabled comments, I really enjoyed also your previous posts about the course and was a little sad that there was no place to express it.

Comments are closed.