Learning from feedback

Processing feedback

Learning from feedback is the shortest route to making progress on OU modules and the best resource for addressing individual issues is the feedback you get from your tutor.  However, without a methodical approach for processing this feedback I’ve found that I haven’t made the most of this resource.  The most explicit OU guidance I’ve found is useful but still too general (what or where is a TMA review form?).  It took me until TMA03 on my final module to come up with a fairly simple approach that I could have applied from the first module I took and which I’m convinced would have accelerated my writing development.

My search for a better way to use feedback started by looking for patterns in my tutor’s comments on TMA01 after getting a disappointing grade.  Although the process was fairly ad hoc at that stage I think it was my major breakthrough.  I’ve now formalised the process a little by doing a mini thematic analysis (hello DD307!) on my tutor’s comments and putting the results in a spreadsheet.  This sounds like more effort than it actually is and the payoff is that you get a clear cumulative, qualitative and quantitative view of your progress than when the comments are sitting alongside your TMA in a document.  I’ll outline the process which should be enough to understand the benefits and create your own spreadsheet if you want to take it that far.

  1. Create three columns: Theme, Positive, Negative.  Take a TMA and work through your tutor’s comments classifying them as positive or negative and giving them a top level theme.  It may take a few passes to produce your final set of themes.  I ended up with themes including ‘Introduction’, ‘Define terms’, ‘Point’, ‘Flow’ etc.
  2. Repeat with as many TMAs as you have/feel like processing. (You could even go back to TMAs from earlier modules).
  3. That’s it!

The results of this process are revealing, even with a single TMA.  Counting the number of positive and/or negative comments for each theme gives a quantitative measure of your strengths and weaknesses.  Reading the comments for a weakness should give you a hint as to which aspect needs improvement.  For example, I consistently get positive comments on the points that I make but equally consistently get negative comments that I don’t fully explain points or follow them through (I can actually trace this weakness back to comments at level 2).  The real benefit comes when you start looking at trends across TMAs.  For example, if you’re doing something right then the positive total should be going up and the negative going down!  If it isn’t, again start looking at the details of the comments for the themes where negatives outweigh positives.  Another interesting thing to emerge is positive/negative reversions.  For example, in TMA01 I have a positive comment under my ‘Introduction’ theme “sets scene” but on TMA03 a negative comment “doesn’t set scene”!  This is great as it clearly indicates that my tutor wants me to set the scene in the introduction, so I should never make that mistake again.  Also, when you get both positive and negative comments on the same theme you can easily trace the comments back to examples from your own writing of what works and what doesn’t.  Together, these are like the extended commentary I mentioned above but personalised for your own writing development.  Finally, by reviewing your analysis before writing each TMA you can highlight your own personal writing issues that you want address.  A good way to do this is to add your own comments to the TMA you’re working on and delete them before submission when you’re happy they’ve been addressed.

Giving Feedback

Each cycle of feedback processing should improve your ability to criticise your own work prior to submission.  This is an ongoing process and I take comfort when I read the gratitude that professional authors give to others that read their drafts.  Another way to develop this skill is to assess other people’s work as it eliminates the difficult problem of having to “step back” from your own writing.  Activity 10.1 in The Good Study Guide is worth doing to get a flavour for this.  I agree with the author’s suggestion on page 248 that

It is also an excellent idea to swap essay plans and marked essays with other students and share writing strategies and tips.

However, this starts to tread the thin line between collaboration and collusion (OU students only) and to avoid the consequences of plagiarism (disciplinary action) I was advised by a staff tutor to only do this “privately, in a very limited way and only well after the particular TMA is due”.  Another useful resource is extended commentary on student work.  This often goes beyond tutor comments by explaining why writing did or didn’t work.  Sadly these aren’t available for all modules (I haven’t found any for DD303) and tend to be hard to track down (see Resources below).

Resources

Advertisements

How to read DD303

In my opinion (and to misquote Kierkegaard) though the OU presents DD303 forwards it’s better understood backwards.  Considering what comes last (the exam) at the start of the module can help optimise your reading strategy.  The approach suggested below encourages active reading by associating each chapter with answering one or more questions.  It also provides a framework for allocating reading time throughout the module.  Although they were formed independently, many of these ideas overlap with the SQ3R approach.  A small amount of preparation is required.  At the start of the module gather as many questions as you can (good sources are specimen exam paper, past exam papers, revision packs and generating your own) and divide the questions based on which chapter they relate to.  This is actually quite a good orienting exercise at the start of the module and I recommend putting each category of questions into a separate document.  Before reading a chapter, check to see whether it relates to a TMA essay or a potential exam question (they’re mutually exclusive).  This should influence the way you read the chapter and take notes.

TMA Chapters

Essay TMAs have two options, so if you’re happy to chose one before reading either chapter you can focus your effort on that chapter.  If you’re reading a TMA chapter then use the question to guide your reading and note taking.  Because you know the question you’re expected to answer (and that there won’t be any other questions directly related to this topic on the exam) you don’t need to create revision notes in a format for answering (unseen) exam questions.  This is both active learning (you’re trying to answer a question) and efficient (you’re creating the notes that will become your TMA).  For the option you’re not doing you can still use the question to focus your reading but you don’t have to worry about turning any notes you take into a TMA.

Exam Chapters

Exam chapters require a different approach.  Because you don’t know exactly what the questions will be, use the questions you prepared from past papers to guide your reading.  I tend to print these out and keep referring back to them so I get a feel for how what I’m reading might answer the type of questions that may be asked.  In contrast with TMA chapters, the notes you take will need to cover the whole chapter to account for the uncertainty associated with exam questions.  As with TMAs, it’s worth taking some time to look at the bigger picture of the exam so you can make an informed decision on how to allocate your reading time.  Part 1 of the exam covers parts 1 and 2 of the book and part 2 of the exam covers parts 3 and 4.  TMA topics won’t appear on the exam but at least one question from each book part will.  If you’re still following me, this means that (after removing TMA chapters) some book parts have fewer chapters to cover to guarantee an associated question on the exam.  In 2014 this potentially reduces (from 3 to 2) the number of chapters you need to cover for exam purposes.  However, taking this strategy does constrain you to topics you may not necessarily find most interesting.

Resources

TMA06

To allow myself to focus on the exam without interruption I decided to write TMA06 before the residential school and use the momentum from that week to power the writing of TMA05.  Even though it took a bit more work after TMA05 to complete TMA06, doing them in this order worked well for me.  I don’t intend to submit either until the deadline to give me a chance to incorporate any feedback I get from a few people offered to read my final drafts.  I’m hoping this breathing space before submission will help address some of the issues I had in TMA04.  I also tried to address the essay issues I identified in my revised approach to processing feedback from my tutor.

In a refinement of an approach I started with DD307, I started TMA06 by turning the student notes into a very rough document outline wherever possible turning points to be considered into questions.   I used this to focus my reading.  Whilst reading chapter 11, I noticed something interesting when I came to add some notes to one of the questions I’d written.  I’d re-worded the student note

Consider also whether experts have a qualitatively different problem solving strategy to novices.

as

Are expert problem solving strategies qualitatively different to those of novices?

An answer to this question came to mind when reading sections 4.1.2 and 4.1.3.  Initially, I was just going to add a couple of bullet points with citations as placeholders of evidence for a qualitative difference, but when I started writing I realised I had the ingredients to produce a some fully formed words for the TMA.  As I seemed to have answered the question I changed it to a proposition

Expert problem solving strategies are qualitatively different to those of novices.

I realised that was a bold claim but saw that a stock phrase at the start of the sentence could tone it down

There is evidence to suggest that expert problem solving strategies are qualitatively different to those of novices.

That claim then invited some words presenting the supporting evidence I’d found

Experts tend to work forwards (Larkin et al., 1980) and categorise potential solutions differently (Chi et al., 1982).

What was most interesting about this was the natural process of moving from a question, via the course material, to a reasonably well articulated answer with supporting evidence.  I’m hoping I’ll be able to repeat this for the remaining questions and that this will allow tasks such as writing linking phrases and moving words around to create a coherent structure to be tackled independently of producing the main points in the essay.

I am writing this TMA alongside my first careful reading of Good Essay Writing: A Social Sciences Guide.  I should have read this years ago!  As this is a final opportunity to demonstrate critical thinking, Chapter 6 seems particularly relevant, as does the OU guide to critical thinking.

After some frustration at not seeing an essay plan emerge from my reading I decided to base my structure very closely on the student guidance notes.  (How I managed to waste so much time before taking this approach given previous epiphanies I don’t know!)  This seems like a breakthrough which is allowing me to (re)read the material (much easier and quicker second or third time around) whilst keeping the question in mind.  What was a homogeneous lump of knowledge in my head now seems much more like a resource that I can draw on to fill in the sections of my plan.  As I do this I’m starting see much more clearly which evidence supports which points in my argument and ways in which those points connect to each other.

Resources

TMA03 Redux

Grade 87% (Pass 1)
Previous DD303 TMA03 grade (2013) 65% (Pass 3)

 

Something worked!

It was satisfying to see a 14% improvement on my TMA01 grade and a 22% improvement on the methods essay on cognitive neuropsychology that I wrote last year.  More importantly, I’m almost certain that I understand the specific changes that I made that resulted in the improved grade.  I put way too much effort into this TMA but in the process I think started to see why that was and how I can reduce the time I spend writing future essays.  This directly addresses a point raised in the comment that inspired this blog.

Speak to your tutor

A golden rule I learnt from TMA03 is to speak to my tutor before I’ve made any major TMA decisions which will be hard to revert.  Had I not done this I would have used a ‘clever’ essay structure which turned out to be explicitly not the structure the module team wanted.  I could have saved about half a day by making this phone call earlier.

Keep it simple

I have a tendency to over-complicate things and it’s crystal clear I need to follow the KISS principle. The essay structure (advantages and issues) suggested by the guidance notes was adequate.  It was the one I started with, changed and then reverted to!  The very simple structuring device I supplemented it with involved grouping advantages and issues into categories.  This made it easy to organise paragraphs and went down extremely well with my tutor!  The other simplification that worked was to initially turn points into very simple sentences.  This saved huge amounts of time struggling over trying to find the write words to make a point.  The flow between sentences happened a lot more organically/naturally doing things this way round.

Use an outliner

Deciding to write using an outliner paid off in a two ways.  First, it saved my bacon when I had to reorganise my essay structure fairly late in the day (see above).  Second it helped to create a clear separation between the structuring and writing phases which I’m now convinced is really important.  Trying to write a finished essay before you know what you think is a recipe for disaster.

Answer the question

Although I still made a few points that weren’t clearly on topic this was probably the biggest issue that I managed to address when compared with TMA01.  In the process I think I discovered the main factors that were causing me problems.  First,maintaining focus alongside other challenging cognitive tasks like understanding the course material itself is simply (lemon) difficult.  Losing sight of the question during this process is natural and if you don’t periodically review the question there’s a high risk of including irrelevant points.  If you do periodically compare your points against the question  you often have the rude awakening that huge amounts of effort expended to understanding material which is irrelevant to this question.  Discarding these points is hard but essential.  I spent days reading and re-reading Henson (2005) but by the end I thought I understood it well.  I then spent a long time using it to make a number of complicated points.  In my final draft there just wasn’t room for most of these and I edited them out.  I also replace my own descriptions of function-to-structure deduction and structure-to-function induction with quotes from Henson (2005) and got the only explicit comment relating to the paper that if there was one place to write in my own words, that was it.  I remember when I was at and beyond the deadline having to “slash and burn”.  It seems that this was the right thing to do but I could have done it much earlier, perhaps before I’d even written a number of points which weren’t (the most) relevant.

Finish before the deadline

This is something I didn’t do for many of the reasons given above.  Consequently I had a couple of negative comments about my introduction and conclusion which were the last two sections I wrote when I was most tired.  Under less pressure I’m sure I could have resolved these issues.

Review feedback

Luckily I seemed to have identified the major issues with my essays before coming up with a more methodical approach to reviewing tutor feedback after submitting TMA03.  Using my TMA03 feedback I’ve extended this approach using a spreadsheet to categorise my tutor’s positive and negative comments.  This simple reorganising process highlights positive and negative trends and produces a clear checklist of points/themes to address in subsequent TMAs.  I already have a simple, targeted list of things to work on for TMA06.

Referencing

My tutor commented on my ‘excellent referencing’ and the negative comments dropped from 6 to 1 which I put down to the application of my citation heuristic.  The one slip I made was just a formatting mistake that slipped past Zotero.  If I’d addressed some of the timing issues I think I would have caught this as I wouldn’t have been so tired when reading the final draft.

TMA03 Timeline

Here’s a journal of the final stages of my TMA process.  There  were ups and downs!  My essay compared fMRI against other brain-imaging techniques.

8/5/2014

Writing has stalled.  I have some but not all of the points I want to make and a couple of ideas for structure.  I tried copying and pasting in key term definitions written in my own words which was only partially successful (some were inccurate!).  I spent a long time (hours) crafting a paragraph and concluded that it may have too much detail.  I wondered why things were dragging.  I think I need to go back to the drawing board.  I need speed at this stage so I have a map for the essay as I’m not completely sure where I’m going.  I’m also a bit concerned that I’m taking my eye off the question as a consequence of this.  I think I should be holding in mind a reader who’s considering using fMRI for an experiment and wants to know if it’s appropriate and what to watch out for (Edit: that was my downfall!). I’m starting to see the benefits of using an outliner to create essay structure .  It allows you to work with structure at the point and paragraph levels and quickly move things around without getting lost in the details.  Looking at the outline view also forces you to think of a title for each level which helps clarify your points (Edit: this saved my bacon).  Switching from imperative e.g. “Explain how fMRI works” to declarative “How fMRI works” seems to move things forward.  Being able to work with larger chunks makes it (practically) easier to move things around but it also starts to suggest grouping and sequence of points (and larger chunks).  If you look at an outline then you (literally, in the outliner) see the forest for the trees. Use technical vocabulary freely and just make notes for each one that needs definition prior to use. Re-read TGSG Chapter 10 and about to read Chapter 11.  Feels like going 2 steps backwards but I think it will take me one step forward.  I should re-read these chapters before every essay.

11/5/2014

Things I’m now fairly certain work

  • Separate structuring and writing phases
  • Write simple sentences first, linking them then becomes easier and faster.  Not doing this is always a drain on my time as I wonder how to write the perfect sentence for points which aren’t yet clear.
    • e.g. simple “Fluctuations in magnetic field strength from sources other than BOLD contrast such as regions of air in the brain”

Makes it more like doing a jigsaw puzzle (you have lots of pieces and there are obvious clues as to how to put them together e.g. you need to define a term before you use it and as sections form you can work with bigger units) from multiple puzzles (i.e. there are pieces you need to leave out) but the huge benefit is that you understand the concept of how to do jigsaw puzzles.  Imagine how hard it would be if you didn’t, that’s what it feels like.  This helps with flow as you tend not to get blocked as you can move from section to section.  Amazing to see this work.  Started off with an arbitrary section for definitions and it became blindingly obvious where to move these to (near to point where they’re introduced) in order to create flow.

12/5/2014

The game of reducing words.  It helps to do a draft to see what you can reduce further.  I just got my description of MRI from about 110 to about 60.  I also re-read the question and learning outcomes and am a bit worried that I’m not 100% focussed on the question.  I’m only supposed to be comparing against other imaging techniques (not cognitive neuropsychology) and I don’t think I’ve identified enough examples to evaluate biological methods.

  • 11:00  Re-writing how fMRI works was a lot smoother with few blockages and I turned the list of points I wanted to make into about 200 good quality words in about an hour. Use comments in word processor in the same way that your tutor might.  This keeps the flow going whilst creating a ‘to do’ list of things which you can work through on your next draft. The only extra advice I can give to TGSG is do it! Arg.  Spoke to my tutor and the clever structure I came up with is explicitly not what they’re looking for!  KISS.
  • 14:00 Lunch
  • 14:50 Starting the restucturing task.  If they give you ruled paper, write the same way.
  • 15:30 Restructuring pretty much done.  Much easier with points still at outline stage.
  • 16:45 Have written 178 word paragraph on fMRI advantages.
  • 18:00 Have written further 177 word paragraph on fMRI advantages. Break.
  • 20:00-21:30 281 words on practical fMRI problems.
  • 22:30 292 words on haemodynamic problems.
  • 22:45 Stop the hard process of writing and plan tomorrow’s work.

13/5/2014 (deadline day)

  • 9:45-11:00 Re-structured what I thought were just problems relating to theory into a new advantage.  Found another problem in the process and became concerned that I’m trying to put in the argument my have possibly at the risk of being off topic, but I don’t want to throw it away as it’s taken me so long to understand and summarise!
  • 12:00 Almost explained function-to-structure deduction!  Break.
  • 12:30-13:00 400 (i.e. too many!) words on function-to-structure dedcation and structure-to-function induction.  Lunch
  • 13:40-14:30  148 words on data analysis problems.  Threw away two points.
  • 16:00 Brain fried.  Just about managed final paragraph of problems.  Break (walk, snack).
  • 16:50-17:45 150 word introduction.
  • 18:00 Now for a quick read of advice on how to write introductions! At 2,300 words but no conclusion written yet.  (Writing good quality headings is good as they show the structure and can be turned into the first sentence of the paragraph.) Thrown away all of the “backup material” I had.  I removed all of my (useful) headings ready for a printed off a copy ready for red-lining/hatcheting (to see if it made sense without them).  Kind of defeats the object of turning them into sentences I just realised.  Still can probably pull that out of the intro.
  • 18:45 Only just printing after a bit of editing.
  • 21:00 Introduction intuitions seemed good but body needs extensive editing.  Had dinner.  Feeling tired and random.  Now have coffee.

14/5/2014 (deadline day +1)

  • 02:00 Irritable, frustrated and tired.  Second draft.
  • 02:24 Sent. Wired. Dead. Sleep.

References

  1.  Henson, R. (2005). What can functional neuroimaging tell the experimental psychologist? Quarterly Journal of Experimental Psychology: Section A, 58(2), 193–233.

TMA01 Redux

Grade 73% (Pass 2)
Best previous DD303 essay grade 64% (Pass 3)

 

I’m writing this the day after submitting TMA03.  Although the mark was an improvement on my best (from last year) I tried hard to address the problems I thought I’d identified and was disappointed with the result.  I read my tutor’s comments a few times prior to TMA03 to identify things I needed to work on and concluded that the main problem (still) was not focussing on answering the question.  However, reviewing the comments again having struggled (again) with TMA03 I now see that there are actually some quite specific themes that, had I noticed sooner I could have addressed more directly.  Actually, I just discovered a new and useful active task — classifying your tutor’s comments.  This is really good value.  It took about 15 minutes and I can now start to really see the patterns, something that didn’t happen each time I just read through the comments.  I wish I’d done this before TMA03!

  • Relevance (to question)
    • Introduction not stating how it will answer question
    • Throwaway sentence
    • Well understood but irrelevant points
    • Not focussing on process words
  • Failing to reference where I should have.  This is one I was mindful of and I think I’ve now found a heuristic to address this.
  • Lack of explanation or justification (for a point)
    • And why it’s relevant
    • “I’d like more context” comments
  • Imprecision (in points).
    • Would like more signposting back to question
  • Don’t use rhetorical questions!
  • Not defining terms (before use)
  • Poor flow
  • Too much (descriptive) detail
  • Conclusion too short and a bit weak

Resources

To cite, or not to cite

One lingering puzzle I’ve had is knowing when points in an essay require a citation.  I’ve asked my tutor and read the OU guidance on plagiarism but have struggled to find a workable rule of thumb.  I did have a useful conversation with someone in the US via the library helpdesk.  They gave me a few practical ideas and pointed me at a different document on plagiarism that does a better job of addressing this issue. Today, a quote from The Good Study Guide from the section ‘Using your own and other people’s words’ came to mind

Yet there is also, paradoxically, a strong sense in Erin’s writing that she is putting forward Layard’s ideas to us, using his terms … and his evidence.  Both Erin and Layard are ‘present’ at the same time in the writing.  It is a three-way dialogue between Erin, Layard, and us her readers.

I think the heuristic I’m looking for is “Whose words are these?”.  If they’re not mine, then it needs a citation.  Another way to do this might be to visualise what you’re writing as a three (or more) way conversation and/or imagine that you’re recounting that conversation.  If you didn’t see yourself speaking the words you additionally need to report who did.  This felt like an insight.  Or an ‘in cite’.