Learning by Spaced Repetition

For me, the medical learning process goes something like this: First I’ll come across something that motivates me to extend my knowledge — whether that’s something I’ve seen at work, something I’ve read, or something I’ve heard about from a colleague or podcast. Then I’ll go away and swot up. I use various sources including journals (via databases like pubmed or MDConsult), books, blogs, podcasts or online videos that cover the subject of interest. Once I’ve collated the necessary information, long-term retention seems to come from relearning the information after I’ve forgotten it and had to find it again (doh!), actively discussing the issues with my colleagues, teachers and students and — for the past few years — writing about it here on LITFL.

When it comes to exams, I confess that I’ve always been a crammer in the build up to test day. Cramming certainly has merits. In the short term it is probably is the most effective way of stuffing a large number of facts into your head. The downside, is that information learned by cramming (or ‘mass presentation’) is poorly retained in the long-term. In fact, the decline in retention over time is exponential! This inevitable decay in our ability to recall memorised information was discovered by a chap called Hermann Ebbinghaus in the late 1800s… Hence the ‘Ebbinghaus Forgetting Curve’:

The Forgetting Curve; from Stahl et al CNS Spectr. 2010;15(8):491-504 --- click image for source

For medical students and doctors (and their patients!), this is important. We should not be learning information for a one-off test. There are many facts that we need to carry in our memories to function as experts in our profession. Also, more prosaically, over the course of our careers we are subjected to repeated examinations that require us to recall facts previously learned and to build on previous knowledge.

So, what is the answer?

In addition to actively discussing, using and teaching the information we learn from clinical practice, I think learning by spaced repetition is a useful strategy. I have been experimenting with this over the past year or so.

What is learning by spaced repetition?

Spaced repetition utilises the spacing effect, which comes from an observation made by Ebbinghaus over a hundred years ago:

 ”…with any considerable number of repetitions a suitable distribution of them over a space of time is decidedly more advantageous than the massing of them at a single time.”
— Ebbinghaus, H. (1885). Über das Gedchtnis. Untersuchungen zur experimentellen Psychologie. Leipzig: Duncker & Humblot; the English edition is Ebbinghaus, H. (1913). Memory. A Contribution to Experimental Psychology. New York: Teachers College, Columbia University (Reprinted Bristol: Thoemmes Press, 1999). [free fulltext online]

www.gwern.net provides an excellent description of how the spacing effect works:

 ”The spacing effect essentially says that if you have a question (“What is the fifth letter in this random sequence you learned?”), and you can only study it, say, 5 times, then your memory of the answer (‘e’) will be strongest if you spread your 5 tries out over a long period of time – days, weeks, and months. One of the worst things you can do is blow your 5 tries within a day or two. You can think of the ‘forgetting curve’ as being like a chart of radioactive half-lives: each review bumps your memory up in strength 50% of the chart, say, but review doesn’t do very much in the early days because the memory simply hasn’t decayed very much!”
— www.gwern.net

In other words, if you are a busy person and need to commit something to memory, you should space out the repetitions to improve retention. This is shown graphically below:

How spaced repetition improves recall (from Wired) — click image for source

It is also clear, that active recall is far more effective than passively rereading information. This means that we need to test ourselves. Indeed, psychologists call this the ‘testing effect‘, and it was well described by Francis Bacon in the Novum Organum centuries ago:

 “If you read a piece of text through twenty times, you will not learn it by heart so easily as if you read it ten times while attempting to recite from time to time and consulting the text when your memory fails.”
— Francis Bacon, 1620

How effective is spaced repetition?

This seems to be difficult to quantify. A meta-analysis in 1999 suggested that those who learn information by spaced repetition will outperform 67% of those who learn by mass presentation given the same number of practice episodes. Of course, this will vary according to the “nature of the task being practiced, the inter-trial time interval, and the interaction between these two variables”. Also, the studies varied in methodological rigour.

  •  Donovan, J. J., & Radosevich, D. J. (1999). A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t. Journal of Applied Psychology, 84(5), 795-805.

How can spaced repetition be put into practice?

The simple answer is: use computer software.

There are a number of programs available. I have used Mnemosyne and Anki and found them to be comparable. Both are free. However, when I decided to get serious about putting spaced repetition software into practice I committed to Anki and paid the additional ~$US25 for the iphone app.

Why did I do this?

Anki uses an algorithm to predict when you need to be re-exposed to a flashcard for optimal retention. When you have tested yourself you can grade the difficulty you had in remembering the answer with a touch of a button, and Anki will put the card back into the virtual pack in the correct place. The algorithm uses an ‘expanding spacing’ approach. This means that the more times you review a card the greater the time interval until the next review (unless you couldn’t remember it, then it moves back to the front of the cue!). There appears to be an ongoing controversy about how best to space trials for longterm recall, but Anki’s algorithm is probably as effective and as efficient as any other.

There is no way I could realistically do spaced repetition without a computer program. Like most people I naturally tend to review information that I find easiest or most interesting. Also I would no doubt lose my cards if they were physical rather than virtual, and I’d certainly have trouble keeping them in sequence. Anki not only helps space the cards appropriately (though I’m sure the algorithm is not perfect) it helps the learner stay disciplined so that you focus on your weaknesses and learn the cards you need to learn.

For me, the iphone app — though somewhat costly for an app — was well worth the price. It enables reviews of cards during the wasted moments of the day — whether that be waiting for a train or standing in a queue, or getting in 60 seconds of revision between patients. Furthermore, the iphone app syncs with ‘the cloud’ and my personal computer, so that it remembers what cards I have reviewed wherever, and however, I want to review them.

 

How to create flashcards for spaced repetition?

It is easy to create flashcards using software like Anki. You can insert or cut-and-paste text, images and even audio files. However, it is important that you create your cards wisely.

Spaced repetition seems to work best for the memorisation of conceptually simple facts and lists. The greater the complexity, the less useful spaced repetition will be. Thus it is most effective for simple facts (e.g. What is the NNT for post-resuscitation therapeutic hypothermia?), OK for word lists (e.g. What are the causes of splenomegaly?)  and much less so for complex, skill-based tasks.

For example, becoming proficient in ECG interpretation is a complex skill-based task. Learners needs to constantly expose themselves to ECG interpretation to become proficient (get over to the ECG library now!). Constant deliberate practice is needed to become an ECG expert (think of Ericsson’s 10,000 hour rule for becoming an expert, as popularised by Malcolm Gladwell). Yet certain facts about ECGs can be learned using spaced repetition (e.g. What is the normal QT interval?, or what is the differential diagnosis of ST elevation?). Hence even the learning of complex, skill-based tasks can be enhanced by the appropriate use of spaced repetition for fact retention.

Flashcards for spaced repetition must use the ‘test effect’. They should be designed to trigger active recall, rather than simple recognition. A question and answer format (a la the case-based Q&As) works well for this. Furthermore the questions and answers should be clear and unambiguous. I also suggest avoiding the use of multi-choice questions in a spaced repetition learning system. MCQs are potentially dangerous for learning as the exposure to false answers runs the risks of retaining incorrect information. When practicing MCQs it is essential that you have access to the answers, or go to the trouble to find out the correct answers soon after testing yourself.

Finally, spaced repetition improves recall, but only once you’ve already learnt something! You need to go to the trouble of really nutting things out and learning the information in the first place, for spaced repetition to help you retain it. Remember to be selective when choosing what facts you need to memorise… Only fill your brain up with facts that you can’t look up in real time, need to know for a test or that will impress your friends and colleagues!

Disclosure:
I have no financial or other interest in any of the products mentioned in this blogpost.

 References

Print Friendly

Comments

  1. says

    Don’t show people in quarantine though…. Wish I’d known about this PRIOR to the ordeal, but will make sure the next crew going through check it out!

  2. says

    Sounds like a cool idea.

    I’ve found the single best thing I’ve done for my learning is to write a blog. If you have to teach it to someone else then you know it. It’s a lot of work but it’s a much more effective way of learning than “hitting the books”

    • says

      Active learning -- teaching, doing, taking concepts apart, attacking them from all angles -- is the key to understanding.
      Spaced repetition can help you remember what you understand.
      I totally agree that blogging is fantastic for learning -- putting your ideas out to dry in full view of everyone really forces you to know your stuff and leave no stone unturned!

      Chris

    • says

      Agree entirely
      The blog acts as an archive of information, an avenue for research and a highway to higher learning
      Making information public ensures a heightened awareness of accuracy, thought and diligence -- strengthening the importance of the iteration
      Mike

  3. couch says

    Do you guys ever sleep ;)

    THANK YOU SO MUCH! Since SuperMemo was never an option for me (way too cumbersome and complicated, not on OSX), Anki now fills the gap.

    The “import” function lets you create hundreds of cards within minutes from a “.txt” file, so easy to create a deck like that. Wish I had known about this during med school….

    Might create a deck for the brain stem rule of 4 and share it

  4. says

    I used this strategy with Mnemosyne for my Step 1. Ended up creating over 9000 flashcards… probably a little bit obsessive, but the work paid off. Right now, I’m mainly learning by listening to podcasts and reading like mad, but I hate the fact that I end up forgetting probably around 90% of what I hear (which I imagine is pretty in-line with current research in education, if I remember what Amal Mattu said correctly… which I’m not sure :P) I just have had a hard time wanting to put forth a serious effort to go through the process of making more flashcards again. Is there any way of collaborating on a deck of Anki cards?

    • says

      9000 cards -- whoa! You are clearly a machine Max…

      I’m little unsure of the utility of sharing cards. Spaced epeitition will work better if the cards are personalised, and reflect facts the individual really wants to know.

      Sharing cards will be useful for topics where there is a fairly standard list of facts that must be remembered -- e.g. much of clinical examination, ECG basics, ACLS, etc. Also, people can always modify and/ or discard cards they don’t like from a shared collection.

      It would be fairly easy to go through websites like LITFL, other blogs, or the EMRAP shownotes and cut-and-paste key facts into Anki. I’ve done it with my own posts that I have trouble remembering!

      Chris

      • Andrew Perry says

        Chris,
        Excellent post on how to best retain large amounts of information. A number of ACEM FE study groups at my hospital have started utilising Anki and sharing our decks which is surprisingly easy to do and helps cut down on time needed to create the decks.
        Having said this making your own does have the benefit of ensuring it is the information you most need to know, is worded in the way your brain likes to think and hence is easier to remember and the creation of the cards themselves is another cycle of the spaced repetition.
        With all of this said I think it would be good if LIFTL were able to offer yet another service to its loyal users by being a central repository of ACEM/CICM/Critical Care digital flashcards in Anki or any other format that people use. I would be happy to share mine to help get things started.
        Andrew

        • says

          Thanks Andrew -- great idea.
          We’ll be more than happy to make a flashcard repository that is free for anyone to download.
          I suggest emailing me a dropbox link or yousendit file to chris AT lifeinthefastlane.com and I’ll get cracking.
          Cheers,
          Chris

  5. Pieter says

    Agree Chris, spaced reputation is very useful. I ended up using a manual version myself using Ominfocus. ANKI looks great.

Trackbacks

  1. [...] This is a great article by Chris Nickson over at LITFL, that goes through some useful techniques that may help your memory and recall of content you are trying to learn for the Fellowship Exam. In it he talks about the use of two programs Mnemosyne (which we’ve mentioned in a previous blog post) and Anki, which has an iphone app. These are both flashcard programs that automatically alert you when it’s time to review a topic or list, and you can use the Anki app anywhere as it syncs with your home computer. [...]

Comments