Ultralearning: Making It Stick

This is the blog post I wish I had read before I started college. My way of learning and studying for classes and tests back then was not the most effective possible. Before I set out to sharpen my axe and learn more about efficient learning strategies, I felt that I forgot most of what I studied or read. Had I known these strategies before, I could’ve learned and retained much more with the same amount of effort.

During that undertaking, I learned about a curious author named Scott Young. He self-proposed the challenge of completing the tests for the entire MIT Computer Science curriculum, which typically takes four years for most students, and did it in just one year. You can read more about his experience here. He also experimented with language learning and even flirted with Quantum Mechanics. In his book Ultralearning, he shares some of the tactics he found most useful to be an effective learner, based on his own experiences.

Here, I present some ideas from Ultrarlearning that most resonated with me and that I still use to this day.

The secret about learning no one had told me before

All my life, my standard approach to studying was to underline parts of the books I thought were more important to learn and reread them later. So, for me, the most useful and surprising advice in the book was that testing yourself by answering questions or trying to recall what you learned is way more effective for retention than just highlighting or reviewing your material.

That is a very empirical finding. In a study, psychologists Jeffrey Karpicke and Janell Blunt separated 80 undergraduate students into four groups. They analyzed their performance in a test after using one of the following studying strategies:

  • Group 1 reviewed the text a single time
  • Group 2 did a concept mapping of what they learned
  • Group 3 reviewed the text repeatedly
  • Group 4 did a free recall of what they read without having access to the material again

Which one would you expect to have the best score in a test afterward?

The researchers found that group 4 (testers) performed way better than the others (reviewers). However, when asked to predict how well they would do in the test, the scores are inverted: reviewers are more confident they will get good grades than testers. That reveals something interesting about learning: when you are exposed to your material, your mind creates a misconception that you can conjure up that piece of knowledge on your own, when in fact, you can’t!

Figures from Karpicke's experiment. Source (pdf)

Dr. Barbara Oakley mentions the same phenomenon in her course Learning How to Learn, and calls it the Illusion of Competence. Having the material open in front of you or having Google at your fingertips creates the illusion that the information is also in your brain. You want to put yourself in a position where you might be wrong as soon as possible to dissipate that undesired state of illusion and make progress.

After going through your book only once, it’s common to feel like you’re not ready to be tested just yet. But if you’re expecting to retain that knowledge for the long term, evidence shows us that’s the way to go.

Tactics for testing

Now we know testing is essential. How to go about it? Here are some ideas:

Free recall

After reading a chapter of a book or sitting through a class, grab a blank piece of paper and write down everything you learned. You can do this just after reading or a couple of days later. Compare what you wrote with the original material to see if you forgot anything or got anything wrong.

Flashcards or spaced repetition

You can use flashcards for things you merely need to memorize or if you want to develop a specific response to a given cue. Write a question or cue on one side and the answer or response on the other. Now, whenever you want to do a study session, go through your flashcards, read the question and try to answer it correctly. You can also chart how many cards you got right in each session to see your progress.

A more modern approach to this technique is using spaced repetition software. Those are apps or programs that test you less often on questions that you get right a lot and concentrate your effort on the ones you fail the most. The most popular choice of spaced repetition software nowadays is probably Anki, available for all platforms. I personally use NeuraCache, which has a great mobile app and integrates with my knowledge base on Roam Research.

The question-book method

This is a great technique to digest and retain concepts and ideas that are more complex than the simple cue-response duo and certainly more useful than just taking notes to be reviewed later.

After you consume your material, write down questions that force you to retrieve the concepts you learned. Try to test for broad ideas instead of specific information like in what year X happened or the value of constant Y - you can use flashcards for those. And don’t forget to check and correct what you answered! I used this method when watching Dr. Oakley’s course and must say I am impressed by how much of the course I still remember, many months later.

You can take this approach one step further and test yourself even before you read anything. Find or come up with questions that you would be able to answer if you already knew the subject you’re about to study. That is called the forward-testing effect and is a way of priming your brain to pay attention to a piece of information it will receive later. That way, when you finally come across the answer to those questions, your brain will be more inclined to value and retain that information.

Self-generated challenges

Imagine that you’re learning how to implement a computer algorithm. In this case, it’s not enough to understand how it works or to memorize its code - you have to apply it to specific problems. For this kind of skill, a useful approach is to propose a challenge to test yourself in the future, based on what you’re learning now. Make up a small project that not only requires you to remember what you read but also tests if you can perform it.

There are many more ideas in Ultralearning that I want to explore in the next blog post. The book goes deep into many aspects of learning, like feedback, research, drills, and experimentation. But if you grasp this single idea of how much testing yourself transforms your performance in learning, I’m sure you’ll want to apply it to your next learning project.

Stay tuned for part 2, where I discuss more ideas from the book! Subscribe below if you want to be notified when it comes out.