How I'm doing spaced repetition for grappling at kenku.org
To be totally honest, at the moment, I don't really know what I'm doing. I've got a rough version working, but it's far from optimal. I thought I would write this to try to figure it out for myself, consolidate my knowledge and see if anyone else has any ideas.
What is spaced repetition?
If you've ever used anki. That's spaced repetition. If you've never used anki, think of spaced repetition as the most effective way to remember things. Flashcards on algorithmically enhanced steroids that make sure you review things at the optimal intervals to deliver enhanced learning.
How it works
If you're trying to learn something by committing it to memory, it's been proven that the best time to review something is just before you forget it; if you review something straight after learning, that's too easy, but wait too long, and you can't review it at all.
Our brain forgets things on a curve, appropriately called the forgetting curve. In our use-case we are recommending grappling techniques to the user. We can use this forgetting curve to figure out when to recommend users review techniques at the optimal time. To do this we need to model the forgetting curve. This can be broken down into two steps.
One. Model how memorable that a certain technique is to a user(the more memorable, the longer they will remember it without review).
Two. Figure out how good a certain user is at remembering things. Some people are going to have inherently worse or better memories than others; for them, I will have to adjust the rate of decay in the algorithm accordingly.
Ultimately, a good spaced repetition algorithm would quickly determine how memorable something is to a user and how good the user's memory is based on how well they have recalled techniques in past reviews. It would then use this information to recommend techniques at the optimal time.
Why is this useful for Sports?
Good question. Unlike sports where the win condition is based purely on physical prowess (think swimming, running etc), grappling is a sport with a large cognitive component. Being analytical and cerebral is an important asset and to cultivate these attributes athletes study for hours outside of training, they also practise and play around with moves until they understand every small detail. Just like studying for academic subjects, consistency plays a huge role. It doesn’t make much sense to study one thing for 10 hours straight and never touch it again. But knowing when to work on what is difficult. This feature of kenku.org hopes to direct users when they should study and review certain techniques to help them learn faster and better.
Problems with relation to Kenku.org
I'm working with skills that aren’t strictly cognitive but also have a kinesthetic component. The forgetting curve for kinesthetic skills is slightly different and depends heavily on how the skill is learned and practised.
I don't have quantitative data; I can't just look at the user doing the technique and tell them if they're doing it right or wrong. I can't even look at them, period. I need to rely on their self-reporting of how well they believe they practised the technique.
What data I do have.
I can infer quite a bit of data based on the single rating users give after they have reviewed a technique given in the form of a self-report from 0-100. From this single rating, I store data on; average rating, amount of time successfully reviewed (reviews higher than 65/100. Communicated to the user through the UI), amount of times reviewed in total, user average review score, time since last review and more.
Using this data I calculate a “review score” This is used to rank how well the user knows a specific technique in comparison to their other techniques. Additionally I run a cron job daily that simulates memory decay, this is an exponential function that recalculates the review-score taking into account memory decay over time. The idea being that techniques that you know better will decay slower than techniques that you don’t know as well. That part of the function looks something like this:
review_score = review_score * e^(-(user_decay_rate) * time_since_last_review) In conclusion
This is a hard problem. A problem that might not even have a solution. But for now, I'm enjoying thinking about and working on it. It feels good to have no restraints and take a problem in whatever direction I want to. If you have any ideas or insights, please email me here. I would love to hear from you.
Published on 2024-01-18