Thank you for all your feedback!
WordPress normally clears transients when the
delete_expired_transients action fires, this is scheduled as a wp_cron event. I’m guessing that if for some reason cron is disabled, transients may not expire. I will investigate if forcing a call to
delete_expired_transients() every now and then helps. Also, if you are very low on memory it is possible that WordPress is crashing before getting to run the wp_cron tasks.