We could finally resolve the issue. It had nothing to do with our mutations or upserts.
After analyzing our badger directories we found that the issue was the export directory. Over time, there were a lot of backups saved there. We expected them to be cleaned up after they were uploaded to S3. It would be great if the docs could be updated so people know that their exports are persisted there as well.
Thanks @matthewmcneely for your help!