EDIT
- Make sure your category base is specified in Permalinks. Save them.
- Remove any custom permalink slugs in the category terms.
- Go to Yoast > Search Appearance > Taxonomies and scroll to the very bottom. Make sure Category URLs is set to Keep.
- Clear any cache and/or flush_rewrite_rules.
- Voila it's fixed.