A catalog can store many items like you say, but usually, catalogs are used to track a user's listens, with some smaller number of explicit likes, and dislikes.
It's not expected that a user will give "ban" feedback to thousands of items.
Could you split this catalog up somehow? I'm not totally clear what the constraints of your use case are, but your idea to start by splitting on genre might be a good first step. Would you be able to create a catalog for each user of your system and track feedback at a user level?
-Eric