I've had the same problem as you. If you remove the ear cushion on the left earcup, you'll find 3 t6 torx screws. Removing these and gently lifting upwards you can seperate the speaker driver from the rest of the internals. (Be warned there is a thin cable attached that is soldered in place, be careful not to damage.) A little fishing around inside the headset and you can remove the rattling plastic piece that came off the hinge, and you won't notice the damage so it'll be good as new.

Just as a head up. For me the screws were t6 torx SECURITY screws. Meaning you need a specialized screwdriver. There's a little dot in the centre preventing you from using normal ones.