Personally I think you can keep your affiliate links up, because the casinos don’t accept US players anyways. I can’t see any wrongdoing in sending a visitor to a gambling site where he/she can’t gamble.
Just make sure you take the links to the 1 or 2 casinos down that are left and actually do accept US players.
With keeping the links up, at least you could make a few bucks from non US traffic if you have any.