I link exchange purely with associates that I know have quality sites and condtent. I DO NOT exchange with people who email me because after tracking them over the years, I have found that about 95% of those exchanges go towards helping out BH spammers. For example, you get a link-back on their WH website, which feeds (gives link juice to) their network of stolen content.
So, since I did my research and that was my conclusion – I don’t bother answering. I sometimes link exchange with members at CAP/GPWA if I feel they have quality sites.
Otherwise, I don’t worry about it. Good content will win out in the end.