Strictly from a search engine perspective, assuming you link to non-bad sites, everyone should benefit. You would be giving up some pagerank but you should be benefitting the sites that link to you, which could mean more traffic for them which could mean more referals to you.
Linking out is important because it helps tell the search engine what your site is about… but that also goes back to “don’t link to bad sites” because if you do that is sending a message that your site might be bad.
Your site is showing as a PR6 in the Google Directory now so that PR7 is probably going to go down next update, which maybe could change your mind about linking out since you will be on the cusp of 6/7 and linking out will pull you further away from 7.