Your question is more about will rankings be effected one way or another. Google as a search engine not only looks at links coming into your site, but also looks at the context of the link (content on that page, content on that site, what other sites are linking to that site). Early on in Google, a link was a link and it didn’t matter, now they look at neighborhoods of sites and want to know if you are in the same neighborhood as other sites you are linked with. My opinion is that you will get no benefit in the SERPS if you have all unrelated links, however, keep in mind that directory sites generally put your link into a page that has alot of other gambling related sites on it, so the page isn’t altogether unrelated. You do get plenty of benefit from those directory sites since they are one-way links, provided you have already established your neighborhood with Google. Bottom line, get related links first, track to see you are getting some sort of traction in the SERPs, then supplement those links with the directory links.