Hi,
I’m no SEO expert but I’d guess that one of two things happened. Your site was down for a bit of time during the changeover … or perhaps more likely you may have just changed at the same time that whatever SE it is that was sending you traffic, happened to have changed their algorythem/updated and somebody else edged you out of your ranking spots … therefore making it appear as though it was due to your changing hosts.
I’ve never heard of a SE punishing someone for simply jumping hosts. But then again I wouldn’t put it past ’em. They have all the cards and sometimes they crap all over ya for what seemingly is no apparent reason.