Here is what I think happened.
The plugin you used to create the sitemap wasn’t that good, so it created a sitemap that, let’s say confused google, by uploading it to the google webmaster tools.That sitemap in fact damaged some of your google rankings, but for those kind of changes take some time for you to notice any results (either positive or negative), and that would be the only question sign on whether that’s what really happened.
I would have one suggestion for you, as you are using wordpress.Install the all in one seo pack, that seems to be the best one.I know a lot of wordpress users that have seen improvements after installing it.
And to answer your question
“I am curious as to why xml (and actually html) pages return the 404 when located in my wordpress directory. Could it be a permissions issue?”
I believe it has something to so with your htaccess file.Where it is placed and the path for the rewriten files.I might be wrong as I don’t know more about your particular case, but from what you told me, that should be the case.