That depends upon the content.
If it is a plain html site, then yes.
If it contains a language (php, perl, ruby, etc…) then you might have to load the libraries (depending upon if your host had the libraries loaded – or not)
If it hooks to a database – then you need a backup (export) of the database and will need to alter the configuration files.
And – don’t forget to do a domain transfer. Which means you will most likely have to set up the name servers in the registrar.