You are correct in most of what you have said.
To dynamically change things the best mechanism is to have key elements of data in a database – and MYSQL is a good choice.
Then your website could be mainly static HTML – with just small snippets of PHP code to call small elements of data from the database.
As you grow more confident with PHP and MYSQL you would soon find that you are using more and more PHP rather than static HTML.
Unfortunately, making the jump from HTML to PHP and MYSQL takes a bit of time to master and seems like a poor payback early on. But you’ll soon be pleased that you did make the move.