@MortimerJazz 153119 wrote:
if I want to list a number of files that the robots shouldn’t be clicking through to, how do I do it? What format should the list take?
Disallow: /file1.html
Disallow: /file2.html
Disallow: /folder1/filename.html
Disallow: /folder2/filename.html
etc.
To disallow everything within a subfolder, use:
Disallow: /banned-folder/