Kids today might not realize that, for about twenty years there, you could go to Google Search and find things you were looking for! Google now features a hilariously unreliable AI summary as the f…
  • ColinHayhurst@lemmy.worldEnglish
    15·
    4 months ago

    Put should put these entries into your robots.txt file.

    To block the Google search crawler use for all of your site:

    User-agent: Googlebot

    Disallow: /

    To block the Google AI crawler use:

    User-agent: Google-Advanced

    Disallow: /