@smurthys@hachyderm.io2025-11-02 07:18:42
*AI* crawlers ignoring *robots*.txt file. The irony. 🤦♂️
#AI #technology #conventions #web
@smurthys@hachyderm.io2025-11-02 07:18:42
@smurthys@hachyderm.io*AI* crawlers ignoring *robots*.txt file. The irony. 🤦♂️
#AI #technology #conventions #web