shellsharks@infosec.pubM to cybersecurity@infosec.pubEnglish · 6 months agoWhat are You Working on Wednesdaymessage-squaremessage-square6fedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1message-squareWhat are You Working on Wednesdayshellsharks@infosec.pubM to cybersecurity@infosec.pubEnglish · 6 months agomessage-square6fedilinkfile-text
minus-squaredubbel@discuss.tchncs.delinkfedilinkEnglisharrow-up0·6 months agoPrivate project, not really security related: Crawling robots.txts to gather some statistics on which bots people are most often excluding - weirdly I couldn’t find any recent/regularly updated stats on this.
minus-squarePaddleMaster@beehaw.orglinkfedilinkEnglisharrow-up0·6 months agoThat’s a neat project. Are you looking for trends, or something specific?
minus-squaredubbel@discuss.tchncs.delinkfedilinkEnglisharrow-up0·6 months agoIt started with a popular mastodon posts on how to block openai crawlers I think, and I’d like to know whether people are actually implementing it.
Private project, not really security related: Crawling robots.txts to gather some statistics on which bots people are most often excluding - weirdly I couldn’t find any recent/regularly updated stats on this.
That’s a neat project. Are you looking for trends, or something specific?
It started with a popular mastodon posts on how to block openai crawlers I think, and I’d like to know whether people are actually implementing it.