if for some reason you find yourself flying from burlington vt to a dc/baltimore airport: this is a perfect time to enjoy larksโ tongues in aspic [the album]
I hate crawlers that donโt advertise theyโre crawlers. โn badly written. They donโt respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Canโt block them theyโre distributed.
I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid โ from robots.txt to cloudflare turnstile โ there has to be a better way!
For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.
It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user canโt be tracked. So weโd have to give up privacy/security :/
But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.
I'm really interested to play around with textlog for a bit. I miss the space that once was Twitter and its constraints. Let's see if we find any #NixOSnotesfollow users here ... ;)
Crazy what happened earlier; one the cats here โthe one Iโm closest withโ started unexpectedly meowing outside wanting to get in, I let it in and it started licking me like never before, as if she sensed Iโm leaving in a couple of days, canโt explain it otherwise.
Went to a party today, the group was mostly strangers, but they were all sociable and knew the art of small talk rather well, something I rather admire.
being a student is great but the tasks are really quite boring. ive worked in academia now a little bit and its genuinely much more fun to work on something real, read papers, do experiments and write. Now i have these last few exams to get through and i cant care for them like i used to.
#Researchnotesfollow#Sciencenotesfollow
Deterministic code generation is fine by me. Quite happy to use templating systems for example.
Problems occur when randomness is introduced. If you can avoid that it should be less "icky".
Before AI, coding was my creative outlet. Now, it seems like I need to find a new one, but hobbies being #ttrpgnotesfollow and #videogamesnotesfollow, and with a family, I'm not sure where to find the time.