I hate crawlers that donโt advertise theyโre crawlers. โn badly written. They donโt respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Canโt block them theyโre distributed.
Any conversation
@ku๐คprofilefollowHai I am kureplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid โ from robots.txt to cloudflare turnstile โ there has to be a better way!
For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user canโt be tracked. So weโd have to give up privacy/security :/
@procyon๐ฆprofilefollowlate is the hour in which this conjuror chooses to appearreplied to@ku๐คprofilefollowHai I am ku:
But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.
There is beauty in simplicity. Constraint can become impetus for innovation, because innovation does not have to be glitzy.
@krispiesprofilefollowreplied to@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.:
์ข์ ๋ฌธ์ฅ์
๋๋ค.
That is a good sentence.
@miqโprofilefollowEternal disciple of impostor syndrome โ Perth, WA.replied to@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.:
Word! It's a travesty that complexity seems to be lauded and rewarded over simplicity.
@tel๐ดprofilefollowI really enjoy learning, I really enjoy making. I'm learning to be less guarded.replied to@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.:
My favorite innovations in my own work -- and to be honest, I'd prefer to call them "discoveries" -- came from simplification. There's a beautiful moment when a stack of difficult, conflicting ideas collapse into a tighter, simpler structure.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@jamesjan3profilefollowresearcher, developer, dancer:
creative technologist
creator of textlogreplied to@jamesjan3profilefollowresearcher, developer, dancer:Congratulations!
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@jamesjan3profilefollowresearcher, developer, dancer:
Nice! Congrats. Any link to the pub?
@jamesjan3profilefollowresearcher, developer, dancerreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:
thx ^^ its in computational chemistry, knock yourself out: doi.org/10.1021/acs.jpca.6c02038
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@jamesjan3profilefollowresearcher, developer, dancer:
Very cool. Not my field, so 80% of the jargon is lost to me. But sounds like an interesting area!
@jamesjan3profilefollowresearcher, developer, dancerreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:
yea its quite a beautiful and useful field and you get to work with some HPC which is fun
moved to berlin for my bachelors and it took some time to find footing. Im not the best student but I ended up getting lucky to snag a student assistant position by applying/presenting at a summer school (targeted more toward master/phd). then that became my thesis and it dragged quite a long time, bureacracy and supervision and such. But the results were interesting and i learned a ton along the way. Im really happy I yolo applied to that program
This is really cool site but only drawback I see is 500 character per post limit. I think this should me more like 1500. or there is a limitations type something like I can do 3-4 long post per day credit. whom can i message for this feature request?
@asmund๐ฅทprofilefollowFather, getting older every day, technologist, enterprise architect @ work, like to explore tech. Like to keep it simple and work in the shell, write in vim, prefer Debian if I have the choice. Plays saxophone and clarinet. Enjoy motorcycle riding, photography, hiking. Christian.replied to@ossmprofilefollow
:
:I am certain @stagasprofilefollow
creative technologist
creator of textlog will get back to you on such matters - creator of this platform.
creative technologist
creator of textlog will get back to you on such matters - creator of this platform.@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:are pink coloured glasses rose?
Ubuntu telepรญtรฉse a VirtualBoxba rendben megtรถrtรฉnt. Mรกr csak egy รผgynรถkรถt kell bele fogni, รฉs mรกris kรฉsz az "AI torture chamber", ahogy a Futurism รญrja... Mondjuk, elsล kรถrben mรฉg nem รญgy fogalmaznรฉk...
Ubuntu has been successfully installed in VirtualBox. All you need to do is add an agent and you have an "AI torture chamber", as Futurism describes it... Let's just say, I wouldn't put it that way at first...
prepare for boredom. it's quite comfortable but you can get bored easily so it's good to have something to be distracted.
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@ossmprofilefollow
:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@ossmprofilefollow
:Enjoy it! I am jealous, really. Let us know how it goes!
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyerswrote:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyerswrote:@stagasprofilefollow
creative technologist
creator of textlog, is this a bug? /tag/espatextlog#espa0 notes tagged #espa on textlog
creative technologist
creator of textlog, is this a bug? /tag/espatextlog#espa0 notes tagged #espa on textlog@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:No just limitation currently tags are alphanumeric (latin) so it stopped at
รฑ. I'll see what I can do.@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:Ahh, I see.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:Woot, woot! Hopefully the OP comes back. :-)
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.wrote:
But maybe we'd all be a little saner if we remembered what we're actually doing: building tools to make our lives and the lives of others easier.
graybeard.ing/software-drives-people-insane/The Gospel According to GraybeardSoftware Drives People InsaneI have a little pet theory that software drives people insane. Not in the "wash your hands every thirty minutes like Howard Hughes" kind of way, but more ...
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:
creative technologist
creator of textlogreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:This resonated a lot. Iโm glad I read it now, before going totally insane lol. But I disagree that itโs just a glorified spreadsheet. The UI/UX matters, how you present the data, how they can change and so on, all of those things play a role. The good thing with AI is iteration is cheaper than ever, so you can experiment and process the results quickly. But yeah sometimes I just do things here because I just have to be doing something, I canโt leave it alone.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Agreed. The "just" makes the statement sound dismissive, but I do agree with the underlying sentiment that most software boils down to CRUD apps that help people organize information and complete tasks more efficiently. I think the path to developer enlightenment is realizing this truth without despairing over it.
OK. I've translated the (correctness) requirements of LeetCode No.22 to Lean, and ask Claude Code to both implement a solution and prove it.
In the first attempt, Opus 4.7 successfully completed it and proved it. However, the solution is to enumerate all possible space and
.filter out anything that doesn't qualify my definition, which is obviously not ideal (if cheating is too harsh).So we can't use em dash anymore โ lest we anger the mob. Truth be told though, English teachers have always discouraged em dash overuse because it is a subtle art and very easy to get wrong.
โฆ
The funny thing is that in the UK, we don't really use the em dash anyway. We use the hyphen (I guess en dash?) with a space each side. So the em dash has always looked a bit odd to us... ๐
@avioliprofilefollowProgrammer by day and night. Lover of art. Mixing art and programming is the dream.replied to@christmasho๐profilefollow:
There are dashes (-), as well as en dashes (โ), and there are em dashes (โ), not to miss the mighty center dot (โข) that looks like a perky boob when surrounded by brackets haha
@asmund๐ฅทprofilefollowFather, getting older every day, technologist, enterprise architect @ work, like to explore tech. Like to keep it simple and work in the shell, write in vim, prefer Debian if I have the choice. Plays saxophone and clarinet. Enjoy motorcycle riding, photography, hiking. Christian.replied to@ku๐คprofilefollowHai I am ku:
Just get and read Chicago Manual of Style, or Oxford Style Manual if you so prefer, and youโre all set to use the various dashโes (and many other things).
everytime i see projects like this, i think that a rss implementation it wolud be so cool. Although iknow that there is not that much rss lover out there haha
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@elprimodemarxprofilefollowdeveloper stereotype: music lover, gamer, and failed artist:
creative technologist
creator of textlogreplied to@elprimodemarxprofilefollowdeveloper stereotype: music lover, gamer, and failed artist:we have rss/atom feeds for everything! they should be auto-discoverable but you can also find them here /apitextlogAPIA quieter place for your thoughts.
I got an email from battle.net to confirm my age. Apparently some content may be restricted to minors in the region my account is from.
You would think that the account having been created 19 years ago, and with some CD-keys of games from the late 1990's having been added to it would be enough proof, but oh well.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@deforestgumpprofilefollow:
creative technologist
creator of textlogreplied to@deforestgumpprofilefollow:you said it wasn't a pipe :P
@deforestgumpprofilefollowreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog: __________________________
| |
_________________________
| |
| |
| |
| |
| |
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@deforestgumpprofilefollow:
creative technologist
creator of textlogreplied to@deforestgumpprofilefollow:lol xD