I hate crawlers that donโt advertise theyโre crawlers. โn badly written. They donโt respect the canonical URLs so they keep getting themselves into a loop instead of actually crawling the site properly. They make the logs really hard to read. Canโt block them theyโre distributed.
Any conversation
@ku๐คprofilefollowHai I am kureplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:I am curious what an actual solution to this problem could be. Everything I've read about it feels like a bandaid โ from robots.txt to cloudflare turnstile โ there has to be a better way!
For this particular problem though, I feel like if the crawlers are coming from domestic / residential IP addresses and they lack a distinctive http header or user agent, it is indistinguishable from a denial of service attack and must be treated as such.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:It would require permaban on IPs which would require permalogging of IPs in order to track them, and we are currently obfuscating them before hitting the DB with daily rotating salt so an anonymous user canโt be tracked. So weโd have to give up privacy/security :/
@procyon๐ฆprofilefollowlate is the hour in which this conjuror chooses to appearreplied to@ku๐คprofilefollowHai I am ku:
But as long as people can still use the site it's OK, right? Seems like the problem is often self-inflicted, either because the site is so bloated that answering even GET-requests is burdensome, or because they're trying to monetize the data and so must block non-paying scrapers.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:
creative technologist
creator of textlogreplied to@ku๐คprofilefollowHai I am ku:My solution was to add an env var to disable logs from anonymous. At least now will be able to read the logs.
#githubdownnotesfollow again. Was in the middle of some timeline experiments, sorry if it appears weird rn. www.githubstatus.com
@eignerchrisprofilefollowTraveler, engineer, founder of personal finance AI, www.heyalistair.comwrote:
Anyone on here fed up with their investment advisor? Interested to hear your frustrations.
#personalfinancenotesfollow
โฆ
@kooiprofilefollowreplied to@eignerchrisprofilefollowTraveler, engineer, founder of personal finance AI, www.heyalistair.com:
We almost had one, but stopped after the main setup but before signing on. Wasn't convinced of the value. Seemed like they were just doing portfolio reallocation for a pretty penny.
What are you experiencing?
@eignerchrisprofilefollowTraveler, engineer, founder of personal finance AI, www.heyalistair.comreplied to@kooiprofilefollow:
Had one for ~8 years. Talked to them maybe 10 times, and they only ever did basic indexing w some tax loss harvesting. We should have just self managed.
Actually, this makes sense to me - www.youtube.com/shorts/-sFwG2lt-nU
I find myself building bespoke tooling for my own #swdevnotesfollow and #ainotesfollow workflows these days, not only because of the obvious reason of software being so much cheaper to build, but also because there are SO MANY options for solving any given problem that it feels impossible to evaluate all of them.
If it's not a crucial piece of tech for a work problem then it often feels like paving my own way is both cheaper mentally (kind of amazing that's true these days) and probably a better fit.
..then of course I have to contend with - "Is this going to be a maintenance nightmare in the future?" and then I still usually conclude "no, the maintenance will be aided by the same approach that the construction was".
It's interesting how dramatically and quickly the scales have changed in this equation - and for such a wide range of software.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@zenprofilefollowfather, cyclist, #haskellnotesfollow, #emacsnotesfollow:
creative technologist
creator of textlogreplied to@zenprofilefollowfather, cyclist, #haskellnotesfollow, #emacsnotesfollow:It will become increasingly harder to find niche markets for software when those people can just build it themselves exactly how they want it. In the end, people want to accomplish tasks, and the AI is going to be building the software on-the-fly exactly for that task, why do we need generic software that may or may not be a fit?
@zenprofilefollowfather, cyclist, #haskellnotesfollow, #emacsnotesfollowreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Yeah. It makes me think that protocols for exchanging data between apps will become even more important in that landscape. But maybe, not - maybe each exchange is another bespoke AI-driven implementation?
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@zenprofilefollowfather, cyclist, #haskellnotesfollow, #emacsnotesfollow:
creative technologist
creator of textlogreplied to@zenprofilefollowfather, cyclist, #haskellnotesfollow, #emacsnotesfollow:Protocols, infrastructure, containers, those kinds of things. I was actually playing around with an idea for a minimal markup language + runtime that can be utilized by AI to spin up apps more easily, and brainstorming it with ChatGPT, I have a feeling something like that will appear soon to complement MCP.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.wrote:
Finally got around to setting up Remux on my phone so I can connect to tmux sessions when I'm not at my laptop. Even just for around the house, this seems like it'll be super useful.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:
creative technologist
creator of textlogreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:Tell us more! What apps are you operating?
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:This is for monitoring/poking agentic coding sessions while I'm not at my computer. So, typically Claude, Emacs, and an overmind process running my app locally in different panes of a tmux window.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.continued:
Call me old fashioned, but instead of adopting one of the "agentic coding orchestration frameworks" that seem to basically be reinventions of tmux, I've been tweaking my tried-and-true tmux-oriented workflow to incorporate agentic coding harnesses in an ergonomic way. I'm currently still using Claude Code, but getting Pi-curious as local models advance.
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.:I will be dead, and many years will pass after that, before local models are capable to match those of the big companies, unless their intended use is banal.
@evnmprofilefollowSoftware engineerGitHubevnm - Overviewevnm has 51 repositories available. Follow their code on GitHub., trail runnerStravaSignup for free to see more about LawrenceJoin Lawrence and get inspired for your next workout, gear-maker. Working on something new for homeownersgetonmain.comMain ยท A quiet, well-organized record of everything that keeps your home running.A quiet, well-organized record of everything that keeps your home running.replied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:It seems like we've reached a point where mid-grade frontier models like Sonnet are good enough for most day-to-day tasks. And Sonnet-grade models are already runnable on the higher end of consumer hardware.
In which case, I can imagine a workflow where local models serve as workhorses for implementing plans that you may bump up to e.g. Opus or Sol to produce.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@unsayprofilefollow#minneapolisnotesfollow:
creative technologist
creator of textlogreplied to@unsayprofilefollow#minneapolisnotesfollow:I imagine something terrible has happened (again) and the world is talking about it. I avoid reading general news. Too depressing.
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyerswrote:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyerswrote:All emails from textlog go to my junk folder, @stagasprofilefollow
creative technologist
creator of textlog. Using iCloud+. Not sure if anyone reported that before, so here I am. :-)
creative technologist
creator of textlog. Using iCloud+. Not sure if anyone reported that before, so here I am. :-)@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:
creative technologist
creator of textlogreplied to@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyers:Yeah it's happening to people. It's because the email contains a link and we don't have the reputation yet. Not sure what to do, have you tried reporting it as 'Not spam' in your client? Maybe if enough people do that it will fix. Gmail works ok.
@fastidiousprofilefollow
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
"Beware those that would deny you access to information, for in their heart they dream themselves your master." โ Sid Meyersreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:Yup, I am moving them out of it. It might take a while, but eventually should be OK. All good, hopefully now with a password I will not have much use for them any more.
You can now change the theme/color scheme appearance by visiting heretextlogenterA quieter place for your thoughts.. Thanks to @paratonerprofilefollow who implemented this #featurenotesfollow first in their #mobilenotesfollow app and which I shamelessly ripped from. #open_sourcenotesfollow
@jamesjan3profilefollowresearcher, developer, dancerreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:nice i dig the dracula
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@jamesjan3profilefollowresearcher, developer, dancer:
creative technologist
creator of textlogreplied to@jamesjan3profilefollowresearcher, developer, dancer:dracula is nice yeah i use that now too
@catastrofer๐ฅธprofilefollowJust another AI guyreplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Thanks for that one!!
@orangefor2profilefollowtry to be a happy engineer of the lifereplied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:cool!
@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.replied to@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.:
It was the only way to reproduce the authentic horrible experience of the offline version.
@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.replied to@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.:
๐คฃ
Did not remember the offline version being this bad, but maybe it went downhill in the past 5 years I have not used it.
@gruntprofilefollowI am just a grain of sand on your beach.
www.gruntsoft.comreplied to@eatziprofilefollowEat, drink, listen to ๐ถ, & watch the fish. Occasionally cook.:
I never used the online version, but the last time I used the offline version it already was a horrible experience.
This was about 5 years ago, before I had moved entirely to Linux. It was so bad I downloaded a pirated version of some old MS Office 2007 Blue Edition to have something acceptable.
I unironically prefer Libre Office.
Devs! Drop your #githubnotesfollow so we can follow each other! Love to see the projects youโre working on. Mine: github.com/stagasGitHubstagas - Overviewstagas has 462 repositories available. Follow their code on GitHub. โ #open_sourcenotesfollow
โฆ
@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Gave up on github after the acquisition. Very glad I did, given the unsurprising decline since then.
@stagas๐คธprofilefollow
creative technologist
creator of textlogreplied to@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.:
creative technologist
creator of textlogreplied to@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.:And what do you use now?
@hippo๐ฆprofilefollowStrictly no LLMs. No exceptions.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:Some gitlab, some codeberg. More recently the latter as gitlab have an annoying tendency to mess with their UI for no discernible reason.
@avioliprofilefollowProgrammer by day and night. Lover of art. Mixing art and programming is the dream.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:I think is time to vibecode an alternative!
I know I would
@semperos๐profilefollowPresbyter, husband, father, technologist, linguist, musician.replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:@keitachikawa๐profilefollowaeres-u99 on github, CodeAtlas \o/replied to@stagas๐คธprofilefollow
creative technologist
creator of textlog:
creative technologist
creator of textlog:
github.com/Aeres-u99
I have created a small tool, codeatlas, that basically generates an index to codebase that your llm can lookup syper quick, github.com/Aeres-u99
there is also a cbz->pdf converter that I wrote for eink devices which is nice
@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.devwrote:
Hi all, this is my first introduction to textlog, stumbled upon it whilst browsing Fdroid.
What do you like to do on here?
@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.dev:
grand and intoxicating innocencereplied to@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.dev:I like to write whatever's on my mind, or questions on the rare occasion!
@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.devreplied to@prophetorit๐งprofilefollow
grand and intoxicating innocence:
grand and intoxicating innocence:Or, as I've seen, budgeting for a certain item?
@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.dev:
grand and intoxicating innocencereplied to@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.dev:caught me there (โ ๏ผ โ _โ ๏ผ โ ;โ )
@mourningdove๐ฆprofilefollowI'm a small web enthusiast working on my blog and working on myself.
themourningdove.bearblog.devreplied to@prophetorit๐งprofilefollow
grand and intoxicating innocence:
grand and intoxicating innocence:You outed yourself
sometimes my devices will make noises that seem to go off for no reason. it really stresses me out though, I can't help but always think something is doing it and if it's not me, then what?
@jamesjan3profilefollowresearcher, developer, dancerreplied to@prophetorit๐งprofilefollow
grand and intoxicating innocence:
grand and intoxicating innocence:what kind of noises? drives spinning up/fans ramping up?
@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@jamesjan3profilefollowresearcher, developer, dancer:
grand and intoxicating innocencereplied to@jamesjan3profilefollowresearcher, developer, dancer:random beeps and a sound that seems like the notification but it just isn't. I don't know maybe I'm crazy
@chanuxprofilefollowI ain't no nothing!replied to@prophetorit๐งprofilefollow
grand and intoxicating innocence:
grand and intoxicating innocence:Would a CO monitor help here? I mean to err on the side of caution etc.
@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:I don't believe I know what that is
@chanuxprofilefollowI ain't no nothing!replied to@prophetorit๐งprofilefollow
grand and intoxicating innocence:
grand and intoxicating innocence:It was a reference to this nypost.com/โฆ/i-thought-a-stalker-โฆwas-even-scarier/
@prophetorit๐งprofilefollow
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:
grand and intoxicating innocencereplied to@chanuxprofilefollowI ain't no nothing!:oh, I may not be able to get this since i am too young to purchase anything online and I lack the funds. I will definitely urge my family to get it though, because I do constantly have issues of hearing things or believing someone is watching me
