Hacker Newsnew | past | comments | ask | show | jobs | submit | Velocifyer's commentslogin

What is the point? As long as you (the service) hash passwords securely, and the users use random passwords with a password manager, you are fine. A password is far easier to type onto another computer or a diffrent web browser.

Can't you just share the TOTP secret key?

TOTP is unfamiliar or hard to use for most people, so they use SMS. Most sites don't support TOTP either.

Even if you use TOTP, it's not designed to be shared, for example look up what hoops you need to jump through to export a single TOTP code in Google Authenticator. And they used to not even have that option; they told you to set up multiple TOTP codes on each website instead. Even on 1password I had to look up a tutorial on how to import a TOTP code cause the menu is in a very non-obvious and deep spot.


I belive that passwords with TOTP are far better than passkeys.

This is confirming my belies that Google is trying to block OEMs that don't pay them to be part of GMS, with the goal of eventually being the only android phone maker.

This will create a social problem of people discrediting images by a niche camera vendor or by a bootloader unlocked phone or from budget cameras or or from niche camera manufactuers from film cameras or from cameras that are old, while still allowing for advanced telecine attacks.

I would add cloudflare, but set it to cache only mode *without* the bot blocking features.


No point, they are all unique requests.


I think he means, get cloudflare to cache your content, so the traffic never reaches your servers to begin with. Assuming your sites content is cacheable by cloudflare. I agree it's a sad state of affairs if you have to rely on a 3rd party..

Maybe I'm not understanding how many requests at a time bots are sending to kernel.org (or how larger kernel is), but couldn't they have a local cache system too, where all it has to do it serve up dumb .html pages, needing next to no compute cycles.


It is server-rendered cgit pages, there are potentially quadrillions of unique. They are not cacheable.


Maybe you're a little hard of hearing. NO POINT CACHING, THEY ARE ALL UNIQUE REQUESTS.


Please remove everything starting from `is` or `si` from youtube links. The stuff after it is just for tracking.


But why don't they just git clone?


These crawlers (in contrast to e.g. googlebot and similar better behaved crawlers) are not very smart: they seem to make very little effort to avoiding crawling useless deep trees of generated pages. About the only thing they seem to put a lot of effort into is avoiding blocking.

(I'll note that while these are generally attributed to AI data gathering because of the timing of when they took off, it's not actually obvious who's running these bots. The big players all have crawlers that identify themselves and are reasonably well behaved, but I don't know if anyone has managed to positively attribute these other ones to any particular group)


> it's not actually obvious who's running these bots

This is fascinating to me. It's a large enough phenomenon that it's affecting the entire Internet and yet nobody seems to know yet who's actually doing it. Which isn't surprising, of course, it's hard to trace back to a source through all these proxies and it's probably a bunch of distinct groups anyways, but still!

Personally I have to wonder how much of it is "scrapers for training data" vs just tool-use LLMs. Even if you use chatgpt in thinking mode you can clearly see it searching and visiting a bunch of different websites to answer a question, presumably faster than any human would. That's got to add up. It's got me wondering why everyone seemingly discounts that as an option


ChatGPT uses a distinct bot type for ChatGPT-initiated web search versus model training:

GPTBot → model improvement / training crawl

OAI-SearchBot → search index / ChatGPT Search

ChatGPT-User → fetch initiated by a user's ChatGPT interaction

OAI-AdsBot → advertising landing-page crawl


Especially since if any crawler IP is in your country, you can get a subpoena to find out who it is.


Meta is an exception. They intentionally ignore robots.txt rules.

(No second chances, I blocked their whole IP range.)


Everyone ignores robots.txt because every robots.txt blocks everyone.


I allow Google, Bing, Yahoo, Kagi, and Apple on my personal blog... who else do I need to allow? Claude, OpenAI, Alibaba, and perplexity can go to hell.


Think about incentives, what benefit does Anthropic obtain by going to hell?


Why do we think that only “good guys” are training LLMs? I imagine organized crime is getting in on the game too.


Because they're crawling a billion webpages, only a tiny fraction of which can be git cloned, and configuring a special case just for that tiny fraction isn't worth the effort (of the crawlers).


Because the crawlers dont care, they are the internets parasites. Their creators care nothing for people or systems downstream of their greed.


vibe coded crawlers run by morally bankrupt trend chasers aren't going to be the most well engineered systems you come across.


What are the advantage of “scroll to content/top” buttons over <main> for screen readers? I think that the use of those buttons is for when the CSS fails to load.


It's essentially the same thing, but a lot of the time people will add a sr-only button at the very top that just goes straight there, and one at the bottom if the screen reader gets stuck in the footer nav, as well. Depending on which screenreader people use they sometimes only work on the actual hydrated markup, so <main> isn't always visible the way you'd expect. It's been a while since I did a11y tho so I'm probably hopelessly out of date.


Such buttons are more visible to (screen reader) users who navigate the web exclusively with arrow keys.

Landmarks are important, but to navigate by landmark, you need to be experienced enough to know the right quicknav keys. Some beginner users might not be.


Why is the word “satan” not allowed?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: