It’s deliberately slow to load
That kinda defeats the goal of feeding AI as much garbage as possible. They will just fetch a page from a different site in that time, instead of spending cycles on this page. It’s not like the crawler works strictly serially.







I’m guessing that Markov chains are pretty efficient computationally compared to AI training. Don’t have a site currently, but I’d love to see a bot rip through hundreds of pages a minute.