Earlier quoted context omitted.
Data scraping. If anything else, the AI machine wants near constant streams of new data, even if it already checked with you 30ms ago. A CDN helps immensely. Also keeps you from getting DDoS'd if you did something like run it off your home connection.
Your webserver most likely have "rate limiting" built in already, which you can configure to act based on lots of variables typically. Set a limit of 1 req/s or whatever, and you've stopped 99% of all DDoS you'll encounter on the public web. If your visitors get cranky, up it to 10 req/s and you still are preventing most of the "abusive traffic", granted your backend/website isn't completely upside down when it comes…
Keeping connection alive take memory.
Rate limit by bytes/sec would hold the connection longer, taking more resources.
Rate limit by ip don't work with sudden surge demand, ddos, etc
Rate limit by user basically means you need to process the request and CPU bound
In very large scale DDoS, incoming SYN alone can cog your down pipe. You need to upgrade the connection just for that (Or something on your upstream to block by ip)
You can do lots of these without using cdn, yes. But it is easier with cdn and it is cheaper than paying for extra capacity "just in case"