Cloudflare magic monitoring

[Dani]

I was just wondering if anyone else also uses Cloudflare magic monitoring?

[jkon]

@Dani I understand that you are asking something completely different, but aren't you tired of all this Cloudflare "magic"?

If our servers (or our DNS servers) had as much downtime as Cloudflare currently has in 2025, then we would have lost some customers, and our S&D department would be on the verge of a nervous breakdown.

[Salem]

"magic" monitoring?

Sounds like "We have no f..king idea how this AI crud works, but it looks good. No idea if it's telling us anything useful, but hey, give us money!"

Or maybe they're tracking what Penn & Teller are up to.

[Dani]

Typing a quick response from my phone. I held out using Cloudflare for many, many years because, especially in their early days, they had a lot of downtime and growing pains.

They now power I think it was about 20% of the web. With that comes a lot of responsibility and much more weight given to keeping their systems running smoothly.

While this latest issue is definitely concerning, their stock plummeted because of it, so they definitely are very financially driven to make sure it doesn’t happen again.

I use them because their edge cache is perfectly suited for my needs, and I also appreciate their ddos protection.

I’m open to you telling me about other edge caches that I can get for dynamic content the same flat $200/mo regardless of traffic.

[jkon]

@Dani I can see cases where edge caching would justify this cost, lowering the burden on servers, but since I don't use such services I have no idea how much of this actually works in practice, or how to propose any alternative to Cloudflare.

All the other things can be done with varying degrees of difficulty. For example, DDoS protection is relatively easy (it's included by most dedicated server hosting companies), while a serious firewall is not so easy, but still doable.

Even when using such services, it would be great to have an automatic bot on a different system that could at least notify you when your web app isn’t working as expected (e.g., not serving the correct content of a specific page), so you can change your A/AAAA/CNAME records manually while the problem exists (if your DNS is hosted somewhere other than Cloudflare) or an even more advanced solution to automate the whole process (which would require another DNS service acting as a router with an API).

I acknowledge that there aren’t easy solutions when your web app receives most of its traffic from different countries or clusters of countries, and that edge caching from such a service might end up being more economical than letting your servers handle that traffic by themselves.

[Dani]

Edge caching has been absolutely huge for us, especially for all our dynamic content (e.g. we don't just cache static files, but rather all logged-out members without a cookie aka 99.9% of our traffic also are fed all of our HTML cached at the edge).

Other CDNs can also handle serving dynamic content at the edge. I believe Fastly is one of them. However, Cloudflare is the only one I know of that is not usage based. Being a fixed cost is very important to me. Especially because there are soooo many bots constantly crawling us and we do get DDoS'ed quite frequently as well. It is not unusual for HTTP requests to fluctuate 100X one month to the next. It would be cost prohibitive if all of those requests were usage-based and the monthly cost was always an unknown.

Our web host does have DDoS protection, but a lot gets through that because Cloudflare also blocks DDoS attacks for us as a second layer of protection. Based on my CF firewall rules, 100K requests were blocked by Cloudflare in the past 24 hours, for example.

We do have monitoring set up in place. I was asleep for the entire duration of this latest Cloudflare outage. However, James, our sysadmin, did see the alerts and temporarily pointed daniweb.com directly to our servers, bypassing Cloudflare. So that did happen.