Website Issues - Slow and "Too Many Requests"

[JerryM100]

Greetings everyone, for relative newcomers I am Jerry the webmaster for the Romeo Riders website and forums.

The website has been very slow and even timing out with the error message "Too Many Requests" for the last couple of days. It took me a while to understand what was happening because it wasn't consistent. The short story is that our server has been hammered with requests for the last day or two - many hundreds of requests per minute! That's why the "Too Many Requests" errors - the server couldn't keep up! When I looked up the requesting IP addresses I see most of them coming from Singapore with many others from Hong Kong, Taiwan, and Moscow. Once I figured that out I knew what we needed was a 'web application firewall.' (WAF)

A WAF is another server that sits virtually in front of our server and intercepts all the traffic and analyzes the IP addresses. It compares the IP addresses to a huge database of known bad IPs around the world and blocks them so the requests never make it to our server. This is what we need!

The downside is this is a subscription service that costs $20/mo. I signed us up for a year. This is a cost that I will have to add to the annual donation request. Mike Lowe manages the annual collection in the fall so you will hear from him about that. For many years your local Ride Coordinators have asked for small donations which they forward to Mick who sends them to me to cover the costs of running the website. The costs have been $500 for many years but now will have to increase for this new service.

This morning I made many phone calls to our hosting service over several hours trying to get to the right department and the right person, and then signing up for the WAF service. The tech support person told me it would be effective in about 30 minutes to an hour (from around noon). I waited until about 2:30 pm (central time) and called them back. It seems the rep sold me the WAF but did not configure it so it wasn't working. The new rep apologized and worked through all the settings and finally got it done (about 45 min).

Sorry for the delay in getting this update out to you all but I was always just a little bit away from getting it to work, only to find another glitch or delay. Now I am convinced it is done and will work.

Jerry M
Romeo Riders Webmaster

[JerryM100]

Update 8/7/2025

After a day of the firewall operating I looked in the control panel. Here are a few statistics from the firewall after just a day. The threats and bots shown were all blocked by the firewall. This is what I was hoping to see. And now you can get an idea of why we were getting errors like "Too Many Requests."

Romeo Riders Firewall - Visitor Statistics.png (42.09 KiB) Viewed 415 times

Romeo Riders Firewall - Browser & Country stats.png (24.73 KiB) Viewed 415 times

[JerryM100]

Here is an update, now almost 3 weeks after I purchased and implemented the firewall to protect our server. Notice the decline in 'total sessions' as the bots are not getting to our website now and are going away. Also notice in the lower panels the incredible number of bots that are scanning our website - 371,000!! And the vast majority of them are coming from Singapore. I post this to show the value we are getting from paying for this firewall ($20/month).

Just a little more insight behind the scenes of running the website and forums. I get a ton of satisfaction in watching all the happy faces in the various cafes every week. Many thanks to the ride coordinators for posting the rides and photos each week.

Jerry M
Romeos Webmaster

Sitelock - Firewall & CDN Visitor Stats - 2025-08-25.png (82.86 KiB) Viewed 373 times

[TomK]

Hey jerry! Hope all is well there. For a long time you have done great work for Romeos. Some of us may know some "stuff" about websites and maintaining them but I bet for most of us you speak Klingon!! Thanks for all that you do!

TomK