Website Has Been Down
Posted: Sun Feb 23, 2025 3:43 pm
Many apologies to everyone for the website being down for the last couple of days. I have been emailing the ride coordinators to keep them informed but I can't email everyone so I am making this post.
Last Friday, Feb 21, I renewed our Internet hosting plan. After many phone calls over the last couple of weeks negotiating the best combination of features and price I finally decided to pull the trigger. As part of the package we had to migrate to a new, better, faster server. That is good but it is a tricky process to move all the files and the database to an entirely new physical server. And, as it turned out, they failed.
So I spent many hours late on Friday and most of the day on Saturday on the phone with tech support. After doing this for many years I can tell after a few questions the level of experience and knowledge a tech has. The challenge is how to not get angry or insult them (it's not their fault) but how to move on and call back to get a different tech and start again.
I'll spare the gory technical details but it involves moving our domain name and security certificate to the new server. The domain name (romeoriders.com) has to be pointed specifically to a physical server, and then if that server changes the connection is broken. That is the first thing they messed up. Then they fixed that but the security certificate was not migrated correctly.
The security certificate (SSL) handles the verification and encryption of all of our connections between our browsers and the server. If the certificate is not properly installed we get an error that says basically, "Be careful of this website, it may not be who you think it is." Which is exactly correct and is the browser doing its job to warn us. It doesn't necessarily mean a website is hacked but you can't assume that.
They fixed the certificate but then it takes several hours (4-5 hours) to propagate over all the routers in the entire Internet all over the world. That took basically overnight on Saturday.
Today (Sunday) the website became available but the forums were giving many errors. Very technical but it meant to me the database had a problem. I couldn't figure it out so I posted for some help on a public support forum where all the experts hang out. A few people responded to my request and one of them finally gave me a clue about a very specific configuration check to change. That made all the difference and the database worked and the forums became available again. Whew!
(Are you still with me??)
BUT, now for the next problem - we are not done yet.
Somehow, in the transfer to the new server, the folder that contains all the attachments (pics) got out of sync with the database where all the posts live. So the posts that have pics attached can't find the pics and thus just display some text.
At least now our ride coordinators can post their rides but I still have work to do to reconnect all the pics to their former posts. That is really a big deal. People like to see their pics from their rides. And we have posted over 65,000 pics over the 16 years or so. I really need to get that fixed and that is my next project.
A lot of detail here but I wanted to explain this to everyone as to what kinds of things go on behind the scenes. When things don't work you deserve an explanation.
Jerry M
Webmaster
Last Friday, Feb 21, I renewed our Internet hosting plan. After many phone calls over the last couple of weeks negotiating the best combination of features and price I finally decided to pull the trigger. As part of the package we had to migrate to a new, better, faster server. That is good but it is a tricky process to move all the files and the database to an entirely new physical server. And, as it turned out, they failed.
So I spent many hours late on Friday and most of the day on Saturday on the phone with tech support. After doing this for many years I can tell after a few questions the level of experience and knowledge a tech has. The challenge is how to not get angry or insult them (it's not their fault) but how to move on and call back to get a different tech and start again.
I'll spare the gory technical details but it involves moving our domain name and security certificate to the new server. The domain name (romeoriders.com) has to be pointed specifically to a physical server, and then if that server changes the connection is broken. That is the first thing they messed up. Then they fixed that but the security certificate was not migrated correctly.
The security certificate (SSL) handles the verification and encryption of all of our connections between our browsers and the server. If the certificate is not properly installed we get an error that says basically, "Be careful of this website, it may not be who you think it is." Which is exactly correct and is the browser doing its job to warn us. It doesn't necessarily mean a website is hacked but you can't assume that.
They fixed the certificate but then it takes several hours (4-5 hours) to propagate over all the routers in the entire Internet all over the world. That took basically overnight on Saturday.
Today (Sunday) the website became available but the forums were giving many errors. Very technical but it meant to me the database had a problem. I couldn't figure it out so I posted for some help on a public support forum where all the experts hang out. A few people responded to my request and one of them finally gave me a clue about a very specific configuration check to change. That made all the difference and the database worked and the forums became available again. Whew!
(Are you still with me??)
BUT, now for the next problem - we are not done yet.
Somehow, in the transfer to the new server, the folder that contains all the attachments (pics) got out of sync with the database where all the posts live. So the posts that have pics attached can't find the pics and thus just display some text.
At least now our ride coordinators can post their rides but I still have work to do to reconnect all the pics to their former posts. That is really a big deal. People like to see their pics from their rides. And we have posted over 65,000 pics over the 16 years or so. I really need to get that fixed and that is my next project.
A lot of detail here but I wanted to explain this to everyone as to what kinds of things go on behind the scenes. When things don't work you deserve an explanation.
Jerry M
Webmaster