Page 1 of 1

Server Problems

Posted: Tue Apr 02, 2019 10:36 pm
by PyTom
This afternoon we had some downtime, that was caused by one of the hard drives in the server that runs the Lemma Soft Forums, renpy.org, and my other websites starting to fail. This is a major problem, as the other hard drive in the server failed a few weeks ago. I've done my best to keep things stable, but things are in a tenuous position at the moment.

Things are not all bad. We have a complete backup of the server as of this morning. The database backup was being made in realtime, so we haven't lost a post yet. In fact, the database seems to have been the source of our problems - I've transferred the serving of forum requests over to the backup server. (Which is why the forum is slightly slower.)

I'd also been building a new server for us. It passed testing, and I overnighted it to the datacenter. That's the reason why I didn't replace the hard drive - I'd been expecting to replace the whole server next week. This just moved up the schedule severely.

The only hitch in getting us back up is I have a minor surgical procedure coming up tomorrow. It's an outpatient thing, so I don't expect to be hospitalized, but I might be on painkillers for a day or two. So that might slow down the process of transferring things over to the new server.

I don't expect the forums to go down, except for a short amount of time when it's being transferred to the new server. But if it happens, don't worry - we have good backups, and we'll be back up as soon as I can get us.

Re: Server Problems

Posted: Wed Apr 03, 2019 2:05 pm
by Zero Chaotic
Good job, man! It's good everything is back at least for now. Thanks for supporting all of us, creators, and good luck to you in your upcoming surgical procedure!

Re: Server Problems

Posted: Wed Apr 03, 2019 2:15 pm
by Donmai
Thank you.

Re: Server Problems

Posted: Thu Apr 04, 2019 4:29 pm
by milksoda
Get well soon!

Re: Server Problems

Posted: Thu Apr 04, 2019 9:42 pm
by LateWhiteRabbit
Hope everything went well with your procedure.

Thanks for all you do, PyTom.

Re: Server Problems

Posted: Thu Apr 04, 2019 10:57 pm
by PyTom
Just updating it - I'm fine, back to 100%. Not sure the procedure (ESWL, if you want to look it up) worked, I'll find out in 3 weeks.

The new server is in place, and I'm working to get it set up properly. DNS has already been cut over, I'll be working on mail next, and then web things, with the forums a priority.

Re: Server Problems

Posted: Fri Apr 05, 2019 1:34 am
by isobellesophia
Thanks you! Now that this forum is not lagging now, that is why i haven't posted at least one yesterday or it is just me..

Re: Server Problems

Posted: Fri Apr 05, 2019 12:47 pm
by PyTom
The major services (renpy.org and lemmasoft.renai.us) have been transferred to the new server. We're still not in the final configuration I intended, so expect a bit more downtime as I move the database over, set up backups, etc, etc, etc.

Some of the lesser services, like renai.us. still need to be restored from the backups. I'll probably take care of that this weekend, when I have some time to chill.

Re: Server Problems

Posted: Sat Apr 06, 2019 9:24 pm
by PyTom
The good news is we're up and running on the new server. The bad news is that fulltext search is totally broken, not just partially broken like it had been. I understand the issue, but I'm not sure how to solve it. (Well, I am - but it involves dumping and reloading the database, which is more than a little freaky. I might also look at using a different tool for fulltext to bypass the problem entirely.)

Re: Server Problems

Posted: Sat Apr 06, 2019 11:50 pm
by PyTom
Okay, fulltext is up and running at the moment. There are some style issues with the forums due to an upgrade that I'll try to get to soon.

If you have any problem with the forums, renpy.org, or renai.us, then please let me know.

Re: Server Problems

Posted: Sun Apr 07, 2019 12:55 am
by Imperf3kt
Thanks for all the hard work Pytom, looking good from my end (though I haven't used the search yet)

Re: Server Problems

Posted: Wed Apr 10, 2019 12:39 pm
by Marionette
PyTom wrote:
Sat Apr 06, 2019 11:50 pm
Okay, fulltext is up and running at the moment. There are some style issues with the forums due to an upgrade that I'll try to get to soon.

If you have any problem with the forums, renpy.org, or renai.us, then please let me know.
Ah i guess that explains why the names and post times are currently showing before the thread titles at the moment.