Between 2:40am and 4:50am internet was down for the site for unexplained reasons.

The vendor does not list any issues overnight but the entire connection was down and it came back without intervention from my part.

It seems the server rebooted around noon CST.  It came back up without issue and I have not had a chance to investigate what may have caused it.  Please report any issues to support@

When I made a firewall adjustment last night I forgot that I had only temporarily disabled traffic shaping (due to issues).  Re-applying the rules after the adjustment re-enabled shaping which eventually leads to total connectivity loss.

The problem is resolved and shaping is permanentlydisabled for the time being so that this does not happen again.

Sorry. My bad.

The authentication database now has replicating backup on another system.  This should prevent the authentication issues that have been occurring when the master database fails to come up after a power related shutdown.

Now at 9:50pm CST Saturday the power has once again gone down due to severe weather.  UPS will run 30 minutes and then systems will be down until power company comes out and repairs.  That could be until the next morning.

Email and DNS backups do exist, delayed mail will be delivered and accessible once power is restored and systems are operational.

UPDATE:  Power is back but authentication is not working again (despite manually shutting the database down before automatic shutdown!)   I have blocked dependent services until I can go out there tomorrow and fix it.   In the mean time, mail will continue to deliver to the backup mx and you’ll be able to retrieve it tomorrow.

UPDATE:  Service is restored.  All should be functional again. Thanks for your patience.

UPDATE:  My bad this afternoon, fixed now.  :(

Around 9:00AM CST Saturday the power was lost during a thunder storm.   It took the power company until after 1:00pm to restore the power and then the authentication database was damaged again.  I drove out to the server location and restored the problem around 4:00pm.

Everything should be functional again now.  Please report any problems to support@

There has been intermittent connectivity loss since around 12:00PM CST today and it has repeated throughout the evening.   The hardware connection between my side and the ISP is failing with severe signal loss.  This looks to be a repeat of an issue that happened in August 2010.  I have opened a hardware repair ticket for the hard line.  Hopefully this will be repaired before 8PM CST tomorrow (the 25th)

Severe weather has taken down the power at the server location.  Backup battery lasts 40 minutes and is already down to 10 minutes.

Systems will be brought back online when power returns, likely a few hours from now.

UPDATE:  Systems are now back online.

I believe I know what happened this time.  I will monitor the system closely to be sure.  Sorry for the second interruption.

Around 2AM CST the server lost network connectivity and CPU idle time jumped higher than normal levels, indicating the system was not responding. Network buffer space messages were witnessed in the logs, the system had to be rebooted to restore connectivity.

Services should be up and available again. I will monitor for problems.