View unanswered posts | View active topics It is currently 19 March 19 3:13 am



Post new topic Reply to topic  [ 7 posts ] 
Site Outage 9/1/2019 
Author Message
850 or more found!!!
850 or more found!!!
User avatar

Joined: 23 August 04 3:17 pm
Posts: 6912
Location: Launceston, TAS
GCA Found: 159
GCA Hidden: 44
100 GCA Finds
Post Site Outage 9/1/2019
Here's the lowdown on today's outage

* reports early AM that site wasn't working.
* Initial thoughts were a full Hard Drive
* Remote diagnostics revealed our two SSDs used for our database were failed. Weird.
* Reboot caused server to go fully offline
* C@W attended site at 5:30pm and him and I worked through recovering the SSDs.
* After a lot of joggery-pokery we determined one SSD was totally NFG.
* Restored RAID config from the second drive (after having to purge a PERC RAID cache)
* Rebooted and tested, however for reasons unknown, databases were an old version from May 2018!!!!
* Restored from Last backup, however this is 24 hours old as the backup at 00:30 on 9/1/2019 hadn't run due to this failure, meaning we lost all data for 8/1/2019
* Tested, back online at ~ 20:35

We now need to replace the faulty SSD(s).


09 January 19 8:36 pm
Profile E-mail WWW
6500 or more caches found
6500 or more caches found
User avatar

Joined: 05 February 11 6:39 pm
Posts: 164
Location: Sandford
GCA Found: 3783
GCA Hidden: 196
100 GCA Finds500 GCA Finds1000 GCA Finds1000 GCA Finds1000 GCA Finds
Post Re: Site Outage 9/1/2019
Thanks for all your time, effort and expertise in getting the site up and running again.


09 January 19 9:09 pm
Profile E-mail
9000 or more caches found
9000 or more caches found

Joined: 09 October 04 7:51 pm
Posts: 1091
Location: Calamvale, Brisbane
GCA Found: 253
GCA Hidden: 0
100 GCA Finds
Post Re: Site Outage 9/1/2019
Thanks for the update and all your hard work to get it up and running again.

I have copies of the imports which I sent through last night so I will resend them.


09 January 19 10:40 pm
Profile ICQ

Joined: 13 November 11 12:16 am
Posts: 105
Location: Hobart
GCA Found: 1358
GCA Hidden: 107
100 GCA Finds500 GCA Finds1000 GCA Finds
Post Re: Site Outage 9/1/2019
Top work guys well done on working thru the issues and getting the site up and running again.

=D>


09 January 19 10:53 pm
Profile E-mail
5500 or more caches found
5500 or more caches found
User avatar

Joined: 17 August 09 10:44 am
Posts: 1748
Location: Sydney, NSW
GCA Found: 1272
GCA Hidden: 78
100 GCA Finds500 GCA Finds1000 GCA Finds
Post Re: Site Outage 9/1/2019
Double your pay this month. :mrgreen:


10 January 19 7:39 am
Profile E-mail
800 or more hollow logs searched
800 or more hollow logs searched
User avatar

Joined: 21 May 10 9:57 pm
Posts: 42
Location: The Channon, NSW
GCA Found: 221
GCA Hidden: 93
100 GCA Finds
Post Re: Site Outage 9/1/2019
We really are lucky have developers with so much talent. What could have been an absolute disaster has been managed and fixed and now we're all back playing the game. Geocaching for me revolves around GCA - the stats, the caches, the games and the people here all put it well ahead of any other geocaching sites, in my opinion. Thankyou to all that make GCA the wonderful place that it is.


10 January 19 8:07 pm
Profile E-mail
10000 or more caches found
10000 or more caches found

Joined: 19 January 10 7:54 pm
Posts: 305
Location: Ulverstone Tasmania
GCA Found: 5186
GCA Hidden: 251
100 GCA Finds500 GCA Finds1000 GCA Finds1000 GCA Finds1000 GCA Finds1000 GCA Finds1000 GCA Finds
Post Re: Site Outage 9/1/2019
Well done in getting everything up and running so soon. We all appreciate the work you do to give us access to so many useful tools. =D> =D>


11 January 19 7:33 am
Profile E-mail
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Forum theme by Vjacheslav Trushkin for Free Forum/DivisionCore.