[Wikia-l] File outage

[Artur Bergman]

Hi,

Today we had a failure of our main file server, unlike all our other  
server this is not fully redundant, we moved to a backup fileserver  
that was set to readonly, we are working on switching back right now.
The reason was a heat surge in one of our datacenter rooms in Market  
Post Tower in San Jose. The file storage array shut down in response  
to this.

1) We are going to investigate the source of the heat surge
2) We are going to move the primary fileserver functionality to our  
new room which is cooler.
3) Work on getting the fileserving system transparently failsafe

Sorry for the problems
Artur

ps. The reason we had the database readonly mode this morning was to  
move primary database server from db3 to db4. db4 is not only faster,  
but located in the new datacenter room.