Subject:  State of the service report |
Author:
Darkstone News
|
[
Next Thread |
Previous Thread |
Next Message |
Previous Message
]
Date Posted: Sat, Sep 07 2002, 03:10pm
State of the service report
MrTact
(Lead Server Engineer, Ultima Online)
09/07/02 10:26 AM
Some of you may have seen the announcement that we will be leaving claiming on and resuming autorefresh at least through the weekend. I thought I would take a moment to try and explain in a little more detail the chain of events that led to this state of affairs.
As we mentioned in a previous FYI, we experienced some problems with the entire service due to a data center move performed the week before last. What this did was expose some issues with the way the shards connect to the housing database. The lag experienced during that time period is directly attributable to this.
Since we had no way of knowing when the networking issues would get resolved (indeed, we were still seeing them intermittently as recently as Wednesday), we determined that it was important to put a fix into production. This is the code that was pushed out via an emergency publish late last night.
For the most part, this has done well. We have even seen reports from players of decreased overall lag due to the networking improvements that were made. However, since doing this publish, many shards have been experiencing intermittent connectivity problems with the central housing server. As a result, several shards have been unable to conduct housing transactions for most of today.
We considered rolling all the shards back to the previous published state, which does not have this connection problem. However, we feel that the issues addressed by last night's publish are severe enough to warrant leaving several shards without the ability to conduct housing transactions, since these issues could potentially cause the entire service to grind to a halt should we experience problems with our west coast facility again.
Contingent on last night's publish was a new release of the housing server which would update players' houses' status as their account status changes in (near) real time. Since we can't put this new housing server code into production until we have fixed the problem with shards being able to connect to it, and we can't say with certainty that all of these issues will be resolved by Tuesday, there are people at risk to lose a house. We are therefore opting to resume autorefresh in order to avoid this undesirable outcome.
To close, I want to apologize; this is certainly not the way I would prefer the game operate. Unfortunately, we've been reduced to making a choice between difficult options at this point. We will continue to work diligently until these problems are resolved, and hopefully we can look at a much cleaner and more pleasant (re)rollout of phase III of housing sometime next week.
MrTact
[
Next Thread |
Previous Thread |
Next Message |
Previous Message
]
| |
Feedback: 