Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have personally never done this. I don't understand why so many sites do, if you do your development on a development server why would you ever need to shut down your production site?

I have always wondered about this.

What are they doing during this time, what requires doing this?

share|improve this question
31  
They're replacing the vacuum tubes in the servers. –  mipadi Apr 26 '11 at 21:04
6  
I thought they were stacking the punchcards. –  Christopher Mahan Apr 26 '11 at 21:37
1  
Keep in mind that the site probably does stay up for most updates. Obviously, you only see the ones where it actually needs to come offline for a while. –  Dean Harding Apr 26 '11 at 22:32
    
perhaps one of the things that made Admiral Grace Hopper popular was rediscovered? –  Joset Apr 27 '11 at 14:05

8 Answers 8

up vote 25 down vote accepted

Big kicker for anything with big scale is that if one is changing database schemas in some way, one typically has some big, nasty maintenance scripts to run. Now, these might take a second or so to run with your development dataset. But when you start measuring data in terabytes and petrabytes, even adding a single column to a table can take hours. So no matter how quick and automated deployment is, you've still got data maintenance issues to get through. If you plan really well, you can put up a read-only mirror of the site while you are undergoing the process, but for many sites read only is so pointless as to not be worth the effort.

share|improve this answer
2  
+1 for mentioning difference in scale –  Covar Apr 26 '11 at 21:33
2  
+1 - a read-only stack overflow wouldn't be much good. There's not going to be much you wouldn't be able to find on google :) –  corsiKa Apr 27 '11 at 3:51
3  
@glowcoder: When you search on Google, you find SO answers. –  Donal Fellows Apr 27 '11 at 7:32
1  
Google is massive and sure to have a massive database; how come I never ever see "down for maintenance" for google? (Google.com homepage) –  alexy13 Jun 12 '12 at 18:36
1  
@alexy13 -- google is in a special category of scale where they can't have a single database or even datacenter, parts of the system are always down and they've written the front end to handle it. I would too if you handed me that kind of time and R&D budget. –  Wyatt Barnett Jun 12 '12 at 20:42

There are a number of reasons why you might want to take a site down for maintenance. To name a few:

  • Database changes
  • DAL changes
  • Updating services

Basically, if your site isn't static, when doing a logic update you want to take it down otherwise people hitting your site may receive errors or unexpected behavior.

Also, if you will be touching the web.config (in ASP.NET) for your site, you should take it down for maintenance first as it will blow out the session for users. Thus, if they were in the middle of something, it would be lost.

share|improve this answer
    
the session would be lost if using "In-Process" session state. If you use out of process session state the session will not be lost if the web.config is changed. –  Anthony Apr 26 '11 at 22:18
    
The last point is only true if you're doing in-process sessions, which I hope you're not on a production site! There's more than just touching the web.config that'll take down the worker process. –  Dean Harding Apr 26 '11 at 22:30

Well this is somehow abstract question - I even seen sites which used "Down for Maintenance" instead of HTTP 500.

For web sites you sometimes need to do some upgrade. For example if you are changing database you don't want any other user to touch the database during that time. If database is offline the site must be gracefully turned off as well because showing SqlException is not very nice. Another reason is some HW failure or system failure (like leaking resources) which requires application or even system reboot.

Once I participated on upgrading of internet banking system in one of the biggest bank in my country. The whole process of upgrade web sites, middle tier and databases took three days where system was offline for customers. It also included full backup of everything so in case of failure the system could be reverted to the old version.

share|improve this answer

Servers need patches to be run, and on many operating systems, those patches require reboots. So that is one category of down time. Many companies schedule reboots from patches for low use times, such as Sunday morning. If there are no patches, they reboot the servers anyway at the regularly scheduled maintenance time (this is a hangover from the NT4 days when certain counters overflowed every week and a half, so rebooting weekly prevented other bugs).

One company I worked for had an e-commerce site back in the late 90s that brought in more than $1,000,000 in sales per month. Someone promoted the wrong tax table to the production database server. The cure was to restore the db server from backup, and apply the transactions since the last backup. This took several hours, during which the website was unavailable to take orders. Since the orders portion and the static sales brochures were running on the same site and were inseparable, both had to come down.

One company I worked for had some wrong text inserted into the wrong place and the CEO flipped out and had the website taken off line "for maintenance" while the layout and text were "fixed" and the appropriate victim blamed and fired.

share|improve this answer

While other answers are correct, you can almost always avoid downtime using right architectures. But this has a cost, and this cost may not worth it: an hour of downtime costs amazon or the infrastructure behind NASDAQ a lot. Stackoverflow ? Most likely not so much.

How to avoid downtimes:

  • shutting down hardware serving pages: if you have proxies in front of your website, you can instead put them offline without any impact to the user
  • reconfiguring servers: same as above
  • updating/changing data in databases: you could put your website in read only mode, etc...

Generally, in a layered architecture, the closer to the "top" you are, the hardest it become to avoid downtime, same for stateful (webserver vs database).

share|improve this answer
1  
Doesn't NASDAQ have about 14 hours a day of scheduled downtime? –  Peter Taylor Apr 27 '11 at 12:24

A site may schedule regular downtime even if there's nothing to do every time the scheduled downtime comes around. By doing so, they get users used to the idea that the site will be down for a certain amount of time every so often so that when work does need to be done, users won't complain so much.

share|improve this answer
    
there's a cure for that: bring down the complaints system during downtime :) I've actually seen companies do that. An MMO company bringing down the website hosting the downtime announcement as well as the support forums together with the game being down for maintenance is a good example of that. Anyone who didn't catch the announcement during the few hours it was up before maintenance would never know what was going on. –  jwenting Apr 27 '11 at 8:07

There also is a psychological and marketing side to this. In some of the cases (I dare to say most of the cases but I'm not that bold *g*) reading "Down for maintenance" can also mean "The server has crashed or gone out of service for any other reason".

I've seen this quite frequently. Normally as a developer you'll want a "real" error messages saying something like "Whoops, we're experiencing a hight load right now and not all requests can be handled" but some people from marketing will tell you "dude, you cannot tell the customer that we're having a problem. Tell them that we're on scheduled maintenance - this will look a lot better".

So "Down for maintenance" often is just another term for "out of service".

share|improve this answer

I don't understand why so many sites do, if you do your development on a development server why would you ever need to shut down your production site?

Shit happens. Unless you are doing some form of mathematical verification of your deliverables (and your specs are valid), no matter how careful you are, shit happens.

Also, there are times when you might have to do an change to a key piece of your infrastructure (say, a change to your database structures) that do require a down-time.

Unless you are developing a critical system (say a five-nine or six-nine system), the responsible and cost-effective thing to do is to build a system with the acceptance of down-times as part of reality.

Furthermore, you take that principle further by making down times manageable and amenable to scheduling (or at least detectable) with a clear understanding and procedure for effective recovery.

share|improve this answer
    
Mathematical verification isn't a panacea either; sometimes you find that what you've verified is not what you wanted to verify. –  Donal Fellows Apr 27 '11 at 7:36
    
True. But then I'd argue that the problem isn't with formal verification of specifications, but with the validation of those specs. If your specs are invalid, then obviously everything will fall apart from there, but validation of specs ("are we really building the right thing needed by the intended user for the intended purpose"), that's not the focus of verification (*"given these specs, are we building this thing right, or can it be built?"), informal or otherwise. I guess I should have put a caveat on that (wrt to validity of the specs.) –  luis.espinal Apr 27 '11 at 10:27
    
I'm not arguing you're wrong to mention it. I just point out that there are limits to what it can do. I used to work on formal verification, and the big problem at the time was how to correctly evolve the specifications so as to take into account changing understanding of requirements. Since that's primarily a human problem, secondarily an engineering problem, and only tertiarily a mathematical problem, I don't imagine it's been solved fully yet. –  Donal Fellows Apr 27 '11 at 15:37
    
Oh. I think then we are of like thinking. Changing requirements (and req. validation) are the Achilles' heels of formal methods. Since it is a creative task (due to its human nature), I don't believe it is solvable, not in the way formalists/purists would like it to be. I think that has been one of the failed promises of FM; they got oversold (I mean, for example, formal methods for web development?) The specs have to be highly scrutinized and not amenable to rapid change (and that's typical of critical systems, not highly malleable ones). The later are the norm rather than the exception. –  luis.espinal Apr 28 '11 at 2:27
    
99% of user interfaces aren't to do with formal methods, but rather applied psychology. The remaining proofs are obvious (“don't deadlock the UI”) even if not always obvious to prove. But if you've separated the webapp according to best practices, then formal methods will make a lot of sense in the business methods layer (also in the data storage layer, but that's usually where the standard advice of “don't write your own DB” applies anyway. :-)) –  Donal Fellows Apr 28 '11 at 14:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.