I am a lead engineer for a highly trafficked eCommerce website (upwards of 1m page views an hour). For various reasons we have the opportunity to rebuild large portions of our infrastructure. This brings up a number of interesting problems in balancing flexibility, stability, speed to market, etc... At a high level I am interested in how others have handled similar situations. In particular I am want to know how others have architected their site to provide stable deploys in fast moving environment.
- One of the main trade offs I am looking at is breaking our site by functional area and providing them each their own sub domain. The primary driver for this is that we are hosting on Azure and deployments are all or nothing.
- I am also interested in how others have maintained architectural integrity in a fast moving environment and team composed of varying levels of experience.
We are primarily a Microsoft shop: SQL Server, Windows Server 2008, .Net 4.0, Visual Studio 10, etc...
We have also made the decision to host our main site on the Azure platform and are making heavy use of table and blob storage as well as app fabric cache. We are considering using multiple Azure data centers but have not made that final decision as of yet.
Our site is built using .Net MVC 3 and is supported by a DDD style architecure. Our data access libraries and business rules are fairly well encapsulated and separate from actual display logic.
While not a true agile shop we iterate very quickly with frequent deployments, multivariate testing, just in time requirements, etc... We are generally very entrepreneurial and have the need to respond quickly to new opportunities as they arise. (code for "it can get chaotic")
Our development team is made up of a couple dozen developers with varying levels of skill and experience. While some of the developers can operate independently others need to have very careful code reviews done.
The QA team has a very thorough manual review process. They are also beginning to build a suite of QTP tests to automate regression testing. The dev team in turn makes use of Unit Testing and BDD testing when appropriate.
I know there are a lot of strong opinions out there so to preempt religious wars here are a couple of responses that will not be helpful to me.
- Just use Java, Oracle, PHP, Ruby, etc...
- Just use EC2
- Just ask your developers to be more careful
- Just tell your developers to code faster
- Just tell "the business" to slow down
- Just use Google Checkout
Who I'm looking for feedback from
I know there are many engineers who, while very skilled, have never worked on systems that supports more than a few hundred concurrent users. The lessons learned from a site supporting 30,000 concurrent users are very different from those of an internal support app (I've worked on both, btw, and am not disparaging internal apps. They just require a different approach).
If you have experience in similar situation I would love to hear how you approached the problem and what the draw backs of your solution where.