Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm going to create my own web server that focuses on scalability, rapid prototyping and the use of JavaScript as the server's scripting language, much like node.js. It will use a Model-View-Controller design pattern so a web application can support more concurrent users just by adding hardware -- and not having to redesign the software. Basically, I'm aiming to produce a framework that allows for fast and easy development of cloud applications without the need to write lots of boiler plate code. I've got some questions about this...

  1. How hard will it be to put MySQL in the cloud? How could I go about implementing this and make the resulting product free? Will I have to write my own engine or modify an existing one, if I do what should I watch out for?
  2. To make this scalable I need to adjust from one server to hundreds of servers this creates the requirement for the servers to be load balancing, how should I do this? If I balance based on the work load per server I would need gateway to handle all the incoming requests. Is it the right idea to have all the servers check into the gateway and update there status.
  3. By having the servers run through a gateway if the gateway dies all the incoming requests are ignored. I'm thinking that having all the servers maintain a list of each other, or at least a few I could rebuild the list of servers and establish a new gateway. Is it worth it? Or should I have a backup gateway that could switch out? Should I let the user choose?
  4. How should I pick which server handles the database and which handles the page serving? Should I spread the database so that queries are preformed on multiple servers? Which would theoretically improve performance.
  5. The servers would need to mirror the database at least once so that if a server goes down the database isn't corrupted. So this brings up writing another question, should I broadcast SQL queries so that all the servers can take a bit of the work load? If I do it that way wouldn't a query clog up the network so that other queries couldn't be preformed? What are my alternatives?
  6. Finally, is there a free solution already out there that might need a little modification that suits my needs?
share|improve this question
add comment

1 Answer

up vote 15 down vote accepted

Please take my advice in the nicest possible way. I believe that you need to use wheels other people have invented for a bit before inventing your own, new, wheel.

I say this because it is clear that you don't know the problem space. Your comments indicate that you don't know the point of MVC. You don't know how to put MySQL in the cloud. You don't understand how people actually do load balancing. You are convinced that you need extreme scalability out of the box. And you want your solution to solve every problem under the sun at once.

This is unlikely to turn out well.

If you still wish to proceed, then I will highly recommend that you focus on just ONE piece of the puzzle. If you have a good architecture, then one machine should be able to happily serve a hundred thousand requests per hour. If you try to avoid sharing state in any machine-specific, then a standard load balancer can address any likely scalability needs. So you don't need to worry about scalability yet. For a first pass you can leave mirroring and backups to the database, they already have solutions for that. Which just leaves making it easy to write MVC without a lot of boilerplate. Why don't you focus on that for a bit? (Or look around for MVC stuff built on top of Node.js. It isn't like you're the first person with that particular need either.)

share|improve this answer
1  
+1 for previously invented wheels. –  user1249 Aug 1 '11 at 12:32
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.