Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We are using Tornado for our system (behind nginx) -- and everything is working fine.

Now we need to integrate a client lib (for Neo4j Graph DB) into our system. The problem is that it is blocking.

My question is:

  • is it OK to use blocking libs in non-blocking environment
  • what are the possible ramifications if we do so
  • how much work is involved if we decide to roll our own async lib

I know there is Node that does everything in non-blocking fashion. But we can't go there because our core is written in Python.

share|improve this question
add comment

1 Answer

up vote 6 down vote accepted

is it OK to use blocking libs in non-blocking environment

No. A non-blocking server works because everything is non blocking. The moment you use any blocking code you create a huge bottle neck.

Non blocking servers use a single process/thread and an event loop. The moment you block inside the process your blocking the entire HTTP server.

This will not scale.

how much work is involved if we decide to roll our own async lib

Not that much. Just put the existing library in it's own process then make a thin async API that talks to that process through stdin/stdout.

share|improve this answer
    
typically the server clones itself at startup via fork/exec and runs N instances of the event loop. So technically the entire HTTP server isn't going to be blocked, but close enough –  Kevin Aug 9 '11 at 14:52
1  
@Kevin that's implementation specific. I don't know what tornado does. I know node.js doesn't fork/exec itself for you. It will just block the entire HTTP server if your in a while(true) loop. Personally I would consider it black magic if my web server forked itself on startup without me telling it to do so. –  Raynos Aug 9 '11 at 15:05
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.