Deeper understanding of event loops and timers [closed]

Question

I have a basic grasp of accepting clients without creating a thread for each connection using tools such as select and modern equivalents (kqueue, epoll).

But where do I go from here? e.g async I/O and (periodic) timers. I could crack open the source to libev and friends but that's a little overwhelming to me to say the least.

I've looked around for textbooks on this subject but didn't come up with anything useful.

The basic objective is to write a small toy implementation of an evented library with timers and async I/O. Just for getting a better grasp on the subjects.

I'm not looking to reinvent the wheel. This is solely for learning/educational purposes. (not homework either)

Answer 1

Briefly, you can find some kind of discussion in Pattern-Oriented Software Architecture vol.2 - you're asking about (and I'm roughly describing below) the Reactor pattern.

You can also read an implementation of this in Doug Schmidt's ACE library, but like most real-world libraries, it's covered in portability warts and is pretty old-style code now anyway.

---

So, you have a simple select/poll/whatever event loop, and want to implement I/O callbacks, periodic timers and async I/O.

1: Basic I/O

The generic (that is, reusable in a library sense) way to dispatch the events is to provide some abstract interface a client can implement. If you look at the select/poll/whatever interface, you'll see the events associated with a file descriptor tell you exactly what you need:

class IOHandler {
public:
  virtual ~IOHandler();

  // lib needs fd for syscall
  virtual int getFD() const = 0;

  // lib needs to know what flags to set
  virtual bool readFlag() const = 0;
  virtual bool writeFlag() const = 0;
  virtual bool errorFlag() const = 0;

  // lib needs to dispatch callbacks
  virtual void onReadable() = 0;
  virtual void onWriteable() = 0;
  virtual void onError() = 0;
};

Now, if you have some vector of IOHandler* or whatever, you should be able to write the loop.

Note that error handling, allowing handlers to remove themselves during the loop etc. will take a little more effort.

2: Timers

You'll notice that your syscall of choice has some timeout parameter. We'll provide another interface for timers:

class Timer {
public:
  virtual ~Timer() {}

  // some kind of period
  virtual unsigned period() const = 0;

  // some kind of callback at the end of each period
  virtual void fire() = 0;
};

Now, your library needs an interface to add & remove those timers. Once you have them, you can maintain some container of Timer*, ordered by next absolute firing time (a heap is traditional).

Then, every time you call select/poll/whatever, you just get the next expiry time from the front of the heap and use that for your timeout. If you get ETIMEDOUT, dispatch timers from the front of your heap until you reach one in the future.

We're talking only about periodic timers here, so each time you pull one off the front, you should add the current time to its period, and re-enqueue it.

Again, handling one-shot or variable-frequency, allowing timers to remove themselves etc. is left as an exercise. If your regular I/O callbacks might be slow (compared to the period of your timers), you may run a timer dispatch cycle after handling those callbacks too.

3: Async I/O

The usual POSIX mechanism for this is to make your sockets non-blocking (this generally doesn't work for files). Once you've done that, you also need to add a lot more state management to handle cases where

• you couldn't complete writing a buffer, so you need to keep it around somewhere (and make sure you check for the fd to become writeable again)
  • any future writes will now have to be queued or appended to that buffer until the socket has caught up
• you couldn't read as much as you want, so you need to keep a partial message around somewhere (until the fd becomes readable) and you can reassamble the message

This can probably be handled within the I/O framework sketched above, but there's no way I'm going to implement it here.

Answer 2

A linux application would use select() or poll() in a loop and then forward to the correct handler for each file-descriptor + event pair. A similar construct with the GetMessage() function is used in windows.

Boost contains a stand-alone version of async IO with event timers (though in classic Boost form it's a bit complicated).