Pipeline arquitectures

Lately I've been developing a real time stream application, that delivers content via http long poll requests.

The details of the inner parts are not important, but overall, you can see it as many other applications, which transform, filter and distribute individual pieces of datastructures.  On the macro level, different components are connected via ZMQ or rabbitMQ, or any queuing system you like. Heck, you could use resque/sidekiq for that also.

Inside the components, although I like to use CLOS OOP (for it's inspection capabilities, and sometimes multiple dispatch), there's the idea of pipelines (or streams) which I like very much.

Here are some pointers for this approach I'd like to review:

-  Rob Pike talks. I love this one, which tells the difference about concurrency and parallelism. Also go concurrency patterns. I guess this one is also cool, but I haven't actually seen it, it just has a cool name.
- Talk about pipes in OCaml. This one isn't particularly enligthening if you have some knowledge of streams, as in SICP streams, but it gives a vision for functional programmers.
- Anything you find about pipeline arquitecture is ok.
- Thanks to native coroutines, lua provides nice uses and docs for pipelines. Also picolisp has support for coroutines.
- Full on functional. Coroutines for streaming in haskell.
- HighScalability article about disqus architecture.
- Also, using just active objects is a nice approach to, at least split responsabilities, and make the producer-consumer dance work. Kudos to the great CTM book that explains them super nicely.