Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I don't fully understand reactive extensions, but my inital reading caused me think about the ETL code I have.

Right now its basically a workflow to to perform various operations in a certain sequence based on conditions it find as it progresses.

I can also imagine an event driven way such that only a small amount of imperative logic causes a chain reaction to occur.

Of course I don't need a new type of programming model to make an event driven collaboration like that.

Just the same I am wondering if ETL is a good fit for potentially exploring Rx further.

Is my connection in a valid direction even? If not, could you briefly correct the error in my logic?

share|improve this question
    
note: i wasn't sure if this should go on SO or here – Aaron Anodide Jun 5 '12 at 23:31
    
If you're still interested in pursuing this, it looks like someone has had the same idea: reactiveetl.codeplex.com – shambulator Nov 30 '12 at 10:28
    
thanks for the link! – Aaron Anodide Dec 3 '12 at 6:17

My first instinct is to say no: Rx is about observable collections and asynchronous programming, whereas ETL is about dynamic data mapping and extraction. I don't see how they are related at all.

share|improve this answer

They seem very related to me too. We're looking at Rhino ETL (of which that Reactive ETL library mentioned by @shambulator is a re-write), but it feels weird having to work with foreach and yield return all the time.

After having done the initial load from the source to the target database, when doing incremental loads using the same logic, the next step would be using your ETL logic on-the-fly, through queues or events: as new data is entered in your sources (databases, files, queues), it's pushed through the pipeline and inserted in the target database.

Seems an ideal candidate for Reactive programming to me!

Actually, I find it very surprising that the Reactive ETL library isn't at all supported or hardly used (there seems to be no NuGet package), or that there don't seem to be any similar libraries! Did you end up using it eventually (or something similar), and how were your experiences with it?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.