The tag has no wiki summary.

learn more… | top users | synonyms

3
votes
0answers
51 views

How to use Messagequeing in a distributed web application

I am curious for solutions to this problem: Assume we are going to build an online-store. To better scale parts of the architecture, it is broken into independend subsystems. The typical scenario ...
2
votes
0answers
144 views

How to schedule large numbers of user-defined tasks?

Say you're running a website which generates a large number of tasks that should be run at particular times (e.g. reminder emails). You could run a cron which SELECTs a database table for unprocessed ...
1
vote
0answers
29 views

Job queue dispatching based on resource constraints

We have a job-queue shaped hole in our architecture, but it isn't immediately obvious that there's a piece of existing software to fill it. There may be a reason for this, or there might not, so let ...
1
vote
0answers
52 views

Improve communication between controller and trackers in a Twitter fetcher tool using RabbitMQ or Apache Flume

I've been working for a time with some researches developing a tool to fetch tweets from Twitter and process them in some way. The first prototype "worked" but became a pain as we used sockets to ...
1
vote
0answers
384 views

Apply filter in Kafka Queue

Is it possible to apply a filter on the Kafka stream of topic? What I want to do is filter out messages while consuming and then run some custom logic on top of it. It is a sort of thing that Storm ...
0
votes
0answers
40 views

Distributed computing - web services vs message queues

I Need to decide on the architecture for a distributed computing implementation. In the server, there is a database, which holds many mathematical problems, that need to be solved by crunching ...
0
votes
0answers
102 views

When to use a message queue as a web service database?

Does it make sense to implement a message queue system such as RabbitMQ or Kafka for the purpose of storing data in a queue? For example, this could be done in order to handle things such as ...
0
votes
0answers
21 views

Logging and analytics from multiople processes

I am setting up a system on OpenStack comprising of several independently running processes and need a logging and analytics process running which will receive very frequent input from the running ...
0
votes
0answers
104 views

Message Queue: Which one is the best scenario?

I write a web crawler. The crawler has 2 steps: get a html page then parse the page I want to use message queue to improve performance and throughput. I think 2 scenarios: scenario 1: ...
0
votes
0answers
791 views

MSMQ Receive Timeout Even With Items in Queue

Tried posting this on main StackOverflow site, but no replies in 5 days. I am having ongoing problems trying to implement a very simple MSMQ solution: 2 PCs (not servers) running Win7 and Win8 ...