is there a good pattern for how to send multiple calls to a web service but without taxing it and ensuring the data is sent back? I don't know enough to correctly describe the problem to even start googling it properly - current google results compare streaming vs. non-streaming wcf answers?
Scenario: I am working on an app (I'm a jr. dev.) that needs to gather information from several sources about a 'customer' and the domains they own
Technical: for one of the sources, I need to send a string array of domains to a web service and this web service returns an entry for each domain name, but this list of domain names will be thousands long - I would like to attempt to divide this list into bite-size chunks (1k domain names each) and then... queue them up to send to that web service, but ensure the web service doesn't skip one
PseudoRequirements: Consumer of web page does not care how long it takes, but would like a list of results up front that does not require pagination to navigate.
Current Theory: Should I take my massive 30k list, break it into 1k chunks, stuff each 1k-sized chunk into a 'request' object, assemble those 'request chunks' into a 'request chunk list' and iterate over that list (sequentially / blocking, so I don't strangle the WS) and for each 'request chunk' get back a 'response chunk' assemble those into a list, and then pass that list back into the front end for viewing? is this a viable method? is there a better way to queue items? Does anyone know off-hand any useful articles for this sort of queuing? are there any 'gotchas' or additional items to consider before I attempt my first pass?
Additional Edits: -I do not have full control to the receiving service, I can not view it's code and the developers that manage it are... less than responsive to email. I do not currently know the stress testing limits of the web service. I emailed the owners of that component but have yet to receive a response - I was going to work up my design while I waited on them.