Repost from here as I think it may be more suited to this exchange. I'm trying to implement DRUM (Disk Repository with Update Management) as per the IRLBot paper (relevant pages start at 4) but as ...
Im running a service that crawls many websites daily. The crawlers are run as jobs processed by a bunch of independent background worker processes, that picks up the jobs as they get enqueued. Now ...
What is the basic process and tools needed for crawling a source code repository for the purpose of data mining?
This all is with respect to Microsoft project CodeBook: CodeBook There is huge amount of code in the repository, many classes , a call hierarchy of functions, testcases etc. I am interested in ...
I am trying to build a system for my company which wants to check for unusual/abusive pattern of users (mainly web scrapers). Currently the logic I have implemented parses the http access logs and ...
A potential freelance client wants a copy of all the phone numbers in an entire country of several million people in CSV format for their telemarketing system. They have sent me a link to an online ...