The tag has no wiki summary.

learn more… | top users | synonyms

-1
votes
1answer
80 views

Suggestion on how to fill a web form (several times) [closed]

I need to fill a form using data from a CSV file. I was planning to use CURL+PHP to do it, but then I realized the form has several steps (one on each page), plus it uses javascript to fill hidden ...
0
votes
1answer
170 views

What is the basic process and tools needed for crawling a source code repository for the purpose of data mining?

This all is with respect to Microsoft project CodeBook: CodeBook There is huge amount of code in the repository, many classes , a call hierarchy of functions, testcases etc. I am interested in ...
5
votes
2answers
352 views

Web Crawler for Learning Topics on Wikipedia [closed]

When I want to learn a vast topic on Wikipedia, I don't know where to start. For instance say I want to learn about Binary Stars, I then have to know other things linked on that pages and linked pages ...
5
votes
4answers
2k views

Development of a bot/web crawler detection system

I am trying to build a system for my company which wants to check for unusual/abusive pattern of users (mainly web scrapers). Currently the logic I have implemented parses the http access logs and ...
10
votes
9answers
2k views

Legality/Ethics of crawling a site to basically copy its database? [closed]

A potential freelance client wants a copy of all the phone numbers in an entire country of several million people in CSV format for their telemarketing system. They have sent me a link to an online ...