Develope small scraper / crawler

$10-30 USD

Terminado

Publicado

hace más de 8 años

$10-30 USD

Pagado a la entrega

I am looking for a guy who can realize the following project: In Google, the following search string is inserted: "[search keyword] site:*/impressum" What then shall be done is the following: - The urls of all findings are extracted - The urls shall be saved in a DB - The urls are cut that way that just the domain name with the tld (with http://www. in front) is saved in the DB Example: - finding is [login to view URL] - saved in DB: [login to view URL] (if the domain is already in DB, it is not saved twice) - I will provide a list of proxies to be used for that project - The project is done when a test with a certain keyword (32,900 matches) was successful

MySQL

PHP

Python

ID del proyecto: 8990966

Información sobre el proyecto

9 propuestas

Proyecto remoto

Activo hace 8 años

¿Buscas ganar dinero?

Dirección de email

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto

Cobra por tu trabajo

Describe tu propuesta

Es gratis registrarse y presentar ofertas en los trabajos

Adjudicado a:

@czrain

I'm an experienced Python developer with 10+ software engineering experience. Worked for a couple years at Google, amongst others. For this, I would probably go with Selenium browser automation, for one very particular reason: Google tends to not like web automation using eg. crawlers and puts script obstacles in the way, whereas a real (or emulated browser) will do just fine. Otherwise, this seems like a decently simple task, the only consideration is that google gives you results gradually so browsing through all the pages will take a while. (Especially if there's 32900 matches!) Also, acceptance criteria is weird, you're probably aware that the number of results google gives you is an estimate and there isn't an actual 32900 individual links.

$50 USD en 1 día

4,9

(4 comentarios)

3,0

9 freelancers están ofertando un promedio de $81 USD por este trabajo

@sknadeem123

hi i am ready to provide this type application for easy to use in which technology you wish to use ? Java or Web apps ?etc ?

$111 USD en 4 días

5,0

(17 comentarios)

4,5

@polaris0621

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

$157 USD en 1 día

5,0

(8 comentarios)

4,5

@deescorp

I already have a product like that. the application is build on C# and takes input of search query in files format. Then search the google,bing, yahoo and save the results in database of mysql. I can give you the product so that you can use it on your own. Let me know and i will modify that according to your needs. Thanks

$55 USD en 3 días

4,9

(10 comentarios)

3,9

@owenzhu

expirenced scraper, has crawled itunes app store, twitter, sina weibo, etc, good at database as well

$55 USD en 3 días

5,0

(8 comentarios)

3,8

@marketiplace

I got 7+years work experience in Data Collection,Bulk Email Campaign,Excel VBA and Internet Research in IT companies here.I can do create script and scrap datas from site Directory and yellowpages using C++,Python and Perl coding as per your requirements in excel with multiple ip rotations.I have dealt with US,UK and Australia companies President,Directors and Managers for web design and development projects successfully and I have Good Communication with writing skills.I am well versed in Internet,MS Office Applications and Phone Etiquette manners with latest Technologies.I can accept your payment terms.

$55 USD en 1 día