Find Jobs
Hire Freelancers

Setting up a free hosted service for scraping data from a website & saving it into CSV file/DB every minute

$10-30 USD

Terminado
Publicado hace más de 8 años

$10-30 USD

Pagado a la entrega
I am looking for a help finding and setting up a fully hosted service (similar to [login to view URL], [login to view URL]) which will scrap data from a website and save it into CSV files or DB. I should be able to access this CSV files or DB when I log into the hosted service. ****What i need?**** I need to save data (that is ready in a API generated format) from 10 links: [login to view URL] remaining 9 links are: curr1= curr2= USD PLN GBP PLN CHF PLN EUR USD GBP EUR EUR CHF GBP USD USD CHF GBP CHF for example: [login to view URL] etc... which is in a format: {"transactions":[{"rate":"4.2205","amount":"4.73","date":"1440884371","dateText":"4 minuty temu"},{"rate":"4.2205","amount":"195.25","date":"1440883443","dateText":"20 minut temu"},...,{"rate":"4.2202","amount":"120.00","date":"1440879931","dateText":"1 godzina temu"} ]} From which I only need extract: 3 columns x 10 rows of data rate, amount and date i.e. 4.2205, 4.73, 1440884371 4.2205, 195.25, 1440883443 (....) 4.2202, 120.00, 1440879931 I need to save this data into 10 separate CSV files EUR_PLN, USD_PLN etc or a Database Since files can get large, the name of the file can change monthly say, EUR_PLN_201508, EUR_PLN_201509 This CSV files (or data from a DB) should be available for me to download from a hosted service. ****How often does the data need to be scrapped?**** All 10 links should be scrapped and data saved to the CSV file EVERY MINUTE. ****Does the data need to be processed?**** Ideally yes, please note that 3rd column is a time stamp, and data between each instance of scraping might not change, or only the newest row(s) of data is/are updated Hence i only need unique entries to be saved into the CSV, at the bottom of the file. For example lets assume there are only 3 column x 3 rows of data 4.2205, 4.73, 1440884371 4.2205, 195.25, 1440883443 4.2202, 120.00, 1440879931 At time t this data should be saved into the CSV in REVERSED order 4.2202, 120.00, 1440879931 4.2205, 195.25, 1440883443 4.2205, 4.73, 1440884371 Now at Time t + 1 MINUTE there are anly 2 rows of data that are new (please note that last 2 rows are no longer visible, as only most recent 3 rows are visible) 4.2211, 4.81, 1440884673 4.2210, 4.80, 1440884572 4.2205, 4.73, 1440884371 As there are only 2 new rows of data is 4.2211, 4.81, 1440884673 and 4.2210, 4.80, 1440884572 only those 2 should be added to the CSV file - please note that they should be added in a REVERSE order. After 2 instances of scraping the following 5 rows should be saved in the file 4.2202, 120.00, 1440879931 4.2205, 195.25, 1440883443 4.2205, 4.73, 1440884371 4.2210, 4.80, 1440884572 4.2211, 4.81, 1440884673 Similar process should be applied to remaining 9 links. ****Is data above processing of data required or optional?**** If the data cannot be processed as described above i am also ok with all data being saved into CSV file (3 columns x 10 rows) every minute, without distinguishing new data. ****Which solutions are acceptable?**** I am looking for a FREE hosted service (probably [login to view URL] will work or [login to view URL]) that will be able to do the above, scrap 10 links EVERY MINUTE and save data to CSV file that I will be able to download myself Google Docs, doesn't refresh data correctly every minute - so is NOT acceptable A small VBS JAVA script that I will be able to run on my Windows server every minute via task scheduler is also acceptable (the code has to be open source, ie not compiled) ***What am I expecting?*** In case of a hosted service, i am looking for a step-by step instruction how to configure it In case of a script, I am looking for such a script to be sent to myself.
ID del proyecto: 8363399

Información sobre el proyecto

4 propuestas
Proyecto remoto
Activo hace 9 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
Adjudicado a:
Avatar del usuario
I can create simple Perl script for you in couple of hours. The code will be really simple and you can edit it later. All you'll need to run it is Perl installed. But it's free. Thanks. Roman
$35 USD en 0 día
4,9 (78 comentarios)
5,4
5,4
4 freelancers están ofertando un promedio de $138 USD por este trabajo
Avatar del usuario
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
$250 USD en 5 días
4,9 (99 comentarios)
6,5
6,5
Avatar del usuario
I can do this using java or python, these two can be configured via windows services and will run automatically depending on the timing you set-up. I can send you both the source code and the set-up file so you only need to edit the set-up file for any changes needed. Send me a message so we can discuss further.
$111 USD en 2 días
0,0 (0 comentarios)
0,0
0,0
Avatar del usuario
Hello, I will create a script for you (in Perl if that matters) that can be run every minute to gather data from that API for those currencies to store the results in a database (MySQL) and then dump the results into CSV files for your download (sorted with the most recent results as the head of the file). I know of a free site where you can receive all these features. I will configure the site for your use along with instructions for all that I did to make it functional. A milestone payment for the full budget for this project must be deposited with this site before your offer can be accepted. Alan Idler Chief Software Architect Idleswell Software Creations
$157 USD en 3 días
0,0 (0 comentarios)
0,0
0,0

Sobre este cliente

Bandera de UNITED KINGDOM
London, United Kingdom
5,0
2
Forma de pago verificada
Miembro desde may 6, 2013

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.