Find Jobs
Hire Freelancers

dynamic CSV to SQL server import - automated using SSIS C#

₹1500-12500 INR

Cerrado
Publicado hace alrededor de 6 años

₹1500-12500 INR

Pagado a la entrega
Project Requirement: I want to create a solution to load a dynamic csv file to SQL table. Following requirements need to be met: 1. The input CSV/TXT file could have any number of columns. 2. Column Delimiter of input file could be comma(,), pipe (|), tab (<tab>), basically it could be different for different file but will be standard in a single file 3. Row delimiter could be [CR][LF] or [LF] or [CR] etc 4. The solution should load all the contents of this file to a SQL server table. (sql version could be 2012/2014 etc) 5. The destination table will be created based on the column names in the header row of the input file 6. All the destination table columns should have sql data type nvarchar(max) 7. Destination table should have PAGE type compression in SQL server 8. Preferably the solution should be a SSIS package or a programatically generated and executed SSIS that loads any files into a table it creates. 9. Solution should just accept a path to the CSV/TXT file and should be able to figure out everything else automatically (number of columns, create table and then load) 10. I should be able to use this to load to SQL 2012, SQL 2014, SQL 2016 Performance requirements 1. The input files I have are very heavy 50GB, 70GB, 80GB etc, so the solution needed should be able to work with such files and also load the the data quickly 2. I already have a few solutions: a) Load csv into a single column table and then parse the column names and data to a table will multiple columns, this process is too slow, so its not acceptable b) use C# script to read header row from csv file, create a table and then load the data using C#. This process is also slow So the requirement is to have the solution as fast as a BCP task in SSIS. I am able to build a static BCP where filename is provided, column structure is static, this works fast but need is to make this dynamic. Copyright requirement: 1. Cannot use any thing that is patented to someone 2. Cannot used a paid library, SSIS addon, every thing should be developed in house 3. you can simply use C#, SSIS, script component etc.... microsoft provided features, so there is no question of using anything copyrighted Documentation Requirements 1. Everything should provide should be documented, so it can be easily reproduced by me on a new installation of SQL/SSIS/Visual Studio etc 2. Code needs to be commented, so every line to code has an explanation
ID del proyecto: 16256939

Información sobre el proyecto

17 propuestas
Proyecto remoto
Activo hace 6 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
17 freelancers están ofertando un promedio de ₹9.376 INR por este trabajo
Avatar del usuario
Hi, I have gone through your REQUIREMENT. I can help you in this as I have much EXPERIENCE in Asp.net, C#, MVC, JSON Web services , ATOM Web services. Can we discuss the REQUIREMENT in detail? We work on different platform like:- . Website designing/ website developing(Asp.net, C#, html, css) .server side scripting language(php) .wordpress Asp.net C# MVC Wordpress Html CSS Web Development and Web Designing. JSON Web services ATOM Web services E-commerce Web development
₹10.000 INR en 20 días
4,9 (16 comentarios)
5,0
5,0
Avatar del usuario
Hi! Read your project proposal about fast import of CSV data to SQL Server. This is a really interesting and challenging task for me, I really want to do this. I have a 15 years experience on MS SQL database development, administration and software development. Have a good knowledge and many completed projects on SSIS. I'd like to know some more about your performance issues with previous attempts - what time on 50GB file (for example) you consider acceptable? Please, message me to discuss this and some other details.
₹8.777 INR en 10 días
5,0 (8 comentarios)
4,8
4,8
Avatar del usuario
I have 8 + years of experience of working on SSIS & SQL server based technologies. I have completed these typed of projects in the past .
₹13.333 INR en 6 días
5,0 (3 comentarios)
4,9
4,9
Avatar del usuario
I have ready package to load a file and import data , i will do for all files as well if you would like to take it . please let me know
₹8.888 INR en 2 días
4,9 (18 comentarios)
4,4
4,4
Avatar del usuario
Please review my profile. https://www.freelancer.com/u/vw7590795vw.html I have 8 years and 3 months experience in SQL and 5 years and 5 months experience in SSRS, SSIS. Please let me know more details and when we will start. Sky pe: jikenjd
₹6.500 INR en 5 días
5,0 (23 comentarios)
4,2
4,2
Avatar del usuario
A proposal has not yet been provided
₹13.333 INR en 10 días
4,4 (9 comentarios)
3,7
3,7
Avatar del usuario
A proposal has not yet been provided
₹12.500 INR en 3 días
0,0 (0 comentarios)
0,0
0,0
Avatar del usuario
A proposal has not yet been provided
₹9.288 INR en 5 días
0,0 (0 comentarios)
0,0
0,0
Avatar del usuario
I have worked similar project where i have to load hedis measures. It is a healthcare quality measure project.
₹6.666 INR en 3 días
0,0 (0 comentarios)
0,0
0,0
Avatar del usuario
I am currently a Business Intelligence Consultant, where I work with ETL and also with reports and Dashboards. With ETL I work with SSIS, where to read the data of a text file to load in a table of SQL, to me it is quiet, probably what will give more work is the question of the file being mutito heavy, but already I worked with tables where they had 100 million records. I am available.
₹8.888 INR en 10 días
0,0 (0 comentarios)
0,0
0,0
Avatar del usuario
Hi... It's Easy to do this work in normal form when we insert data from any type of file like (CSV,text,docx) to database....But when we insert 50GB of data it mean it carry lots of rows ... and lots of time... so don't worry I will do this work in proper way ... Now I tell something about myself I'm Akash and I'm software developer (.net Specialist) I currently working on CBS(Core banking solution) project . in that project Also have same cases when we insert lots of customers data from Database to file and file to Database... so I have good experience for how we do your work... I'm already done this job.. Thanks..
₹8.888 INR en 3 días
0,0 (0 comentarios)
0,0
0,0

Sobre este cliente

Bandera de INDIA
India
0,0
0
Miembro desde feb 5, 2013

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.