Find Jobs
Hire Freelancers

AWS Lambda to extract static assets from CodePipeline Artifact to S3

€30-250 EUR

Cerrado
Publicado hace casi 7 años

€30-250 EUR

Pagado a la entrega
Dear freelancers, we need a simple AWS-Lambda function that may be attached to AWS-CodePipeline. The CodePipeline-artifact passed to the Lambda-function is round about 200 MB to 300 MB in size with about 10.000 assets that need to be extracted to S3. Only files with definable file-extensions within definable folders and their subfolders should be extracted and synced with an S3-bucket. Therefore, an JSON-String will be set at CodePipeline invocation of Lambda-Function. *Parameters that should be set in CodePipeline:* - Folders (array of Strings containing foldernames, each with trailing slash) - Fileextensions (array of Strings containing f.e. ".js", ".css" and so on) - Target-Bucket (string - may be in another region as the codepipeline) *Rules* - files outside the defined folders should be ignored. - files with other file extensions should be ignored. - the file-extension must exactly match - while *.js is allowed, *.json might not. *.[login to view URL] would be okay, if *.js is allowed. - files existing on S3 (that match the rules for folder and file-type above) but do not exist in the Artifact should be removed from S3 (they may have existed in a previous version). - Only changed files should be copied to S3 (because of the amount of assets) - The folders in S3 may contain more file / objects then AWS API command listObjects() will return in a single request *Performance / Speed is really important.* - The whole process should take as less time as possible best much below 1 minute. - The listObjects() command should only be executed against the listed folders because the whole bucket may contain 250k files and more in folders, that are irrelevant for the sync-process. - The sync should be processed async so that putObjects & deleteObjects operations are processed in parallel but the number of parallel putObject and deleteObject operations should be capped so that they don't exceed AWS limits or result in a out of memory exception. - On production, the lambda-function will get max Memory possible assigned. Node.JS ore Python may be used (if possible with one of them) to develop this functionality. If you have any questions, feel free to ask.
ID del proyecto: 14309805

Información sobre el proyecto

7 propuestas
Proyecto remoto
Activo hace 7 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
7 freelancers están ofertando un promedio de €361 EUR por este trabajo
Avatar del usuario
Hi I really don't recommend to add this much heavy processing in Lambda, as it'll definitely timeout even if you give it maximum execution time of 5 min. Instead I'd want to know the entire pipeline process, so that I can propose a better solution for your use case. BTW I'm a super expert in AWS. Ranked #1 in AWS Freelancer Category. You can have a look at my profile to view my past AWS experience. https://www.freelancer.com/u/joystick220.html Thanks Joy
€235 EUR en 3 días
4,9 (70 comentarios)
6,7
6,7
Avatar del usuario
Hi sir, I am scraping expert, I have did more than 350+ scraping project, please check my feedback then you will know. Can we discuss more details about this project? then I will provide example data/script for you. Thanks, Kimi
€311 EUR en 6 días
5,0 (39 comentarios)
6,0
6,0
Avatar del usuario
Hey there, I have extensive experience of writing Lambda functions using Nodejs/Python and Java. I've also configured Lambda to use PHP as well, so I'm very much well versed in developing Lambda functions. You can confirm this by checking out my profile page, you will see lots of Lambda related projects. I've worked with almost all of the Amazonn AWS services as well including DynamoDB, CodeDeploy, ElasticBeanstalk, Elastic Load Balancer, etc. Please feel free to ask if you've any questions. Thanks
€399 EUR en 15 días
4,8 (54 comentarios)
5,7
5,7
Avatar del usuario
Though I am new here but my team has 7 years of experience into AWS Services. Can very well execute this Project
€555 EUR en 20 días
4,6 (5 comentarios)
2,2
2,2
Avatar del usuario
Hi there, First point I would like to make is your requirement of less than 1 minute upload time is not practical / technically feasible. Apart from this particular requirement I will give you a lambda function written with Python that will satisfy all other requirements.
€250 EUR en 15 días
5,0 (3 comentarios)
1,6
1,6
Avatar del usuario
we can start ur work immediately. we have 10 years experience in coding and development. we are expert in writing programs in JAVA, .NET, C, C++,C# we have experience in hadoop,python coding,raspberry Pi django PHP Arduino Magento metatrader web design HTML,HTML5 graphic design wordpress Android unity3d
€555 EUR en 3 días
5,0 (1 comentario)
0,1
0,1

Sobre este cliente

Bandera de GERMANY
Lüdinghausen, Germany
0,0
0
Forma de pago verificada
Miembro desde abr 5, 2017

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.