I have got experience in big data development with Apache Spark and Scala on on-premise and AWS cloud infrastructure.
Have delivered 6 big data projects for different business domains such as Banking & Finance, Credit Bureau , Media communication, Hospitality.
Have designed, build and deployed 30 + Data Pipelines using tools like Apache Airflow, AWS Data Pipeline, SOS Berlin.
Have worked on CI/CD pipelines integration with Jenkins and GoCD.
Currently working on Data Pipelines to ingest data from data sources like, Adobe Analytics, Google Analytics, Apple, Google, Amazon and derive business KPI's for reporting and dash-boarding.
Tech skill: Apache Spark, Scala, Python, Hadoop, Hive, PrestoDB, AWS Services : EC2, EMR, S3, SNS, Data Pipeline, Athena, Glue, Spectrum, Redshift, API Gateway, Kinesis Firehose, Lambda.