What are the responsibilities and job description for the Data engineer position at Nalashaa LLC?
Job Details
Role: Data Engineer
This position will focus on developing various database and data loading/transformation efforts. Initiatives could include multiple source files and/or files sourced both internally and externally. Candidate is required to interact with databases using AWS Lambda, AWS Glue (for transformations) Python and Spark. Possible work leveraging SnapLogic for ETL, depending upon use case. Work will be applicable to multiple applications. Knowledge of automated testing of data a plus.
5-7 years of experience required.
Python, Spark and database technologies
ETL: Glue
Database: snowflake
AWS Glue, IAM, RDS, S3, SQS, SNS
Responsibilities:
Design and built systems for analytical reporting
Analyze data and design ETL frameworks using Python and Spark technologies
Collaborate with business partners to gather requirements and translate that to technical specifications use.
Adhere to lean agile methodologies
Production Support including trouble shooting and problem-solving capabilities
Implement data pipelines to meet business requirements and adhere to best practices. Required Skills:
Proficient in Python, Spark and database technologies
Experience with various AWS services like AWS Glue, IAM, RDS, S3, SQS, SNS, Lambdas, Cloud Formation, Cloud Watch, Event Bridge, CLI
Experience with Unix/Linux
Experience with SQL
Experience with source code control, preferably Git.
Experience in CI (Continuous Integration)/ CD (Continuous Delivery) software development pipeline stages like Commit, Build, Automated Tests, and Deploy