What are the responsibilities and job description for the Data Engineer position at ATC?
- US CITIZENS ON W2 BASIS ONLY*****
- Create and upd
- Ingest data into the Enterprise Data Lake from various database/file sources
- Transform data and create common tables using Pyspark and SQL
- Load data into MySQL from the Enterprise Data Lake
- Analyze data using SQL and Pyspark
- Create and maintain updates to AWS resources
- Work closely with the front end developer on the API specification
- Build test scripts in Python to test data replication processes
- Work closely with the product manager and end users to understand requirements
- Experience requirements - at least 3 - 5 years of experience with SQL, Python and cloud computing (AWS preferred)
- Languages and Frameworks: Python, SQL, Pyspark
- Core understanding of AWS services including IAM, Lambda, S3, RDS, ECS and EC2
- Experience developing Databricks jobs and pipelines
- Other: Docker, containerization concepts, REST API concepts (calling REST APIs using Python)
Terraform, Github Actions, Datadog, AWS Database Migration Service, Apache Airflow, REST API development in Python
Data Engineer
Berkley -
Urbandale, IA
Data Engineer
Integrated Resources, Inc ( IRI ) -
Johnston, IA
Data Engineer
MDA Edge -
Johnston, IA
