What are the responsibilities and job description for the Data Engineer position at Rapport IT?
Job Description
Roles and Responsibilities:
- End to End responsibility for data modeling (OLTP, OLAP)
- Data analysis experience with writing complex queries, unions, joins and aggregations.
- Data analysis experience using major RDBMS solutions like snowflake, redshift etc
- Experience with ELT, ETL processes using glue, glue brew transformations.
- Use airflow to schedule and time data transfers.
- Understanding of deep JSON structures and partitioning using Kafka, spark, Scala, s3 etc.
- Work with Data Scientist team to build segmentations, ML use cases, forecasting etc
- Experience working with sagemaker, jupyter notebooks for deep data analysis.
- Work with the BI specialists to design develop and enhance connectors to get closer to business use cases.
- Migration of existing custom pipelines to a normalized connector approach
- Help educate CT teams on data integration, validation standards and drive clean ingestion egestion patterns for the platform.
Minimum Qualifications:
- Work with APIs to extract and ingest data.
- Work with virtual warehouses and configure them for optimal performance and efficiency.
- Conduct ETL data integration, cleansing transformations using glue spark script.
- Work on aggregations of data coming from applications and apis to store results in a historic table.
- Minimum Experience of 5 years with streaming data analytics and building of streaming pipelines and connectors.
- Experience with connections to BI solutions like tableau which include configurations of roles, policies within aws
- Leverage Lambda, glue and step functions to cleanse and transform data.
- Work with DWH technologies like EC2, s3, redshift, athena, snowflake to churn large data sets and partition them in readable formats and in real time
- Bachelor’s degree in Computer Science, Electronics Engineering, Information Technology, or a related field.
Salary : $60 - $70
Talend Data Engineer
NTT DATA Americas, Inc -
Plano, TX
Mechanical Engineer
Aligned Data Centers -
Dallas, TX
Support Engineer
Smart Data Solutions LLC -
Plano, TX