Azure Data Engineer

Job Title: Azure Data Engineer

Key Skills: Azure Synapse/ADF/Databricks/ADLS/MSBI/SQL

Note - Upon Selection the Consultant will be on our W2

Job Description 

• Should possess expertise in data engineering tools and storage such as Azure Synapse, Databricks, Azure Data Factory and Azure Data Lake
• Experience in implementing automated Synapse pipelines
• Strong Experience in developing Databricks Notebooks
• Strong Understanding of Databricks Architecture, API Integration
• Ability to develop ADF ETL pipelines for transforming and ingesting data into the data warehouse
• Experience with migration of conventional ETL (SSIS) from on-premise to Azure (ADF) environment
• 5 years’ experience with relational database technologies (Azure SQL, Azure Synapse, SQL Server, MySQL or similar)
• Ability to debug data load issues, transformation/translation problems etc.
• Familiarity with Azure infrastructure and the ability to integrate with various services
• Experience working with high volume data and large objects
• Experience working in DevOps environments integrated with GIT for version control and CI/CD pipeline
• Ability to understand data transformation and translation requirements and which tools to leverage to get the job done
• Strong Python/ Spark, SQL scripting skills
• Strong understanding Devops Methodolgy
• Strong understanding of various data formats such as CSV, XML, JSON, PARQUET etc.
• Working knowledge of data quality approaches and error handling techniques
• Good understanding of data modelling for data warehouse and datamarts
• Strong verbal and written communication skills
• Ability to learn, contribute and grow in a fast phased environment
• Good to have: Working knowledge of Azure Logic Apps, Azure Functions