Azure Data Engineer

Job Description

The Data Engineering Consultant has the following responsibilities and duties: 

• Create and maintain optimal data architecture pipeline.
• Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.
• Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into a data warehouse
• Understand existing implementation and should enhance it as needed.
• Work with other data engineers, data ingestion specialists, and experts across cross-functional teams.
• Work on large datasets to meet functional and business requirements

Requirements

The Data Engineering Consultant should have the following skills, education, and experience: 

Skills

      Required:

• Excellent SQL and Python skills
• Experience with Spark programming language(s)
• Experience with Docker/containerization
• Expertise in designing efficient Data Models
• Experience with ETL/ELT and familiarity of Data Warehouse concepts
• Strong knowledge across Microsoft Azure Tools.
• Data pipeline and workflow management tools: Databricks (Spark Python), ADF, Dataflow
• Knowledge of API-based integration and security (Restful and SOAP)
• Experience with Data Visualization tools such as Power BI etc.
• Ability to learn fast and translate data into actionable results
• Excellent written and communication skills with client stakeholders

Preferred:

• Experience with JSON Programming
• Experience developing and supporting scalable data pipelines
• Experience with Snowflake Data Warehouse
• Solid Linux programming skills
• Cosmos DB, Teradata, SQL Server (all nice to have)
• Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)

Education

      Required:

• Bachelors in a quantitative field such as computer science, computer engineering other related disciplines.

      Preferred:

• Masters or PhD in a quantitative field such as computer science, computer engineering other related disciplines. 

Experience

• • An experienced professional will have at least 3 years of professional experience in Data Engineering role with a Masters Degree or 5 years of experience in a Data Engineering related role with a Bachelors degree

Location: Anywhere in Canada

#LI-REMOTE