Data Engineer with Design Skills Salary

Develop software and utilize commercial and open source software to interface big data and relational solutions Design and develop schemas and data acquisition, transformations, and data integration for large scale media consumption and advertising analytics Design and implement solutions for metadata, data quality, privacy management Develop, maintain, and document ETL processes Audit data loading process and monitor data anomalies Develop and maintain third party API processes Maintain and support multi terabyte data stores and data marts Work with data scientists to support a scalable data visualization platform Design, develop and maintain data aggregation, summarization jobs.

set up, tune, design, as well as newer unstructured data tools.

Demonstrated an understanding of design best practices and operational requirements of data warehouse solutions such as sizing, scaling, fault tolerance, performance tuning, security etc.

Design, develop, document, and test advanced data systems that bring together data from disparate sources, making it available to data scientists, analysts, and other users using scripting and/or programming languages (Python, Java, C, etc).

Hold architecture and design reviews with other team members.

Attend design meetings and recommend security and data improvements – and will be empowered to work on those prioritized changes.

Lead the design, implementation, and continuous delivery of pipelines using distributed AWS based big data technologies supporting data processing initiatives across batch and streaming datasets.

Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions.