Senior Data Engineer

SENIOR Data Engineer

Location: NYC, NY

s Holders

3 days a. week onsite

Pyspark expertise, Scala, Databricks and AWS are a must have.

Qualifications:
Expert-level understanding of Agile, SDLC, CI/CD, DevOps, GitOps, and ProdOps is required.
Expert-level expertise in SQL, Python, Scala or similar; other object-oriented programming experience a plus
Expert-level experience in data architecture, data types/formats, optimizing code and ETL jobs, data architecture, data modeling, data contracts a key requirement for successful collaboration with data architects and visualization developers.
Expert-level professional hands-on experience in extensible data pipeline development using Databricks, Alteryx, AWS Glue and Lambda functions.
Expert-level professional hands-on experience with RDBMS, ETL scripting tools and IDEs, data pipeline development, and management using orchestration/monitoring tools.
Expert-level knowledge and experience in implementing software engineering security best practices and contributing to design/engineering for DevSecOps.
Expert-level experience with master data management strategies, rules-based engines, and tools for multi-stage data correlation on large data sets.
Must have experience managing deployment and leading the release of code to production.
Expert-level understanding of data visualization tools and requirements for business customers.

Responsibilities:
Work on migrating applications from an on-premises location to the cloud service providers.
Develop products and services on the latest technologies through contributions in development, enhancements, testing and implementation.
Develop, modify, extend code for building cloud infrastructure, and automate using CI/CD pipeline.
Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
Perform problem analysis, data analysis, reporting, and communication.
Work with peers across the system to define and implement best practices and standards.
Assess applications and help determine the appropriate application infrastructure patterns.
Use best practices and knowledge of internal or external drivers to improve products or services.

Additional Skills & Qualifications

Expert-level experience working in agile setting or being an agile best-practice mentor.
Expert-level experience working in Jira, confluence, and contributing to engineering team documentation/playbooks.
Proven experience influencing and partnering with product management, Product Owners in translating business requirements into data and engineering specifications.