Senior Manager of Engineering

Job Overview:

The Sr. Manager of Engineering will play a crucial role in designing, developing, and maintaining large-scale data applications in a big data environment. This position requires a strong technical leader with hands-on expertise in Spark, Scala, and Hadoop, and experience managing off-shore development teams. The ideal candidate will have a deep understanding of ETL processes, work closely with various cross-functional teams, and ensure the timely delivery of data-driven solutions. This is a highly technical role, requiring hands-on development work as well as leadership in an agile environment.

Key Responsibilities:

• Design, develop, and maintain highly scalable ETL applications in Hadoop using Spark, Python, Scala, SQL, and Hive.
• Lead and manage off-shore development teams, ensuring requirements are effectively captured and translated into deliverables.
• Collaborate with Product Management, Data Science, QA, Operations, and other internal teams to deliver features on tight deadlines.
• Actively participate in daily agile/scrum meetings, providing technical direction, and conducting code reviews.
• Optimize and tune performance of ETL pipelines in complex, concurrent environments to meet SLAs.
• Lead big data projects within retail and supply chain verticals, managing data in the billions and working on mobile applications such as Coin Out.
• Mentor and guide junior engineers, providing technical expertise, delegation, and leadership.
• Hands-on coding and development (80% of the time) with 20% focusing on team leadership, mentorship, and code reviews.

Requirements:

• 8 years of software development experience in big data environments.
• 3 years of hands-on experience developing and optimizing Spark applications in PySpark or Scala.
• Strong background in Spark, Scala, and Hadoop, with at least 3 years of practical experience in these technologies.
• Java development experience, with exposure to frameworks like Spring Boot.
• Experience managing off-shore development teams, preferably based in India, and collaborating across onshore/offshore teams.
• Proven experience in the retail or supply chain sectors delivering data-driven solutions.
• Experience working in highly dynamic environments with evolving processes and demands.
• Hands-on leadership experience, including mentoring and leading projects for at least 3 years.
• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Excellent knowledge of Spark architecture, data frames, and tuning Spark for performance.
• Experience with SQL and relational databases.

Preferred Qualifications:

• Experience in mobile application development.
• Experience working with Azure cloud-based technologies.

Soft Skills:

• Strong problem-solving skills, with the ability to address issues directly and efficiently.
• Excellent communication and interpersonal skills, with the ability to work cross-functionally with product management, data science, and other technical teams.
• Ability to mentor and lead by example, providing hands-on technical guidance while fostering growth and collaboration within the team.