What are the responsibilities and job description for the Spark Developer position at Ascendion Inc.?
Job Description
Description:
Day to day:
• Provides input to technology strategy implementation based on leading-edge industry standards, best practices, and a comprehensive understanding of business operations.
• Write code, mostly for new programs, for the most complex system designs, and provides consultative guidance on projects with extensive size, scope, and risk. Write programs that span platforms. Code to and/or create Application Programming Interfaces
• Write detailed technical specifications for entire systems (made up of several subsystems). Identify integration points. Review technical specs created by other developers. Ensure sufficient quality and compliance of documentation to architectural standards
• Consult and partner with other Developers, Business Analysts, Systems Analysts, and Project Managers. “Scope” time, resources, etc. required to complete programming projects. Seek review from other Developers, Business Analysts, Systems Analysts, or Project Managers on estimates. Incorporate other Developer’s scope information into their own scope document (for integration points)
• Perform unit testing and debugging. Set test conditions based on code specifications.
• Provides insights/input on integration issues.
Must haves:
• 5 years of software development coding experience
• Extensive hands-on coding experience in Spark
• Experience writing complex SQL queries and extensive data warehousing knowledge
• Experience working with different file formats including XML, JSON, and Parquet.
• Experience developing code to process large volume data in batch and real-time using Spark Streaming.
• Experience in Hadoop stack
• Experience building data flow and process flow diagrams
• Extensive experience working with SQL & NoSQL databases (Mongo, Postgres)
Spark,Scala,ETL,Data Pipelining,Data Warehousing
