PySpark-Lead

nEssential Job Functions: n tDesign and development of data ingestion pipeline. n tPerform Data Blending and Data harmonization. n tPerform data migration and conversion activities. n tDevelop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures. n tApply knowledge and experience to develop, test, and document enhancements or extension to the Palantir (Lava) Datalake as needed. n tCollaborate with Business Analysts, Architects and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments). n tPerform end to end automation of ETL process for various datasets that are being ingested into the big data platform. n tDevelop and execute test scripts independently or in concert with testing personnel. n nRequired Tools/Languages: n nMust Have n tPySpark n tHadoop n tCore Java n tStrong SQL n tUnix / Linux Shell Scripting nNice to Have n tAzure Databricks, ADF n tNoSQL databases (Mongo, Couchbase, Cassandra) n tJavaScript UI frameworks (Angular, NodeJS, Bootstrap) n tJenkins n t nMinimum Qualifications and Job Requirements: n tMust have a Bachelor s degree in Computer Science or related IT discipline n tMust have at least 5 years of IT development experience ( for Lead Role) n tMust have 3-5 years relevant professional experience working in PySpark, Big Data/ Hadoop Python or equivalent scripting language ( Lead Role) n tMust have strong, hands-on J2EE development n tMust have experience with ETL tools n tFamiliar with Agile and Scrum. n tStrong communication skills. n "As a Lead, you are responsible for managing a small team of analysts, developers, testers or engineers and drive delivery of a small module within a project (Delivery/Maintenence/Testing) You may serve as entry level specialist with expertise in particular technology/industry domain/a process / application / product. You are responsible for functionalechnical track of a project."

Salary : $0