What are the responsibilities and job description for the Lead Data Engineer position at Optomi?
Experience of the right candidate:
- 5 years of experience in programming language Java or Scala.
- 2-5 years of experience in ETL projects.
- AWS Cloud native technologies (AWS EMR, EKS, Dynamo DB, Lambda/Glue, etc).
- 2-5 years of experience in big data projects.
- 2 years of experience with API development (REST API’s).
- Believes in Scrum/Agile, and has deep experience delivering software when working on teams that use Scrum/Agile methodology.
- Strong and creative analytical and problem-solving skills.
Desired Technical Expertise:
- Strong experience in Java or Scala.
- Strong experience in big data technologies like Apache Spark.
- Strong experience with serverless technologies.
- Strong experience in processing with JSON and CSV files.
- Must be able to write complex SQL queries.
- Experience in performance tuning and optimization.
- Familiar with columnar storage formats (ORC, Parquet) and various compression techniques.
- Experience in writing Unix shell scripts.
- Unit testing using JUnit or ScalaTest.
- Git/Maven/Gradle.
- Code Reviews.
- Experience with CI/CD pipelines.
- Agile.
Data Engineer Lead
HireRightAway.AI -
Reston, VA
Data Engineer Lead
Steampunk -
Mc Lean, VA
Engineer, Lead Data
Loudoun County Public Schools -
Ashburn, VA
