What are the responsibilities and job description for the Senior Data Engineer position at firstPRO, Inc?
Job Description
As a Senior Data Engineer, you will work closely with both software engineers and data scientists to:
- Build and optimize data pipelines and ETLs to generate value for our organization and our client base.
- Investigate new technologies to improve the availability, performance, cost, and reliability/fault tolerance of our data pipelines and workloads.
- Optimize the performance of the current data stores and pipelines.
- Create data pipelines to power AI/ML workloads.
- Ensure data pipelines comply with retention policies.
- Prepare data to optimize embedded reporting/BI for customer and internal needs.
Familiarity - You have at least seen the technology and possibly written a few small programs utilizing the technology.
Proficiency - Able to use the technology with little to no special training and should be very comfortable with the technology based on past experience.
Mastery - Adept with all the advanced features of the technology and would be a good candidate to mentor those with less experience.
Required Qualifications:
- Mastery of SQL
- Mastery with Python, Java 8 , or Scala
- Mastery with distributed data/computing tools, such as Hadoop, Hive, or Spark
- Proficiency working with any Non SQL based data store, including but not limited to
- Graph Databases such as Neo4j or AWS Neptune
- NoSQL Datastores such as MongoDB, DynamoDB, or Apache Cassandra
- Proficiency with open source ETL frameworks, such as Apache Airflow, Luigi, or similar
- Proficiency with Git
- Proficiency with AWS Services including
- Glue
- EMR
- Lambda
- RDS
- S3
- Athena
- Familiarity with Data streaming services, such as Apache Kafka, Amazon Kinesis, or Apache Flink
- Familiarity with code quality tools such as SonarQube
- Familiarity with CI/CD pipelines
Desired Qualifications:
- AWS Certified Database Specialty (DBS-C01) or AWS Certified Data Analytics Specialty (DAS-C01)
- Familiarity with AWS Services including
- Redshift
- DynamoDB
Salary : $140,000 - $160,000
Senior Software Engineer - Data Engineering
Data Engineering - Rokt -
Sydney, FL
Senior Marine Mechanical Engineer Auxiliary Systems
Pantheon Data -
Curtis, MD
Senior Software Engineer, Real Time Engine
Treasure Data -
Vancouver, WA
