Senior Data Engineer

Role.  An overview of the opportunity:

At Skupos, data is everything. Our data integrates the forgotten, fragmented world of mom-and-pop corner with the glitzy, gigantic world of CPG conglomerates.

With tens of millions daily transactions, Skupos is looking for a data engineer to wrangle and tame our data flows, expand and optimize our data and data pipeline architecture.

Team.  The team and our people:

This role will be part of the Data Platform team, the core functional team building our data products. The team works closely with the Data Infrastructure, Product and BI teams.

Responsibilities.  Your responsibilities will include:

• Building and maintaining the data and reporting layers for customer facing products.
• Collaborate closely with Data Infrastructure and Analytics teams to build complex data pipelines to deliver CICD complete deployment, move data cross - platforms including real time systems.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Work with the Data Infrastructure team to build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Create data tools, analytical datasets for Data Analytics and Machine Learning teams that assist them in building and optimizing our product
• Maintain and deliver continuous improvement of core projects through automation and process enhancement
• Work with all data teams and consumers to strive for greater functionality in our data systems
• Partner with engineers, product managers, and data scientists to breakdown data requirements, analyze source data sets, address data quality issues and effectively build and automate ETL pipelines at scale.
• Partner with ML engineers, data infrastructure team to define and own Data engineering tools, products and processes in place and define/set SLAs for each.
• Have deep understanding of existing data integration challenges and solutions with optimal ETL solutions and querying techniques

Experience and Skills. Candidates should have:

• Bachelor’s degree or equivalent, ideally in a technical or quantitative field (advanced degree is a plus).
• 4-6 years of experience working with SQL
• Advanced with proficiency with Python, Java, or Scala in a production environment
• 1-3 years of experience building and optimizing large-scale data pipelines, architectures and data sets.
• Recent experience (2 years) with a modern data warehouse (e.g., Snowflake, BigQuery, Redshift, Pentaho)
• Experience with data warehousing architecture and understanding of data modeling concepts and best practices (e.g., normalization/denormalization).
• Production experience with Spark
• Familiarity with stream-processing platforms and message-queues (e.g. Flink, Hadoop, Kafka) is a plus

Salary is based on experience and location.

Salary range: $115,000 - $135,000.