Data Engineer

For more than 50 years, NISC has worked to develop technology solutions for our customers, who we call our “Members”. Those Members are comprised primarily of 900 utility and broadband companies across the country and abroad, and we strive to provide services and technology to help them operate efficiently and better serve their end users. Our Members have over 16 million end customers (residential and businesses who receive power, internet, television and/or telephone services) that our enterprise software solution enables our Members to compete effectively in the industry, while excelling in customer service.

We are seeking an experienced Data Engineer to join our growing team of data analytics experts. The hire will be responsible for curating and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for various application teams. The Data Engineer will support our application experts, software developers, database architects, and data analysts on a Data Roadmap strategy and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be comfortable supporting the data needs of multiple teams, systems, and products.  We look for individuals who will thrive in a team environment, be committed to accomplishing a common goal and excited by the prospect of optimizing or even re-designing our company’s data architecture and have some fun along the way.

Essential Duties and Responsibilities: 

• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Understanding of Data Warehouse and Data Lakehouse paradigms.
• Design and build optimal data pipelines from a wide variety of data sources using AWS technologies.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing a unified data stream.
• Work with other data engineering experts to strive for greater functionality while making data more discoverable, addressable, trustworthy, and secure.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Create and maintain a culture of engagement and one that is conducive of NISC’s Statement of Shared Values. 
• Commitment to NISC’s Statement of Shared Values.    

Desired Experience and Skills: 

• Experience building and optimizing data pipelines, architectures, and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build ETL processes supporting data transformation, data structures, metadata, dependency, and workload management.
• Working knowledge of message queuing, stream processing, and highly scalable data stores.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Candidate with 4 years of experience in a Data Engineer role, who has attained a BS or MS degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field. They should also have experience using the following software/tools:
  • Experience with AWS: Lambda, S3, SQS, SNS, CloudWatch, etc.
  • Experience with Databricks and Delta Lake.
  • Experience with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases, including Oracle, Postgres Cassandra, and DynamoDb.
  • Experience with data pipeline and workflow management tools: Hevo Data, Airflow, etc.
  • Experience with AWS cloud services: EC2, Databricks, EMR
  • Experience with stream-processing systems:  Apache Spark, Kafka Streams, Spring Cloud, etc.
  • Experience with object-oriented languages: Java, Scala.
  • Nice-to-have
    • Experience with scripting languages: Python, JavaScript, Bash, etc.
• Strong verbal and written communication skills. 
• Ability to demonstrate composure and think analytically in high pressure situations.  

Desired Qualifications: 

• Bachelor’s degree in Computer Science, Statistics, Informatics, Information Systems or similar discipline, preferred.
• Certification in Database Administration, along with relevant experience in lieu of 4-year degree.

Minimum Physical Requirements:  

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this position. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. While performing the essential functions of this position, employees must be able to see and communicate.  Employees are regularly required to maintain a stationary position, move, and operate computer keyboards or office equipment.  

Disclaimer:  

Management may modify this job description by assigning or reassigning duties and responsibilities at any time.   

Key Words:   

SQL | Data | Big Data | Databricks | ETL | Spark | Scala | DBA| Lakehouse | Postgres | Python | AWS