What are the responsibilities and job description for the Lead Data Engineer position at 3B Staffing LLC?
Job Details
Job: Lead Data Engineer
Location: Encino, CA (Onsite)
Lead Data Engineer
Must have:
- Azure exp.
- Synapse Data Engineering experience
- Databricks
- Logical architecture prospective:
- Data vault
- Lead must have done data models with Kimball and data vault type structures as lake houses.
Team and Responsibilities
We are currently looking for an experienced and detail-oriented candidate to join our team in for one of our Financial Services Client based out of Encino, CA as Principal Data Architect. We are looking for this position as a self-motivated contributor working closely with software development teams.
Responsibilities:
Define software feasibility, run data reviews, assess current databases to identify areas in need of improvement, and oversee data development teams
Build and maintain Enterprise Data Model to meet the company s needs
Understand the Industry latest trends on data strategy and create Innovative solutions
Provide updates to stakeholders on product development processes
Drive innovation and data strategy by building and improving the data modeling and database design and best practices around Data Modeling/Architecture
Lead the data committee, gather requirements, and define data models and data solutions
Collaborate with stakeholders to understand the organization's data requirements, business objectives, and long-term goals.
Develop a data strategy that aligns with the overall business strategy and ensures data-driven decision-making.
Design and develop data models that cater to the specific needs of the organization.
Establish efficient and robust data integration processes to extract, transform, and load (ETL) data from various sources into the Azure environment. Ensure data quality, consistency, and accuracy during the integration process.
Design and implement data warehousing solutions using Azure services to enable scalable storage and efficient data retrieval for analytical processing.
Establish data governance policies and procedures for data access, usage, and retention.
Design data visualization solutions for reporting and business intelligence purposes.
Work closely with cross-functional teams, including developers, data engineers, business analysts, and project managers, to ensure successful implementation of data solutions.
Document the data architecture, data flows, and processes for reference and training purposes, enabling other team members to understand and work with the data architecture effectively.
Requirements:
Experience with Azure / Any Cloud and On-Prem Services Apache Spark Analytics, Microsoft Purview, loud Hadoop, R Server, HBase, and Storm clusters, Azure Stream Analytics, Azure Dev Ops, and standard Microsoft development environments
Experience with Data Bricks and Azure Synapse
Low code/No Code, ERP, APIM, Event hubs and application integration technologies
Experience working in various technical environments including data architecture, programming experience, preferably in building enterprise web solutions
Familiarity of the SDLC and common development practices
Bachelor s degree in computer science or related field or equivalent experience and/or training
Experience in integration enterprise solutions using Services and/or APIs
Experience on Cloud, Relational, NoSQL and Big Data technologies, techniques, and trends
Strong Experience with data analytics data integration (ETL/ELT), data delivery, data preparation, data discovery, data processing, data storage, and data science
The following experience is considered as a strong plus:
Understanding / Experience in Agile Development
Python