What are the responsibilities and job description for the Lead Data Engineer position at Prudential?
Job Classification:
Technology - Engineering & CloudPrudential’s Global Technology team is the spark that ignites the power of Prudential for our customers and employees worldwide. Our organization plays a critical and highly visible role in delivering customer-driven solutions across every area of the company. The Global Technology team is made up of diverse, agile-thinking, and highly-skilled professionals; we use our combined capabilities to enable the organization with innovation, speed, agility, scalability and efficiency.
The Global Technology team takes great pride in our culture where digital transformation is built into our DNA. When you join the Global Technology organization at Prudential, you’ll unlock a challenging and impactful career – all while growing your skills and advancing your profession at one of the world’s leading financial services institutions.
Responsibilities
Provide Infrastructure and Operational Support for
- End-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS
- Data lake data intake/request/onboarding services
- Data lake ingestion services for batch/real time data ingest
- Data processing services (ETL/ELT) for batch/real time (Glue/Kinesis/EMR)
- Data storage services for data lake(S3)/ data warehouses (RDS/Redshift)/ data marts
- Data services layer including Athena, Redshift, RDS, microservices and APIs
- Pipeline orchestration services including lambda, step functions, MWAA(optional)
- Data security services (IAM/KMS/SM/encryption/anonymization/RBAC)
- Data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education
- Data provisioning services for data consumption patterns including microservices, APIs and extracts
- Metadata capture and catalog services for data lake(S3/Athena), data warehouses (RDS/Redshift), Microservices/APIs
- Metadata capture and catalog services for pipeline/log data for monitoring /support
- Release support through CI/CD pipelines
- Prepare operation support documentation and SOPs for data platform support
- Support data stream migration from traditional sources (RDBMS, NAS, MFT) to AWS relational databases such as Amazon RDS, Aurora, and Redshift
- Provide governance/audit reporting for data platform access
- Serve as technical lead for the DataOps / MLOps product owner
- Partner with immediate engineering team, product owner, IT, partners on EDP agenda
- Provide technology thought leadership, consulting, and coaching/mentoring
- Establish and enforce best practices for production release / support processes
- Work with scrum master to develop and own backlog, sprints, stories, features, and epics
Qualifications
- Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
- Solid understanding of DataOps / MLOps concepts and goals
- Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
- Programming experience with Java, Python/Scala, Shell scripting
- Solid experience with AWS services such as CloudFormation, S3, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
- Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB.
- Experience in AWS data lake/data warehouse/business analytics
- Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift
- Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS
- Experience with DevOps and Continuous Integration/Delivery (CI/CD) concepts and tools
- Experience with business intelligence tools such as Tableau, Power BI or equivalent
- Knowledge of ETL/ELT
- Experience in production support from Level 1 to Level 3
- Awareness of Data Management & Governance tools
- Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired
- Experience working on Agile projects
Prudential Financial, Inc. of the United States is not affiliated with Prudential plc. which is headquartered in the United Kingdom.
Prudential is a multinational financial services leader with operations in the United States, Asia, Europe, and Latin America. Leveraging its heritage of life insurance and asset management expertise, Prudential is focused on helping individual and institutional customers grow and protect their wealth. The company's well-known Rock symbol is an icon of strength, stability, expertise and innovation that has stood the test of time. Prudential's businesses offer a variety of products and services, including life insurance, annuities, retirement-related services, mutual funds, asset management, and real estate services.
We recognize that our strength and success are directly linked to the quality and skills of our diverse associates. We are proud to be a place where talented people who want to make a difference can grow as professionals, leaders, and as individuals. Visit www.prudential.com to learn more about our values, our history and our brand.
Prudential is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, ancestry, sex, sexual orientation, gender identity, national origin, genetics, disability, marital status, age, veteran status, domestic partner status , medical condition or any other characteristic protected by law.
The Prudential Insurance Company of America, Newark, NJ and its affiliates.
Note that this posting is intended for individual applicants. Search firms or agencies should email Staffing at staffingagencies@prudential.com for more information about doing business with Prudential.
PEOPLE WITH DISABILITIES:
If you need an accommodation to complete the application process, which may include an assessment, please email accommodations.hw@prudential.com.
Please note that the above email is solely for individuals with disabilities requesting an accommodation. If you are experiencing a technical issue with your application or an assessment, please email careers.technicalsupport@prudential.com to request assistance.
