What are the responsibilities and job description for the Data Bricks Consultant position at Trinity Technologies?
Job Title: Lead Databricks Engineer
Location: NYC, NY, Iselin, NJ || Hybrid
Job Type: Contract
Job Description:
Major Responsibilities:
· Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
· Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.
· Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.
· Optimize Databricks jobs for performance and scalability to handle big data workloads.
· Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
· Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
· Demonstrated proficiency with Data Analytics, Data Insights
· Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process
· Azure Synapse/Bigquery/Redshift is good to have.
· Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.
Skills:
· 5 years’ – Strong experience in Finance / Banking industry – Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
· 10 years - Enterprise Data Management
· 10 years - SQL Server based development of large datasets
· 5 years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.
· Azure Synapse/Bigquery/Redshift experience is good to have
· 3 years Python(numpy, pandas) coding experience
· Experience with Snowflake utilities such as SnowSQL and SnowPipe - good to have
· Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
· Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills
· Capable of discussing enterprise level services independent of technology stack
· Experience with Cloud based data architectures, messaging, and analytics
· Superior communication skills
· Cloud certification(s)
· Any experience with Regulatory Reporting is a Plus.
Job Type: Contract
Salary: $36.68 - $98.45 per hour
Benefits:
- 401(k)
- Dental insurance
- Health insurance
Experience level:
- 10 years
Schedule:
- 8 hour shift
- Day shift
- Monday to Friday
Experience:
- Data Bricks: 10 years (Required)
- Banking: 6 years (Required)
Work Location: On the road
Salary : $37 - $98