What are the responsibilities and job description for the AWS architect position at Talent Groups?
AWS Databricks Architect
Location: O’Fallon, MO- Onsite
Full-Time
Job Description:
Candidates are required to have these mandatory skills:
Strong hands on in Pyspark and Apache Spark.
Experience in Native Spark Migration to Databricks.
Experience in Hadoop Migration to Databricks.
Experience in Building Data Governance Solutions like Unity Catalog, StarBust …etc
Build Very Strong Orchestration Layer in Databricks/ADF…. Workflows.
Build CICD for Databricks in Azure Devops.
Process near Real time Data thru Auto Loader, DLT Pipelines.
Implement Security Layer in Delta Lake.
Implement Massive Parallel Processing Layers in Spark SQL and PySpark.
Implement Cost effective Infrastructure in Databricks.
Experience In extracting logic and from on prem layers like SSIS, Stored procedures, Informatica, Vertica, Apache Hudi, Filesystems. etc into Pyspark.
Ability to Build Solutions as Cloud Agnostic.
Proven experience in designing and implementing complex data solutions aligned with business objectives.
Expertise in data modelling, integration, security, and governance
Hands-on experience with guiding the virtual data model definition, defining Data Virtualization architecture and deployment with focus on Azure, Databricks, PySpark technologies.
Prior experience with establishing best practices for business optimizations.
