What are the responsibilities and job description for the Data Engineer position at BasisPath?
The Data Engineer should have knowledge and experience in leading and developing data products such as data pipelines and architectural design within the software development life cycle from requirements definition to architecture, design, implementation, and test. The Data Engineer will be collaborating with other engineers, data analysts, data scientists, and data visualizers to bring impactful analytical solutions and insights to our mission partners. The Data Engineer should also be experienced in preparing data for analytical and operational uses.
Responsibilities and Duties:
Analyze and organize raw data
Build data systems and pipelines
Evaluate business needs and objectives
Prepare data for prescriptive and predictive modeling
Combine raw information from different sources
Explore ways to enhance data quality and reliability
Develop analytical tools and programs
Collaborate with data scientists and architects on several projects
Required Qualifications and Skills:
Work within the full software development lifecycle, utilizing emerging technologies and big data design principles to develop data pipelines, interfaces, and architectures.
Define the data requirements and structure for the application. Model and design the application data structure, storage and integration.
Knowledge of multiple databases and big data frameworks.
Ability to apply/implement data and database technologies, preferably cloud based.
Knowledge and skills to apply data standards and best practices while adhering to applicable Government laws, policies, directives, mandates, and guidance, and the ability to reconcile data models and de-duplicating data stores.
Desired Qualifications and Skills:
Technical expertise with data models, data mining, and segmentation techniques
Knowledge of programming languages (e.g., SQL, Python, Spark)
Hands-on experience with SQL and No SQL database design