What are the responsibilities and job description for the Big Data Developer position at Technology Ventures?
Job Description
Location: Must be on-site 3x a week in Mclean- Only
Duration:12 months, possible extension
Required Skills:JD is up to date Bigdata platform experience, plus if they have infosec exposure
Years of experience:2
Qualifications:
• 2 years’ experience in the following: Big Data and distributed programming models and technologies and techniques such as Hadoop, Spark, Map Reduce, Sqoop, Hive, Hadoop Distributed File System (HDFS), Distributed Indexing and Databases (HBase, Hive, Cassandra, Vertica), Serialization Formats (JSON, Avro, Parquet), Pyspark, Spark, Scala, Hive Tuning, Bucketing, Partitioning.
• Knowledge of database structures, theories, principles, and practices (both SQL and NoSQL).
• Experience and working knowledge in Kafka, Spark streaming, Sqoop, Oozie, Airflow/ Control-M/Autosys.
• Experience with Python and Shell Scripting and basic system admin knowledge on Linux environments.
• Bachelor’s degree in Information Technology, Computer Science, Engineering or related field or equivalent combination of education and work experience.
• Well versed with Information and application security including LDAP, certificates, public key encryption, SSH, access credentials, etc.
• Knowledge or experience in Jira, Confluence, Bitbucket & ServiceNow.
• Ability to assist with production incidents, triage across multiple teams, and apply short-term / long-term fixes.
• Ability to independently perform all duties from analysis to deployment to postproduction defect fixing.
• Experience with data corrections made to production data stores.
• Experience in ELK stack, Splunk or any other monitoring systems a plus.
• Experience working in Agile, Lean/Kanban, or Scaled Agile organization a plus.
• Experience on one or more ETL/ELT tools like DataStage, Talend etc. a plus.
Preferred
• Experience in any of Cloud technologies like AWS S3, EC2, Docker, OpenShift, Kubernetes/Amazon EKS, EMR, Cloudwatch, Kinesis
• Experience working on database technologies Oracle, DB2, Mongo DB, NoSQL DBs, PostgreSQL, Snowflake
• Experience in Attunity, Kafka, AMQ or similar streaming tools is a plus
• Experience with Talend and/or Informatica Intelligent Cloud Services (IICS) is a plus
• Experience with Okera or similar is a plus
• Any AWS or Snowflake certification is a plus
