Senior Software Engineer - Massive Graph Processing

LiveRamp is the leading data connectivity platform. We are committed to connecting the world’s data safely and effectively, advancing innovation, and empowering people to do good. Our platform powers customer experiences centered around the needs and concerns of real people, keeping the Internet open for all. We enable individuals around the world to connect with the brands and products they love. LiveRampers thrive on solving challenging problems for the good of humanity—and we’re always looking for smart, kind, and creative people to help us get there.

Mission: LiveRamp makes it safe and easy for businesses to use data effectively.

We on the Identity Engineering team maintain the graph processing engine that connects the trillions of different identifiers for consumers and the devices they use online. Help us process this trillion edge graph as quickly and efficiently as possible.

You will:

• Collaborate with a team of supportive and passionate software engineers, product, infrastructure and other internal partners remotely to tackle challenging problems, plan and execute on development projects that build, enhance and customize our high-performance data ingest, partitioning, graph and edge building systems.
• Foster a positive environment of integrity, operational excellence, initiative and empower the team to achieve a high level of technical productivity, reliability, and simplicity by developing in Java and leveraging tools like Docker, Kubernetes, Terraform, Gradle, Jenkins, GCP, Hadoop and Spark.
• Design with privacy and security-first principles and err on the side of shipping fast and iterating.

Your team will: 

• Find feedback to be important in helping us grow as individuals and as a team; we’re always looking for chances to kindly share positive and constructive feedback and we aim to accept feedback with grace.
• Follow a Scrum-based devops model with a few guiding principles:
• Support, trust and motivate the people involved
• Continuous software delivery and process improvement
• Minimize barriers to development and leverage automation
• Work on:
• Edge Ingestion and Partitioning Framework: We could never process all trillion edges at once and luckily we don’t have to. Instead we process subgraphs that contain specific types of edges. Our edge ingestion and partitioning leverages LiveRamp’s Seek MSJ framework to efficiently incorporate new data into existing edge stores.
• Path Computation as a Service: We provide a service to other LiveRamp engineering teams for finding specific types of paths within our massive graph. It services 80,000 requests, 10PB data per day and this is possible due to its use of highly scalable Spark jobs and intelligent caching mechanism.
• Project categories: besides product requests and enhancements, we work on projects in performance and cost optimization, security and legal compliance, reliability and quality initiatives, as well as architectural refreshes such as POC in restructuring our data to be more streaming/real time friendly, new REST API for relevant services, migration of apps to Spark, Parquet, and other new technologies that could make our processing faster, more scalable, maintainable and cost effective.

About you: 

• 5 years of experience writing and deploying object oriented production code.
• 2 years of experience writing and optimizing large-scale batch processing pipelines in Spark/Hadoop.
• 2 years of experience using cloud based platforms such as GCP, AWS, Azure or similar technologies.
• Strong ability to break down complex problems to their essential components, design and implement elegant solutions for them.
• Have a passion for building large-scale distributed systems and are comfortable writing maintainable and high performance code.
• Comfortable evaluating and adapting to the latest tools and technologies.
• You love mentoring junior and senior engineers and helping a team deploy software that conforms to best practices.
• Startup personality and enjoy working as part of a team: smart, ethical, friendly, hard-working, and productive.

Bonus Points:

• Experience with GCP products such as BigTable, BigQuery, Dataproc, PubSub or other data tools like Kafka, Hive, or Apache Beam
• Experience with development tools such as Terraform, Kubernetes, Helm, or Gradle
• Experience with designing and implementing interfaces and infrastructure for large scale systems and RESTful APIs

Benefits:

• People: work with talented, collaborative, and friendly people who love what they do.
• In-Office Food: enjoy catered meals, boundless snacks, and the occasional food truck.
• Fun: we host in-person and virtual events such as game nights, happy hours, camping trips, and sports leagues. 
• Work/Life Harmony: flexible paid time off, options for working from home, and paid parental leave.
• Whole Health Package: medical, dental, vision, and disability insurance. Plus mental health support (via Talkspace) and fitness reimbursement up to $100 per month. 
• Savings: our 401K matching plan helps you plan ahead.
• RampRemote: a comprehensive program to assist you in setting up a home office that works for you
• Location: Work remotely from home

More about us:

LiveRamp’s mission is to connect data in ways that matter, and doing so starts with our people. We know that inspired teams enlist people from a blend of backgrounds and experiences. And we know that individuals do their best when they not only bring their full selves to work but feel like they truly belong. Connecting LiveRampers to new ideas and one another is one of our guiding principles—one that informs how we hire, train, and grow our global team across eight countries and four continents.

LiveRamp is an affirmative action and equal opportunity employer (AA/EOE/W/M/Vet/Disabled) and does not discriminate in recruiting, hiring, training, promotion or other employment of associates or the awarding of subcontracts because of a person's race, color, sex, age, religion, national origin, protected veteran, disability, sexual orientation, gender identity, genetics or other protected status. Qualified applicants with arrest and conviction records will be considered for the position in accordance with the San Francisco Fair Chance Ordinance. 

California residents: Please see our California Personnel Privacy Policy for more information regarding how we collect, use, and disclose the personal information you provide during the job application process.

To all recruitment agencies: LiveRamp does not accept agency resumes. Please do not forward resumes to our jobs alias, LiveRamp employees or any other company location. LiveRamp is not responsible for any fees related to unsolicited resumes.

#LI-Remote