Senior Software Engineer - Data Collection

The Data Collection team owns the ingestion pipeline -- starting with a lightweight agent to collect, compress, encrypt and ship the data back to the Sumo Logic cloud. This agent runs on a variety of operating systems including the most popular Windows & Linux variants. They comprise of enterprise IT infrastructure, including servers, personal computers and cloud services. These systems can generate petabytes of platform-specific log data, including Event Logs and Event trace Logs, which must be ingested in near-real time and with a minimal resource footprint.  

As a  collection Engineer, you will architect, implement, and scale the collection systems to ingest this data locally (from the machine generating the events), remotely (from a different machine within our customers’ infrastructure), and from the cloud (Azure, AWS, GCP, etc...).

You have strong Operating system fundamentals with knowledge about Windows & Linux internals & APIs. You are passionate about handling very large volumes of data on a distributed system. You care about producing clean, elegant, maintainable, robust, well-tested code; you do this as a member of a team, helping the group come up with a better solution than you would as individuals. Ideally, you have experience with performance, scalability, and reliability issues of 24x7 commercial services.

Responsibilities:

• Design and implement collection system that is extremely high-volume, fault-tolerant, scalable backend systems that process and manage petabytes of customer data.
• Analyze and improve the efficiency, scalability, and reliability of our backend systems.
• Write robust code; demonstrate its robustness through automated tests.
• Work as a member of a team, helping the team respond quickly and effectively to business needs.   

Requirements:

• B.Tech, M.Tech, or Ph.D. in Computer Sciences or a related discipline
• 4 years of industry experience with a proven track record of ownership and delivery if you only have a B.S. degree.
• Object-oriented experience, for example in Java, Scala, Ruby, or C .
• Understand performance characteristics of commonly used data structures (maps, lists, trees, etc).
• Desire to learn Scala, an up-and-coming JVM language (scala-lang.org).

Desirable:

• Experience in multi-threaded programming and distributed systems is highly desirable.
• Experience in big data and/or 24x7 commercial service is highly desirable.
• Experience with Win32 API, DCOM, WMI and the Windows Event Viewer
• Agile software development experience (test-driven development, iterative and incremental development) is a plus.

About Us: https://app.box.com/v/SLGeneralDossier

• What we do:

We are a cloud-native SaaS machine data analytics platform, solving complex monitoring problems for DevOps, SecOps and ITOps teams. Customers love our product because it allows them to easily monitor and optimize their mission critical, large scale applications.

• Massive Scale:

Our microservices architecture in AWS ingests hundreds of terabytes daily across many geographic regions. Millions of queries a day analyze hundreds of petabytes of data.

• Mission:

Democratize machine data analytics through the Sumo Logic platform, bringing real-time data insights securely through the cloud.