Staff Quality Engineer - Machine Learning

The Team:

At Celonis, the Quality Engineering team consistently works to improve processes across agile teams and shift testing left so that we surface quality issues early and often. We focus on integrating our toolsets and examining our processes to make sure we don't have any more manual effort than necessary, that our documentation is sufficient but also lean, and that our work is repeatable. In addition, we work individually as embedded members of agile teams to do the day-to-day work of testing strategy, execution and automation. You'll work with the team to ensure this product is clear, easy to use and high quality. 

The Role:

As a Staff Quality Engineer, you have a keen understanding of what it takes to deliver a high-quality product as well as strengths in the area of automation coding and delivery. You are responsible for the overall effort to ensure that all innovations from our development team will be implemented in the right way in our software, including the creation and monitoring of automated tests. You are not bothered by getting your hands dirty with manual testing, but you are bothered enough by repetitive tasks to think creatively about how to not have to repeat them again. You believe in doing the job well but you also understand the balance between elegant solutions, time to market, and product quality. In this role, you support our core developers by ensuring a high quality of our Celonis EMS. On the other side, you are the link between development and product management. You have a need for quality assurance perfectionism, and it drives your passion for technology. 

The work you’ll do:

• Operate independently as a member of a cross-functional team, consistently advocating to push quality left and drive customer satisfaction of our product offerings


• Own responsibility for one or more complex areas of testing


• Drive robust and high-quality test coverage with minimal overlap in the team, ensuring the definition of done includes minimal but sufficient automation and recognize the level of automation necessary to reach the goals


• Research, recommend and implement tools and processes as needed with the goal of increasing automation and test coverage


• Collaborate across the EPE Chapter, mentoring other team members and contributing to efforts to improve overall process and quality


• Live by data not opinions, ensuring your team consistently meets auditability goals and that your team KPIs reflect the truth of overall quality status


• Serve in a capacity as a test coach to developers; reviewing unit tests and suggesting methods for better coverage

The qualifications you need:

• Strong understanding of Quality Assurance, Agile Processes and Test Automation


• Proficiency in traditional ML algorithms as well as large language models (LLMs)


• Proficiency in Model Evaluation and Validation as well as knowledge bias detection and mitigation techniques


• Knowledge of prompt engineering techniques


• Familiarity with AI/ML performance metrics


• Competence in programming languages and test automation tools (Python, PyTest, Java, RestAssured or similar), knowledge about LLM response evaluation tools (e.g. Promptfoo) will be considered a plus


• Competence with CI/CD as well as monitoring tools (e.g. DataDog)


• Strong analytical and problem-solving skills

Visa sponsorship is not offered for this role.