AI Operations & QA Tester – Overnight Coverage

Our AI agents are directly serving live policyholders across chat, phone, and email in real time. We operate at rapid startup speed without bloated sprint cycles, meaning bad outputs or edge-case failures carry immediate compliance risks. You are the critical quality shield ensuring uninterrupted operational excellence during European overnight hours.

You will own end-to-end overnight operational quality, prompt analysis, and QA monitoring across all live customer channels.

• Monitor live AI customer conversations and system alerts across chat, phone, and email channels in real time.
• Evaluate AI responses against stringent insurance guidelines and exact policy wording for compliance and accuracy.
• Inspect system prompts, rules, and underlying logs to diagnose root causes for model failures and edge cases.
• Run regression test suites across new prompt iterations and system version updates.
• Log detailed bug reports with reproducible context, evidence, and clear severity ratings in Jira or Linear.
• Deliver concise, actionable handover shift reports for the European morning team.

• 2+ years of QA or structured software/AI testing experience.
• AI-native professional comfortable reading system prompts and evaluating LLM outputs.
• Highly autonomous operator thriving in dedicated asynchronous ~21:00–06:00 UTC shifts.
• Strong written communicator who logs clean, evidence-backed bug reports.
• Regulated industry background (fintech, insurance, legal) is a strong plus.
• Professional German proficiency (C1+) is a significant advantage.

We operate at startup speed with corporate backing: direct decisions, minimal meetings, no sprints or steering committees, and rapid execution where ideas go live the same day. As part of our overnight operations, you will work with high autonomy and ownership, ensuring uninterrupted quality for our live AI insurance agents.

• Extreme operational autonomy with high ownership over critical live systems.
• Zero corporate red tape: no steering committees, no endless sprints, same-day deployments.
• Direct impact working on cutting-edge production AI applications serving real customers.
• Flexible remote schedule ideally suited for daytime/afternoon hours in APAC and Americas timezones.

QA Testing · Software Testing · Regression Testing · Prompt Analysis · AI Tooling · Bug Tracking · Async Communication · Insurance Domain Knowledge · Regulated Industry Experience · German Language · AI Evaluation · LLM Output Rating · Jira · Linear

This probably isn't for you if:

• You rely on rigid sprint rituals, micromanagement, or synchronous daily standups to stay productive.
• You cannot reliably commit to a consistent ~21:00–06:00 UTC overnight schedule.
• You are unfamiliar with prompt structures and diagnosing why an LLM produced an incorrect answer.

If you are ready to own the operational line and keep our production AI running flawlessly, apply now to start the process with our paid test task.