Senior QA Engineer

Requirements

• Senior / Principal, 5+ years of manual QA - a high level of ownership and self-direction: the ability to take on the project as a whole, be fully accountable for quality, and get up to speed on new things independently.

• Experience testing AI agents / assistants: checking scenario and prompt following (prompt / instruction following), response quality and relevance.

• Experience testing "under the hood" via AI observability / tracing tools (LangSmith and analogs): analyzing an agent's traces - tool calls, memory handling, the model that ran, inputs/outputs, token consumption, spotting bottlenecks.

• Experience with mobile and/or TV-app testing; experience specifically with TV devices is less critical.

• Experience testing backends and integrations: API testing (REST / gRPC), verifying the seams between services, reading logs and traces (Kibana / Grafana / Loki, etc.).

• Solid QA fundamentals: test design, test documentation, defect handling.

• Hands-on experience using AI tools in the day-to-day QA process.

Nice to have

• Experience testing speech technologies (ASR / TTS) and working with audio.

• Evals - building and running AI-assistant quality evaluations: scenario-based self-play, LLM-as-judge, manual validation as human-in-the-loop.

• CI/CD / deployment / test environments: configuring stands, building and deploying applications (GitHub / GitLab, etc.).

• Device testing experience / ADB.

• Willingness to move into test automation over time.

Responsibilities

AI assistant:

• Assessing assistant quality: prompt / instruction following, compliance with requirements; catching regressions between prompt and model versions.

• Testing the assistant "under the hood" via observability / tracing tools: correctness and parameters of tool calls (MCP / integrations with external APIs), memory handling, the bot's reasoning, traces, etc.

Devices and applications:

• End-to-end manual testing of the assistant in the mobile app and on the devices (TV, speaker).

• Testing voice interaction with the assistant, recognition (ASR) and synthesis (TTS) quality, voice UX.

• Testing the assistant's product scenarios: agentic e-commerce (voice order -> cart -> confirmation on TV via remote), media content and a movie showtimes guide, flight tickets, basic scenarios (weather, currency rates, time, timers, Q&A).

Platform:

• Testing the platform / backend part the assistant runs on - a distributed backend of many services: dialog orchestration, LLM agent, ASR / TTS, etc.

• Verifying the request's end-to-end path through these components, the correctness of integrations and the seams between services, working with backend logs and traces.

Process:

• Maintaining test cases and bug reports.

• Close collaboration with the team: TPM, Prompt Engineer, DEV, DevOps, ML.

• First-line QA of critical scenarios ahead of demos.

What we offer

• The team has built award-winning AI products for tech corporations - devices, voice assistants, products that are actually in the world 

• Cutting-edge tech stack: Speech Technologies, NLP, Generative AI (LLMs, diffusion models), voice-first agentic architecture with privacy-first and on-premises deployment

• High engineering bar and real ownership - the team cares about what actually works in production, not what looks good in a demo, and you'll see the impact of your work directly 

• Fast career progression - a senior-heavy team and a high volume of real problems means you grow faster than you would anywhere else 

• Startup pace with enterprise stability - real clients, real revenue, no bureaucracy 

• Fully remote across Europe

• 21 vacation days + public holidays + 5 sick days 

• Private English lessons via Preply