Loading ...
Oyster AI is our assistant for customers and team members, and we want it to be genuinely useful: accurate, fast, reliable and safe with the data it touches. In this role you’ll own complex improvements to it end to end, from defining the problem and designing the solution through evaluation, rollout and learning from production.
Your work will span agent capabilities and tools, knowledge retrieval, answer quality and infrastructure, with real trade-offs between model quality, latency, reliability and cost. This is a hands-on product engineering role, not a research-only or prompt-only position. You’ll also help other product teams build AI features well by sharing patterns, pairing with them and improving the reusable foundations they rely on.
Our backend is Ruby on Rails and our frontend is React. Strong AI engineers from other stacks are very welcome, and Rails experience is a bonus, not a prerequisite.
End-to-end AI improvements. Connect user needs to architecture, implementation, evaluation and safe rollout for complex Oyster AI work.
Reliable agents and tools. Design clear tool contracts, robust error handling and bounded execution, with appropriate human control over consequential actions.
Better knowledge access. Choose retrieval, embedding, indexing and freshness strategies that improve answers while respecting data access boundaries throughout retrieval and generation.
Evaluation you can trust. Build evaluation loops with representative datasets, experiments, regression checks and, where useful, automated judges calibrated against human judgment.
Root-cause fixes in production. Diagnose failures across prompts, retrieval, tools, models and infrastructure, fix the underlying cause and verify the fix with evidence.
Explicit trade-offs. Choose models and system designs deliberately, and document decisions and rollout criteria.
AI capability across teams. Support other product teams through technical guidance, code review, pairing and mentoring, so they can deliver AI features independently.
Senior-level engineering judgment and a track record of independently leading complex production work from design through operation and improvement.
Hands-on experience shipping LLM-powered product capabilities to production, with clear personal responsibility for implementation and delivery decisions.
Real depth in at least some of the following: retrieval/RAG and indexing, evaluation datasets and experiments, LLM-as-judge, or agents and tool calling.
The ability to tell model, retrieval, tooling and application failures apart, and to validate improvements with evidence.
Strong fundamentals in coding, debugging, testing, API integration and system design, plus willingness to work in Ruby/Rails and React.
Sound judgment about sensitive data, access control, model limitations and operational risk.
Clear communication of trade-offs, effective async collaboration with product and engineering partners, and practical mentoring.
Nice to have
Production experience with Ruby on Rails, especially integrating AI capabilities into an established backend.
Full-stack delivery of conversational or assistant experiences.
Slack or in-app assistant integrations with clear user and permission context.
Operating retrieval pipelines, agent tools, evaluation infrastructure, or model routing and cost controls at production scale.
Helping multiple teams adopt reusable AI patterns while balancing shared foundations against local needs.
Real AI in production. You’ll ship to real users, and success is measured by evaluations and user outcomes, not plausible-looking demos.
Ownership of hard problems. You’ll work across agents, retrieval, evaluation and infrastructure, where engineering judgment matters at every layer.
Multiplier impact. Your patterns and mentoring will shape how AI is built across multiple product teams.
Stack-agnostic hiring. We hire for AI engineering depth, and you’ll have room to grow into Rails and React.
Thoughtful remote culture. You’ll join a fully remote, async-friendly team that values documentation and clear decision-making.
Published on: 11/6/2026
Oyster
Oyster HR is a global employment platform (Employer of Record/EOR) designed for companies to hire, pay, and manage remote teams in over 180 countries without setting up local entities.
Please let Oyster know you found this job on Wantapply.com. It helps us to get more jobs on our site. Thanks!
