Piloting GenAI search answers safely with Adobe Target and Analytics

The challenge

Our client had a clear vision and the search expertise to back it: bring GenAI-powered answers into its Coveo search experience, showing direct LLM responses alongside traditional results. The client owned the strategy and handled the Coveo implementation itself.

Switching the feature on for everyone at once carried real risk. The client needed a phased, instrumented, recoverable rollout that protected the user experience while the team validated the feature in the real world.

What we did

Trellist came in on the Adobe Target side, focused on two things: the pilot infrastructure that controlled who saw the new experience, and the tooltip layer that helped users understand it.

• Pilot audience controls. We built pilot infrastructure in Adobe Target that limited the GenAI answer to a percentage of logged-in users during validation.
• An always-on tester group. A dedicated internal tester group always received the experience, keeping QA continuous without affecting the broader audience.
• Suppression logic. The GenAI answer yields to Smart Snippets or custom content when present, so existing best results are never overridden.
• Guided tooltips. A tooltip series orients users on first encounter, reducing friction and building confidence without any product changes.
• Full instrumentation. We tracked GenAI answer display events and tooltip interactions in Adobe Analytics so the client could evaluate the pilot and decide how to scale.