Start with a baseline
Record your current support workload, common product questions and storefront conversion rate before enabling a new assistant. Use a consistent period and record promotions, stock changes and traffic sources. If the assistant is already active, document the earliest reliable period rather than inventing a historical baseline. Keep the same definitions for subsequent reviews.
Check answer accuracy
Review a sample of conversations across product discovery, comparisons, policies and order questions. Mark answers as supported, incomplete or incorrect against the store information available at the time. Divide supported answers by the number reviewed to calculate your sample’s accuracy rate. Keep the sample size with the result; a small sample should not be presented as representative of every conversation.
Test product discovery and cart journeys
Record whether the assistant suggests relevant products, asks useful clarifying questions and selects the intended variant. Follow a complete conversation through cart review and checkout navigation on desktop and mobile. An unavailable variant should not be presented as ready to purchase. Investigate repeated failure patterns rather than treating every product link as a successful recommendation.
Review handoff and support workload
Count requests that need a person and track how long customers wait for the first useful human response. Review whether the transcript gives your team enough context to continue. Compare routine support contacts and unresolved questions across consistent periods. A reduction in handoffs is only beneficial when customers still get correct answers and can reach your team when necessary.
Read sales from chat carefully
Dejor’s dashboard includes sales from chat, orders from chat and conversations this month. These attribution signals help you identify journeys to review. They do not establish that the assistant caused every purchase. Compare them with your store’s broader traffic and sales context. For a stronger causal conclusion, use a properly designed comparison with similar visitors and consistent conditions rather than a simple before-and-after claim.
Turn the review into improvements
Keep a monthly record of the review period, conversation sample, accuracy, recurring questions, handoff times and observed cart problems. Assign each repeated issue to a catalog correction, policy update, assistant setting or team workflow. Retest the affected journey after the change. Prioritize useful answers and completed customer tasks before claiming a return on investment.
Explore this workflow in Dejor.
See how dejor shopping assistance fits into your store, including setup steps and common questions.