Skip to main content

Advance notice before AI model changes

Notify customers in advance before the underlying AI model behind the support agent is changed, including the lead time and the channel used, and disclose which AI model(s) are currently in use. Prompts and knowledge bases tuned to the current model's behavior can degrade in answer quality when the model changes without warning, making performance changes hard to trace. The primary use case is investigating changes in answer quality rather than gating rollout approvals, but advance notice remains a desired feature. As a possible approach, the data science team evaluating the product suggests handling model changes like software versioning: release a new version of Featurebase that uses the different model while still supporting the current version, let customers evaluate and tune their prompts for the new model, then switch to the new version, after which support for the old version ends.

What AI feature?

Log in to comment and vote

No comments yet

Be the first to share your thoughts.