Dialogflow CX Generative Playbook Ignores Correct Data Store URL and Hallucinates Non-Existent URLs (Grounding Failure)

Google Cloud Build with AI Conversational Agents (Dialogflow CX)
1

We are experiencing a severe and critical bug within the Dialogflow CX Generative Playbook integration with Vertex AI Search. The system is actively hallucinating and generating non-existent URLs for information that is correctly indexed in the Data Store.

This issue is occurring despite setting the strictest grounding rules and renders the chatbot unusable for production, as it directs users to 404 error pages, severely damaging trust.

  1. The Critical Issue (URL Hallucination)

  2. System Action: When asked about indexed content (e.g., “Group Insurance”), the Generative Playbook successfully extracts the correct content text from the Data Store.

  3. Failure: The Playbook fails to quote the correct, indexed source URL (e.g., .../insurance.html).

  4. Hallucination: Instead of quoting the correct URL, it invents a non-existent, plausible-looking URL (e.g., .../insurance/index.html,.../insurance_service/index.html) and attempts to output it, resulting in a Status 400 (Bad Request) error on the webhook and a 404 for the user.

  5. Content Creation: The system also exhibits related hallucination by inventing content not present in the Data Store (e.g., mentioning “Safety Mutual Aid” in the context of the “Union Overview”).

2

There is a critical bug in the Dialogflow CX Generative Playbook integration with Vertex AI Search, where the system hallucinates non-existent URLs and content despite strict grounding rules. While it correctly extracts content from the Data Store, it generates plausible-looking but incorrect URLs, leading to 404 errors, and invents content not present in the source. This issue makes the chatbot unusable in production and undermines user trust, requiring urgent attention to fix URL and content hallucinations.