Last quarter, our small engineering team made the call to move our internal customer support knowledge assistant off a SaaS-hosted agent platform to a self-hosted stack. We’d grown frustrated with the platform’s opaque usage caps and limited control over data access, so we expected to spend time reconfiguring the LLM and vector database layers. What we didn’t anticipate was how much work would be required to rebuild our existing knowledge assets, which were tightly tied to the SaaS’s proprietary formatting and workflows. All of our curated knowledge base entries—including tagged source documents, optimized text chunks, and linked prompt templates—were stored in a database schema that had no official export tool compatible with other systems. Manually reformatting each of the thousands of entries would have taken weeks, so we shifted gears to build a migration pipeline instead.
We started by inventorying every type of knowledge asset we relied on: source PDFs linked to specific support workflows, custom retrieval thresholds set per product topic, and role-based access rules that limited which team members could view certain knowledge groups. We first pulled raw exports from the SaaS platform, only to find that the exported data was locked into a non-standard JSON format with nested fields that didn’t align with common RAG tooling standards. We wrote a series of lightweight Python scripts to parse these exports, extract the core chunked text and associated metadata, and convert them into a consistent, flat file structure. We referenced the FastGPT documentation to confirm how to structure our metadata files so that the self-hosted retrieval pipeline could properly index and serve the correct chunks during queries.
Once the asset formatting was standardized, we deployed the self-hosted stack on our internal Kubernetes cluster, pointing the vector database to our new directory of markdown files and JSON metadata. We tested the retrieval accuracy against a sample set of past support tickets, and found that we needed to adjust the chunk overlap settings slightly to match the relevance rankings we’d grown accustomed to from the SaaS platform. We also rebuilt the access control layer, moving the SaaS’s role-based rules into our cluster’s existing identity provider, since the self-hosted stack didn’t include built-in user management that matched our team’s existing workflows. We updated our internal Slack bot integration to point to the new stack’s API endpoint, making sure the response formatting matched what our support team was used to.
The biggest takeaway from this project was how easy it is to overlook the hidden dependencies tied to your knowledge assets. Even small choices made when setting up a SaaS platform—like how text is chunked, how metadata is stored, and how prompts are linked to specific data subsets—become critical when moving to a self-hosted stack. We spent extra hours fixing edge cases, like duplicate metadata entries and truncated source text, that we hadn’t accounted for during the initial planning phase. There’s no single universal process for this migration, but taking the time to fully inventory your knowledge assets and their associated workflows before starting will cut down significantly on unplanned work.
