LLM Business Assistant

LLM Business Assistant

This LLM assistant answers KBLI business-classification questions with cited sources, not guesses. KBLI is Indonesia's standard business classification system; every company needs the right code during licensing (OSS). Getting a code wrong isn't a typo. It can mean the wrong permits, the wrong tax treatment, or a licensing office kicking a filing back.

Challenge

A generic chatbot answering from an LLM's training data produces plausible-sounding KBLI codes with confidence. That's true whether or not the code actually exists or fits the business. The corpus itself doesn't help.

Thousands of codes span two active KBLI versions, cross-referenced against OSS risk levels, sector regulations, and 2020-to-2025 migration mappings. That's far more than fits in a single prompt, and far too easy for a model to summarize from memory instead of the source text.

Solution

I built a RAG pipeline over the classification corpus first. Official code definitions get parsed into retrieval-ready evidence cards. Those cards are indexed with hybrid search: SQLite full-text search for exact terminology, plus dense embeddings for informal descriptions. Everything is reranked deterministically before it reaches the model.

On top of that runs an agentic tool-calling loop I wrote from scratch. The model doesn't answer from memory; it calls tools instead.

It searches candidate codes, pulls a cited detail record, and compares codes side by side. It assembles a full code suite for a new company's stated business activities, or audits a client's existing code list. Every tool result carries its source page forward into the answer.

The grounding check goes further than "does this look right." A separate relevance-judge pass makes the model quote a verbatim span from the official code text. Only then can it call a code relevant to the business. A claim it can't quote gets dropped back to the deterministic retrieval score instead of trusted outright.

Cost is controlled per task, not per conversation. The recommendation pass, the relevance judge, and the enrichment pass each get their own model choice. A daily budget cap and a per-turn deadline mean a heavy session degrades gracefully. It returns an honest "at capacity, try again" instead of a slow call or a rushed, ungrounded answer.

Impact

The assistant now answers classification questions with cited sources, not confident guesses. It separates a business's primary activities from its supporting ones. It also audits a client's existing code list against what the business actually does. Anything outdated or unsupported by the official text gets flagged instead of rubber-stamped.

Every turn logs its verification status. Grounding failures are something the system can measure, not something a user discovers later.